Getting Started with Think Models

This guide walks you through using Think Models with the evroc CLI. Think Models provides access to leading open-source LLMs through two deployment options: Shared Models (multi-tenant, instant access) and Dedicated Models (single-tenant, isolated instances).

Prerequisites

1. Install the evroc CLI
2. Authenticate with evroc login

Shared Models

Shared models are pre-configured, multi-tenant endpoints that provide instant access to popular open-source LLMs. They're ideal for development, testing, and production workloads that don't require dedicated resources. Even though Shared Models are running on shared infrastructure, we guarantee that all your (output)prompts are isolated per tenant.

Step 1: List Available Shared Models

View all shared models available in your project:

evroc think shared

Step 2: Create an API Key

Create an API key to authenticate your inference requests:

evroc think apikey create my-dev-key

To create a key with an expiration date:

evroc think apikey create my-dev-key --expires-at 2025-12-31T23:59:59Z

Save the returned token—this is your API key for inference requests.

Step 3: List Your API Keys

View all API keys in your project:

evroc think apikey list

Step 4: Make Inference Requests

Use your API key with any OpenAI-compatible client. The Think API follows the OpenAI API specification.

Using curl:

curl -X POST "https://models.think.cloud.evroc.com/v1/chat/completions" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-oss-120b",
    "messages": [
      {"role": "user", "content": "Explain Kubernetes in one sentence."}
    ],
    "max_tokens": 100
  }'

Using the OpenAI Python SDK:

from openai import OpenAI

client = OpenAI(
    base_url="https://models.think.cloud.evroc.com/v1",
    api_key="your-api-key"
)

response = client.chat.completions.create(
    model="openai/gpt-oss-120b",
    messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)

Step 5: Delete an API Key

When you no longer need an API key:

evroc think apikey delete my-dev-key

Dedicated Models

Dedicated models provide isolated, single-tenant instances with configurable resources. Use dedicated models when you need guaranteed capacity and custom configurations.

Step 1: List Available Models

View all models available for dedicated instances:

evroc think models list

Or use the shorthand:

evroc think models ls

Step 2: List Available Sizes

View available instance sizes and their resource allocations:

evroc think sizes list

Step 3: Create a Dedicated Instance

Create a new dedicated model instance:

evroc think modelinstance create my-gpt-oss \
  --model openai/gpt-oss-120b \
  --size 1-b200-27c-240g \
  --token my-instance-auth-token

Or use the shorthand:

evroc think mi create my-gpt-oss -m openai/gpt-oss-120b -s medium -t my-instance-auth-token

Step 4: Check Instance Status

List your instances to check their status:

evroc think mi list

Wait for the instance status to show Running and ready: true before making inference requests.

Step 5: View Instance Logs

Monitor your instance startup and runtime logs:

evroc think mi logs my-gpt-oss

Step 6: Make Inference Requests

Once your instance is ready, use the endpoint with the token you specified during creation:

curl -X POST "https://models.evroc.com/<org>/<project>/my-gpt-oss/v1/chat/completions" \
  -H "Authorization: Bearer my-instance-auth-token" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-oss-120b",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Step 7: Manage Instance Lifecycle

Stop an instance (preserves configuration, stops modelinstance billing):

evroc think mi stop my-gpt-oss

Start a stopped instance:

evroc think mi start my-gpt-oss

Update an instance (change size):

evroc think mi update my-gpt-oss --size 1-b200-27c-240g

Delete an instance:

evroc think mi delete my-gpt-oss

Quick Reference

Additional Resources

• evroc CLI Reference
• API Key Management
• Model Instance Management