> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/opencode-ai/opencode/llms.txt
> Use this file to discover all available pages before exploring further.

# Self-hosted model providers

> Configure OpenCode to use self-hosted and local model providers

OpenCode supports using self-hosted model providers that implement an OpenAI-compatible API. This allows you to run models locally or connect to custom inference servers.

## Overview

The local provider enables you to:

* Run models on your own hardware
* Use custom inference servers (Ollama, LM Studio, vLLM, etc.)
* Experiment with fine-tuned or custom models
* Maintain full control over your data and privacy

<Note>
  Your self-hosted provider must implement an OpenAI-compatible API for tool calling and streaming.
</Note>

## Configuration

### Setting the endpoint

Configure your self-hosted provider endpoint using the `LOCAL_ENDPOINT` environment variable:

```bash theme={null}
export LOCAL_ENDPOINT="http://localhost:1234/v1"
```

The endpoint should include the full base URL including the API version path (usually `/v1`).

### Configuring models

Once the endpoint is set, configure your agents to use local models in your `.opencode.json`:

```json .opencode.json theme={null}
{
  "agents": {
    "coder": {
      "model": "local.granite-3.3-2b-instruct@q8_0",
      "maxTokens": 5000
    },
    "task": {
      "model": "local.llama-3.3-70b-instruct",
      "maxTokens": 4096
    },
    "title": {
      "model": "local.granite-3.3-2b-instruct@q8_0",
      "maxTokens": 80
    }
  }
}
```

<Info>
  Model names use the format `local.<model-name>` where `<model-name>` matches the model identifier in your inference server.
</Info>

## Popular inference servers

### Ollama

[Ollama](https://ollama.ai) provides an easy way to run models locally.

<Steps>
  <Step title="Install Ollama">
    Download and install Ollama from [ollama.ai](https://ollama.ai)
  </Step>

  <Step title="Pull a model">
    ```bash theme={null}
    ollama pull llama3.3:70b
    ```
  </Step>

  <Step title="Configure OpenCode">
    Set the Ollama API endpoint:

    ```bash theme={null}
    export LOCAL_ENDPOINT="http://localhost:11434/v1"
    ```

    Update your configuration:

    ```json .opencode.json theme={null}
    {
      "agents": {
        "coder": {
          "model": "local.llama3.3:70b",
          "maxTokens": 5000
        }
      }
    }
    ```
  </Step>
</Steps>

### LM Studio

[LM Studio](https://lmstudio.ai) provides a desktop application for running models.

<Steps>
  <Step title="Install LM Studio">
    Download and install LM Studio from [lmstudio.ai](https://lmstudio.ai)
  </Step>

  <Step title="Load a model">
    Use LM Studio's interface to download and load a model
  </Step>

  <Step title="Start the server">
    Enable the local server in LM Studio (usually runs on port 1234)
  </Step>

  <Step title="Configure OpenCode">
    ```bash theme={null}
    export LOCAL_ENDPOINT="http://localhost:1234/v1"
    ```

    ```json .opencode.json theme={null}
    {
      "agents": {
        "coder": {
          "model": "local.your-model-name",
          "maxTokens": 5000
        }
      }
    }
    ```
  </Step>
</Steps>

### vLLM

[vLLM](https://github.com/vllm-project/vllm) is a high-throughput inference server.

<Steps>
  <Step title="Install vLLM">
    ```bash theme={null}
    pip install vllm
    ```
  </Step>

  <Step title="Start the server">
    ```bash theme={null}
    python -m vllm.entrypoints.openai.api_server \
      --model meta-llama/Llama-3.3-70B-Instruct \
      --port 8000
    ```
  </Step>

  <Step title="Configure OpenCode">
    ```bash theme={null}
    export LOCAL_ENDPOINT="http://localhost:8000/v1"
    ```

    ```json .opencode.json theme={null}
    {
      "agents": {
        "coder": {
          "model": "local.meta-llama/Llama-3.3-70B-Instruct",
          "maxTokens": 5000
        }
      }
    }
    ```
  </Step>
</Steps>

### Text Generation WebUI

[Text Generation WebUI](https://github.com/oobabooga/text-generation-webui) provides an OpenAI-compatible API extension.

<Steps>
  <Step title="Enable OpenAI extension">
    Enable the `openai` extension in the WebUI settings
  </Step>

  <Step title="Configure OpenCode">
    ```bash theme={null}
    export LOCAL_ENDPOINT="http://localhost:5000/v1"
    ```

    ```json .opencode.json theme={null}
    {
      "agents": {
        "coder": {
          "model": "local.your-model-name",
          "maxTokens": 5000
        }
      }
    }
    ```
  </Step>
</Steps>

## Advanced configuration

### Reasoning effort for reasoning models

Some self-hosted models support reasoning modes. Configure reasoning effort if your model supports it:

```json .opencode.json theme={null}
{
  "agents": {
    "coder": {
      "model": "local.deepseek-r1-70b",
      "maxTokens": 5000,
      "reasoningEffort": "high"
    }
  }
}
```

Valid values:

* `low` - Faster responses with less reasoning
* `medium` - Balanced approach (default)
* `high` - More thorough reasoning

### Custom headers

If your inference server requires custom headers, you can pass them through OpenCode's provider options (requires code modification).

### Authentication

For self-hosted endpoints that require authentication, you can set an API key:

```json .opencode.json theme={null}
{
  "providers": {
    "local": {
      "apiKey": "your-api-key-here"
    }
  }
}
```

## Model requirements

For the best experience with OpenCode, your self-hosted model should support:

<AccordionGroup>
  <Accordion title="Tool calling">
    OpenCode relies heavily on tool/function calling for file operations, code execution, and more. Your model must support the OpenAI tools API format.

    Models known to work well:

    * Llama 3.3 70B Instruct
    * Qwen 2.5 Coder
    * Granite 3.1 (IBM)
    * Mistral Large
  </Accordion>

  <Accordion title="Streaming">
    Streaming responses provide a better user experience. Your inference server should support server-sent events (SSE) streaming.
  </Accordion>

  <Accordion title="Context window">
    A larger context window (32K+ tokens) is recommended for complex coding tasks. Configure `maxTokens` according to your model's capabilities.
  </Accordion>
</AccordionGroup>

## Troubleshooting

<AccordionGroup>
  <Accordion title="Connection refused error">
    **Symptoms:** Cannot connect to local endpoint

    **Solution:**

    * Verify your inference server is running
    * Check the endpoint URL is correct (including port and `/v1` path)
    * Ensure there are no firewall rules blocking the connection

    ```bash theme={null}
    # Test the endpoint
    curl http://localhost:1234/v1/models
    ```
  </Accordion>

  <Accordion title="Model not found">
    **Symptoms:** Error indicating the model doesn't exist

    **Solution:**

    * Check your model name matches exactly what's loaded in your inference server
    * List available models:

    ```bash theme={null}
    curl http://localhost:1234/v1/models
    ```
  </Accordion>

  <Accordion title="Tool calling not working">
    **Symptoms:** The model doesn't use tools or returns invalid tool calls

    **Solution:**

    * Ensure your model supports function/tool calling
    * Check that your inference server properly implements the OpenAI tools API
    * Try a different model known to support tool calling (e.g., Llama 3.3)
  </Accordion>

  <Accordion title="Slow response times">
    **Symptoms:** Responses take a long time to generate

    **Solution:**

    * Use a smaller, faster model for the `task` and `title` agents
    * Enable GPU acceleration in your inference server
    * Reduce `maxTokens` for faster responses
    * Consider quantized models (Q4, Q8) for better performance
  </Accordion>

  <Accordion title="Out of memory errors">
    **Symptoms:** Inference server crashes or returns memory errors

    **Solution:**

    * Use a smaller model or quantized version
    * Reduce the context window size
    * Allocate more RAM/VRAM to your inference server
    * Enable CPU offloading if using GPU inference
  </Accordion>
</AccordionGroup>

## Performance tips

<CardGroup cols={2}>
  <Card title="Use appropriate model sizes" icon="gauge-high">
    * Large models (70B+) for complex tasks (coder agent)
    * Small models (7B-13B) for simple tasks (title agent)
  </Card>

  <Card title="Enable GPU acceleration" icon="microchip">
    Most inference servers support GPU acceleration for significant speed improvements.
  </Card>

  <Card title="Quantization" icon="compress">
    Use quantized models (Q4\_K\_M, Q8\_0) to reduce memory usage while maintaining quality.
  </Card>

  <Card title="Batch processing" icon="layer-group">
    Configure your inference server for optimal batch size and parallel requests.
  </Card>
</CardGroup>

## Example configurations

### Development setup (fast iteration)

```json .opencode.json theme={null}
{
  "agents": {
    "coder": {
      "model": "local.qwen2.5-coder:7b-instruct-q4_K_M",
      "maxTokens": 4096
    },
    "task": {
      "model": "local.qwen2.5-coder:7b-instruct-q4_K_M",
      "maxTokens": 2048
    },
    "title": {
      "model": "local.granite-3.3-2b-instruct@q8_0",
      "maxTokens": 80
    }
  }
}
```

### Production setup (high quality)

```json .opencode.json theme={null}
{
  "agents": {
    "coder": {
      "model": "local.llama-3.3-70b-instruct",
      "maxTokens": 8192
    },
    "task": {
      "model": "local.qwen2.5-coder:32b-instruct",
      "maxTokens": 4096
    },
    "title": {
      "model": "local.llama-3.3-70b-instruct",
      "maxTokens": 80
    }
  }
}
```