Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -413,6 +413,49 @@ If your LiteLLM catalog mixes naming conventions from different teams or environ
- Filtering happens before the on-disk cache is written, so each provider's cached view respects its own filters.
- Provider IDs beginning with `litellm-` are recognized automatically. Add `settings.litellm: true` to mark a provider with another ID.

### Declaring providers in the plugin options (`providers`, OpenCode 2)

On OpenCode 2, providers declared under the config's `providers` key are registered by the host *after* plugin setup. A one-shot command such as `opencode run` resolves its model before the plugin has published anything, so with a cold cache it fails with `Model unavailable`. Declaring the provider in the plugin's own options avoids that, because the plugin then discovers models during setup. The catch is that the flat options describe a single provider, and when the same package is listed twice only one of the entries takes effect.

To get several providers (for example one proxy split by prefix, or two different proxies) and still start on a cold cache, declare them under `providers` in the plugin options, keyed by provider id and written the way the config's `providers` key is:

```jsonc
{
"plugins": [
{
"package": "opencode-plugin-litellm",
"options": {
"providers": {
"litellm": {
"name": "Prod",
"settings": {
"baseURL": "http://localhost:4000/v1",
"apiKey": "{env:LITELLM_API_KEY}",
"excludeModels": ["staging/*"]
}
},
"litellm-staging": {
"name": "Staging",
"settings": {
"baseURL": "http://staging.internal:4000/v1",
"apiKey": "{env:STAGING_LITELLM_API_KEY}",
"includeModels": ["staging/*"]
}
}
}
}
}
]
}
```

- A provider declared here goes through the same code as one declared in the config, so a block can be moved between the two places as is. It reads `name` (defaults to the id), `package` (defaults to the OpenAI-compatible runtime), `settings` (everything documented above, including `baseURL`, `apiKey`, `includeModels` and `excludeModels`) and `headers`.
- Each provider is self-contained. Its address, key, headers and filters come from its own declaration, and the plugin-wide options (the flat keys next to `providers`) are not applied to it, so one provider can never pick up another's credentials.
- Providers are discovered at the same time, so a slow proxy costs the longest wait instead of the sum.
- If the config's `providers` key declares the same id, the plugin options win.
- Entries that are not objects are ignored. Without `providers`, nothing changes.
- This applies to OpenCode 2 only. On OpenCode 1, declare the providers under the config's `provider` key as shown above.

### Correcting capability flags (`modelCapabilities`)

Model classification (tool-call badge, attachments, reasoning, input modalities) leans on the capability flags LiteLLM reports via `/v1/model/info`. If your proxy doesn't expose that endpoint, or reports a flag that doesn't match your deployment, override flags per model id:
Expand Down
57 changes: 55 additions & 2 deletions src/plugin/v2.ts
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,38 @@ function asRecord(value: unknown): Record<string, unknown> {
: {}
}

/**
* Providers declared in the plugin options under `providers`, keyed by provider
* id, in the same shape OpenCode 2 accepts under the config's `providers` key
* (`name`, `package`, `settings`, `headers`). They go through the same code
* path as a provider declared in the config, with one difference that matters:
* they exist during `setup`, so a one-shot `opencode run` on a cold cache can
* already use them. The host only registers the config's `providers` after
* plugin setup, and it keeps a single instance per plugin id, so a second
* entry of this package cannot declare them.
*
* Each provider is self-contained: its address, credentials, filters and
* headers come from its own declaration, and the plugin-wide options are not
* applied to it. Anything that is not an object is ignored.
*/
function readDeclaredProviders(options: Record<string, unknown>): ProviderInfo[] {
return Object.entries(asRecord(options.providers)).flatMap(([id, value]) => {
if (typeof value !== 'object' || value === null || Array.isArray(value)) return []
const entry = value as Record<string, unknown>
const info = Provider.Info.empty(Provider.ID.make(id))
return [
{
...info,
name: readString(entry.name) ?? id,
activation: 'enabled',
...(readString(entry.package) ? { package: readString(entry.package)! } : {}),
settings: asRecord(entry.settings),
headers: readHeaders(entry.headers),
} as ProviderInfo,
]
})
}

function readHeaders(value: unknown): Record<string, string> {
const result: Record<string, string> = {}
for (const [key, header] of Object.entries(asRecord(value))) {
Expand Down Expand Up @@ -638,9 +670,14 @@ async function reconcileProviderSources(
context: Context,
pluginOptions: Record<string, unknown>,
sources: Map<string, ProviderSource>,
declaredIds: ReadonlySet<string> = new Set(),
): Promise<ProviderSource[]> {
const changed: ProviderSource[] = []
for (const provider of await listMatchingProviders(context)) {
// A provider declared in the plugin options is explicit plugin config. The
// host reflects the plugin's own publication back under the same id, and a
// config declaration of the same id is a conflict the options win.
if (declaredIds.has(provider.id)) continue
const existing = sources.get(provider.id)
// Skip sources that are both configured (not a fallback) and already
// published; a pending configured source must be returned so it is retried.
Expand Down Expand Up @@ -720,7 +757,12 @@ const definition = Plugin.define({

initV2Logging()

const pluginOptions = asRecord(context.options)
const rawOptions = asRecord(context.options)
const declaredInfos = readDeclaredProviders(rawOptions)
// `providers` is consumed only when it declares something. Any other value
// keeps being handed on with the rest of the options, as before.
const { providers: _declared, ...withoutProviders } = rawOptions
const pluginOptions = declaredInfos.length > 0 ? withoutProviders : rawOptions
const sources = new Map<string, ProviderSource>()
// Providers the dialog replaced under a new id; the transform below
// removes them from the registry on the next reload.
Expand All @@ -731,10 +773,21 @@ const definition = Plugin.define({
// integration API, or a failed registration, keeps every other auth path.
const integrationRegistration = await registerLiteLLMIntegration(context)

// Providers declared in the plugin options are discovered at once, so a slow
// gateway costs the longest wait, not the sum (the host gives setup ~5 s).
// They are self-contained, hence no plugin-wide options.
const declaredIds = new Set<string>(declaredInfos.map((info) => info.id))
if (declaredInfos.length > 0) {
const declared = await Promise.all(
declaredInfos.map((info) => makeProviderSource(context, info, {})),
)
for (const source of declared) if (source) sources.set(source.id, source)
}

// First pass. On OpenCode 2 the host has not registered user-configured
// providers yet, so this typically finds only built-ins; the event loop
// below reconciles again when they appear.
await reconcileProviderSources(context, pluginOptions, sources)
await reconcileProviderSources(context, pluginOptions, sources, declaredIds)
// With no configured provider, fall back to purely option/env-driven
// discovery so `LITELLM_BASE_URL` and plugin options keep working.
if (sources.size === 0) {
Expand Down
Loading
Loading