Skip to content

feat: multi-provider routing + /v1/usage reporting endpoint#105

Merged
devin-ai-integration[bot] merged 5 commits into
developfrom
feat/agentic-multi-provider
Jun 15, 2026
Merged

feat: multi-provider routing + /v1/usage reporting endpoint#105
devin-ai-integration[bot] merged 5 commits into
developfrom
feat/agentic-multi-provider

Conversation

@pyramation

@pyramation pyramation commented Jun 14, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds multi-provider routing + /v1/usage reporting endpoint to the agentic server, and extracts the MetaSchema-based table resolution into a shared @constructive-io/usage-loader workspace package.

Multi-provider routing:

  • Routes by X-Provider header, model prefix (anthropic/claude-3.5-sonnet), or known patterns (claude-* → anthropic, gpt-* → openai, llama-* → ollama)
  • Anthropic fully supported: x-api-key auth, system message extraction, response transforms to OpenAI format
  • GET /v1/providers — lists configured providers
  • POST /v1/usage — fire-and-forget endpoint for Python functions to report inference usage

@constructive-io/usage-loader package:

// Before: hardcoded schema in 3 separate files
pool.query(`INSERT INTO "constructive_usage_public".platform_usage_log_computes ...`)

// After: single canonical loader, MetaSchema-resolved, 60s TTL cache
import { UsageLoader } from '@constructive-io/usage-loader';
const loader = new UsageLoader(pool, databaseId);
loader.logComputeUsage(entry);   // resolves table from metaschema_public.table
loader.logInferenceUsage(entry);
loader.logStorageUsage(entry);

All three consumers (worker/compute-meter, agentic-server/inference-meter, fn-runtime/storage) now import from this shared package. job/worker/usage-client.ts is a thin re-export for backward compat.

Tests: 119 integration + 25 unit, all green.

Link to Devin session: https://app.devin.ai/sessions/b2291a8e333e445aa125a2efd1996206
Requested by: @pyramation

- Refactor agentic-server router for multi-provider support:
  - Route by X-Provider header, model prefix (anthropic/claude-3), or
    known model patterns (gpt-* → openai, claude-* → anthropic, llama-* → ollama)
  - Anthropic API support: x-api-key auth, system message extraction,
    response transform to OpenAI-compatible format
  - LLM_PROVIDERS env var for multi-provider standalone config
  - GET /v1/providers endpoint to list configured providers
  - Backward-compatible: single provider config still works

- Add POST /v1/usage endpoint for external usage reporting:
  - Python functions (LlamaParse, HuggingFace, etc.) can report inference
    usage for billing without proxying through the agentic server
  - Accepts model, provider, tokens, latency, status; fires metering INSERT
  - Returns 202 Accepted (fire-and-forget)
  - Validates model is required, defaults service to 'chat'

- 17 new integration tests covering all routing + usage scenarios
@devin-ai-integration

Copy link
Copy Markdown

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

- All metering (compute/inference/storage) now resolves table names from
  MetaSchema module registration tables instead of hardcoding schema names
- Each meter uses a 60s TTL cache for resolved table references
- Falls back to well-known defaults when MetaSchema is unavailable
- fn-runtime/storage.ts: self-contained MetaSchema resolution for S3 metering
- inference-meter.ts: self-contained MetaSchema resolution for LLM metering
- storage-meter.ts in worker: delegates to UsageClient (same directory)
- Updated storage-meter tests to account for MetaSchema lookup queries
- Create packages/usage-loader/ as canonical MetaSchema table resolver
- UsageLoader class: resolves table names from metaschema_public with 60s TTL
- All metering (compute, inference, storage) imports from shared package
- job/worker/usage-client.ts is now a thin re-export for backward compat
- inference-meter.ts (agentic-server): delegates to UsageLoader
- storage-meter.ts (worker): delegates to UsageLoader
- compute-meter.ts (worker): delegates to UsageLoader
- fn-runtime/storage.ts: uses UsageLoader for createMeterCallback
- 119 integration + 25 unit tests passing
- Create packages/module-loader/ with TtlCache, ComputeModuleLoader,
  UsageLoader, BillingLoader, and unified ModuleLoader facade
- Multi-database support: all loaders accept databaseId for platform vs tenant
- Add GraphExecutionModuleConfig to resolve node_states, complete_node,
  fail_node table/function names from MetaSchema
- Remove hardcoded constructive_compute_public/private refs from compute-worker
  graph operations (markNodeRunning, completeGraphNode, failGraphExecution)
- Migrate compute-worker to import from shared package (cache, module-loader,
  billing, compute-log, discovery, invocation all delegate to module-loader)
- usage-loader is now a thin re-export from module-loader for backward compat
- 119 integration + 25 unit tests passing
- Update job/worker/src/graph-complete.ts to use ComputeModuleLoader
  for resolving complete_node and fail_node function names
- Remove last hardcoded constructive_compute_private refs
- Fix inline-nodes tests to handle MetaSchema resolution queries
- Add _resetGraphCompleteCache() for test isolation
@devin-ai-integration devin-ai-integration Bot merged commit 4aca61c into develop Jun 15, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant