Expand description
Dynamic model metadata discovery via provider APIs.
This module queries provider APIs at runtime to discover actual model capabilities (context length, output limits, features) rather than relying on static mappings.
§Supported Providers
| Provider | Endpoint | API field(s) used | Stored as context_length |
|---|---|---|---|
| Gemini | GET /v1beta/models | inputTokenLimit + outputTokenLimit | input + output |
| Anthropic | GET /v1/models | max_input_tokens + max_output_tokens | input + output |
| Grok (xAI) | GET /v1/language-models | input_modalities.text.token_limit + output | input + output |
| Ollama | POST /api/show | model_info.*.context_length (combined) | as-is (already combined) |
| OpenAI | N/A | Uses error-driven learning | N/A |
For providers that expose separate input and output limits, discovery stores
input_limit + output_limit as context_length so that
ModelCapabilities::max_input_tokens() (which subtracts max_output_tokens)
recovers the correct prompt budget without double-subtracting.
Functions§
- discover_
model_ capabilities - Discover model capabilities from provider API and store in cache.