Struct ModelCapabilitiesCache 

pub struct ModelCapabilitiesCache { /* private fields */ }

Thread-safe cache for learned model capabilities.

Capabilities are discovered dynamically and cached for future use. The cache persists for the lifetime of the application.

Implementations

impl ModelCapabilitiesCache

pub fn new() -> Self

Create a new empty cache

pub fn get(&self, provider: &str, model_id: &str) -> ModelCapabilities

Get capabilities for a model, using cache or falling back to heuristics

pub fn learn_from_error( &self, provider: &str, model_id: &str, error_message: &str, ) -> Option<ModelCapabilities>

Learn model capabilities from a context_length_exceeded error message.

Parses error messages like:

• “maximum context length is 16385 tokens”
• “your messages resulted in 17063 tokens”

Returns the learned capabilities if parsing succeeded.

pub fn store_from_api(&self, caps: ModelCapabilities)

Store capabilities discovered from API

pub fn store_user_override( &self, provider: &str, model_id: &str, context_length: usize, )

Store user-configured override

pub fn clear(&self)

Clear the cache (useful for testing)

Trait Implementations

impl Clone for ModelCapabilitiesCache

fn clone(&self) -> ModelCapabilitiesCache

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ModelCapabilitiesCache

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for ModelCapabilitiesCache

fn default() -> ModelCapabilitiesCache

Returns the “default value” for a type.