Custom Config

aana.core.models.custom_config

CustomConfig

CustomConfig = dict

A custom configuration field that can be used to pass arbitrary configuration to the deployment.

For example, you can define a custom configuration field in a deployment configuration like this:

class HfPipelineConfig(BaseModel):
    model_id: str
    task: str | None = None
    model_kwargs: CustomConfig = {}
    pipeline_kwargs: CustomConfig = {}
    generation_kwargs: CustomConfig = {}

Then you can use the custom configuration field to pass a configuration to the deployment:

HfPipelineConfig(
    model_id="Salesforce/blip2-opt-2.7b",
    model_kwargs={
        "quantization_config": BitsAndBytesConfig(
            load_in_8bit=False, load_in_4bit=True
        ),
    },
)