Model Masters
What a model master stores, how a Managed Inference Job uses one, and where you manage them.
A model master stores what CosmicAC knows about a model. That includes its runtime image and default serving parameters. It saves you from configuring every field each time you serve the model.
How a job uses one
When you select a model while creating a Managed Inference Job, its model master prefills the Serving configuration. You can adjust the prefilled values before you submit the job, except the runtime image. The web interface offers the model master's runtime image as the only runtime image for that model, so change it in the model master rather than on the job.
A model master also sets which parameters raise a job creation warning, and in which direction. See Set the job creation warnings.
A vLLM job serves any model vLLM itself supports, once that model has a model master. For a model with a published vLLM recipe, CosmicAC builds the model master from the recipe rather than asking you for each value.
Where you manage them
You manage model masters in two places. From the Models page in the web interface, you edit the model master of any model your jobs serve. Through the cosmicac-app-node API, you reach every model master, including one for a model you don't serve yet.
Use the web interface to add a model master from a vLLM recipe, or to update one. For everything else, including adding a model master manually and archiving one, use the API.
After you deploy CosmicAC on your host machine, add a model master for each supported model. See Set up model masters. For the values to use, see Recommended model parameters.
In the web interface
Through the CosmicAC API
Next steps
- Set up model masters after you deploy CosmicAC
- Recommended model parameters for each supported model
- Model commands in the CLI reference
- Managed Inference Job for what a model master feeds into