Manage Packaged Models #
A packaged model describes the model and user code that you want to deploy as an inference service.
Model Packaging Options #
The following model types and registry options are supported:
Model Type | Registries |
---|---|
Bento Archive | S3, PFS |
Custom | OpenLLM, PVC, S3, PFS, None |
NIM | NGC, PVC |
OpenLLM | OpenLLM, S3, PFS |
vLLM | HuggingFace, S3, PFS |
Service Deployment Journey #
Adding a packaged model is the third step in the service deployment journey. It requires that you have already created a compatible image available.