Manage Packaged Models

A packaged model describes the model and user code that you want to deploy as an inference service.

Model Packaging Options

The following model types and registry options are supported:

Model TypeRegistries
Bento ArchiveS3, PFS
CustomOpenLLM, PVC, S3, PFS, None
NIMNGC, PVC
OpenLLMOpenLLM, S3, PFS
vLLMHuggingFace, S3, PFS

Service Deployment Journey

Adding a packaged model is the third step in the service deployment journey. It requires that you have already created a compatible image available.