From Registry (UI)
Before You Start #
- Set up a registry
- Confirm that your model is available in your chosen registry
Basic Details #
- Sign in to HPE Machine Learning Inferencing Software.
- Navigate to Packaged Models.
- Select Add new model.
- Input details for the following:
- Name: The name of the model within HPE Machine Learning Inferencing Software.
- Description: A brief description of the model.
- Select Next.
Storage Details #
- Input details for the following:
- Registry: The registry where the model is stored.
- Model Format: Options are
OpenLLM
,Bento archive
,NIM
,Custom
. - Image: The container image servicing the model; must be the name of the image + a release tag. For NIM, see the NGC catalog for the image options.
- URL/Path: The URL to the model object in the registry. (e.g.,
s3://<BUCKET_NAME>/models/<MODEL_NAME>.tar.gz
)Prefix Description openllm:// An openllm model name from huggingface.co dynamically loaded and executed with a VLLM backend. s3:// An openllm model path which will be dynamically downloaded from an associated s3 registry bucket and executed with a VLLM backend. pvc:// A PVC model path that can be used for pre-downloaded NIM and Custom models.
- Select Next.
Resource Templates #
-
Choose a Resource Template or define custom resources.
Name Description Request CPU Request Memory Request GPU Limit CPU Limit Memory Limit GPU cpu-tiny 1 cpu, 10Gi memory, no gpu per replica 1 10Gi 1 10Gi cpu-small 4 cpu, 20Gi memory, no gpu per replica 4 20Gi 6 40Gi cpu-large 8 cpu, 40Gi memory, no gpu per replica 8 40Gi 10 60Gi gpu-tiny 1 cpu, 10Gi, 1 gpu per replica 1 10Gi 1 1 10Gi 1 gpu-small 2 cpu, 20Gi, 2 gpu per replica 2 20Gi 2 6 40Gi 2 gpu-large 8 cpu, 40Gi, 4 gpu per replica 8 40Gi 4 10 60Gi 4 GPU TypeSpecifying a GPU type requires heterogenous GPU support be enabled. -
Select Next.
Environment Variables & Arguments #
Environment variables and arguments are advanced configuration options that you can set for your packaged model. These inputs will vary based on your model’s requirements. For more information, see the Advanced Configuration reference article.
- Provide any needed Environment Variables.
- Provide any needed Arguments.
- Select Create model.