PFS Registry Setup

HPE Machine Learning Data Management offers a built-in registry for storing your models through its Pachyderm File System (PFS). This guide explains how to set up and use PFS as a model registry.

Before You Start #

Review the official HPE Machine Learning Data Management documentation to understand the prerequisites and configuration options.
Review how Data Versioning works in HPE Machine Learning Data Management to understand how to manage your model versions.
Complete the Beginner Tutorial to get a sense of how HPE Machine Learning Data Management works.

How to Set Up a PFS Registry #

The following steps are minimal instructions to get you started with a PFS registry – namely, creating a project and a repository to store a model.

Install and Deploy HPE Machine Learning Data Management #

Refer to the official setup guides based on your environment:

Self-Signed Certificates

If your pachd installation uses HTTPS with a self-signed certificate, you may need to configure TLS for the external repository before adding it as a registry.

To obtain the authentication token for your HPE Machine Learning Data Management cluster, run the following commands:

pachctl connect <PACHD_ADDRESS>:<PORT>
pachctl auth login
pachctl config get context <PACHD_ADDRESS>:<PORT> | jq -r '.session_token'

Example:

AUTH_TOKEN_ENV=$(pachctl config get context http://34.44.25.160:80 | jq -r '.session_token')

Create a Project #

Create a project in HPE Machine Learning Data Management to store your models.
```
pachctl create project <PROJECT_NAME>
```
Set the project as current.
```
pachctl config update context --project <PROJECT_NAME>
```
See the pachctl create project and pachctl config update context CLI commands for more details.

Create a Repository #

Create a repository to store your model.
```
pachctl create repo <REPO_NAME>
```
Verify that the repository was created.
```
pachctl list repo
```
See the pachctl create repo and pachctl list repo CLI commands for more details.

Upload a Model #

Warning

For best integration between PFS and MLIS, we recommend that you store only one model per repo and place it in the top-level path of the master branch.

Upload the model to the repository.

pachctl put file -r <REPO_NAME>@<BRANCH_NAME>:<NEW_DIR_NAME> -f <PATH_TO_MODEL_DIR>

Verify that the model was uploaded.
```
pachctl list files <REPO_NAME>@<BRANCH_NAME>
```
See the pachctl put file and pachctl list file CLI command for more details.

Next Steps #

You are now ready to Add a registry to MLIS using HPE Machine Learning Data Management’s PFS.