Connect your Superb.ai project to HPE Machine Learning Data Management to automatically version and save data you’ve labeled in Superb AI to use in downstream machine learning workflows.

This integration ingests the data into HPE Machine Learning Data Management on a cron schedule. Once your data is ingested into HPE Machine Learning Data Management, you can perform data tests, train a model, or any other type of data automation you may want to do, all while having full end-to-end reproducibility.

Before You Start #

• You must have a Superb.AI account
• You must have a HPE Machine Learning Data Management cluster
• • Download the example code and unzip it. (or download this repo. gh repo clone pachyderm/docs-content and navigate to docs-content/docs/latest/integrate/superb-ai)

How to Use the Superb AI Connector #

1. Generate an Access API Key in SuperbAI.
2. Put the key and your user name in the secrets.json file.
3. Create the Pachyderm secret

   pachctl create secret -f secrets.json

   Copy
4. Create the cron pipeline to synchronize your Sample project from SuperbAI to HPE Machine Learning Data Management. This pipeline will run every minute to check for new data (you can configure it to run more or less often in the cron spec in sample_project.yml).

   pachctl create pipeline -f sample_project.yml

   Copy
5. HPE Machine Learning Data Management will automatically kick off the pipeline and import the data from your sample project.