In this blog post, you'll learn how to use our new Github Action to continuously deploy models from an ML repository to Algorithmia.

In our recent blog post about building and deploying XGBoost models, we showed how you can do the following from within the notebook code:

  • Build a simple XGBoost model in a Jupyter notebook
  • Create the inference code that runs on Algorithmia
  • Push the inference code and requirements file and uploading the saved model file to Algorithmia

In this blog post, we’ll introduce our new Github Action that’s capable of:

  • Executing this Jupyter notebook on a Github worker machine
  • Taking the created model file that’s saved on the worker machine and uploading it to Algorithmia
  • Pushing the inference code and the requirements file to the algorithm repository
  • Linking the algorithm with the newly uploaded model file through a “model manifest”
  • Triggering a new algorithm build on Algorithmia

But why would you want this CI/CD automation in the first place?

In a typical workflow, you have a repository on Github, where you’re developing your ML models using a Jupyter notebook, and you want to continuously deploy your model to your serving layer as you iterate on it.

So when it’s time to put your model into production, you create an algorithm on Algorithmia so that you can have a scalable endpoint to serve your ML model. Algorithmia hosts your serving and inference layer—that is to say, the algorithm—in a repository that’s either backed by Github or by Algorithmia (per your own preference). So your algorithm has its own repository and you push your changes to this repository whenever you want to have a new algorithm build.

Non-automated workflow

In a non-automated workflow, when you develop your ML model, you upload it to your data collection on Algorithmia. You do this either manually via Algorithmia’s web interface, or programmatically via the data API.

If you don’t want to override your existing model files on Algorithmia, you’ll need to upload your model with a new name. This means you’ll want to change your inference script to load and use this new model file in your algorithm. (Or perhaps you have a different reason for needing to change your inference script, in which case this is also applicable.)

When you make changes in your Jupyter notebook to create a new model file, you’ll need to:

  1. Commit and push your changes in your repo containing your Jupyter notebook
  2. Take the saved model file and upload it to Algorithmia
  3. Switch to your algorithm repository, update your inference script, and commit and push in this repository to trigger a new algorithm build in Algorithmia

Automated workflow

With our Github Action integrated into your Github repository’s workflow file, you can reduce the three steps above into a single step. Once you commit and push your changes in your ML repo, your automation workflow can be triggered to deploy your model file and inference code to Algorithmia.

Sounds cool, right?

Benefits of automation

Besides reducing the number of manual operations you do in different repositories, automated model deployment ensures that your deployments are traceable to their origins. This is an important component of model governance. As a policy, some organizations require that deployments involve multiple people’s approval, and are only done through automated processes. This helps with tracing the deployed artifacts to their triggering events while also making them repeatable.

In addition, with the new model_manifest.json file we introduce with this automation, you can see where your uploaded model came from and what the MD5 hash of your model file was at the time of this automated upload. So essentially, this manifest file contains your model’s signature. In addition to having access to model metadata, your algorithm can recalculate your model file hash at inference time, compare it to the uploaded time hash, and make sure the model file hasn’t been tampered with in between.

When to use this automation

This Github Action would be a good fit for your workflow if you are either using a Jupyter notebook to train and export your ML model or if you’re checking your saved model file into your repository on Github, possibly as the result of a separate CI/CD workflow.

Optionally, you might have your inference (algorithm) script and dependencies in this repo, but this is not a requirement for you to benefit from this workflow. If your algorithm files aren’t in your repo, you’ll still benefit from the automation by having your model file deployments automated as you iterate on your notebook or your checked-in model file.

How it works

 

Workflow overview

 

Whenever you do a git push to your master branch (or any other triggering event as per your configuration), your Github workflow integrating this Github Action will kick off.

 

Workflow flowchart

 

If you’re developing your ML model in a Jupyter notebook, you can configure the workflow with the notebook path to execute. In this case, the Github Action will:

  • Install the required packages in your requirements.txt file on a Github worker machine
  • Run your configured Jupyter notebook file on the worker machine’s from-scratch environment

If you’re not developing your model in a Jupyter notebook—and if you have a saved model already checked into your repository—you can configure the workflow with the existing model file path.

In both scenarios, the Github Action will:

  • Take the ML model file from the configured path and upload it to your data collection in Algorithmia
  • Copy your inference (algorithm) files to your algorithm repository
  • Create or update the model_manifest.json file with the information related to your newly uploaded model file

Additional benefits

In order to get your inference endpoint to use this newly uploaded model, the workflow will make the connection between your algorithm and the uploaded model file using the key-value pairs in the model_manifest.json file.

 

Diagram of what's in the model manifest file

In addition to the file path of the model, the manifest file will contain certain metadata such as:

  • The Github repository and ref the model file was uploaded from
  • The MD5 hash of the model file when it was first uploaded
  • The Github commit SHA and the commit message triggering the CI/CD workflow
  • When the upload happened

 

Diagram showing how to use the model manifest

 

So, by using this manifest, your algorithm will know which model to load and use. As we mentioned previously, your algorithm can also recalculate the MD5 hash of the model file at inference time and compare it with the original MD5 hash that was calculated at the time of the upload to make sure that the model file hasn’t been changed.

How to configure the Github Action

The inputs to this Github Action can be found in our repository’s documentation. As you’ll notice, some of them will be different depending on whether you host your Algorithmia algorithm on Github or on Algorithmia.

To help you save time, we’ve created default values for some of the inputs. Make sure to review these and update them if you want a different configuration.

How to get started with the integration

Check out Algorithmia’s Template ML Repository on Github

To help you get you started with this integration, we created a Github template repository containing some necessary pieces.

Diagram showing what the template repo contains

The template repository comes with the following files to get you up to speed with this integration:

  • .github/workflows/algorithmia_deploy.yml: Workflow definition file containing Github Action inputs.
  • YOUR_ALGORITHM_DIR: Algorithm directory containing your algorithm files, to be pushed to the Algorithm repo. Should be renamed with your algorithm name.
  • model_training.ipynb: Notebook file, to be optionally executed by the Github Action. Can be omitted if the model file is checked into this repository.
  • requirements.txt: File containing Jupyter notebook requirements.

Create your own repository

Let’s walk through the steps to set up an example ML repository that creates an XGBoost sentiment analysis model to be deployed to Algorithmia via a CI/CD workflow. Each step contains a link to the related file in the repository, so you can examine it in more detail.

  1. Either create a new repository using our ML template repo on Github, or clone this repository and copy the files over to your existing repository. To generate a new repository from this template, you can use the green Use this Template button.
  2. In your repository, navigate to Settings > Secrets and input the required secret variables. Secrets are environment variables that are encrypted and only exposed to the actions you configure. The secret variables you input will depend on where your algorithm repository is hosted.
  3. Edit the algorithmia_deploy.yml file with your algorithm’s name. Remember, if you want to change the default values of the other inputs, go ahead and update those too at this time.
  4. Rename your YOUR_ALGORITHM_DIR directory and src/YOUR_ALGORITHM_NAME.py to your own algorithm name.
  5. Work on your notebook and Python script files until you’re happy with your model and your inference script.
    • Add your model training code in your notebook.
    • Once you’re done with your training, evaluation, and testing, export your ML model object to a file on the local disk.
    • Edit your algorithm files in your favorite way.
    • Test your inference script in your preferred way. If you don’t want to leave your notebook to call and check your algorithm output, you can call your inference script with the %run macro from within a cell.
  6. Edit your requirements.txt file to contain the required packages for your notebook execution.
  7. Commit and push your changes. Or, if you’re working on a development or feature branch, create a PR for someone else in your organization to review and merge your changes to the branch that in turn triggers your CI/CD workflow.

Check that your workflow runs

Unless you edited it, the default trigger of the automated workflow will be the git push event to your repository’s master branch. Once the trigger event happens, you can go to the Actions tab of your repository and check that your workflow has run.

 

Picture of confirming that the workflow runs

 

Check your artifacts on Algorithmia

Once the workflow run completes successfully, you should see:

  1. Your model file uploaded to the configured data collection on Algorithmia. Unless you changed the default data collection path in your workflow file, your model should be uploaded to _data://ALGORITHMIA_USERNAME/ALGORITHMIA_ALGONAME_.
  2. Your algorithm’s model_manifest.json file updated with the information related to the origins of this automation.
  3. Your algorithm’s source code updated with the edits you’ve made.

How does this fit into a mature organization’s workflow?

Some mature organizations have a separation of roles and responsibilities for DevOps, data science, and product.

 

Diagram of roles and responsibilities for DevOps, data science, and product

 

The DevOps (or IT) team can be the admins of the Github repositories and the Algorithmia organizations. In this case, they would be the ones setting up the ML repository generated by our template and providing the secret inputs as the workflow’s environment variables afterward.

After the initial setup, they can hand over the repository to the data science (or ML engineering) team to work on the models and inference code. Typically, teams work on feature or development branches and do not work on the master branch of repositories.

Once the data science or ML engineering team is ready to deploy the developed model and inference code to Algorithmia, they can ask for the product or business team’s review and approval. With their approval, the changes can be merged into the master or a prod branch, triggering the automated workflow for deployment.

This is just one example of how this could fit into your organization’s workflow. Needless to say, your organization can create its own workflow to align with any existing methodology or preferences.

Now it’s your turn!

We hope you’ve enjoyed learning how to use our new Github Action to continuously deploy models and inference algorithms from an ML repository to Algorithmia. We’d love to know if you have any questions, comments, or feedback to improve this automation even further!

As you get started, here are a few resources to help you use this automation:

Aslı Sabancı