Github Actions and Algorithmia—building a continuous integration pipeline

What is continuous integration?

In the event that you haven’t used continuous integration systems in the past, let’s do a quick run through of what it is and how it can be useful. Development can be really difficult, especially when working with complex systems that require multiple applications. Bugs always seem to find their way into critical sections if you aren’t careful. Continuous integration, at a high level, creates a process where modifications can be made to your code in smaller, easily tested portions. This allows your production environment to contain fewer bugs at any time and an overall more efficient output. Algorithmia’s CI for machine learning means you can define end-to-end tests in your TEST_CASES.json file and restrict algorithm publishing to just the automated deployment tool. You can ensure that any new version of your algorithm that reaches production must have passed at least the tests you’ve set up before doing so. This can be a really powerful function, as it means you can be more confident when bringing on engineers from different teams who might have less experience with your source code than you do, and build faster without sacrificing safety and reliability.

Hello world with Algorithmia CI—a procedure

Let’s take a quick look at how you can implement Algorithmia CI into your algorithm today!A full working example can be found here: https://github.com/algorithmiaio/algorithmia_ci1. Start by using our GitHub CI workflow to create a new algorithm using the Algorithmia wizard. Make sure to select GitHub as the repository host:

2. After your algorithm is created, there are two files you will want to copy into your algorithm’s repository. The easiest way to do this is to git clone your algorithm to your local machine.

3. Create the following directory structure in your algorithm repo at the root level: .github/workflows and then create a main.yml in the workflows directory. Copy the following into the file:

This is an example using the Algorithmia CI actionname: CI# Controls when the action will run. Triggers the workflow on push or pull request
# In this instance we're triggering this workflow whenever a push to master is performed.
on:
push:
branches: [master]jobs:
# This workflow only contains a single job, however if you'd like you can split this processing across multiple jobs.
algorithmia-ci:
runs-on: ubuntu-lateststeps:
- name: Checkout
uses: actions/checkout@v2.0.0
with:
ref: ${{github.sha}}
path: algorithm
- name: Algorithmia CI
uses: algorithmiaio/algorithmia-ci-action@v1.0
with:
# Your master Algorithmia API key
api_key: ${{ secrets.api_key }}
# The API address for the Algorithmia Cluster you wish to connect to
api_address: https://api.algorithmia.com
# identifier to describe how to promote this release ('major', 'minor', 'revision')
version_schema: revision
# the path variable you defined in the actions/checkout action triggered before this one.
path: algorithm

4. Next we need to create a TEST_CASES.json file at the root level of your algorithm repository (if you can see the algorithmia.conf file, that means you’re there!), then you can copy the following into the file:

[
 {
    "case_name": "hello world",
    "input": "world",
    "expected_output": "hello world"
  }
]

5. Commit these changes and push your commit to your algorithm repo. If you check the actions tab of your GitHub repository page, you can see that a job has started:

Note that this job will fail and that’s normal! We aren’t finished yet.

6. We need to add your Algorithmia API Key to your GitHub repository as a secret. This API key must have full algorithm access, which your default key might not have. Let’s create a new API key and make sure your settings are the same as those shown below:

7. Once you have that API key, go back to your GitHub repo and click the settings button. Then navigate to secrets and add a new secret called api_key as shown below:

And with that, you should be all set! The next time you push any kind of commit to your algorithm, it should now trigger a GitHub Action job you’ll be able to walk though and see the final result. If all goes well, your algorithm will be published on Algorithmia with a new version, based on the version_schema you define in the main.yml file.

Security and access

Algorithmia has a robust access and permissions management system to ensure that your algorithms and data are safe and secure. However, what about deployment access? With GitHub Enterprise support, you can now leverage GitHub’s access control systems to restrict merge access, commit access, and PR acceptance criteria to certain individuals within your organization, ensuring accountability and sufficient oversight for mission-critical applications.

Give it a try and see what you can build with Algorithmia CI and GitHub enterprise. Let us know what you think, and if you run into any problems or have suggestions on how to improve the workflow, please feel free to file an issue against either of these GitHub repositories mentioned herein: https://github.com/algorithmiaio/algorithmia_ci https://github.com/algorithmiaio/algorithmia-ci-action.

Github Actions and Algorithmia—building a continuous integration pipeline

What is continuous integration?

Hello world with Algorithmia CI—a procedure

Security and access

James Sutton