Algorithmia Blog - Deploying AI at scale

Shorten deployment time with Algorithmia

Pie graph of ML projects deployed

Every year, millions of dollars are wasted planning, cleaning, and training machine learning (ML) models that will never get to production. This means that more than half of data science projects are not fully deployed—and some never will be, resulting in zero generated revenue.

When organizations are asked about their machine learning business challenges, deployment difficulties are cited as the biggest obstacle. 

Reduce waste; increase value

The solution looks simple at first:

  • Make it fast and simple to deploy ML models
  • Reduce the learning curve
  • Stop asking data scientists to do DevOps
  • Automate where possible and measure the results
  • Reduce model deployment time from months (or years) to minutes

But let’s deep dive into how to make these solutions feasible.

Remove barriers to deployment

If a data science department is isolated, it may not have a reliable DevOps team and must learn to deploy its own models. However, when data scientists are tasked with deploying models, they face a number of challenges: 

  • They must learn a wide range of DevOps-related skills that are not part of their core competencies.
  • They will spend a lot of time learning to properly containerize models, implement complex serving frameworks, and design CI/CD workflows.

This pulls them away from their primary mission of designing and building models, and often working on the challenges above have varying degrees of success.

But let’s say an IT or DevOps team is available, now the data scientists are faced with a new set of challenges:

  • IT is used to working with conventional application deployments, which differ from ML in a number of ways, often requiring a unique “snowflake” environment for each individual model. 
  • Information security restrictions further complicate deployment, requiring various levels of isolation and auditing. Because ML models are opaque, and data science teams follow non-standard development practices, IT is often unable to add sufficient tooling, such as fine-grained logging or error-handling. 

From there, developers are typically borrowed from other teams to help solve these problems—for example, writing wrapper code to add company-standard authentication—which can cause further slowdowns and resource consumption.

ML infrastructure layout

Reduce the learning curve

To succeed in ML efforts, companies must reduce the breadth of knowledge each individual team is responsible for, allowing them to specialize in their own practices. When they are able to do so, the learning curve for each team can be reduced and they can quickly scale up their own activities and projects.

Stop asking data scientists to do DevOps

A key mechanism for enabling this is a managed platform for the deployment, serving, and management of ML models. This platform provides the following benefits:

  • Separation of concerns: data scientists can focus on model building and app developers can focus on integration.
  • Low DevOps: managed platforms require minimal oversight and DevOps never need to be involved in the deployment of an individual model.
  • Reduced manual tool-building: authentication, data connectors, monitoring, and logging are built-in.

Selecting the right platform for ML is critical. At minimum, it should:

  • Provide deployment, serving, and model management within a single environment to enhance accessibility and reduce tool thrashing.
  • Allow app developers and other consumers to quickly locate, test, and integrate appropriate models.
  • Support any language and framework so data scientists are not constrained in their development.
  • Allow data scientists to write code, not containers, so they can remain focused at the model level.
  • Not require any deep proprietary changes to models, cause vendor lock-in, or tie model-serving to a specific model-training platform.
  • Embed within a choice of private cloud, with full portability between infrastructure providers.
  • Work with existing choices for CI/CD and other DevOps tooling.

ML infrastructure diagram

Go the last mile

With the problems of model deployment and serving contained, your company can focus on creating and tracking value and ROI.

By providing application developers with a searchable, structured way of locating models, cross-departmental communication barriers can be reduced to zero. The instant that the ML team releases a model for general consumption, it becomes discoverable and usable by developers. Developers can consume the model with a global API and have cut-and-paste code ready to drop into any application or service.

As models are added and consumed, oversight becomes key. Monitoring, logging, and showbacks allow for seamless continuous operation while demonstrating the value of each model. Companies can properly allocate resources across teams and prove ROI on each individual project.

ML infrastructure plugging into apps and code

Start deploying today

Don’t become the alchemy Gartner warned about: “Through 2020, 80 percent of AI projects will remain alchemy, run by wizards whose talents will not scale in the organization” (Gartner). 

Take stock of your company-wide ML initiatives. If you’re not deploying all of your models into production, Algorithmia can help you get there. If your data science teams are running their own DevOps, or your IT team is being overloaded with ML needs, our managed solution is the right tool to get your model productionization back on track.

Algorithmia is the leader in machine learning deployment, serving, and management. Our product deploys and manages models for Fortune 100 companies, US intelligence agencies, and the United Nations. Our public algorithm marketplace has helped more than 90,000 engineers and data scientists deploy their models.

Developer Advocate for Algorithmia, helping programmers make the most efficient and productive use of their abilities.

More Posts

Follow Me:
TwitterLinkedIn