Algorithmia developers unveiled a new UI on our user platform this week. Now, when you log into Algorithmia, you’ll see a personalized dashboard with your account information and recent activity, enabling easy account management and navigation.
This change comes about as we mark the four-year anniversary of the Algorithmia website, but the redesign of the platform was tied to creating a more user-friendly and intuitive experience for our users.
A Mini Tour
There are two new menus: global and navigation. The global menu (the purple one at the left) is designed to keep primary actions readily available (creating new algorithms, data sources, searching, accessing your notifications, and profile actions).
The navigation menu (the light gray one) provides quick access to the main pages of the site (your user pages, algorithms, data, and the accompanying technical docs).
We were excited to move to a drawer-based navigation style, and surfacing application functionality to be more consistent. We’re making the app more intuitive and usable for everyone.
-Ryan Miller, Algorithmia Frontend Engineering Lead
We designed the new UI with user control in mind. Some of the features include:
- Lists organization: the new UI provides easy access to saved work, API Keys, and set organizations for model sharing.
- Dashboard navigation: easy access via two nav panes.
- Mobile integration: much of the updated drawer navigation features of the site are now available for mobile users as well.
“A good UI should get out of the way, but as the app grew we began to see the opposite—we were inhibiting user effectiveness. So we broke it down and are iteratively building it back up with usability at the foundation so users can focus on their work—not how to use the app.”
-James Hoover, Algorithmia Product Designer
We welcome any feedback you have as we continue to make the Algorithmia user platform ever more intuitive and usable.
Check out our updated UI today!
Asking a data scientist to work with only one framework is like asking a carpenter to work with only a hammer. It’s essential that professionals have access to all the right tools for the job.
It’s time to rethink best practices for leveraging and building ML infrastructure and set the precedent that data scientists should be able to use whichever tools they need at any time.
Now, certainly some ML frameworks are better suited to solve specific problems or perform specific tasks, and as projects become more complex, being able to work across multiple frameworks and with various tools will be paramount.
For now, machine learning is still in its pioneering days, and though tech behemoths have created novel approaches to ML (Google’s TensorFlow, Amazon’s SageMaker, Uber’s Michelangelo), most ML infrastructure is still immature or inflexible at best, which severely limits data scientists and DevOps. This should change.
Flexible frameworks and ML investment
Most companies don’t have dozens of systems engineers who can devote several years to building and maintaining their own custom ML infrastructure or learning to work within new frameworks, and sometimes, open-source models are only available in a specific framework. This could restrict some ML teams from using them if the models don’t work with their pre-existing infrastructure. Companies can, and should, have the freedom to work concurrently across all frameworks. The benefits of doing so are multifold:
Increase interoperability of ML teams
Often machine learning is conducted in different parts of an organization by data scientists seeking to automate their processes. These silos are not collaborating with other teams doing similar work. Being able to blend teams together while still retaining the merits of their individual work is key. It will de-duplicate efforts as ML work becomes more transparent within an organization.
Allow for vendor flexibility and pipelining
You don’t want to end up locked-in to only one framework or only one cloud provider. The best framework for a specific task today may be overtaken next month or next year by a better product, and businesses should be able to scale and adapt as they grow. Pipelining different frameworks together creates the environment for using the best tools.
Reduce the time from model training to deployment
Data scientists write models in the framework they know best and hand them over to DevOps, who rewrite them to work within their infrastructure. Not only does this usually decrease the quality of a model, it creates a huge iteration delay.
Enable collaboration and prevent wasted resources
If a data scientist is accustomed to PyTorch but her colleague has only used TensorFlow, a platform that supports both people’s work saves times and money. Forcing work to be done with tools that aren’t optimal for a given project is like showing up with knives to a gunfight.
Leverage off-the-shelf products
There’s no need to constantly reinvent the wheel; if an existing open-source service or dataset can do the job, then that becomes the right tool.
Position your team for future innovations
Because the ML story is far from complete, being flexible now will enable a company to pivot more easily come whatever tech developments arise.
How to deploy models in any framework
Attaining framework flexibility, however, is no small feat. The main steps to enable deploying ML models from multiple frameworks are as follows:
It’s fairly simple to run a variety of frameworks on a laptop, but trying to productionize them requires a way to manage all dependencies for running each model, in addition to interfacing with other tools to manage compute.
Containerization and orchestration
Putting a model in a container is straightforward. When companies have only a handful of models, they often task junior engineers with manually containerizing, putting the models into production, and managing scale-up. This process unravels as usage increases, model numbers grow, and as multiple versions of models run in parallel to serve various applications.
Many companies are using Kubernetes to orchestrate containers—there are a variety of open-source projects on components that will do some of the drudge work of machine learning for container orchestration. Teams who have attempted to do this in-house have found that it requires constant maintenance and becomes a Frankenstein of modular components and spaghetti code that falls over when trying to scale. Worse still, after models are in production, you discover that Kubernetes doesn’t deal well with many machine learning use cases.
API creation and management
Handling many frameworks requires a disciplined API design and seamless management practice. When data scientists begin to work faster, a growing portfolio of models with an ever-increasing number of versions needs to be continuously managed, and that can be difficult.
Languages and DevOps
Machine learning has vastly different requirements than traditional compute, including the freedom to support models written in many languages. A problem arises, however, when data scientists working in R or Python sync with DevOps teams who then need to rewrite or wrap the models to work in the language of their current infrastructure.
Down the road
Eventually, every company that wants to extend its capabilities with ML is going to have to choose between enabling a multi-framework solution like the AI Layer or undertaking a massive, ongoing investment in building and maintaining an in-house DevOps platform for machine learning.
Algorithmia’s AI Layer Powers the UN Methods Service
Algorithmia is renewing its commitment to global humanitarian efforts by making powerful ML tools available to everyone.
Economic and population data are key elements of planning and decision-making in first-world countries, but access to sophisticated analytic and compute power is limited or non-existent in developing countries.
Meeting the Problem Head On
Working in conjunction with the United Nations Global Platform for Official Statistics, Algorithmia built a repository of algorithms that are readily available to any data scientist of any member state at any time. There are models for predicting economic, environmental, and social trends to enable smarter decision-making for strategies like agricultural planning, flooding probabilities, and curbing deforestation.
The United Nations Global Platform for Official Statistics sought to build the algorithm repository as part of the Sustainable Development Goals (SDGs). The SDGs aim to meet global challenges in healthcare, poverty, environmental degradation, and inequality by 2030. The algorithm repository will serve member states to “establish strategies to reuse and adapt algorithms across topics and to build implementations for large volumes of data.” UN Big Data
Building an Algorithm Marketplace for the Developing World
The UN wanted a way to share models with underdeveloped countries to curate economic, environmental, and social data to save lives and improve health and environmental conditions. Using the UN algorithm repository, for example, a developing country could model farmland satellite imagery to predict draughts, urbanization trends, or migration patterns.
Such statistical information can be used in myriad ways by both humanitarian organizations and policy-making, governmental bodies to make smarter resource-allocation decisions, better understand urban planning needs from population data, and even predict migration crop cycles using geospatial imagery.
The UN’s partnership with Algorithmia demonstrates our dedication to leveraging AI and machine learning to seek solutions to global problems. We are so looking forward to empowering the developing world, one algorithm at a time.
Machine learning (ML) will drastically alter how many industries operate in the future. Natural language processing will enable seamless and instantaneous language translation, forecasting algorithms will help predict environmental trends, and computer vision will revolutionize the driverless car industry. The possibilities are almost literally endless, and organizations are beginning to realize the ways in which machine learning can benefit them. Companies have begun implementing machine learning into their business processes, and there is a lot to be learned from their experiences.
Nearly all companies that have initiated ML programs have encountered challenges or roadblocks in their development. Despite efforts to move toward building robust ML programs, most companies are still at the nascent stages of building sophisticated infrastructure to productionize ML models. There is room for improvement, and Algorithmia has outlined the challenges that have arisen thus far along with potential solutions to help organizations avoid the common pitfalls of enterprise machine learning.
Machine Learning Roadmap Overview
After surveying hundreds of companies, Algorithmia has developed a roadmap that outlines the main stages of building a robust ML program as well as tips for avoiding common ML pitfalls. We hope this roadmap can be a guide that companies can use to position themselves for ML maturity. Keep in mind, the route to building a sophisticated ML program will vary by company and team and require flexibility.
This roadmap is intended to act as a framework for organization management and data scientists to successfully implement AI within their companies. Follow each step to avoid the challenges ML pioneers faced and chart your course to machine learning maturity through ML workflow optimization.
Whether you plan to use deep learning algorithms, neural networks, a combination of programming languages such as Python, and any other machine learning tools, the roadmap will ensure your project has value.
The following is a preview of our roadmap, but if you’re ready to jump right in, you can download the full roadmap here.
Using the Machine Learning Roadmap
Every company or data science team is situated at a different maturity level in each stage. After locating your current position on the roadmap, we suggest the following:
- Chart your path to maturity – Chart a path to get from your current location on the roadmap to the goal destination you’ve pinpointed for your organization. Defining the path you will take is crucial to reaching maturity. You must be sure of your process but also willing to be flexible to accommodate any challenges you encounter on your unique journey.
- Orient and align stakeholders – Enterprise ML projects require buy-in from DevOps, IT, product teams, data scientists, ML engineers, and the organization’s leadership. Gaining support from each of these groups needs to be part of your plan, which is why it’s included in the roadmap.
- Navigate common pitfalls – Surveying hundreds of companies that have implemented ML has shown us the most common mistakes organizations make in this endeavor. For example, many organizations strive to be perfect or reinvent the wheel. These flawed goals will delay your process and impede your progress toward ML maturity.
The roadmap comprises four stages: Data, Training, Deployment, and Management. The stages build on one another but could also occur concurrently in some instances. Whether you follow these stages in order or pursue some simultaneously, it is important to address every stage at some point on your machine learning path.
Data: Developing and maintaining secure, clean data
Training: Using structured datasets to train models
Deployment: Feeding applications, pipelining models, or generating reports.
*Models begin to generate value at this stage.*
Management: Continuously tuning models to ensure optimal performance
Pinpointing Your Location on Algorithmia’s Roadmap to Machine Learning Maturity
Getting started on your journey to ML maturity is exciting, but don’t get ahead of yourself. The first step, even before charting your path or aligning your stakeholders, is to pinpoint your organization’s current location on the Roadmap to Machine Learning Maturity. To figure out how to reach your destination, you have to know where you stand right now.
At each stage, the roadmap charts three variables to gauge ML maturity: people, tools, and operations. These are the variables you should use to determine your organization’s current location on the ML roadmap. In the full Roadmap to Machine Learning Maturity, you will learn how to rate these factors within your organization as either low, medium, or high maturity. That way, you will be aware of the strengths and weaknesses that you have to work with, and you will better understand how to get each of these variables to high levels of maturity. These variables develop further at every stage as an ML program becomes more sophisticated so you can use this framework to track your progress as you get closer to maturity.
For more information about building a sophisticated machine learning program and to use the roadmap, read our whitepaper, The Roadmap to Machine Learning Maturity.
As 2018 comes to a close, we’d like to take a look back to see how our readers have interacted with our blog, which articles were the most read, and what that could tell us about the field of machine learning writ large.
We know 2019 will be a year of tremendous progress in tech, and we’re relentlessly curious and eager for it. We look forward to adding more algorithms for our marketplace, expanding our AI Layer to more industries, producing interesting articles about novel tech applications, and engaging with innovators in the AI and machine learning fields.
Let’s take a look back on our year:
In March, we published Introduction to Machine Learning to give readers an in-depth look at what machine learning is at the macro and micro level. We got great engagement from this piece and know it will have staying power even as the world of AI morphs and grows.
Machine learning applications in sentiment analysis are becoming more and more popular, and conducting sentiment analysis can provide a company with continuous focus group feedback to gauge customer satisfaction and contentment. The explanation of a specific data use case in How to Perform Sentiment Analysis with Twitter Data was our ninth most read article of 2018.
A post from April on how computer vision works was insanely popular this year. Introduction to Computer Vision was shared more than 4,000 times by our readers, and provides a big-picture overview of the field of machine learning concerned with training computers to identify elements in images. It’s a hot topic in AI because of the pervasiveness of this technology. As our CEO said last year,
Used to be if the product was free you were the product , now if a product is free you are the training set.
— Diego Oppenheimer (@doppenhe) October 6, 2017
Introduction to Emotion Recognition was another tech overview article that was of much interest to curious tech readers in 2018. Like computer vision, emotion recognition trains computers to read the facial expressions of people in images to decipher their moods. This technology has many possible applications, including criminal justice: polygraph analysis, juror psychology, security surveillance systems and interrogation tactics, or in industry for fatigue monitoring for pilots and drivers.
Haven’t you always wanted to know how deep learning works without ground truth? Introduction to Unsupervised Learning is for you (and for the more than 7,500 other avid AI news consumers who have read this post since April. And no, before you ask, unsupervised learning is not about classrooms without teachers present; actually it kind of is.
Our intro posts sure were popular this year! (Perhaps in 2019 we’ll move on to intermediate posts.) Introduction to Optimizers comes in at the number four most-read article. Optimizers shape and mold machine learning models into their most accurate possible forms, and they’re the cousin of loss functions (see below).
Much is still unfolding in the machine learning software field; but hardware is just as important when running multivariate algorithms at scale. Learning the different compute modes and which is best for building and deploying ML applications was a topic of supreme interest for nearly 12,000 savvy readers out there this year. Make some time today to read Hardware for Machine Learning.
Facial recognition software was in the news a lot in 2018 so it makes sense that our post, Facial Recognition Through OpenFace was so popular. This article gives a good technical run-down of how OpenFace, a facial recognition machine learning model works.
Remember optimizers from above? Loss Functions can also evaluate machine learning models by determining how well an algorithm is modeling a dataset. Learn more about this tool in Introduction to Loss Functions, which helped educate more than 17,000 people this year.
And finally! Our number one most-read post of 2018 is Convolutional Neural Networks in PyTorch! Convolutional neural networks are algorithms that work in tandem on large projects #convoluted (typically computer vision). Check out this deep dive into the Python-based framework, PyTorch, and how it easily enables development of machine learning work flows.
We hope you’ll join us in 2019 as we take a deeper look into the most cutting-edge technology.