All posts in Blog Posts

Machine learning use cases

A brain with two segments, one like a neural network drawing

Machine learning is a vast field, composed of many model types, subsets, and use cases. In our forthcoming 2020 State of Enterpriser Machine Learning report, we dig into the use cases that are used most often by businesses today, but as there are new advances made in ML every day, there are also advances in number and complexity of ML use cases. This post will walk through some common machine learning use cases and how they enable businesses to leverage their data in novel ways.

What is machine learning?

Machine learning is the subset of artificial intelligence that involves the study and use of algorithms and statistical models for computer systems to perform specific tasks without human interaction. Machine learning models rely on patterns and inference instead of manual human instruction. Most any task that can be completed with a data-defined pattern or set of rules can be done with machine learning. This allows companies to automate processes that were previously only possible for humans to perform—think responding to customer service calls, bookkeeping, and reviewing resumes. 

To extract machine learning value, a model must be trained to react to certain data in certain ways, which requires a lot of clean training data. Once the model successfully works through the training data and is able to understand the nuances of the patterns its learning, it will be able to perform the task on real data. We’ll walk through some use cases of machine learning to help you understand the value of this technology.

What is machine learning used for?

Machine learning has many potential uses, including external (client-facing) applications like customer service, product recommendation, and pricing forecasts, but it is also being used internally to help speed up processes or improve products that were previously manual and time-consuming. You’ll notice these two types throughout our list of machine learning use cases below.

Seven machine learning use cases list

1. Voice assistants

This consumer-based use for machine learning applies mostly to smart phones and smart home devices. The voice assistants on these devices use machine learning to understand what you say and craft a response. The machine learning models behind voice assistants were trained on human languages and variations in the human voice, because it has to translate what it hears into words and then make an intelligent, on-topic response. 

Millions of consumers use this technology, often without realizing the complexity behind the tool. The concept of training machine learning models to follow rules is fairly simple, but when you consider training a model to understand the human voice, interpret meaning, and craft a response, that is a heavy task. 

2. Dynamic pricing

This machine–based pricing strategy is most known in the travel industry. Flights, hotels, and other travel bookings usually have a dynamic pricing strategy behind them. Consumers know that the sooner they book their trip the better, but they may not realize that the actual price changes are made via machine learning. 

Travel companies set rules for how much the price should increase as the travel date gets closer, how much it should increase as seat availability decreases, and how high it should be relative to competitors. Then, they let the machine learning model run with competitor prices, time, and availability data feeding into it. 

3. Email filtering

This is a classic use of machine learning. Email inboxes also have a spam inbox, where your email provider automatically filters unwanted spam emails. But how do they know when an email is spam? They have trained a model to identify spam emails based on characteristics they have in common. This includes the content of the email itself, the subject, and the sender. If you’ve ever looked at your spam inbox, you know that it wouldn’t be very hard to pick out spam emails because they look very different from real emails.

4. Product recommendations

Amazon and other online retailers often list “recommended products” for each consumer individually. These recommendations are based on past purchases, browsing history, and any other behavioral information they have about consumers. Often the recommendations are helpful in finding related items that you need to complement your purchase (think batteries for a new electronic gadget). 

However, most consumers probably don’t realize that their recommended products are a machine learning model’s analysis of their behavioral data. This is a great way for online retailers to provide extra value or upsells to their customers using machine learning.

5. Personalized marketing

Marketing is becoming more personal as technologies like machine learning gain more ground in the enterprise. Now that much of marketing is online, marketers can use characteristic and behavioral data to segment the market. Digital ad platforms allow marketers to choose characteristics of the audience they want to market to, but many of these platforms take it a step further and continuously optimize the audience based on who clicks and/or converts on the ads. The marketer may have listed 4 attributes they want their audience to have, but the platform may find 5 other attributes that make users more likely to respond to the ads. 

6. Process automation

There are many processes in the enterprise that are much more efficient when done using machine learning. These include analyses such as risk assessments, demand forecasting, customer churn prediction, and others. These processes require a lot of time (possibly months) to do manually, but the insights gained are crucial for business intelligence. But if it takes months to get insights from the data, the insights may already be outdated by the time they are acted upon. Machine learning for process automation alleviates the timeliness issue for enterprises. 

Industries are getting more and more competitive now that technology has sped up these processes. Companies can get up-to-date analyses on their competition in real time. This high level of competition makes customer loyalty even more crucial, and machine learning can even help with customer loyalty analyses like sentiment analysis. Companies like Weavr.ai provide a suite of ML tools to enable this type of analysis quickly and deliver results in a consumable format.

7. Fraud detection

Banks use machine learning for fraud detection to keep their consumers safe, but this can also be valuable to companies that handle credit card transactions. Fraud detection can save money on disputes and chargebacks, and machine learning models can be trained to flag transactions that appear fraudulent based on certain characteristics. 

Machine learning can provide value to consumers as well as to enterprises. An enterprise can gain insights into its competitive landscape and customer loyalty and forecast sales or demand in real time with machine learning. 

If you’re already implementing machine learning in your enterprise or you’d like to start...

Continue learning

Harnessing Machine Learning for Data Science Projects

Data science and machine learning: their differences and how they are used in the enterprise

Improving Customer Retention Analytics With Machine Learning

Executive summary: the 2020 state of enterprise machine learning report

Read it 12 December >

Reflects data only from survey Group B. Respondents were allowed to choose more than one answer.

In the last 12 months, there have been numerous developments in machine learning (ML) tools, applications, and hardware. Google’s TPUs are in their third generation, the AWS Inferentia chip is a year old, Intel’s Nervana Neural Network Processors are designed for deep learning, and Microsoft is reportedly developing its own custom AI hardware.

This year, Algorithmia has had conversations with thousands of companies in various stages of machine learning maturity. From them we developed hypotheses about the state of machine learning in the enterprise, and in October, we decided to test those hypotheses.

Following the State of Enterprise Machine Learning 2018 report, we conducted a new two-prong survey this year, polling nearly 750 business decision makers across all industries at companies that are actively developing machine learning lifecycles, just beginning their machine learning journeys, or somewhere in between. Sign up to receive the full 2020 report on 12 December 2019 when it publishes.

2020 key findings and report format

The forthcoming 2020 report focuses on seven key findings from the survey. In brief, they are:

  1. The rise of the data science arsenal for machine learning: most all companies are building data science teams to develop ML use cases. There are discrepancies in team size and agility, however, that will affect how quickly and efficiently ML is applied to business problems. 
  2. Cutting costs takes center stage as companies grow in size: the primary business use cases center on customer service and internal cost reduction. Company size is the differentiator. 
  3. Overcrowding at early maturity levels and AI for AI’s sake: the pool of companies entering the ML arena is growing exponentially but that could bring about an increase in “snake-oil AI” solutions. 
  4. An unreasonably long road to deployment: despite the rapid development in use cases, growth in AI/ML budgets, and data science job openings, there is still a long road to model deployment. We offer several hypotheses why. 
  5. Innovation hubs and the trouble with scale: we anticipate the proliferation of internal AI centers (innovation hubs) within companies designed to quickly develop ML capabilities so the organization can stay current with its competition. Machine learning challenges still exist, however, stymying the last-mile to sophisticated levels of ML maturity. 
  6. Budget and ML maturity, an emerging disparity: AI/ML budgets are growing across all company sizes and industries, but several industries are investing more heavily. 
  7. Determining machine learning success across the org chart: hierarchical levels within companies are determining ML success by two different metrics. The director level will likely play a large role in the future of ML adoption.

The report concludes with a section on the future of machine learning and what we expect in the short-term. 

What to expect in the 2020 report

Our findings are presented with our original hypotheses, as well as our analysis of the results. Where possible, we have provided a year-on-year comparison with data from 2018 and included predictions about what is likely to manifest in the ML space in the near term.

We have included graphics throughout to bring the data to life (the banner graphic of this post is a bubble chart depicting the use cases of machine learning and their frequency in the enterprise).

We will continue to conduct this annual survey to increase the breadth of our understanding of machine learning technology in the enterprise and share with the broader industry how ML is evolving. In doing so, we can track trends in ML development across industries over time, ideally making more informed predictions with higher degrees of confidence.

Following the report and future-proofing for machine learning

We will soon make our survey data available on an interactive webpage to foster transparency and a greater understanding of the ML landscape. We are committed to being good stewards of ML technology.

This year’s survey report should confirm for readers that machine learning in the enterprise is progressing at a lightning pace. Though the majority of companies are still in the early stages of ML maturity, it is incorrect to think there is time to delay ML efforts at your company. 

If your organization is not currently ML–oriented, know that your competitors are. Now is the time to future-proof your organization with AI/ML.

Sign up to receive the full 2020 State of Enterprise Machine Learning report when it publishes on 12 December.

Using machine learning for sentiment analysis: a deep dive

Sentiment analysis depicted as four faces with different facial expressions

Sentiment analysis invites us to consider the sentence, You’re so smart! and discern what’s behind it. It sounds like quite a compliment, right? Clearly the speaker is raining praise on someone with next-level intelligence. However, consider the same sentence in the following context.

Wow, did you think of that all by yourself, Sherlock? You’re so smart!

Now we’re dealing with the same words except they’re surrounded by additional information that changes the tone of the overall message from positive to sarcastic. 

This is one of the reasons why detecting sentiment from natural language (NLP or natural language processing) is a surprisingly complex task. Any machine learning model that hopes to achieve suitable accuracy needs to be able to determine what textual information is relevant to the prediction at hand, have an understanding of negation, human patterns of speech, idioms, metaphors, etc, and be able to assimilate all of this knowledge into a rational judgment about a quantity as nebulous as “sentiment.” 

In fact, when presented with a piece of text, sometimes even humans disagree about its tonality, especially if there’s not a fair deal of informative context provided to help rule out incorrect interpretations. With that said, recent advances in deep learning methods have allowed models to improve to a point that is quickly approaching human precision on this difficult task.

Sentiment analysis datasets

The first step in developing any model is gathering a suitable source of training data, and sentiment analysis is no exception. There are a few standard datasets in the field that are often used to benchmark models and compare accuracies, but new datasets are being developed every day as labeled data continues to become available. 

The first of these datasets is the Stanford Sentiment Treebank. It’s notable for the fact that it contains over 11,000 sentences, which were extracted from movie reviews and accurately parsed into labeled parse trees. This allows recursive models to train on each level in the tree, allowing them to predict the sentiment first for sub-phrases in the sentence and then for the sentence as a whole.

The Amazon Product Reviews Dataset provides over 142 million Amazon product reviews with their associated metadata, allowing machine learning practitioners to train sentiment models using product ratings as a proxy for the sentiment label.

The IMDB Movie Reviews Dataset provides 50,000 highly polarized movie reviews with a 50-50 train/test split.

The Sentiment140 Dataset provides valuable data for training sentiment models to work with social media posts and other informal text. It provides 1.6 million training points, which have been classified as positive, negative, or neutral.

Sentiment analysis, a baseline method

Whenever you test a machine learning method, it’s helpful to have a baseline method and accuracy level against which to measure improvements. In the field of sentiment analysis, one model works particularly well and is easy to set up, making it the ideal baseline for comparison.

To introduce this method, we can define something called a tf-idf score. This stands for term frequency-inverse document frequency, which gives a measure of the relative importance of each word in a set of documents. In simple terms, it computes the relative count of each word in a document reweighted by its prevalence over all documents in a set. (We use the term “document” loosely.) It could be anything from a sentence to a paragraph to a longer-form collection of text. Analytically, we define the tf-idf of a term t as seen in document d, which is a member of a set of documents D as:

tfidf(t, d, D) = tf(t, d) * idf(t, d, D)

Where tf is the term frequency, and idf is the inverse document frequency. These are defined to be:

tf(t, d) = count(t) in document d

and 

idf(t, d, D) = -log(P(t | D))

Where P(t | D) is the probability of seeing term t given that you’ve selected document D.

From here, we can create a vector for each document where each entry in the vector corresponds to a term’s tf-idf score. We place these vectors into a matrix representing the entire set D and train a logistic regression classifier on labeled examples to predict the overall sentiment of D. 

Sentiment analysis models

The idea here is that if you have a bunch of training examples, such as I’m so happy today!, Stay happy San Diego, Coffee makes my heart happy, etc., then terms such as “happy” will have a relatively high tf-idf score when compared with other terms. 

From this, the model should be able to pick up on the fact that the word “happy” is correlated with text having a positive sentiment and use this to predict on future unlabeled examples. Logistic regression is a good model because it trains quickly even on large datasets and provides very robust results. 

Other good model choices include SVMs, Random Forests, and Naive Bayes. These models can be further improved by training on not only individual tokens, but also bigrams or tri-grams. This allows the classifier to pick up on negations and short phrases, which might carry sentiment information that individual tokens do not. Of course, the process of creating and training on n-grams increases the complexity of the model, so care must be taken to ensure that training time does not become prohibitive.

More advanced models

The advent of deep learning has provided a new standard by which to measure sentiment analysis models and has introduced many common model architectures that can be quickly prototyped and adapted to particular datasets to quickly achieve high accuracy.

Most advanced sentiment models start by transforming the input text into an embedded representation. These embeddings are sometimes trained jointly with the model, but usually additional accuracy can be attained by using pre-trained embeddings such as Word2Vec, GloVe, BERT, or FastText

Next, a deep learning model is constructed using these embeddings as the first layer inputs:

Convolutional neural networks
Surprisingly, one model that performs particularly well on sentiment analysis tasks is the convolutional neural network, which is more commonly used in computer vision models. The idea is that instead of performing convolutions on image pixels, the model can instead perform those convolutions in the embedded feature space of the words in a sentence. Since convolutions occur on adjacent words, the model can pick up on negations or n-grams that carry novel sentiment information.

LSTMs and other recurrent neural networks
RNNs are probably the most commonly used deep learning models for NLP and with good reason. Because these networks are recurrent, they are ideal for working with sequential data such as text. In sentiment analysis, they can be used to repeatedly predict the sentiment as each token in a piece of text is ingested. Once the model is fully trained, the sentiment prediction is just the model’s output after seeing all n tokens in a sentence. 

RNNs can also be greatly improved by the incorporation of an attention mechanism, which is a separately trained component of the model. Attention helps a model to determine on which tokens in a sequence of text to apply its focus, thus allowing the model to consolidate more information over more timesteps. 

Recursive neural networks
Although similarly named to recurrent neural nets, recursive neural networks work in a fundamentally different way. Popularized by Stanford researcher Richard Socher, these models take a tree-based representation of an input text and create a vectorized representation for each node in the tree. Typically, the sentence’s parse tree is used. As a sentence is read in, it is parsed on the fly and the model generates a sentiment prediction for each element of the tree. This gives a very interpretable result in the sense that a piece of text’s overall sentiment can be broken down by the sentiments of its constituent phrases and their relative weightings. The SPINN model from Stanford is another example of a neural network that takes this approach.

Multi-task learning
Another promising approach that has emerged recently in NLP is that of multi-task learning. Within this paradigm, a single model is trained jointly across multiple tasks with the goal of achieving state-of-the-art accuracy in as many domains as possible. The idea here is that a model’s performance on task x can be bolstered by its knowledge of related tasks y and z, along with their associated data. Being able to access a shared memory and set of weights across tasks allows for new state-of-the-art accuracies to be reached. Two popular MTL models that have achieved high performance on sentiment analysis tasks are the Dynamic Memory Network and the Neural Semantic Encoder.

Sentiment analysis and unsupervised models

One encouraging aspect of the sentiment analysis task is that it seems to be quite approachable even for unsupervised models that are trained without any labeled sentiment data, only unlabeled text. The key to training unsupervised models with high accuracy is using huge volumes of data. 

One model developed by OpenAI trains on 82 million Amazon reviews that it takes over a month to process! It uses an advanced RNN architecture called a multiplicative LSTM to continually predict the next character in a sequence. In this way, the model learns not only token-level information, but also subword features, such as prefixes and suffixes. Ultimately, it incorporates some supervision into the model, but it is able to acquire the same or better accuracy as other state-of-the-art models with 30-100x less labeled data. It also uncovers a single sentiment “neuron” (or feature) in the model, which turns out to be predictive of the sentiment of a piece of text.

Moving from sentiment to a nuanced spectrum of emotion

Sometimes simply understanding just the sentiment of text is not enough. For acquiring actionable business insights, it can be necessary to tease out further nuances in the emotion that the text conveys. A text having negative sentiment might be expressing any of anger, sadness, grief, fear, or disgust. Likewise, a text having positive sentiment could be communicating any of happiness, joy, surprise, satisfaction, or excitement. Obviously, there’s quite a bit of overlap in the way these different emotions are defined, and the differences between them can be quite subtle. 

This makes the emotion analysis task much more difficult than that of sentiment analysis, but also much more informative. Luckily, more and more data with human annotations of emotional content is being compiled. Some common datasets include the SemEval 2007 Task 14, EmoBank, WASSA 2017, The Emotion in Text Dataset, and the Affect Dataset. Another approach to gathering even larger quantities of data is to use emojis as a proxy for an emotion label. 🙂 

When training on emotion analysis data, any of the aforementioned sentiment analysis models should work well. The only caveat is that they must be adapted to classify inputs into one of n emotional categories rather than a binary positive or negative. 

To see some NLP models in action with Algorithmia, start here, and discover many more sentiment analysis models at algorithmia.com/algorithms.

Further reading

MonkeyLearn – A guide to sentiment analysis functions and resources.

Stanford  – Reading Emotions From Speech Using Deep Neural Networks, a publication

Coursera – Applied Text Mining in Python video demonstration

Machine learning methods with R

iris virginica in a field

(Wikispecies)

R is an excellent language for machine learning. R primarily excels in the following three areas:

  1. Data manipulation
  2. Plotting
  3. Built-in libraries

R manipulates data using a native, in-built data structure called a data frame. Let’s load some data now and see how R can help us to work with it.

The data

Luckily for us, R comes with some built-in datasets that we can simply load and have ready to go. We’ll use one such dataset called iris to test some of R’s machine learning capabilities. This dataset contains 50 flowers, each one of three different different species: iris setosa, iris versicolor, and iris virginica. We can classify these data points using a data frame containing 4 different measurement attributes: sepal length, sepal width, petal length, and petal width. Let’s load the data and get started.

data("iris")
head(iris)

The R head command lets us inspect the first few elements of a dataset. You’ll see that the first elements all have the same class label, given by the Species column. When training a machine learning model, we want our data to be randomized so let’s fix this.

Shuffling the data

The elements of our data frame are the rows, so to shuffle the data we need to permute the rows. To do this we can use the built-in R functions, nrow and samplenrow(iris) returns a range the size of the iris dataset. Applying sample to this returns a permutation of the range. We can then index into the dataset using the shuffled row indices as shown below.

shuffled <- iris[sample(nrow(iris)),]
head(shuffled)

Now, when we take a look at the head of the dataset, we see that we’re getting a mix of different flowers, which is great!

Train and validation splits

Now that we’ve got the data shuffled, let’s partition it into a training and validation set. We’ll hold out 20 percent of the data points for the validation set. We can get the two partitions as follows:

n <- nrow(shuffled)
split <- .2 * n
train <- shuffled[1:split,]
validate <- shuffled[split:n,]

Defining a model

Once that we have our data divided into train and validation splits, we can start training a model on it. We’ll first explore using boosted decision trees for classification with an R package called xgboost. Let’s first install it.

install.packages("xgboost")
require(xgboost)

Now, we can construct the model. The XGBoost interface requires that we section the training data into the actual X data and the species labels. To do this, we just select the corresponding columns from the data frame. We also need to convert the Species labels into numerical values in \(\{0, 1, 2\}\) as XGBoost only handles numbered class labels. We’ll do the same for the validation set.

# Training set
dcols <- c(1:4)
lcols <- c(5)
train$Species <- as.character(train$Species)
train$Species[train$Species == "setosa"] <- 0
train$Species[train$Species == "virginica"] <- 1
train$Species[train$Species == "versicolor"] <- 2
X <- train[,dcols]
labels <- train[,lcols]

# Validation Set
validate$Species <- as.character(validate$Species)
validate$Species[validate$Species == "setosa"] <- 0
validate$Species[validate$Species == "virginica"] <- 1
validate$Species[validate$Species == "versicolor"] <- 2
Xv <- validate[,dcols]
labelsV <- validate[,lcols]

Training the model

Now that we’ve gotten that out of the way, we can begin training the model. The interface for xgboost is super simple; we can train the model with a single call.

booster <- xgboost(data = as.matrix(X), label = labels, max.depth = 2, eta = 1, nthread = 2, nrounds = 2, objective = "multi:softmax", num.class = 3)
[1] train-merror:0.000000 
[2] train-merror:0.000000 

The parameters we used deserve some explanation.

  1. We set max.depth = 2 to use decision trees that have a maximum depth of 2. Deeper trees can give higher accuracy but are also higher in variance. Depth 2 should be a good balance for our purposes.
  2. eta specifies the learning rate for our model. This is a hyperparameter that requires some experimentation and tuning to get right. We’ll use 1 for this example.
  3. nthread specifies the number of CPU threads to use while training the model. This is a small dataset so we don’t need many threads, but we would for larger datasets.
  4. nrounds specifies the number of training epochs (passes) to perform over the data. We’ll start with 2.

Evaluating model performance

Now that we’ve trained our boosted tree model, we can evaluate its performance on the validation set. To do this, we can use the predict function that comes packaged with xgboost in order to generate our predictions.

preds <- predict(booster, as.matrix(Xv))
head(preds)
[1] 1 1 0 0 1 0

You can see that our boosted model is predicting nicely. All that is left is to calculate is its accuracy. We will calculate the mean of all entries of preds that are not equal to the corresponding entries of labelsV.

err <- mean(preds != labelsV)
print(paste("validation-error=", err))
[1] "validation-error= 0.0495867768595041"
print(paste("validation-accuracy=", 1 - err))
[1] "validation-accuracy= 0.950413223140496"

Nice! The validation accuracy on the dataset is equal to about 95 percent, which is a great performance from the model.

Visualizing our results

To finish off the project, we’ll just visualize our predictions. First, we will plot the native data using a library called ggvis. Before that though, we must install the package.

install.packages("ggvis")

Now, since we’re visually limited to two dimensions, we will choose to plot our classes vs. the Sepal Length and Width attributes. This can be done as follows.

library(ggvis)
Registered S3 method overwritten by 'dplyr':
  method           from
  print.rowwise_df     
data <- iris
data %>% ggvis(~Sepal.Length, ~Sepal.Width, fill = ~Species) %>% layer_points()

To visualize how our model predicts in comparison, we’ll quickly run it across all data points, not just the validation set. To combine the training and validation data, we use the rbind function, which just joins the two data frames vertically. We also undo the work we did before, converting from numeric labels back to species names.

all_data <- rbind(X, Xv)
preds <- predict(booster, as.matrix(all_data))
all_data["Species"] <- preds
all_data$Species[all_data$Species == 0] <- "setosa"
all_data$Species[all_data$Species == 1] <- "virginica"
all_data$Species[all_data$Species == 2] <- "versicolor"
head(all_data)

Everything looks good! Let’s plot the final result, using the same code as before.

all_data %>% ggvis(~Sepal.Length, ~Sepal.Width, fill = ~Species) %>% layer_points()

We can visually compare the true plot with the plot of our predictions. There should only be a few points misclassified between the two!

To learn more about machine learning models in R, visit https://algorithmia.com/developers/algorithm-development/languages/r.

Algorithmia at Big Data London

People walking on Millennium Bridge in London. Bridge built by Architect Norman Foster

Recently, Algorithmia ventured from Seattle to London to discover what was happening at the Big Data London (BDL) conference in Kensington. We had great conversations with data engineers, data analysts, and business leaders about how Algorithmia makes it easy to deploy machine learning models into production. Our platform handles the MLOps portion of the data science pipeline, so data scientists can focus instead on solving business problems with data.

Highlights from the booth

At BDL, we got the opportunity to talk with many companies about where they are in their ML journeys. While some are just starting to evaluate use cases and consider infrastructure requirements, it’s very encouraging to hear about how they are planning to put their models into production. This is an important step and it is often overlooked. You don’t want to choose a training platform, for instance, that locks you into a specific ecosystem. It’s better to use the best possible platforms and services for each stage of your data science pipeline rather than get locked into one that tries to do everything, without excelling at any portion of it.

We also talked to many data scientists who are at the stage where they have several models sitting on laptops waiting to be utilized in production but don’t know where to go from there. This is a very common scenario, and Algorithmia has white glove customer support to help you get models off laptops and into operation.

Of course, there are also engineers and business owners who are experiencing the same friction points that Algorithmia helps address in the MLOps workflow. This includes: versioning, model updating, centralized repositories, and of course dependency management and scaling.

If any of these stages of the ML roadmap resonate with you, come talk to us at AWS re:Invent where we can go into more detail about getting your models deployed to a scalable, reliable infrastructure today.

Special topics in big data

There were several core themes at the conference, and ones that turned out to be very popular were: Data Governance, Self-Service Analytics, DataOps, Customer Experience Analytics, and of course Machine Learning and AI. 

Some crowd favorites included A GDPR Retrospective: Implementation by a Large-Scale Data Organization in Reality which covered GDPR compliance from a technical standpoint, rather than a business point of view like some of the other talks within that track. Another popular talk within Data Governance, focused on how data management is a customer service story, not just a technical one in Data Governance as a Customer Service. Here at Algorithmia we feel the same way about model management!

To be expected, there were some standout talks in the Keynote Theater. One of our favorites was from EY’s Chief Scientist Harvey Lewis, a leader in applied ML in business, who talked about the need for humans in the loop in the AI pipeline. Lewis covered use cases that showed how important it is to combine humans with machine learning algorithms to ensure that inferences are accurate when it comes to safety, compliance, and risk in the realm of accounting, auditing, and consultancy firms

Another big hit in the Keynote Theater was Making Everyone A Data Person At Lloyd’s. This talk focused on empowering all users across various teams within an organization to be more data-informed. The speakers talked about their initiative called the Data Lab within Lloyd’s Data, which focuses on making everyone within their company data-literate through mentorship and training.

Our tracks of interest

One of the tracks with the longest queues was the Self-Service Analytics tracks. We know because the Algorithmia booth was right near it so we got a chance to chat with many folks waiting in line. A crowd favorite came from our friends at Tableau, who served up a great talk on how to explore data and gain actionable insights with natural language processing

And of course, our favorite track: AILab, which hosted talks on everything from ethics in AI, to extracting actionable insights from machine learning. It also covered infrastructure and scaling modern machine learning systems. 

We’ve thought a lot about these subjects to, so be sure to read up on racial bias in AI,, gaining insights on customer churn data and scaling your machine learning models.

What was missing from the talks, was substance surrounding the difficulty in the deployment cycle. While scaling is important, making sure you can automate your ML deployment lifecycle is crucial. We’ve covered everything from shortening your deployment time to what makes cloud infrastructure crucial to machine learning

That wraps up our take on our first experience at the Big Data London data conference. And if you’re going to re:Invent next month, check out Diego Oppenheimer’s talk on continuous deployment, and don’t forget to set up a meeting to see how Algorithmia can enable your model deployment, serving, and management at scale.