Machine Learning in Google

Table of Contents

AI First

Think about the complex pipeline, to be successful at Machine Learning, besides creating models, you need to think about serving out machine learning predictions using notebooks, dashboards, applications, reports, etc. Operationalizing a machine model is hard. Batch data and stream data should be treated the same way. To be good at Machine Learning, you need to be good at data engineering.

Two Stages

Artificial Intelligence is a discipline, which is about theory and methods to build machines that think and act like humans. Machine Learning is a tool set, which helps solve certain kinds of problems. There are 2 stages in Machine Learning. The first stage is to train an model with examples. The models have parameters. When we train a model, we are making tiny adjustment to the model, so that output of the model (actually math functions) is as close as possible to the true answer for any given input. The second stage is inference which is rather important. After the model is trained, we can use it to “predict” the label.

Neural networks

Neural networks are important technology. In the past, neural networks contain only 1 hidden layer. Just adding layers will run into issues. In the last few years, neural networks have proven themselves to be the best in a wide variety of tasks. In practice there are many models built to solve a use case. There is no one monolithic model to solve the whole problem solutions.

Replacing heuristic rule by Machine Learning. That’s what Machine Learning is about. Google thinks of Machine Learning as a way to scale, to automate, to personalize.

Pre-trained Models

There are 2 ways to add Machine Learning models to your apps. First way is you build everything all by yourself, the second way is there are pre-trained machine learning services available on Google Cloud.

Custom ML models
- Tensorflow
- Cloud AI Platform Notebook
Pre-trained APIs
- Vision
- Speech-to-Text
- Translation
- Natural Language
- Video Intelligence

It is all about data

The idea behind machine learning is to take a bunch of examples and convert that knowledge into future predictions. Machine learning is about collecting appropriate data and finding this right balance of good learning and trusting examples. In some cases (for example, routing in map) is not that complex once you have data, the problem can be solved by a set of rules. Machine learning is about scaling beyond the handwritten rules. An machine learning strategy is actually a data strategy.

Training and serving skew

The typical successful journey involves going from manual data analysis to machine learning. If you can not do manual analysis first, then there is no point of doing machine learning. Manual analysis help you fail fast, try some new ideas. You need to build a streaming pipeline in addition to a batch pipeline. Many machine learning projects fails because of training-serving skew.

The training-serving skew problem is that unless model sees exact same data in “serving”, as it was used to seeing during training, the model prediction is going to be off (which means the predictive power wanes as data profile changes). The result of batch processing and stream processing have to be the same. The data pipeline has to process both batch and stream data.

During training, the key performance aspect is scaling to a lot of data, but during prediction the key performance aspect is speed of response. Tensorflow is equally capable in both aspects.

MLOps

MLOps is lifecycle management discipline for Machine Learning. Some of concepts translate directly from DevOps to MLOps, for example: Continuous Integration(CI) and Continuous Delivery(CD). But there is another important aspect in ML: Continuous Training(CT). It is the process of monitoring, measuring, retraining and serving the models.

MLOps differs from DevOps in a few important ways:

In CI, MLOps also involves testing and validating of data, data schema and models.
MLOps considers a system: the training pipeline, instead of a single software package.
MLOps Constantly monitors, retrains and serves the models.

Machine learning has be described as high-interest rate credit card of “technical debt”. It can be fast and cheap to develop and deploy an ML system, but expensive and difficult to maintain. The real challenge is not building a model, but continuously operating it in production.

There are a few challenges specific to machine learning projects:

It includes Multi-functional teams.
It is experimental in nature, you must constantly try new approaches with data, models and parameter configurations.
Testing a ML system is more involved than testing a software package.
Deployment is more complex, it may require you to deploy a multi-step pipeline.
Model decay due to constantly change of data profile.

Bias

When you are trying to teach a computer, you may end up exposing it to your own bias. By machine learning, computer learns to find patterns in data, human bias become part of technology we create. Some examples include:

interaction bias
latent bias
selection bias

Transform Business with ML

The magic of machine learning comes with quantity, not complexity. Freedom to experiment and fail is important. You want to have an environment where you fail fast and get the ability to iterate to try new ideas. For ML to be magic, you should keep in mind the word “delight” as delight your users. Your business can benefit from ML in three ways:

infuse your apps with ML
fine tune your business
delight your users, anticipate their needs

ML Surprise

Machine learning is the process of a computer writing a computer program to accomplish a task. The computer figures out the best program to write by only looking at a set of examples. Most people really tend to focus just on optimizing the ML algorithm, in fact, in Google optimizing the ML algorithms takes a much smaller segment of effort than people expect. We should really pay close attention to:

data collection
building infrastructure
integration

Avoid these top 10 ML pitfalls

ML requires just as much software infrastructure
No data collected yet
Assume the data is ready for use
Keep the human in the loop
Product launch focused on the ML algorithms
ML optimization for the wrong thing
Is your ML improving things in the real world
Using a pre-trained ML algorithm VS. building your own
ML algorithms and trained more than one time
Trying to design your own perception or NLP algorithms

The path to ML

Organizations must continually improve business processes through a feedback loop. Understanding that loop becomes really important to understand the role ML plays inside a large organization. Business processes that eventually end in ML, typically go through 5 phases:

Individual contributor (eg. a receptionist)
Delegation (eg. store checkers)
Digitization (eg. ATM)
Big data and analytics (eg. car manufacturing)
Machine learning (eg. recommendation)

Confusion matrix

Instead of just looking at how your model perform over entire data set, we need to break the performance down to the subgroups. You want to look at the performance over all subgroups to identify the areas of improvements. A common way to evaluate performance is to use Confusion matrix to look at inclusion. There are both Positive and Negative for both labels and model predictions. You compare the labels and model predictions.

True Positive – when model accurately predicts.
False Negative (Type II error) – when label says something exists but model does not predict it
True Negative – model does not predict it and label says something does not exist
False Positive (Type I error) – when label says something does not exist, but model predicted it

Sometimes False positive is better than False negative, sometimes vice versa. Once you figure out what right evaluation metrics to focus on, then find the threshold that bring “Precision” and “Recall” to acceptable values. Then check “Precision” and “Recall” you obtain with the threshold with each subgroups.

Equality of Opportunity

This is an approach striving to give each individual an equal chance of desired outcome. If you keep everything about a person the same, and changing them from being member of 1 subgroup to another, the chance of qualifying the desired outcome remain the same. This approach gives you way to scrutinize your model in order to discover possible areas of concern. A better trade off between accuracy and non-discrimination can make your machine learning model more inclusive.

Simulating decisions without constraints may result in unequal distributions among subgroups.
Similarly, simulating decisions maximizing profits results in unequal standards.
“Group unaware” approach holds all the subgroups in the same standard, but which can be unfair to some subgroup.
“Equality of opportunity” approach, which result in identical True Positive rate for all subgroups, will do equally well among subgroups.

Use visualization to help unlock nuances and insights into large data sets. Facets will help you make machine learning model more inclusive.

My Certificate

For more on Machine Learning in Google, please refer to the wonderful course here https://www.coursera.org/learn/google-machine-learning

My #57 course certificate from Coursera

I am Kesler Zhu, thank you for visiting. Check out all of my course reviews at https://KZHU.ai