Although many organizations recognize the importance of machine learning for making the most of their businesses, very few actually know how to get started. It is also required to develop a machine learning strategy for the business, moving beyond individual successes into a long-term growth plan.
Machine learning impacts entire business from processes and staff to products and services. This means it’s critical to know if you really are machine learning ready and understand how to get there.
Assess Machine Learning Readiness
The three dimensions of ML adoption are data, ML strategy and ML expertise. It important for organizations to assess their ML readiness, because there likely will be cross-organizational impact, the readiness affects short-term and long-term goals in the organization and it’s difficult to see the path of AI adoption if you don’t know where you are in readiness.
To assess readiness, first, you will need to identify your short and long-term goals with machine learning. “Where do you want to be at the conclusion of a project?” These goals will be constantly evolving as you and your organization gain experience with machine learning solutions. You’ll discover new goals and opportunities and be able to refine your starting assumptions.
Second, consider how these goals fit into your larger strategy. How aligned are these goals with your current organizational strategy, your current life goals and career targets as an individual, or as a team? Honestly assess where you are now, it is helpful to consider the five progressive stages of machine learning adoption:
| Initial (Exploratory stage) | No ML expertise and/or limited skills. Determine if ML is the right approach for the organization long-term goals. |
| Developing (Investment stage) | The organization is committed to investing in basic adoption of ML solutions to drive business goals. Develop your first proof-of-concepts. Assess the different approaches to adopting ML: build, buy or partner? |
| Competent (Experience stage) | You have usable data, you have deployed proof-of-concepts, you’re capable of building more. Your organization has experience with ML and is developing an AI strategy using this experience. |
| Proficient (Experimental stage) | You have built multiple ML systems and have begun driving revenue based on these solutions. Apply ML for various and diverse types of solutions as an organization. Implement a cohesive strategy that is guiding your ML investments in data, infrastructure and expertise. |
| Advanced (“You are ML” stage) | Not everyone needs to end up here. Your business is machine learning. Your organization is a leading machine learning organization that guides the direction of applications and / or research. |
Risk Mitigation
Risk for organizations manifest in many different ways: legal, political, and strategic. This high level of unpredictability makes the assessment of risk for an organization applying machine learning challenging if not intractable.
An organization can take steps to mitigate potential hazards, by identifying and prioritizing which type of risks are key to preventing serious issues with their machine learning solutions. Particularly, mitigate functional risks from a strategic perspective. For instance, when our machine learning model does not answer the question we defined, that presents a functional risk.
Functional risks in machine learning can be caused by:
- problem definition
- data
- too much or too little variability in your model
- how the model’s output is used and interpreted
Even before thinking about developing models, you should have some sense of the risk in the context of your short-term and long-term goals. We can mitigate risk by controlling how critical the machine learning application is to your business functions. You also need to consider the impact of machine learning on your processes, your products or services, and how your employees work.
Experimental Mindset
An experimental mindset is an organizational culture that moves you beyond what you already do well to experiment with new ideas and directions. It is an essential part of your machine learning strategy. Experimentation is the best way to identify if your machine learning problem and data can drive your business objectives. We recommend using a few steps to iterate through this approach:
- Prerequisites: identify your organizational challenges and objectives.
- Opportunities: knowledge and experience that you’ve already acquired or can easily access.
- Experiment: Take note of what goes well, what breaks and why.
- Evaluation: document, discuss, and sit with all the facts to improve model development and deployment.
We should also note that experimentation is not just about success. There will be failure and it is a necessary part of the process.
Build, Buy, or Partner
The phrase ‘Build, Buy, or Partner’ describes three approaches to acquiring solutions and skills that you need but don’t already have. Most of the time, when we’re building models, it’s to increase the effectiveness or efficiency of some existing business processes.
Developing your own machine learning solutions doesn’t happen in isolation. It depends on having the machine learning infrastructure that supports the ability to repeatedly and efficiently build and deploy solutions. So there are two things you might need to build / buy / partner:
- supporting infrastructure
- machine learning solution
Be honest about your skills and expertise right now, most likely through a combination of build / buy / partner, you’ll find a way forward to machine learning excellence.
Teamwork and Communication
It’s essential to fully understand your existing business processes, the problem you’re trying to solve using machine learning, and the data you have to work with. This information helps you come up with the best approach considering available resources, and identify who has the most relevant skills and experience.
A major part of running a machine learning project is managing expectations and deliverables. Aim for a simple yet reliable machine learning solution that your business can get value from, then it will be easier to convince other team members and executives to go for more ambitious plans from there.
To ensure success in your team, three distinct skill sets need to be covered:
- technical expertise
- machine learning scientist and analysts
- data experts (data warehousing, ETL processing, and streamlining)
- software development experts
- domain knowledge
- internal or external clients
- understand problem domain really well
- business oversight
- project manager, relationship manager
- non-tech view of project to ensure smooth execution
Working together the right mix of people can use machine learning to transform even supercharge your organization.
There are two main audiences for machine learning developers. You need both to be persuaded of the value of your project and for both, you need a clear communication plan:
- the people who make the decisions about budget and deployment
- focus on the value in the project, procedural issues and how it impacts the company.
- he front-line people who are going to be interacting with the system
- focus on why they care. How they’re going to use it and spend the necessary time to integrate it into their workflow.
Don’t rush your head without stopping to check whether you’re actually addressing relevant objectives, values and needs, and you need to make sure that you’re considering all the people who will be using or affected by your system.
If you don’t get buy-in from one group or both, don’t assume it’s because of their stubbornness or lack of vision. Take a close look at your product design and your communication:
- Are you providing value to the organization?
- Are you providing value to the users?
- Are you clearly communicating relevant plans and facts?
- What are you doing so that they can really feel the importance of this project?
- Are you being clear about how they’ll interact with the system and experience the potential going forward?
It’s important to accept that you are moving along the 5-stage machine learning adoption spectrum. You’re now fundamentally concerned with data and there will be new employees and technologies to incorporate into your tech stack. Make sure that all your leads and business units are aware of this change.
Responsible Machine Learning
Machine learning has its potential to change society in a negative way increasing inequality, oppression, exploitation, poverty, and desperation, as great as to change society in a negative way, increasing opportunity, justice, happiness, and growth.
As developers and machine learning practitioners, we need to make AI for good and for everyone, putting in the time and effort to think about:
- How our Machine Learning projects can benefit society
- How we can ensure that they’re inclusive
- How we might prevent or mitigate any harm they might cause
To help understand the AI ethics, there are frameworks, guidelines, and declarations by different government and non-government organizations. Most of the current guidelines and frameworks are carefully and thoughtfully done.
Positive / Negative Feedback Loops
Machine learning finds patterns in data. Finding the pattern doesn’t guarantee it’s useful. They only identify features correlated with the labels you give it. “Correlation is not causation.” This illustrates a danger in using these patterns or systems based on them to make decisions.
You can create runaway feedback loops, where the decision creates data that skews the system even further. This reinforces the correlations our machine learning system found. It becomes a self-fulfilling prophecy. The feedback creates a self-reinforcing bias and the cumulative results can be disastrous.
A related danger comes from using patterns identified by machine learning systems without understanding the reasons why those patterns exist. It’s impossible to totally remove bias from our machine learning models, since we rely on imperfect data from the start.
As much as we try, it’s not possible to completely identify and remove bias from our data before training. You have to use human judgment to identify potential issues your system will create and aim to proactively address them.
Seriously consider potential feedback patterns, look for runaway feedback loops where the predictions made by the system creates data that directly reinforces those predictions, and watch for cases where prediction and decision-making need to be separated.
Metrics Design
Metrics are important in machine learning, they are necessary as we compare different models and decide which does best. While good metrics can bring us important information and guide decisions, badly done metrics can lead us astray and even cause a whole lot of harm.
The key to creating good metrics is:
- First talk to domain experts and users.
- Never forget that you are measuring a proxy, not the real thing. The best metric possible is still a proxy – something measurable that you hope reflects the thing you care about. Sometimes the proxy is great and captures what you need it to, sometimes it seems like it should be a good proxy but actually isn’t.
- Moreover, metrics can be gamed. Gaming the metric means figuring out how to get undeservedly high score, finding ways of behaving that boost your metric without doing a good job of the underlying task.
- Don’t rely on or optimized for a single metric. Optimizing for a single metric is a frequent cause of negative secondary effects.
- Don’t put so much emphasis on the quantitative metric that’s easy to measure, that you forget about the qualitative, which is less easy to measure but at least as important.
- Don’t optimize short-term gains over long-term benefits.
Metrics should not be your goal and goals should not be metrics.
Secondary Effects of Optimization
Secondary effects mean anything that’s a direct result of your system that isn’t the primary intended effect (what we meant to do). They’re also sometimes called unintended consequences. There are three major kinds of secondary effects:
- unexpected benefits
- unexpected drawbacks
- perverse results
Regulatory Concerns
There are already many regulatory bodies that exist to establish safety standards and to protect consumers and users. When we develop machine learning applications in domains which have existing regulations, we have to comply with their standards and safety requirements. Many of these authorities are now adapting new measures to accommodate machine learning-based technologies in their respective fields.
Other than domain specific regulatory requirements, there can be regulatory concerns across different domains. Privacy of user data is one such concern. There are different laws introduced by different regulatory bodies across the world to safeguard user privacy in this era of ubiquitous data.
My Certificate
For more on Machine Learning: Readiness, Responsibility and Regulatory Concerns, please refer to the wonderful course here https://www.coursera.org/learn/optimize-machine-learning-model-performance
Related Quick Recap
I am Kesler Zhu, thank you for visiting my website. Check out more course reviews at https://KZHU.ai
All of your support will be used for maintenance of this site and more great content. I am humbled and grateful for your generosity. Thank you!
Don't forget to sign up newsletter, don't miss any chance to learn.
Or share what you've learned with friends!
Tweet
