Busting the 5 myths of AI with the Cortana Intelligence Gallery
This post was authored by Rimma Nehme, Technical Assistant, Data Group at Microsoft.
Today, the business potential of Machine Learning and AI is real. Businesses can apply ML and AI to transform, optimize and automate their businesses having previously relied only on human intelligence. You may wonder, ‘how can I apply AI to my business?’ This blog post describes some of the ways you can do that using the resources in the Cortana Intelligence Gallery, without a PhD in Machine Learning or AI or even a deep expertise in these subjects. A recent blog post on 7 Myths of AI by our partner CrowdFlower inspired us to write this blog to show how customers can easily use examples in the Cortana Intelligence Gallery to build ML and AI applications.
What is the Cortana Intelligence Gallery?
Figure 1: Cortana Intelligence Gallery
The Gallery is a community hub built on the cloud to discover, learn, and share samples built using the Cortana Intelligence Suite, Microsoft's end to end advanced analytics offering on the cloud. If you are looking for sample data science solutions, starter machine learning experiments or tutorials to skill up, you should look no further. Predicting breast cancer, sentiment analysis, online fraud detection and predicting wine quality – these are just a few of the interesting live examples that the user community has shared on the gallery. Anyone can leverage these community assets in their own work and anyone can contribute their own live examples.
Now let us see how we can dispel some of the AI myths using the Cortana Intelligence Gallery.
Myth 1: AI is all magic
Many people think of AI as magical and mystical and the domain of ivory tower researchers in machine learning and intelligence. This portrayal is a disservice. AI, in its simplest form, is data and algorithms (with some math) and lots of experimentation with algorithmic recipes for predicting from data. The Cortana Intelligence Gallery is a community driven site for discovering and sharing recipes for machine learning and AI that anyone can use. It is a destination for users looking to learn from others, to get started developing their own AI and ML solutions, or to contribute their own work to the AI, ML and advanced analytics community. Some of the interesting examples our community has shared are:
Titanic Survival Predictor (Would you survive the Titanic disaster? Given information such as your gender, age, and accommodation class, you’ll be able to figure out your odds of survival from the doomed liner. Don’t you want to know?):
Figure 2: Titanic survival predictor
Restaurant recommendations (Wouldn’t you want to know how to build recommenders that predict similar things (like restaurants) that users might like based on past habits and item characteristics):
Figure 3: Restaurant recommendations
If you go to the Experiments section on the Cortana Intelligence Gallery you will see hundreds more – from Predictive Maintenance, Fraud Detection, and Churn Prediction to Retail Forecasting, Heart Disease Prediction, and Face Detection.
Myth 2: AI is only for the elite technology companies
Many articles and blog posts online imply that AI is only the domain of elite technology companies that can afford to assemble large teams of machine learning experts and invest large amounts of money. This perception is wrong. Today it is possible to apply AI to one of your business processes with just a click of a button (click “Deploy” in Cortana Intelligence Gallery – and it literally deploys the entire solution with a single click of a button to your Microsoft Azure account). AI is for every business.
Figure 4: One-click deploy in Cortana Intelligence Gallery
If you come to the Cortana Intelligence gallery, you will see examples that are applicable not only to large enterprises but also to small businesses and startups.
Any business or organization can leverage examples from the gallery -
- IT departments within large organizations can quickly detect and fix issues based on underlying health metrics like CPU and Memory (from their IT infrastructure) and Timeouts, SLA variations and Brownouts (from their IT services)
- Airline companies can do predictive maintenance for their aircraft and aircraft engine components
- Real estate agencies can predict the real estate sales price of a house
- A company can monitor real-time data like weather data from a sensor, or financial data from the stock market or social media’s reaction to a newly launched product or campaign
Figure 5: Cortana Intelligence Gallery examples
Let me give you some concrete examples of some of our varied customers:
Portland Trail Blazers,an NBA franchise known for its passionate fans used data-driven insights to increase ticket sales. With big data and advanced analytics , the Trail Blazers were able to better identify leads and significantly increase their marketing efficiency.
The Tacoma public school system, through an intense effort leveraging data and predictive analytics, boosted graduation rates from 55 to 82.6 percent. Today, the district is recognized nationally for its educational achievements.
Track Revenue, a Silicon Valley startup offering comprehensive advertising and marketing campaign analytics boosted customer revenue-per-click by thirty-eight percent using machine learning.
The stories are endless and many of them began with the Cortana Intelligence Gallery.
How did these companies achieve their successes? If you said things like “advanced analytics”, “big data technology” or “predictive analytics”, you are partly right. Yes, they used advanced analytics and big data technology. But more importantly there were one or more highly motivated individuals inside these institutions that thought big and thought about how they could do things better. Driven by their passion for making the lives of their customers better or ensuring that they do everything they can to turn their students into rock stars, they walked a path not walked before and chose to explore technology not explored by their institutions before. Well, if they can do it, you can do it too. Each and every one of you working in any business can benefit from data science, machine learning and make a meaningful contribution to your business. All it takes is the initiative to explore how data science can transform your business.
Myth 3: AI is only for billion $ futuristic problems
Many of the stories online that tend to get covered sound too futuristic, like self-driving cars or autonomous drones or virtual reality. Big corporations are investing millions of dollars into being the first to market. This can give the impression that AI and machine learning are only for new problems. But that’s a mistake. AI is equally applicable to many of the existing problems. For example, one of the core essentials of any business is to understand its customers. This was true before and it’s still true today. Many companies are sitting on a treasure trove of data about their customers that is waiting to be explored and understood. AI can be applied to understand that data and convert it into intelligence. So AI is not only for cool new problems like driverless cars but also for many existing “boring” problems such as understanding your customers better through social media sentiment analysis or purchase history and making their experience with your business an unforgettable one. Just type “customer” in the search bar in the Cortana Intelligence Gallery and you will get 160+ solutions examples at your fingertips.
Below are some canonical examples to get you thinking. Do you have a scenario that makes sense for your business? Take a look in the gallery.
- Employee hiring and retention
Figure 6: Examples of customer-related solutions in the gallery: customer segmentation and classification, customer relationship prediction, customer propensity to buy, customer churn prediction, sentiment analysis for email, email marketing visit predictor
Myth 4: Machines are better than humans
The Internet and media love to portray machines with AI as being better than humans. But this is a misrepresentation of what is actually happening. Machines and Humans have complementary capabilities, and ideally you would bring both skills together in software to create powerful applications. This is called “human in the loop” AI. So what is "human in the loop" and why does it matter? Imagine that I'm a data scientist and I want to build a predictive model in a completely new domain. That means I need new labelled training data to teach my machine learning system. Some may get stuck here because getting a lot of historical data correctly classified so that one can learn from it takes time and effort. But what if we could get people on the internet to build the training data set for you? All you need to do is write down your data labeling task exactly how you want it to be, and overnight, you can get very large numbers of records labeled through crowdsourcing. Yet another application is when you need to increase the accuracy in online production deployments of machine learning models. Here a human-in-the-loop solution can help again. You can take the confident predictions from ML algorithms and use them directly and take less confident predictions and send them to a human to get them classified with higher confidence through CrowdFlower. In this way, you can get the best combination of accuracy and automation that you need. So the right framing within a business context is to realize that Machines and Humans can now complement each other’s intelligence within a software application. Check out the two examples in the Cortana Intelligence Gallery - Logistic Regression for Text Classification (Sentiment Analysis) and Consumer Complaint / Support Ticket Classification where we have partnered with CrowdFlower to build such human in the loop solutions.
Figure 7: Machine Learning and human-in-the-loop as a single platform examples in Cortana Intelligence Gallery
Myth 5: AI is all work and no play
This brings me to the final myth that AI is all boring, hard work and no play at all. We’ve introduced the concept of AI gamification by introducing competitions in the Cortana Intelligence Gallery. You do not need to be an expert to compete. We’ve recently hosted two competitions: Decoding Brain Signals and Women’s Health Risk Assessment. Our tutorials, videos, and data set descriptions make it easy for those with an interest in analytics, data science, machine learning and AI to compete. You can explore unique data sets by participating in varying levels of competitions. These datasets are being released publically for the first time, so don’t miss out. You can compete with data science experts from all over the globe online. What better way to learn than some healthy competition? Can you build predictive models with a higher accuracy than the experts? Cortana Intelligence Gallery allows you to submit multiple entries, so keep learning and iterating. By contributing your ideas and creativity towards meaningful challenges and sharing results with others your submission may advance the data science field, and in many cases science in general. Finally, you can win prizes worth thousands of dollars or find yourself on the coveted Top 10 public leaderboard.
Figure 8: Gamification of AI through competitions in the Cortana Intelligence Gallery
So now to replace the 5 myths with the 5 truths…
Truth 1: AI = f(data, algorithms, humans) , and you can find a lot of AI examples and resources in the Cortana Intelligence Gallery.
Truth 2: AI is for every business, and any business can find useful resources in the Cortana Intelligence Gallery.
Truth 3: AI is for many existing business problems, and you can find templates and pre-built solutions for many of those problems in Cortana Intelligence Gallery.
Truth 4: AI is Machines augmenting Humans, and you can find solutions for “human-in-the loop” AI in Cortana Intelligence Gallery.
Truth 5: AI can be learned through play. You can participate in competitions hosted in the Cortana Intelligence Gallery, learn AI and make a difference.
What else can you find in the Gallery?
The Cortana Intelligence Gallery contains a variety of examples for industries leveraging big data, data warehousing, and machine learning. For example:
Industry-specific solutions - The Industries section of the Gallery brings together various resources that are specific to such industries as retail, manufacturing, banking, and healthcare.
Figure 9: Industry-specific examples in Cortana Intelligence Gallery
Solution templates - Solutions provide a starting point to quickly build a complete end-to-end solution for an industry-specific problem. They include extensive documentation and all the tools you need to get you going on your own solution.
Figure 10: Quick-to-Start solutions in Cortana Intelligence Gallery
Experiments - The Gallery contains a wide variety of experiments that have been developed in the Azure Machine Learning Studio. These range from quick proof-of-concept experiments that demonstrate a specific machine learning technique, to fully-developed solutions for complex machine learning problems.
Figure 11: Experiments in Cortana Intelligence Gallery
Machine learning APIs - A solution developed in Azure Machine Learning can be launched as a web service so that the analytics model can be accessed by others through a set of REST APIs. A variety of these finished solutions are available in the Gallery, such as a product recommendation engine or cloud-based face and speech recognition.
Figure 12: Machine Learning and Cognitive APIs in Cortana Intelligence Gallery
Jupyter Notebooks - Jupyter Notebooks include code, data visualizations, and documentation in a single, interactive canvas. Notebooks in the Gallery provide tutorials and detailed explanations of advanced machine learning techniques and solutions.
Figure 13: Notebooks in Cortana Intelligence Gallery
Tutorials - A number of tutorials are available to walk you through machine learning technologies and concepts, or to describe advanced methods for solving various machine learning problems.
Figure 14: Tutorials in Cortana Intelligence Gallery
Collections - A collection allows you to group together experiments, APIs, and other Gallery items that address a specific solution or concept for future reference.
Figure 15: Collections in Cortana Intelligence Gallery
I wish you a great journey into the rich and fascinating world of AI and Machine Learning with the Cortana Intelligence Gallery.