Machine Learning 101
Introduction – The Freight Train
The business headlines are dominated with buzzwords like machine learning, data analytics, and predictive analytics. You hear about case studies like “Chase uses predictive analytics to predict credit worthiness in customers”, “Netflix uses machine learning to recommend movies to its subscribers”, “Amazon segments customers using AI [artificial intelligence]”, and, fantastic as it sounds, “Most hedge funds allow machine learning algorithms to pick and trade stocks… with better success than the experts.”
Machine learning and artificial intelligence is everywhere. It’s pervasive in every aspect of life from you’re your big store purchases, to Google search recommendations, to Amazon delivery logistics, to Netflix movie recommendations. It’s also everywhere you’re not seeing. Literally, hundreds of millions of dollars of R&D money are being poured into virtual reality, augmented reality, autonomous drones, manual labor robots, gene sequencers, self-driving cars and, coming real soon, domestic robots. The convergence of all this research will create a reality in the next two decades that will make the previous 10,000 years look like we were standing still. And it’s going to happen at the speed and momentum of a runaway freight train: At one moment it’s off in the distance, and the next moment you don’t know what hit you.
Thing is… we hear so much about how machine learning is enabling Fortune 1000 businesses, but we hear nothing about how machine learning is helping Mary’s restaurant down the block attract new customers. We hear how Chase is savings hundreds of millions using predictive analytics, but we hear naught about how predictive analytics is helping Sam’s used car lot, Bob’s gyms, or Dr. Smith’s Chiropractic office.
If you are Mary, Sam, Bob, or Dr. Smith, pay close attention. The machine learning revolution will have a major impact on your business by 2020. Similar to the days when no business had or needed a web site, to today where a web presence is an important part of any businesses marketing strategy. Within 3 years, every business will need machine learning… but the impact will be far greater than the comparison of not having a web site. Those businesses using machine learning will lower costs and increase sales, almost in a sense of having an “unfair competitive advantage”. Those who choose not to adopt machine learning will struggle at best, and go out of business at worst. I liken it to football, where a business without machine learning is like a football player competing without a helmet. You may get lucky, but most likely you’ll lose bad.
OK, at this point you get the critical strategic advantage machine learning can provide for your business. But, you may have two burning questions:
- What IS machine learning?
- HOW exactly can it help me?
The remainder of this article is dedicated to addressing those two questions. You don’t need a math, statistics, or programming background to understand the core concepts of machine learning anymore than you need a PhD in rocket science to understand what a rocket does. All that’s required is an understanding of how to run and manage a business.
Part I – What is Machine Learning?
Machine learning (ML) is the field of artificial intelligence (AI) that is concerned with (a) finding patterns in your data, so you can (b) make better and more profitable business decisions. Let’s look at each in turn:
- The “learning” in “machine learning” simply means that a computer looks for, finds, and learns patterns in your data.
- When a computer finds patterns in your data, you can make “predictions” that will help your business prosper… that’s where the buzzword “predictive analytics” (PA) comes from.
Let’s look at a concrete example. A restaurant gathers number data points every day. For each customer, the restaurant gathers:
- Individual items ordered
- Amount paid for each item
- Total amount paid for all items
- Day and time of visit
Let’s see what a machine learning algorithm can do with a fraction of that data.
First, let’ create a simple plot of wine sales. We want to plot the time of day that a customer purchased a glass of wine, and the amount paid for that glass of wine. The plot below represents wine sales by customer by hour of day and price paid per glass:
We can clearly see that as the day progresses, the price a customer is willing to pay for a glass of wine steadily increases. In fact, we can even draw a line through the center of the plot that helps capture and visualize the relationship of time of day and price paid for a glass of wine, like this:
There is a clear upward trend, as depicted by the line we drew through the points on the plot. We can then ask the ML algorithm questions like, “What is the predicted price a customer is willing to pay for a glass of wine at 7:00 PM?” The ML algorithm uses the trend line to answer the question, as we can see in the next plot:
It appears that a customer who dines at 7:00 PM will be willing to pay about $13 for a glass of wine. Of course, some customers are willing to pay more, and some will pay less, but on average, the restaurant should expect about $13 per glass of wine.
This insight helps the restaurant:
- Predict sales of wine
- Manage its wine inventory
- Assist servers in recommending wine based on the time of day
For example, at Noon, a server knows to offer wines around $8 to ensure a sale; while at 7:00 PM knowing to offer wines around $13 so as to not leave any money on the table.
While valuable, this was a simple example in only two dimensions: “time of day” and “amount paid for a glass of wine”. The true power of ML algorithms comes in the analysis of multiple dimensions. While humans are limited to visualizing in only three dimensions, ML algorithms can, and often do, work in 10’s to 100’s of dimensions.
For example, let’s assume that we take all the data the restaurant has, and feed it to the ML algorithm. This includes all sales data that encompasses all customers, what they ordered, what they paid, and time of day. For example:
As you can see, there can potentially be thousands of variations. Finding simple patterns, like “More coffee is sold before 9:00 AM” is easy for humans, but finding complex patterns is the domain of ML algorithms. These complex patterns are exactly what gives a business that employs ML a competitive advantage. For instance, a ML algorithm may find a pattern of: “People who order sweet potato fries as a side are more likely to order cheesecake as a dessert.” This fact can better assist the server in targeting a desert option changing the familiar “Anyone save room for dessert?” to “We have a homemade cheesecake today”.
That’s another sale for the business using machine learning.
Part II – Give Me More
We took a very quick look at the basics of machine learning and how it can help a business. I hope this brief intro in Part I convinced you of the power of machine learning and predictive analytics. The more existing news is that the examples previous just scratch the surface. We can make even better predictions using additional data outside our own data. For example, and these are all true actual case studies:
- A national call center found that people are more willing to buy the day after their local sports team won a game, and less likely to buy of their sports team lost, or if the weather is gloomy.
- A 9-year study showed that people spend more during the new moon, and the least during a half-moon. (sorry full-moon enthusiasts)
- The audio wavelength pattern of a song can predict the song’s success or failure.
There are hundreds more examples. They all use “corollary data”. Corollary data is data mapped to the original data to help the machine learning algorithm find patterns. As noted, a ML algorithm can look at hundreds of separate data items at once and find patterns. Us humans can usually handle two or three at a time. This is why most hedge funds use machine learning algorithms to choose which stocks to buy and sell. Yes, stock trading algorithms work, and the majority of all stock trades are done by computers. That’s right… the majority of all stocks traded are chosen by and executed by machine learning algorithms. The ML algorithms start by looking at the price of a stock, let’s say Apple Computer. It then looks at corollary data, like the price of hundreds of other stocks, volume traded, the weather, political events and more. All this corollary data is crunched by the algorithms to predict the price of Apple’s stock. And they perform well.
Many businesses create their own corollary data. For example, a camera using artificial intelligent computer vision can anonymously note the approximate age and gender of each customer, thus correlating that information to the purchases made by the customer. The business owner can then make better recommendations to customers based on age range and gender, and increase sales. For example, “Men between 35-45 who order black coffee before 9:00 am are more likely to also purchase a bagel”… “Would you like a bagel with your coffee this morning, sir?”
That’s another sale for the business using machine learning.
Of course, recommendations and predictions are just one segment of what machine learning can do for a business. Other advanced applications include logistic planning, cash flow analysis, resource allocation, inventory management, marketing and sales, and operations stragey. All of this… from your data.
Specifically, let’s look at customer segmentation as it pertains to machine learning. In the past, customer segmentation was performed by hand, over many months, at a great cost, and may or may not be particularly accurate. Machine learning algorithms can examine multiple dimensions of data and segment customers in unique ways that could assist business owners in marketing and sales. For example, gym members swipe a membership card or enter a key code to gain access to the gym. As such, a local gym has years of data from thousands of members. This data includes age, gender, address and zip code, and the exact visits to the gym by day and time, as well as other data, like special classes the member may have signed up for or personal training. Let’s visualize these multidimensional data points in a 3D plot, where each dot represents the gym member’s age, gender, zip code, gym visits by day and time, and any special classes or training signed up for :
A machine learning “clustering” algorithm can then examine and find similarities among the data points, and “cluster the customer segments” far superior than most humans. This is the way customer segmentation is performed today by savvy businesses, helping them identify customers in ways never possible before. In our example, let’s suppose the machine learning clustering algorithm discovered four customer segments:
Armed with this knowledge, the gym can now laser focus its marketing on the four customer segments; or focus on just one or two segments that have the potential to drive revenue. In addition, the types of gym memberships offered no longer need to be a “one size fits all” approach. Instead, a gym using machine learning can offer four distinct membership types that would appeal to each customer segment, thereby ensuring new member sign up and retention.
Speaking of customer retention, yep, you guessed it, machine learning can assist with customer retention. As the machine learning algorithm finds and learns patterns in your data, it comes to expect the data to behave in a certain way. When the data deviates from expected patterns, the machine learning algorithm detects the deviation as an anomaly, and creates an alert. This is called “anomaly detection”, and is effective in detecting when a cooling tower is malfunctioning, an anomalous purchase has been charged to your credit card, or when Jane stops going to the gym 5 days a week… creating an alert for the gym manager to reach out to Jane and intervene before she cancels her membership.
That’s another sale for the business using machine learning.
Part III – Getting Started
The field of artificial intelligence got its official start in the summer of 1956. Since then there have been advances and setbacks. Advances came in the way of more and more clever and accurate machine learning algorithms and techniques. But the software far exceeded the capabilities of the hardware. Today, we have low cost cloud computing and cheap data storage. These two ingredients are key to the democratization of machine learning for everyone. Those factors are what convinced Google, IBM, HP, Microsoft, Amazon and more to create “machine learning in the cloud”.
Analyzing data and creating a machine learning model for a business is rather different than traditional computer programming. For one, machine learning models can generally be created faster and less expensive than a typically web application. Second, the accuracy of a good model is partially dependent on the business owner’s domain knowledge. In other words, he may be able to provide insights into corollary data that can uncover hidden gems in his data, and catapulting his sales.
The process typically consists of:
- High-level analysis of the business’s data
- This is usually sufficient to determine how machine learning can help a business
- A detailed analysis of the business’s data
- This phase involves a data scientist and/or a machine learning engineer to examine the business’s data from a variety of angles, as well as corollary data, to determine a strategy.
- Test <-> Learn
- This phase repeats until the machine learning engineer finds and optimizes the best machine learning model. This is markedly different from application development which involves a plan upfront and execution of the plan. Machine learning is much more akin to the scientific method than application development.
- This is the final phase which delivers the machine learning model to the business. Equipped with model that can now empower the business owner to capitalize on the findings, it’s like hiring an artificial intelligent manager that works 24-7 and little more than the price of electricity.
Less than 1% of the world’s data has been analyzed by machine learning algorithms. With the daily proliferation of data from web sites, IoT devices, and every brick and mortar business, that number is actually growing smaller every day. Th data that has been analyzed usually contains insightful data worth tens of thousands to millions, depending on the size of the business.
There could be gold hidden in your data. You may be leaving money on the table, or worse, allowing a competitor to capitalize on the machine learning revolution. Machine learning will be ubiquitous soon. That’s a fact. The question is, when will your business cash in?
SerpicoDEV offers a “free data checkup”. This is Phase I of the process: “High-level analysis of the business’s data”. During the evaluation, we’ll discuss your specific business and goals to align with our evaluation of your data. The checkup usually results in one of two outcomes:
- We analyze the data and notice patterns emerging that can be refined.
- We ascertain that there is not sufficient data, and we recommend collecting additional data, and provide specifics.
Either way, there’s no charge for the initial consultation. When you’re ready, contact us here, and we’ll get started: