# Learn Data Science for Business ## Become a Data Scientist in our online courses Earn 6 figures or more in 6 months or less by learning R, Shiny, Machine Learning, Time Series, Web Apps, AWS, Cloud, and more! 5-10 Hours Per Week. 80/20 Skills. End-To-End Business Projects. [Start Your Courses Now! 👉](https://university.business-science.io/p/5-course-bundle-machine-learning-web-apps-time-series?el=website) Join over 100,000+ Data Scientists #### DATA SCIENCE CHEAT SHEETS FOR FREE! #### _Choose your interest:_ Interested in R Interested in Python Interested in Segmentation Interested in Machine Learning Click Here to Download! # K-Means Clustering in R the Tidy Way (Feat. Tidyclust) _Written by Matt Dancho_ * * * To be successful as a Data Scientist, you’re often put in positions where you need to find groups within your data. One key business use-case is finding clusters of customers that behave similarly. And that’s a powerful skill that I’m going to help you with today: **It’s called Customer Segmentation with K-Means Clustering.** ### Table of Contents Today you’re going to become better at clustering customers. Here’s what you’re learning today: - What K-Means Clustering is (and why it’s absolutely essential to your data science career) - How to use an amazing R package to make K-means clustering a breeze (and _insantly_ improve your modeling workflow) - **Bonus: Steal my brand NEW data visualization code (for the Customer Segmentation Plot we make today)** * * * # R-Tips Weekly This article is part of R-Tips Weekly, a [weekly video tutorial](https://learn.business-science.io/r-tips-newsletter) that shows you step-by-step how to do common R coding tasks. Pretty cool, right? Here are the links to get set up. 👇 - [Get the Code](https://learn.business-science.io/r-tips-newsletter) - [YouTube Tutorial](https://youtu.be/QYcrjrwKswc) # This Tutorial is Available in Video I have a companion video tutorial that walks you through how to do the full K-Means Clustering Analysis and create the Marketing Cluster Plot. And, I’m finding that a lot of my students prefer the dialogue that goes along with video-based coding. So check out this video to see me running the `R` code in this tutorial. 👇 K-Means Clustering In R (feat. Tidyclust) - YouTube [Matt Dancho (Business Science)](https://www.youtube.com/channel/UCqQ_cxcNu1ekqHXDc-MbU_w) # K-Means Clustering: What it is AND why it’s important I remember in my days of working as the Director of Sales for Bonney Forge, a company that did roughly $400,000,000 per year selling valves and fittings to Energy companies. Bonney Forge had tons and tons of customers AND tons and tons of product SKUs. Each customer had different purchasing patterns AND because of this… ## There was a big problem: we couldn’t easily target them for marketing campaigns. Some customers would buy specific products for “Upstream” applications (where they pulled oil out of the ground). Some would purchase products for “Midstream”, or transmission from an oil patch to a refinery. And some would buy products for “Downstream”, focusing on refining the oil into useful products. To make matters worse, some customers would have weird buying patterns where they’d buy only valves. And others would buy only fittings. And some would buy both. And, with over 1,000,000 SKUs, it became really tough to market to them. In fact, for the longest time we didn’t. And we lost out. ## Enter K-Means Clustering I needed a way to group our customers based on what they were buying in real time. The algorithm that showed the most promise was **K-Means Clustering**. [According to Wikipedia](https://en.wikipedia.org/wiki/K-means_clustering)… > **K-Means clustering** is a method that aims to partition n observations into k clusters in which each observation belongs to the cluster with the nearest mean (cluster centers or cluster centroid), serving as a prototype of the cluster. When the K-means algorithm converged, it found centers that minimized the “within-cluster variances” of the groups. ## The main challenge with K-means: how to pick K Picking how many clusters (the value for “K”) is the toughest part. Nowadays there are more rigorous methods, but often in business data I had to test a few values (e.g. 4 to 8 clusters). Then I’d settle on the number of clusters that made the most business sense. Other methods include the [“Elblow Method”](https://en.wikipedia.org/wiki/Elbow_method_(clustering)) and the [“Silhouette Method”](https://en.wikipedia.org/wiki/Silhouette_(clustering)). I won’t dive into those methods here. But you can find more detailed discussions on Wikipedia. ## Armed with K-means, here’s what it did for me **So what did K-means get me?** Well, it turns out that I didn’t know a lot about our customers. And running K-means gave me a better understanding of which customers behaved similarly. This in and of itself wasn’t the end. But rather it helped me with a much larger project, which was customer lead scoring. Armed with K-means, I knew which products customers preferred. And combining with Lead Scoring, I could determine the probability of purchase. And I used this information to develop a Lead Scoring System that **grew our team’s revenue from $3,000,000 per year to $15,000,000 per year in about 2 years.** Even more remarkable, this was during 2014-2016 Oil Recession which was the time when [Oil Prices dropped from $100 per barrel to $25 per barrel](https://www.bls.gov/opub/btn/volume-4/pdf/the-2014-plunge-in-import-petroleum-prices-what-happened.pdf). **And our company went through 4 rounds of layoffs.** This was the worst economic period in our company’s history. And, somehow I was promoted twice, doubled my salary (went from $80,000 per year –> $155,000 per year), and went from overseeing 5 people to 60+ people inside the organization. ## So what did K-means really do for me? **K-means gave me job security.** A common theme in data science: there was always more to be done. And each problem more often than not had a positive outcome financially. And even with a 5X growth, **I was just scratching the surface with Data Science.** Had we used a marketing strategy that integrated customer purchasing behaviour, I guarantee there’s more growth we could have achieved. But such is life. I’ll take a 5X ROI. And my CEO was happy. I’ll take that too. Ok, onto the K-means clustering tutorial… # Tutorial: How to Cluster Customers with K-Means Clustering and `tidyclust` I’m going to share exactly how to make the customer segmentation that is shown in this Customer Segmentation Plot. And as a BONUS, I’ll share how you can get the Customer Segmentation Plot (both static and interactive versions) for FREE. ## Step 1: Get the Libraries and Data First, load in the following libraries. The data is stored in a CSV file. We’ll read it in. The raw marketing data is composed of 2,240 rows and 29 columns containing: - Personal Attributes - Customer History & Marketing Campaign Data - Removed Columns ## Step 2: Prepare the Data In this step, we need to get the dataset ready for modeling with K-means. To do so, we will: 1. Remove `NA` values (could also impute) 2. Add a feature, `Dt_Customer_Age` that identifies how long the customer has been with the company (and remove the `Dt_Customer` feature) 3. Add a feature, `Spent` that calculates the Customer’s total spend 4. Remove features we don’t want to influence the grouping The prepared dataset looks like this: ## Step 3: Preprocess the Data with `recipes` Next, we will use `recipes` to further preprocess the data: 1. One-hot encoding character and categorical data 2. Scaling & center all numeric data to mean zero and standard deviation 1 (also called normalizing) 3. Removing the “ID” feature (not predictive) The transformation when applied looks like this: # Step 4: K-Means Clustering (feat. `tidyclust` + `tidymodels`) Next, I’ll perform K-Means Clustering using `tidyclust` and `tidymodels`: 1. Make a K-Means Model. Be sure to identify the number of clusters. 2. Train the K-means Model (using a Tidymodels Workflow that combines the model, recipe, and fits to the prepared data.) With a trained (fitted) model, we can get the clusters with `predict()`: # BONUS: Steal My Customer Segmentation Library + Visualization Code for this R-Tip Want all the code I just showed you? [Steal my R-tip library.](https://learn.business-science.io/r-tips-newsletter) # Need to advance your business data science skills? I’ve helped 6,107+ students learn data science for business from an elite business consultant’s perspective. I’ve worked with Fortune 500 companies like S&P Global, Apple, MRM McCann, and more.