Welcome to my newsletter, which I call Drop-In Class because each edition is a short, fun intro class for technology concepts. Except unlike many instructors, I'm not an expert yet: I'm learning everything at the same time you are. Thanks for following along with me as I "learn in public"!
Why recommendation engines matter right now
Recommenders are behind so many things we interact with: Google searches, Amazon shopping, Spotify playlists, Hinge dates, even your Chick-fil-A order.
The better the recommendation, the more you consume. The more you buy, the more music you listen to, the more chicken sandwiches you order.
But that’s all old news, right? Recommendation engines have been around for a long time, so why talk about them now?
Because we are in the midst of an exciting recommendation battle! Everyone from streaming platforms to fashion retailers are competing for clicks and views. Spotify and Apple Music are dueling for users. Facebook and TikTok are fighting to get people hooked on a video feed. And they see better recommendations as an answer.
So recommendation engines are getting more personalized and more pervasive. At least 30% of songs streamed on Spotify are recommended by AI!
Let’s learn a little more about how recommendation engines decide what choices to show us. Here’s today’s music selection!
How do recommendation engines work? Fancy filtering
A recommendation is a prediction. When you tell your friend about a new workout class they should try, you’re not 100% sure they’ll like it, but you feel pretty confident based on what you know about them. And you’ve filtered out all the other possible workout classes that you could have recommended.
Same goes for the machine learning algorithms behind recommender systems — also called recommenders, or recsys for short, if you want to sound like a pro.
Peloton, for instance, wants to be that friend telling you about a workout class. They’ve got a massive library of workouts to pick from, and Peloton’s Personalization team wants to narrow it down for you.
So to narrow down the choices, every recommender does some kind of filtering. They have to sort through all the possible content to find options. Then they rank the best options and choose which ones to show you.
There are different approaches to filtering. Here are the three main ones:
Content-based filtering: This one’s straightforward. The recommender will filter based on product characteristics, like restaurant cuisine, film genre, or clothing brand. It’ll recommend content with similar attributes to other content you’ve previously liked.
In our Peloton example, a workout class has attributes like duration, music genre, instructor, and difficulty level. If you like 20-minute rides with a pop-heavy playlist, they’ll surface more 20-minute pop rides. (Now if only Peloton did Progressive Rock Progressive Runs. That would get me on their platform every day).
Collaborative filtering: This one’s a little weirder and cooler. The algorithm filters for products based on preference information from many users.
For instance, Spotify uses collaborative filtering for Discover Weekly playlists. Spotify knows your listening patterns, and they know who else has similar patterns. If another user likes Glass Animals, Tame Impala, and Foster the People, and you like Glass Animals and Tame Impala, Spotify will recommend Foster the People.
Interestingly, Spotify has a playlist-centric approach. They analyze which songs are commonly grouped together in playlists. Because playlists are a good way to understand songs that are contextually similar, in a way only humans can curate.
Hybrid filtering: What it sounds like: A combination of the two! Netflix is famous for its hybrid filtering system. I think of Netflix as the OG recommender, with its “You May Also Like…” feature.
They’ve got different ranking systems for different purposes. “Trending Now” shows you what’s popular; “Continue Watching” calculates the probability that you’ll go back to something you started; and so on.
Are recommenders really that helpful?
Listen, there are good and bad things to say about The Algorithm. I’ve got my own thoughts on where I appreciate recommenders (thank you, Netflix, for telling me a new season of The Great British Bake-Off is out) and where they suck (no thanks, Instagram feed that only shows me the same five accounts over and over).
This line from a SiliconANGLE story about Chick-fil-A’s use of recommendations kind of gave me Black Mirror vibes:
“Most people don’t think twice when they open a mobile app and order food from a menu. They don’t have to think much about it because, in the case of Chick-fil-A Inc., the app is doing a lot of the thinking for them.”
This is a sci-fi novel I’d like to write: Nobody makes decisions anymore. You think your sandwich order is up to you, but it’s not! Free will is an illusion!
So whether you like recommenders or not probably depends on what service you’re using. But what we do know is recommendations really do make a difference, and it’s good for business.
E-commerce stores see an almost 23% increase in purchases when a customer clicks on a product that was predictively recommended. Facebook saw a 10% increase in video watch times when it started experimenting with a newly revamped recommender.
As consumers we have freedom of choice, but in today’s apps, perhaps freedom from choice is really what we want. In the wise words of DEVO.
Or in other words, don’t make me look through Peloton’s entire catalog of 10,000 classes. I’m good with the 20-minute Metal Ride you picked out for me 🤘
The cooldown: Extra reading
This is a 101-level drop-in, so I keep things at a high level! For deeper reading, check out these articles.
Facebook is using AI to supercharge the algorithm that recommends you videos (Engadget)
What Is a Recommendation Engine? How Recommenders Work (Amplitude)
How We Built: An Early-Stage Machine Learning Model for Recommendations (Peloton)
See you in the next Drop-In!
Cheers,
Alex