Recommendation systems: how YouTube and TikTok know you

How recommendation systems work: the signals YouTube and TikTok listen to, the "people like you" idea, and lessons from the 1 million dollar Netflix Prize.

9 phút đọcCơ bảnEyeBlogs

Đọc bản tiếng Việt →

A "taste map" chart on a dark background: three clusters of points far apart. Each cluster has squares (videos) and hollow circles (viewers) of the same colour: gold for maths, blue for cooking, pink for football. Viewers sit right next to the videos that match their taste, even though the machine was never told which video belongs to which topic.
Mục lục

You watch one video about cooking pho, and your YouTube home page instantly fills with cooking videos. You pause for two seconds on a football clip on TikTok, and the next ten clips are all football. Behind this is a recommendation system: a machine that learns from the behaviour of millions of people to guess what you want to watch next. Its core idea is surprisingly simple: what did people with tastes like yours go on to like?

This article explains how recommendation systems work: what signals they pick up from you, why a machine can understand “taste” without anyone teaching it what videos are about, the lessons of Netflix’s 1 million dollar prize, and how YouTube picks a few dozen videos out of millions in the blink of an eye. At the end there is a piece of code so you can build a tiny recommendation system yourself.

What does a recommendation system learn from you?

YouTube says its recommendation system learns every day from more than 80 billion signals: which videos you click, how long you watch, how you answer surveys, what you share, like and dislike[1].

That list has a history. At first, the click was the main signal. But in 2011 YouTube realised that clicking a video does not mean you actually watched it, because a sensational title can trick anyone into clicking. In 2012 they added watch time. Then they realised watch time was not enough either: you can watch a whole video and still feel it wasted your time. So YouTube asks some viewers directly to rate videos from one to five stars, and only counts as “valued watch time” the videos rated four or five stars. Since most people do not answer surveys, they trained a model to predict the answers for everyone[1].

TikTok describes its system as based on three groups of signals[2]:

  • How you interact: the videos you like or share, the accounts you follow, the comments you write, the content you create.
  • Video information: captions, sounds, hashtags.
  • Device and account settings: language, country, device type. These carry less weight, because they are not things you actively choose.

What the two have in common: the system does not read your mind. It only looks at what you do, then compares it with what millions of other people have done.

“People like you”: the core idea of a recommendation system

Suppose An and Binh both watched five baking videos all the way through and both abandoned three football videos. Today Binh watched a bread-making video to the end that An has not seen. Will An like it?

That is collaborative filtering: you do not need to know what a video is about, only who watched what. But comparing every pair of people is far too slow when there are billions of users. A more efficient approach is to place every person and every video as a point on a taste map, so that people sit close to the videos they like. To recommend something to An, just find the videos near An that An has not watched yet.

The remarkable part is that the map learns itself from the data. The image at the top of this article is an example produced by the code at the end: 30 viewers and 24 videos from three topics, maths, cooking and football. The machine is not told which video belongs to which topic, only what share of each video each person watched, for about a quarter of the videos. Even so, the videos sort themselves into three clusters, and each person sits next to the cluster they like.

In real systems, each person and each video is not a point on a plane but a list of tens to hundreds of numbers, called an embedding. Its dimensions have no names. One dimension might happen to correspond to “likes long content”, another to “likes humour”, but nobody names them in advance. The machine finds by itself which axes best explain behaviour.

Netflix’s 1 million dollar prize

In 2006 Netflix, then still mostly renting DVDs by post, opened a competition: whoever could improve the accuracy of its system for predicting the star ratings users give films by 10% would win 1 million dollars. They released about 100 million (anonymised) ratings for everyone to work with.

One of the most important discoveries of the competition was exactly the taste map above, under the name matrix factorisation: treat the “people × films” table as a huge, almost empty matrix, then find two small tables (people vectors and film vectors) that, multiplied together, best match the known cells. Members of the winning team later wrote the method up in a classic paper[4].

In 2009 the team BellKor’s Pragmatic Chaos reached the 10% mark and won. But the most interesting part came later. In 2012 Netflix wrote that it had put into production the two best algorithms from the 2007 progress prize, including matrix factorisation, but did not use the solution that won the grand prize. That solution blended more than 800 models, and the extra accuracy was not worth the engineering effort to run it in a real system. Besides, Netflix had by then moved to streaming, where people pick a film instantly rather than waiting days for a disc, and the recommendation problem had changed. In the same post, Netflix said that about 75% of what people watch comes from some form of recommendation[5].

The lesson is familiar to anyone who builds systems: the most accurate model on paper is not necessarily the best model to run for millions of people every day.

Picking a few dozen videos out of millions

YouTube has millions of candidate videos for each person and must answer within a few hundred milliseconds. Carefully scoring every video is impossible. In 2016 YouTube engineers described their system as split into two stages[3]:

  1. Candidate generation: from millions of videos, quickly pick a few hundred that look relevant. This stage uses the taste map idea itself: turn the user into a vector, then find the videos whose vectors are closest, an extremely fast search.
  2. Ranking: for those few hundred candidates, a more careful model, using many more signals, scores each video and sorts them.

At the ranking stage, YouTube chose to predict watch time rather than the chance of a click. The reason is stated plainly in the paper: ranking by click-through rate tends to promote deceptive videos that viewers do not finish[3]. The paper also describes a small trick: during training they tell the model how many days old each example is; when making real recommendations they set that age to zero. That way the model does not favour videos that have been popular for a long time, and new videos get a chance to be recommended[3].

The “quick rough filter first, careful ranking later” architecture appears everywhere in large systems, in the same spirit as the Bloom filter standing guard in front of a database: use a cheap step to rule out most possibilities, and only then use the expensive step on what is left. Medium even used a Bloom filter to remove articles you had already read from its recommendations.

One more piece: training the model usually runs periodically over a huge data store, while recommending must happen instantly whenever you open the app. The two are usually split into separate systems: one slow, heavy and batch-based; the other fast, light and working from precomputed results.

New users, new videos and the filter bubble

Cold start. Someone who just installed the app has no history, and a video that was just posted has no views: the system does not know where to place them on the map. TikTok says that for new users it starts from the interests you pick when signing up; if you pick none, it shows a general feed of popular videos to get some initial data[2]. A new video is tried with a small group of viewers before the system knows who it suits. The experiment at the end of this article shows the problem clearly: wherever data is thinnest, the guesses are worst.

The filter bubble. A recommendation system optimises for what you have already liked, so it can easily show you more of exactly that, narrowing as you watch. That is why platforms deliberately mix in some content that differs from your habits, accepting slightly “less accurate” recommendations to learn more about you and keep you from being boxed into one corner. TikTok says it has mechanisms to diversify content in its “For You” feed[2]. This is the classic tension between exploitation (recommend what you certainly like) and exploration (try something new to learn more).

Build a tiny recommendation system yourself

The Python below creates a small world: 30 viewers, each with a hidden taste for three topics, and 24 videos. Each viewer has watched only about a quarter of the videos, and the “score” is the share of the video they watched. The system only sees the watched cells, then uses matrix factorisation to guess the rest.

The learning method is alternating least squares: keep the video vectors fixed and find each viewer’s vector that best fits what they watched, in the spirit of Gauss’s least squares; then keep the viewers fixed and find each video’s vector again; repeat. Large systems usually use gradient descent instead of this formula, but the idea is the same. The code produces both the image at the top of the article and Figure 1. You need numpy and matplotlib.

recsys.py
import numpy as np
import matplotlib.pyplot as plt
plt.rcParams.update({"figure.facecolor": "#16181c", "axes.facecolor": "#16181c", "text.color": "#d4d7dc",
"axes.edgecolor": "#2a2e35", "xtick.color": "#878d97", "ytick.color": "#878d97", "font.size": 12})
rng = np.random.default_rng(2026) # fixed seed so reruns give the same result
# 30 viewers, 24 videos in 3 topics. Each viewer has a hidden "taste": how much they like each topic.
TOPICS = ["maths", "cooking", "football"]
n_users, n_videos = 30, 24
video_topic = np.repeat(np.arange(3), n_videos // 3)
user_main = rng.integers(0, 3, n_users) # each viewer's favourite topic
taste = np.full((n_users, 3), 0.15) + rng.uniform(0, 0.1, (n_users, 3))
taste[np.arange(n_users), user_main] = rng.uniform(0.75, 0.95, n_users)
# True score: share of the video watched (0…1) — people finish videos that match their taste.
truth = np.clip(taste[:, video_topic] + rng.normal(0, 0.05, (n_users, n_videos)), 0, 1)
# Each viewer has watched only 25% of the videos. The rest is what the system must guess.
seen = rng.random((n_users, n_videos)) < 0.25
print(f"known {seen.sum()} / {seen.size} cells ({seen.mean():.0%})")
# Matrix factorisation: each viewer and each video is a 3-dimensional vector; predicted score = dot product.
# Learn by alternating: fix the videos, find the viewer vectors that best fit the known cells (least squares),
# then fix the viewers and find the video vectors; repeat.
K, reg = 3, 0.1
U = rng.normal(0, 0.3, (n_users, K))
V = rng.normal(0, 0.3, (n_videos, K))
for step in range(50):
for u in range(n_users):
Vs = V[seen[u]]
U[u] = np.linalg.solve(Vs.T @ Vs + reg * np.eye(K), Vs.T @ truth[u, seen[u]])
for v in range(n_videos):
Us = U[seen[:, v]]
V[v] = np.linalg.solve(Us.T @ Us + reg * np.eye(K), Us.T @ truth[seen[:, v], v])
pred = np.clip(U @ V.T, 0, 1)
unseen = ~seen
mae = np.abs(pred - truth)[unseen].mean()
baseline = np.abs(truth[seen].mean() - truth)[unseen].mean() # guess that everyone watches the average amount
hit = np.mean([video_topic[np.argmax(np.where(unseen[u], pred[u], -1))] == user_main[u] for u in range(n_users)])
print(f"mean error on unwatched cells: {mae:.3f} (guessing the average: {baseline:.3f})")
print(f"top recommendation is in the viewer's favourite topic: {hit:.0%} of viewers")
# Figure 1: the known scores (left) and the table the system fills in (right).
order_u = np.argsort(user_main, kind="stable")
fig, axes = plt.subplots(1, 2, figsize=(11, 5.2))
shown = np.where(seen, truth, np.nan)[order_u]
for ax, data, title in [(axes[0], shown, "What the system knows: 25% of cells (grey = unwatched)"),
(axes[1], pred[order_u], "The system fills in the rest")]:
ax.set_facecolor("#2a2e35")
ax.imshow(data, cmap="magma", vmin=0, vmax=1, aspect="auto")
ax.set_title(title, color="#d4d7dc", fontsize=12)
ax.set_xticks([4, 12, 20], TOPICS)
ax.set_yticks([])
ax.set_xlabel("24 videos, sorted by topic", color="#a4a9b2")
axes[0].set_ylabel("30 viewers, sorted by preference", color="#a4a9b2")
fig.tight_layout()
fig.savefig("bang-diem-tu-dien-en.png", dpi=150, bbox_inches="tight")
# Cover image: each viewer and each video becomes a point on the "taste map" the machine learned by itself
# (3-dimensional vectors, projected onto a plane with PCA for plotting).
both = np.vstack([U, V])
_, _, axes_pc = np.linalg.svd(both - both.mean(0), full_matrices=False)
U2, V2 = (U - both.mean(0)) @ axes_pc[:2].T, (V - both.mean(0)) @ axes_pc[:2].T
colors = ["#e8b05a", "#8fa9c9", "#c98f8f"]
fig, ax = plt.subplots(figsize=(9, 6))
for t in range(3):
ax.scatter(V2[video_topic == t, 0], V2[video_topic == t, 1], marker="s", s=90, color=colors[t], label=f"{TOPICS[t]} video")
ax.scatter(U2[user_main == t, 0], U2[user_main == t, 1], marker="o", s=45, facecolors="none", edgecolors=colors[t], lw=1.5,
label=f"likes {TOPICS[t]}")
ax.set_title("The taste map the machine learned: viewers sit next to the videos they like", color="#d4d7dc", fontsize=12)
ax.set_xticks([]); ax.set_yticks([])
ax.legend(frameon=False, labelcolor="#a4a9b2", fontsize=10, ncol=3, loc="lower center", bbox_to_anchor=(0.5, -0.16))
fig.savefig("ban-do-khau-vi-en.png", dpi=150, bbox_inches="tight")
known 185 / 720 cells (26%)
mean error on unwatched cells: 0.133 (guessing the average: 0.289)
top recommendation is in the viewer's favourite topic: 100% of viewers
Two heat maps side by side, rows are 30 viewers sorted by preference, columns are 24 videos sorted by topic: maths, cooking, football. The left map is mostly grey cells (unwatched), with only about a quarter of cells coloured, bright where a viewer watched their favourite topic. The right map is completely filled: three bright blocks along the diagonal, each group of viewers bright on its own group of videos. In the cooking area there are a few unusual dark stripes.
Figure 1. Left: what the system knows, only about a quarter of the cells. Right: the system fills in the rest with matrix factorisation. Three bright blocks appear in the right places, even though the machine was never told the videos' topics. The few dark stripes in the cooking area are where the data is thinnest: the cold start problem.

With only about a quarter of the data, the mean error on unwatched cells drops from 0.289 (guessing everyone watches the average amount) to 0.133, and the first video recommended to every viewer is in their favourite topic. The dark stripes in the right-hand table all sit in the cooking area, where the data is thinnest: of the 10 viewers who like cooking, 7 have watched just one cooking video, and one cooking video was watched by only one cooking fan. That area has only 14 known cells, against 23 and 25 in the other two, and its error is four to five times higher (0.48 against 0.09 and 0.12). That is too little data for the machine to place a few viewers and a few videos correctly on the map.

You can try changing the share 0.25 to 0.1 to see what happens with less data, or change K = 3 to K = 2 to see what happens when the taste map has fewer dimensions than there are topics: the machine no longer has room to separate the three groups, and its recommendations go badly wrong.

You are teaching it too

A recommendation system does not know who you are. It only knows where you stand on a map drawn by millions of other people, and every time you watch to the end, skip or tap like, you shift your position a little. Knowing that, you can take more control: tap “not interested” when needed, search for new things instead of only scrolling, and remember that your feed is the result of what you have shown it.

Free geometry illustrations for students and teachers: EyeViz.

Phát video: Video: Recommendation systems: how YouTube and TikTok know you
Video: Recommendation systems: how YouTube and TikTok know youVideo bằng tiếng Việt

Tài liệu tham khảo

  1. [1]Cristos Goodrow. On YouTube’s recommendation system. YouTube Official Blog, 2021.
  2. [2]How TikTok recommends videos #ForYou. TikTok Newsroom, 2020.
  3. [3]Paul Covington, Jay Adams and Emre Sargin. Deep Neural Networks for YouTube Recommendations. RecSys ’16, 2016.
  4. [4]Yehuda Koren, Robert Bell and Chris Volinsky. Matrix Factorization Techniques for Recommender Systems. IEEE Computer, 42(8), 30–37, 2009.
  5. [5]Xavier Amatriain and Justin Basilico. Netflix Recommendations: Beyond the 5 stars (Part 1). Netflix Technology Blog, 2012.