Cosine similarity: why AI measures angles, not distance
What is cosine similarity? From a cup of coffee to an experiment on 47 articles: why AI compares text by angle, and when this measure breaks down.

Mục lục
Which article is most like Schrödinger’s cat?
I had a computer read 47 articles on this blog and asked it a simple question: which article is most like Schrödinger’s cat? Using ordinary distance between two articles, the machine answered: the one about image search. That article is about shoes and pixels, and has almost nothing to do with cats or quantum physics.
Switch to cosine similarity, with the same data, and the machine answers: the article about quantum computers. Cosine similarity measures how closely two arrows point the same way, and does not care which arrow is longer. AI turns text, images and songs into arrows like these. Pointing the same way means similar content, while length mostly says how big the thing is.
This article explains why measuring angles beats measuring distance, starting with a cup of coffee. Then we look at the experiment on this blog, meet the person who turned text into arrows in the 1960s, and see where cosine gives the wrong answer.
A cup of milk coffee: why not measure distance?
A café has three drinks. The small Vietnamese milk coffee has 30 ml of coffee and 15 ml of milk. The large one follows the same recipe at double the size: 60 ml and 30 ml. The third is bạc xỉu, the milky Vietnamese coffee, with 15 ml of coffee and 30 ml of milk.
Write each drink as two numbers and you can draw it as an arrow on paper, with coffee on the horizontal axis and milk on the vertical one. A list of numbers like this is called a vector. Think of it simply as a recipe written in numbers.
Now ask: which drink tastes most like the small milk coffee? Anyone would say the large one, since only the size differs. But Euclidean distance, the length of the straight line joining the tips of two arrows, says otherwise:
| Compared with the small cup | Distance | Angle | Cosine similarity |
|---|---|---|---|
| Large milk coffee | 33.5 ml | 0° | 1.00 |
| Bạc xỉu | 21.2 ml | 37° | 0.80 |
Distance decides bạc xỉu is more alike, because it mixes two questions: what is the recipe and how much is there. The angle answers only the first. Two cups with the same recipe sit on the same direction whatever their size, as the left of Figure 1 shows.
Text works the same way. A long article and a short one on the same subject are like the large cup and the small cup: same recipe, different size. To compare content, we need a measure that cup size cannot fool.

What is cosine similarity?
The name says what it does: it takes the cosine of the angle between two arrows. You do not need a protractor. Multiply the matching numbers, add them up and divide by the lengths [9]:
The top line is the dot product. Think of it as a taste-match score between two people: dishes both love a lot add many points, and dishes only one of them likes add almost nothing. The bottom line divides away the size of each arrow, like converting every cup to the same size before tasting.
Try the small cup against bạc xỉu. The dot product is . Both lengths are . So the cosine is , which matches an angle of about 37°.
The result always falls between −1 and 1:
- 1: same direction, same recipe.
- 0: perpendicular, nothing in common. An article about pho and one about aircraft, if they share no words, get exactly 0.
- −1: exactly opposite. Word counts cannot do this because they are never negative, but AI embeddings can go below zero.
Why does it never exceed 1, even with thousands of numbers? That is the Cauchy–Schwarz inequality, mentioned in the article on Cauchy’s inequality. A piece of school mathematics guarantees that the word “angle” still means something in a space nobody can draw. If you subtract each list’s average before computing cosine, you get the Pearson correlation coefficient, the number in the article on regression to the mean (article in Vietnamese).
The person who turned text into arrows
Using this measure for words is tied to Gerard Salton. He was born in 1927 in Nuremberg as Gerhard Anton Sahlmann. War forced his family to flee Germany, and he and his brother were led across a border at night to avoid German guards [2]. Salton reached the United States in 1947, studied mathematics, and became Howard Aiken’s last doctoral student at Harvard [2] [3].
In the early 1960s, Salton designed SMART, a text retrieval system that served for about 30 years as a test bed for search methods [3]. At the time, search was usually true-or-false logic: a document that matched the keywords was kept, and one that did not was dropped, like a doorman who can only say yes or no.
Salton’s group thought differently: each word in the language is an axis, each document is an arrow, and a query is an arrow too. Search became ranking by similarity between arrows, with cosine as the typical measure, rather than a yes-or-no answer [1] [3] [9]. The 1975 paper by Salton, Wong and Yang set out this view as the vector space model. Cornell’s memorial to him includes the line: “Salton was information retrieval” [2]. He passed away in 1995.
TF-IDF: why the word “the” gets no vote
Raw word counts have a problem: every article is full of “the”, “of” and “and”. Two articles can look alike just because they are written in the same language.
In 1972, Karen Spärck Jones proposed a neat idea: the fewer documents a word appears in, the more it should count [4]. It is like finding someone in a crowd: “wearing a shirt” does not help, but “wearing a red pith helmet” finds them at once. Combine how often a word appears in an article (TF) with how rare it is overall (IDF) and you get TF-IDF, a formula that has been used widely for decades. Common words nearly fall silent, while distinctive words such as “quantum” or “Galois” speak up.
An experiment on 47 articles
Back to the opening question. I took the 47 Vietnamese articles on the blog at the time of writing, not counting this one, and removed code and formulas. Each pair of consecutive syllables became an axis, because many Vietnamese words have two syllables. The result was a set of arrows in 63,800 dimensions, weighted by TF-IDF.
| Article | Closest by cosine | Closest by Euclidean distance |
|---|---|---|
| Schrödinger’s cat | Quantum computing (0.105) | Image search |
| Galois | Abel and the letter that came too late (0.160) | Consistent hashing |
| The chicken and dog puzzle | Computing with light (0.063) | Image search |
The cosine column is not perfect, but it makes sense: Galois and Abel both deal with the fifth-degree equation, while the chicken and dog article and the one on computing with light both talk a lot about matrices. The Euclidean column barely changes. Across the whole collection, the articles on image search, the 3nm chip and consistent hashing each appear in the top 5 for 46 of the other 46 articles. 97% of Euclidean top-5 picks land on the 10 shortest articles. For cosine, it is 15%.
The reason is the coffee cup again. A short article has a short arrow near the origin, so it is “close” to everything. A long article has a long arrow, so it is “far” from everything. Distance is sorting by cup size, not by content.
One more test: paste the Schrödinger’s cat article three times over. The content has not changed, and cosine with the original stays at 1.00. Euclidean distance to the original is 477, while distance to its Euclidean nearest neighbor is only about 278. By distance, that article differs from itself more than it differs from an article about shoes.
From counting words to embeddings
TF-IDF has a big blind spot: “car” and “automobile” are two different axes, so two sentences meaning the same thing can score a cosine of 0.
In 2013, Tomas Mikolov’s team at Google published word2vec, a way for a machine to learn a vector for each word by reading text [5]. Vectors of this kind are called embeddings. Instead of one axis per word, each word becomes a few hundred numbers, like a flavor profile of sweetness, strength and richness, except that the machine picks the qualities itself and they have no names. Words used in similar contexts point in nearby directions, so “car” sits close to “automobile”.
The most famous example: take the vector for “king”, subtract “man”, add “woman”, then look for the word with the highest cosine, and you get “queen” [6]. This example has been oversold. The code for such analogies usually removes the input words from the possible answers, and Nissim and colleagues showed that this rule can make results look better than they are [7].
Today, embeddings can represent sentences, passages and images. OpenAI’s text-embedding-3-small model turns a passage into an arrow with 1,536 dimensions [8]. When a chatbot answers from a company’s documents, it turns your question into an arrow, then looks for the passages pointing in the nearest direction, as the article on image search describes. The recommendation systems behind YouTube and TikTok compare arrows like these as well.
The trick that makes it fast: one cup size
Computing square roots for billions of vectors is hard work. So people do one step in advance: divide each vector by its length, so every arrow has length exactly 1. At the café, that means converting every cup to one size from the start.
After this normalization, the bottom of the fraction equals 1, and cosine is just the dot product, a multiply-and-add that GPUs do very quickly. OpenAI normalizes its embeddings to length 1 and says so in its documentation [8].
There is a surprising consequence: once all arrows are the same length, Euclidean distance and cosine produce exactly the same ranking, because [8]. So Euclidean distance was never wrong. What threw it off in the experiment was length. Remove length, and the two measures become one.
Is a cosine of 0.1 high or low?
In the table above, 0.105 is the closest match to Schrödinger’s cat. That sounds low, but in this collection the median cosine between two articles is only about 0.018, so 0.105 is nearly six times higher.
Spaces with many dimensions have a counterintuitive property: two random arrows are almost always nearly perpendicular. The right of Figure 1 is my simulation. In 2 dimensions, random cosines spread from −1 to 1. In 1,536 dimensions, 95% of random pairs fall within ±0.05. There are so many directions to go that two people choosing at random almost never take the same road.
So there is no universal rule like “above 0.8 means similar”. The number depends on how the vectors were made. For each system, test on real data and then choose a threshold, rather than copying one from another system.
When does cosine similarity break?
When size carries meaning. Someone who rates every film 1 star and someone who rates every film 5 stars have vectors pointing the same way, with a cosine of 1, even though one dislikes everything and the other likes everything. When size carries information, such as quantity bought or strength of interest, the dot product or distance may fit better. The dot product has its own bias: in the experiment above, it tended to pick long articles.
When one word reverses the meaning. With word counts, “the pho is good” and “the pho is not good” have a cosine of 0.89, even though they say opposite things. Modern embeddings do better than counting words, but two sentences on the same subject with opposite meanings often still sit fairly close. Cosine can only measure what the vectors record.
When comparing vectors from two different models. Two models use two different sets of axes, like mixing a recipe in milliliters with one in spoonfuls. Change the model and you must recompute every vector.
When a high score is taken as truth. A high cosine only says two things use similar words or contexts. A passage that closely matches a question can still be wrong or out of date.
Run cosine similarity yourself
The Python below recomputes the coffee table, runs two tests on short reviews, measures random cosines against dimensions, checks the formula , and draws Figure 1. The seed is fixed at 42. You need numpy and matplotlib. The 47-article experiment uses the same cosine function on the blog’s text, so it is not part of this code.
from collections import Counter
import matplotlib.pyplot as pltimport numpy as np
plt.rcParams.update({"figure.facecolor": "#16181c", "axes.facecolor": "#16181c", "text.color": "#d4d7dc", "axes.edgecolor": "#2a2e35", "axes.labelcolor": "#a4a9b2", "xtick.color": "#878d97", "ytick.color": "#878d97", "font.size": 12})rng = np.random.default_rng(42)
def cosine(a, b): """Cosine similarity: the dot product divided by the product of the two lengths.""" return a @ b / (np.linalg.norm(a) * np.linalg.norm(b))
# Part 1: three coffees, each one an arrow (ml of coffee, ml of milk)SMALL = np.array([30.0, 15.0]) # small milk coffeeLARGE = np.array([60.0, 30.0]) # large milk coffee, same recipeBAC_XIU = np.array([15.0, 30.0]) # bạc xỉu, mostly milkprint(f"small - large: distance {np.linalg.norm(SMALL - LARGE):.1f}, cosine {cosine(SMALL, LARGE):.2f}")print(f"small - bạc xỉu: distance {np.linalg.norm(SMALL - BAC_XIU):.1f}, cosine {cosine(SMALL, BAC_XIU):.2f}")print(f"angle small - bạc xỉu: {np.degrees(np.arccos(cosine(SMALL, BAC_XIU))):.1f} degrees")
# Part 2: turn text into arrows by counting wordsdef count_words(*sentences): vocabulary = sorted({w for s in sentences for w in s.split()}) return [np.array([Counter(s.split())[w] for w in vocabulary], float) for s in sentences]
a, b, c = count_words("the pho is good", "the pho is good " * 3, "the pho is not good")print(f"a review and itself repeated three times: cosine {cosine(a, b):.2f}, distance {np.linalg.norm(a - b):.1f}")print(f"'good' and 'not good': cosine {cosine(a, c):.2f}")
# Part 3: two random arrows get closer to perpendicular as dimensions growDIMENSIONS = [2, 10, 100, 1536]random_cosines = {}for d in DIMENSIONS: x, y = rng.normal(size=(2, 20_000, d)) x /= np.linalg.norm(x, axis=1, keepdims=True) y /= np.linalg.norm(y, axis=1, keepdims=True) random_cosines[d] = (x * y).sum(axis=1) print(f"{d:5d} dimensions: 95% of pairs have cosine within ±{np.quantile(np.abs(random_cosines[d]), 0.95):.3f}")
# After normalizing, distance and cosine rank things the same way: d² = 2 - 2·cosineprint(f"check: d² = {np.sum((x[0] - y[0]) ** 2):.4f}, 2 - 2·cosine = {2 - 2 * x[0] @ y[0]:.4f}")
# Plotfig, (ax, ax2) = plt.subplots(1, 2, figsize=(13, 5.8), gridspec_kw={"wspace": 0.2, "width_ratios": [1, 1.12]})for v, colour, label, shift in [(SMALL, "#e8b05a", "small", (2, -7)), (LARGE, "#e8b05a", "large", (-12, 4)), (BAC_XIU, "#8fa9c9", "bạc xỉu", (-11, 3))]: ax.annotate("", xy=v, xytext=(0, 0), arrowprops={"arrowstyle": "-|>", "color": colour, "lw": 2.8, "mutation_scale": 20}) ax.text(v[0] + shift[0], v[1] + shift[1], label, color=colour, fontsize=12)ax.plot(*zip(SMALL, BAC_XIU), color="#a4a9b2", ls=(0, (3, 3)), lw=1.2)ax.text(24, 24.5, "21.2 ml", color="#a4a9b2", fontsize=11)ax.text(56, 15, "small to large: 33.5 ml", color="#e8b05a", fontsize=11, ha="center")arc = np.radians(np.linspace(26.57, 63.43, 40))ax.plot(15 * np.cos(arc), 15 * np.sin(arc), color="#d4d7dc", lw=1.3)ax.text(10, 10.2, "37°", color="#d4d7dc", fontsize=12)ax.set_xlim(0, 75)ax.set_ylim(0, 45)ax.set_aspect("equal")ax.set_xlabel("coffee (ml)")ax.set_ylabel("milk (ml)")ax.set_title("Distance sees cup size, the angle sees the recipe", color="#d4d7dc", fontsize=13)
colours = ["#878d97", "#8fa9c9", "#e8b05a", "#d4d7dc"]for d, colour in zip(DIMENSIONS, colours): ax2.hist(random_cosines[d], bins=np.linspace(-1, 1, 161), density=True, histtype="step", lw=2, color=colour, label=f"{d} dimensions")ax2.set_yscale("log")ax2.set_ylim(0.2, 40)ax2.set_xlim(-1, 1)ax2.set_xlabel("cosine between two random arrows")ax2.set_ylabel("density, log scale")ax2.set_box_aspect(0.6)ax2.set_title("In many dimensions, random arrows are nearly perpendicular", color="#d4d7dc", fontsize=13)ax2.legend(frameon=False)for a_ in (ax, ax2): for sp in ("top", "right"): a_.spines[sp].set_visible(False)fig.savefig("cosine-similarity-en.png", dpi=150, bbox_inches="tight")Output:
small - large: distance 33.5, cosine 1.00small - bạc xỉu: distance 21.2, cosine 0.80angle small - bạc xỉu: 36.9 degreesa review and itself repeated three times: cosine 1.00, distance 4.0'good' and 'not good': cosine 0.89 2 dimensions: 95% of pairs have cosine within ±0.997 10 dimensions: 95% of pairs have cosine within ±0.598 100 dimensions: 95% of pairs have cosine within ±0.196 1536 dimensions: 95% of pairs have cosine within ±0.050check: d² = 2.0650, 2 - 2·cosine = 2.0650The line about the review repeated three times shows at small scale what the 47-article experiment showed at large scale: cosine stays at 1.00, while distance jumps to 4.0 with no change in content. You can change BAC_XIU to the recipe of your usual drink and see how the angle moves.
Back to the coffee cup
Gerard Salton had no embeddings and no GPUs, only axes for words and a division by length. Six decades later, chatbots have thousands of dimensions learned by machines, but the final question is still whether two arrows point the same way.
Cosine’s job fits in one short sentence: do not let cup size fool you when you are asking about the recipe. On these 47 articles, that change was enough for the machine to see that Schrödinger’s cat is about quantum physics, not shoes.
Free geometry illustrations for students and teachers: EyeViz.
Tài liệu tham khảo
- [1]G. Salton, A. Wong, C. S. Yang. A vector space model for automatic indexing. Communications of the ACM 18(11), 613–620, 1975.
- [2]In Memoriam: Gerard Salton. Department of Computer Science, Cornell University, 1996.
- [3]Gerard Salton. IEEE Computer Society, Computer Pioneers.
- [4]Karen Spärck Jones. A statistical interpretation of term specificity and its application in retrieval. Journal of Documentation 28(1), 11–21, 1972.
- [5]T. Mikolov, K. Chen, G. Corrado, J. Dean. Efficient Estimation of Word Representations in Vector Space. arXiv, 2013.
- [6]T. Mikolov, W. Yih, G. Zweig. Linguistic Regularities in Continuous Space Word Representations. NAACL-HLT 2013, 2013.
- [7]M. Nissim, R. van Noord, R. van der Goot. Fair Is Better than Sensational: Man Is to Doctor as Woman Is to Doctor. Computational Linguistics 46(2), 487–497, 2020.
- [8]Vector embeddings. OpenAI API documentation.
- [9]Cosine similarity. Wikipedia.


