What is a vector space? A plain-language explanation

In this article

1. Before you start
2. Why interviewers ask this
3. Vector spaces from first principles
4. Worked example
5. Trade-offs and edge cases
6. How to answer it in an interview
7. Common mistakes
8. Follow-up questions
9. Sources

Before you start

Why interviewers ask this

A candidate who treats "vector space" as a polite word for "a pile of numbers" cannot say why embedding arithmetic works, what "384 dimensions" counts, or why a set of length-one vectors does not behave like a space. Those gaps show up in follow-ups on embeddings and vector search, where the interviewer moves from "what is an embedding" to "what kind of object is the space it lives in".

The question usually arrives in one of three forms: define a vector space, explain what the dimension of an embedding means, or say whether two embeddings can be added. A weak answer lists numbers. A strong answer states the two operations, the closure test, the basis, and what extra structure an embedding space adds on top: a ruler for length and angle. This page builds each part from zero.

Vector spaces from first principles

Vector space

A vector space is a set of objects called vectors, together with a way to add two vectors and a way to multiply a vector by a number, such that the results always stay in the set and the usual rules of arithmetic hold. Lists of real numbers are the standard example.

Subspace

A subspace is a part of a vector space that is itself a vector space, which happens exactly when that part contains the zero vector and adding two of its vectors, or scaling one, never leaves it. A flat sheet through the origin of three-dimensional space is a subspace.

Basis

A basis is a smallest list of vectors from which every vector in the space can be built by scaling and adding, with no vector in the list buildable from the others. Every basis of a space has the same number of vectors.

In plain words

Think of a huge drawing board with a pin at the centre. Every point on the board is an arrow from the pin. You can put two arrows tip to tail to get a third arrow, and you can stretch or flip an arrow. However many times you do this, the result is always another arrow on the board. That unbreakable "you never fall off the board" property is the heart of a vector space.

The analogy breaks in two places. A board has two directions, while the space of a sentence embedding has hundreds you cannot picture. And the objects need not be arrows at all: lists of numbers, polynomials and functions all qualify, as the Wikipedia article on vector spaces notes, because only the rules matter.

In one precise sentence: a vector space is a set with addition and scaling in which every sum and every multiple stays in the set and the usual rules hold. Axler's Linear Algebra Done Right (4th edition, read 2026-10-24) defines it this way, then builds span, independence, basis and dimension.

1 · Vectors2 · Operations3 · Closure4 · Basis5 · RulerReplay

Step-by-step description

1. Lists of numbers you can picture as arrows.
2. Add two vectors; scale one.
3. Neither operation may leave the set.
4. Independent vectors that reach everything.
5. An inner product adds length and angle.

• A set is a vector space when adding two members, or scaling one, never leaves the set.
• The dimension is the number of vectors in a basis.

Step 1: Start with vectors you can picture

A vector in this page is an ordered list of real numbers, and the number of entries is how many directions it has. The list (1, 2, 3) is one vector in three-dimensional space; picture an arrow from the origin to the point one step across, two up and three deep. A sentence embedding is the same kind of object with more entries: the all-MiniLM-L6-v2 model card says the model maps sentences to a 384-dimensional dense vector space as of October 2026, so each sentence becomes a list of 384 numbers.

Order matters: (1, 2, 3) and (3, 2, 1) are different vectors. The output of this step is only a way of writing things down; no rule has been checked yet.

Step 2: Learn the two operations, adding and scaling

A vector space needs exactly two operations, adding two vectors and multiplying one by a number, called a scalar, and for lists both work entry by entry. Adding (1, 2, 3) and (0, 1, -1) gives (1, 3, 2). Scaling (1, 2, 3) by 2 gives (2, 4, 6). Scaling by -1 flips the arrow and scaling by 0 gives the zero vector, the list of all zeros.

The formal definition adds eight rules, such as "addition can be done in any order" and "there is a zero vector that changes nothing" (Wikipedia, read 2026-10-24). Lists of numbers satisfy all eight automatically, so for lists the working question is the next step: does the set you chose stay closed?

Step 3: Test whether the set stays closed

A set passes the closure test when adding any two of its vectors, and scaling any one of them by any number, always lands back inside the set. One failed example is enough to disqualify a set, so you can test by hunting for a counterexample.

Positive-entry vectors fail: take (1, 2, 3), which is inside, multiply by -1 and you get (-1, -2, -3), which is not.¹ Unit-length vectors fail: (1, 0, 0) and (0, 1, 0) each have length 1, but their sum (1, 1, 0) has length about 1.41.¹ A plane that misses the origin fails because the zero vector is not on it. A plane through the origin passes, and it is then a subspace, a vector space inside a bigger one.

Text description of the diagram

• Positive-entry vectors fail because multiplying by minus one leaves the set.
• Unit-length vectors fail because the sum of two has length about 1.41, not 1.

Step 4: Count the dimension with a basis

A basis is the smallest set of vectors that reaches everything, and the number of vectors in it is the dimension. The Wikipedia article states that all bases of a vector space have the same size, so the count is well defined. The three arrows (1, 0, 0), (0, 1, 0) and (0, 0, 1) form the standard basis of three-dimensional space. A 384-number embedding lives in the space of all 384-number lists, whose standard basis has 384 vectors, so that space has dimension 384.

Any three arrows that are independent also work: with the basis (1, 1, 0), (0, 1, 1), (1, 0, 1), the vector (2, 3, 4) has coordinates one half, five halves and three halves, because half the first plus five halves of the second plus three halves of the third rebuilds it.¹ The vector did not change; only the way of writing it did. By contrast (1, 2, 3) and (2, 4, 6) are not independent, since the second is twice the first, and together they reach only a line, a space of dimension 1.¹

Step 5: Add a ruler: length, angle and the embedding space

A bare vector space has no lengths or angles, and an inner product is the extra rule that adds them: it turns two vectors into one number, written as the dot product for lists. With it, the length of (3, 4) is 5, and the angle between (1, 0) and (1, 1) has cosine about 0.71, which is 45 degrees.¹ That cosine is the cosine similarity used to compare embeddings.

An embedding space is a vector space plus this ruler, filled by a trained model. The model decides what each direction means. Mikolov, Chen, Corrado and Dean report in the introduction of their word-vector paper that vector("King") minus vector("Man") plus vector("Woman") gives a vector closest to that of "Queen" (arXiv 1301.3781 v3, 2013). That is a property of the trained vectors, not of vector spaces in general.

Worked example

Take five sets in three-dimensional space and run the closure test on each, using exact arithmetic.¹ Only two pass, and each failure comes with the single counterexample that proves it.

The plane through the origin has dimension 2 because it needs two independent vectors, such as (1, -1, 0) and (0, 1, -1), to reach all of it. The last row also shows the practical use of dimension: it counts the independent directions a set allows.

Now attach a cost to dimension. A float32 number takes 4 bytes, so a vector of d numbers takes 4d bytes, before any index overhead.¹ At 384 dimensions that is 1,536 bytes, or 1.5 KiB; at 1,536 dimensions, 6 KiB; at 3,072 dimensions, 12 KiB. The 1,536 and 3,072 figures are the default dimensions OpenAI's embeddings guide lists for text-embedding-3-small and text-embedding-3-large as of October 2026. A million stored vectors of 1,536 dimensions is about 5.72 GiB.¹

Text description of the diagram

• One float32 vector takes 1.5 KiB at 384 dimensions.
• It takes 6 KiB at 1,536 dimensions and 12 KiB at 3,072.

Trade-offs and edge cases

Normalised embeddings are not a vector space. OpenAI states that its embeddings are normalised to length 1 (OpenAI embeddings guide, October 2026). A set of length-one vectors fails the closure test exactly as in the table: two perpendicular unit vectors sum to a vector of length 1.4142.¹ The vectors live on the surface of a sphere. You can still add them; normalise the result if you need it back on the sphere. Cosine similarity is the standard comparison because it measures the angle and ignores length; for length-one vectors it equals the plain dot product.

Higher dimension is not automatically better. Each extra dimension adds storage and search cost, as the chart shows. The same OpenAI page says a text-embedding-3-large vector can be shortened to 256 numbers and still beat an unshortened text-embedding-ada-002 vector on the benchmark the guide cites (OpenAI, October 2026), so quality depends on how the model was trained, not on size alone.

"Vector space" has two senses in machine learning. In mathematics it is the structure above. In information retrieval the vector space model is a method that represents documents and queries as vectors with one dimension per term and scores them by cosine (Wikipedia, vector space model, read 2026-10-24). Say which sense you mean. Weak semantic sensitivity is that method's stated limitation: documents with similar meaning but different vocabulary are not linked.

Dimension can be infinite. Functions and polynomials form vector spaces too, and the Wikipedia article gives polynomial rings as countably infinite-dimensional. Embeddings are always finite.

How to answer it in an interview

Say it in this order: the closure rule, the basis and dimension, the ruler, then the sphere caveat.

Common mistakes

"A vector space is just a set of vectors." (wrong) Without addition and scaling that stay inside the set, you only have a collection. (right) "A vector space is a set closed under adding and scaling, with the usual arithmetic rules; for lists of numbers the rules come free and closure is the test."

"The dimension is the number of vectors in the set." (wrong) A plane contains endless vectors and has dimension 2. (right) "The dimension is the size of a basis, the independent directions."

"Any three arrows make a basis of three-dimensional space." (wrong) (1, 2, 3), (2, 4, 6) and (0, 0, 1) include a dependent pair and reach only a plane. (right) "Three independent arrows make a basis."

"Embeddings form a vector space, so I can add any two and stay valid." (wrong) Normalised embeddings sit on a sphere, and the sum of two of them is generally not of length 1 (two perpendicular ones give 1.41), so the sum leaves the set. (right) "I can add them, then renormalise."

Follow-up questions

Is the set of normalised embeddings a vector space?

No. Normalised embeddings all have length 1, and the sum of two perpendicular ones has length about 1.41, so the set is not closed under addition.¹ The ambient space of all lists with the same number of entries (the same dimension) is a vector space; the normalised vectors are a curved subset of it.

What is the difference between a vector and a vector space?

A vector is one object, such as a list of numbers. A vector space is the whole set of such objects together with the two operations and the closure property. An embedding is a vector; the embedding space is the vector space it belongs to.

Can you add two sentence embeddings?

Yes, entry by entry, and the result has the same number of entries, but its meaning is only sensible if the model was trained for it. Mikolov et al. report the King, Man, Woman, Queen offset for word vectors; for sentence embeddings, treat addition as an experiment to test, not a sure thing.

What is the vector space model in information retrieval?

It is a method that represents each document and query as a vector with one dimension per term and ranks documents by cosine; the classic formulation is Salton, Wong and Yang (1975), according to the Wikipedia article. Modern dense embeddings keep the idea but learn the dimensions, and many systems run a keyword ranking beside them, which is hybrid search; see also the vector database overview.

Why does a higher embedding dimension cost more?

Every vector stores one number per dimension, so storage and comparison work grow with dimension. A 3,072-dimensional float32 vector is 12 KiB against 1.5 KiB at 384.¹ An approximate index cuts the comparison work but not the storage (similarity search), and OpenAI's text-embedding-3 models can return shorter vectors, as the trade-offs above note; the transformer hub shows where the vectors come from.

Sources

1. Linear Algebra Done Right, 4th edition (open access) — Sheldon Axler (date = read date), 24 Oct 2026
2. Vector space — Wikipedia (date = read date), 24 Oct 2026
3. Efficient Estimation of Word Representations in Vector Space (arXiv 1301.3781 v3) — arXiv (Mikolov, Chen, Corrado, Dean), 7 Sep 2013
4. sentence-transformers/all-MiniLM-L6-v2 model card — Hugging Face (date = read date), 24 Oct 2026
5. Vector embeddings guide — OpenAI (date = read date), 24 Oct 2026
6. Vector space model — Wikipedia (date = read date), 24 Oct 2026