An embedding is a dense numerical vector that represents the semantic meaning of a piece of text (or other data) in a continuous vector space where similar concepts are located closer together. Cosine similarity is used to compare embeddings because it measures the angle between two vectors, quantifying their semantic closeness based on the direction they point, regardless of their magnitude.
In the context of RAG, an embedding is a dense numerical vector representation of a piece of text, where proximity in the vector space reflects conceptual similarity[reference:12]. These vectors are generated by specialized embedding models and encode the semantic meaning of the input text. To compare two embeddings, cosine similarity is the preferred metric. It calculates the cosine of the angle between two vectors, producing a value between -1 and 1, where 1 indicates identical direction and thus high semantic similarity[reference:13]. It is favored over metrics like Euclidean distance because it focuses on the orientation (direction) of the vectors rather than their magnitude, making it robust to differences in vector length and better at capturing relative semantic similarity[reference:14].