Skip to content
Devgo

Embedding

A numerical vector representing the meaning of text, images, or other data. Similar things have similar vectors. Embeddings let you measure semantic similarity (cosine similarity) and power search, recommendations, and RAG. The bridge between unstructured data and math.

When to use
  • Semantic search.
  • RAG (retrieve relevant docs for LLMs).
  • Recommendations.
  • Clustering/classification.
When not to
  • Exact keyword search (use full-text search).
  • When you need human-readable explanations.

Definition

An embedding is a list of numbers (vector) that captures the semantic meaning of input. Text like 'dog' and 'puppy' have similar vectors; 'dog' and 'car' have different ones. Models (OpenAI embeddings, sentence-transformers) produce embeddings. Similarity is measured by cosine distance.

Why it exists

To represent unstructured data (text, images) as numbers so machines can compare, search, and cluster them by meaning, not just exact keyword match.

How it works

Pass text through an embedding model (e.g., text-embedding-3-small). It outputs a vector (e.g., 1536 dimensions). Similar meanings → vectors close in space. Store vectors in a vector DB. Search by embedding a query and finding nearest neighbors by cosine similarity.

Scaling limits

Embedding quality depends on the model. Dimensionality affects storage and search speed. Domain-specific text may need fine-tuned embeddings. Multilingual support varies.

Cost shape

Embedding API: cheap per 1M tokens. Self-hosted models: free but GPU cost. Vector storage: free (pgvector) to paid (Pinecone).

Alternatives

  • TF-IDF/BM25 — Keyword-based, no semantic understanding, but free and fast.
  • Full-text search (Postgres tsvector) — Keyword search, no meaning.

Failure modes

  • Poor quality on domain-specific text.
  • High dimensionality storage cost.
  • Stale embeddings after content changes.

, ,