Embeddings & Vector Search
Convert text, images, or audio to dense numeric vectors and search by semantic similarity - powering RAG, recommendation engines, and duplicate detection.
skill: Embeddings & Vector Search category: AI/ML tags: [Embeddings, Vector Search, Semantic Search, pgvector, Pinecone] readingTime: 7
Embeddings are the foundation of modern semantic search and RAG. Use them when keyword search fails on paraphrased queries or cross-lingual content. For simple exact-match search, full-text search (Postgres FTS, Elasticsearch) is faster and cheaper. For hybrid search, combine both.
Workflow
- Generate embeddingsEmbeddings are floating-point vectors that capture semantic meaning - similar meanings cluster nearby in vector space, regardless of exact wording.
- Chunk documents for optimal retrievalEmbedding an entire document loses specificity - a query matches only if the exact paragraph is retrieved. Smaller chunks (256-512 tokens) improve precision but lose context.
- Store in a vector databaseVector databases use ANN (Approximate Nearest Neighbor) indexes (HNSW, IVF) for sub-millisecond similarity search over millions of vectors.
- Build a RAG pipelineRAG (Retrieval-Augmented Generation) combines vector search with an LLM - retrieve relevant chunks, inject them into context, generate a grounded answer.
- Evaluate retrieval qualityRAG quality depends on retrieval - a great LLM on irrelevant context produces wrong answers. Measure retrieval precision and recall separately from generation quality.
Output Format
RAG pipeline: embed → chunk → store in vector DB → retrieve → generate grounded answer.
Pipeline:
1. Document ingestion: chunk → embed → upsert to vector DB
2. Query time: embed query → ANN search → top-k chunks
3. Generation: LLM with retrieved chunks as context
4. Evaluation: faithfulness, relevancy, recall (RAGAS)Examples
Input
Output
Edge Cases & Gotchas
- Embedding model dimension mismatch - Switching embedding models changes vector dimensions - you must re-embed all documents. Never mix embeddings from different models in the same index.
- Chunking too small or too large - Chunks < 100 tokens lose context; chunks > 1000 tokens dilute the signal. Start with 512 tokens, 64 overlap, and adjust based on retrieval evaluation.
- Hallucination from irrelevant context - If the retrieved chunks don't contain the answer, the LLM may hallucinate. Add a 'no answer' path: if max similarity < threshold, return 'I don't have information on that'.
- Stale index after document updates - Deleting and re-inserting with the same ID is the simplest update strategy. For partial updates, use soft deletes and re-index periodically.
When to Break These Rules
- Use full-text search (Postgres FTS, Elasticsearch) for exact keyword matching, boolean queries, and faceted filtering - these don't need semantic search.
- Use hybrid search (vector + BM25) when users mix keyword and semantic queries - rerank with RRF (Reciprocal Rank Fusion).
Reference Files
https://platform.openai.com/docs/guides/embeddingshttps://github.com/pgvector/pgvector