Backend6 min read

Elasticsearch & Search

Build fast, relevant search with Elasticsearch - indexing, full-text queries, aggregations, relevance tuning, and Kibana for visualization.

ElasticsearchSearchOpenSearchFull-Text SearchAnalytics

skill: Elasticsearch & Search category: Backend tags: [Elasticsearch, Search, OpenSearch, Full-Text Search, Analytics] readingTime: 6

Elasticsearch is the standard for full-text search - it powers the search on major e-commerce, documentation, and logging platforms. Don't use it as a primary database; use it as a read-optimized search index synchronized from your primary database (PostgreSQL, MongoDB).

Workflow

Design index mappingMapping defines how Elasticsearch indexes fields - wrong mappings (text when keyword needed, or vice versa) degrade search quality and performance.
Search with bool queriesBool queries combine must (AND), should (OR), filter (AND, no scoring), and must_not (NOT) clauses - the foundation of all Elasticsearch queries.
Aggregate for faceted searchAggregations compute facet counts (category: 120 results, brand: 45 results) without additional queries - the backbone of e-commerce filter UIs.
Index documents from PythonThe elasticsearch-py client handles connection pooling, retry, and bulk indexing for high-throughput sync from your primary database.
Tune relevance scoringDefault Elasticsearch scoring (BM25) may not match business relevance - newer products, popular items, or exact matches may need boosting.

Output Format

Elasticsearch: typed index mapping, bool query with filter, aggregations for facets, bulk indexing from Python, and function_score for business relevance.

Index mapping: text (analyzed) + keyword (exact) fields
Search: bool { must: [match], filter: [term, range] }
Aggregations: terms (facets) + range + stats
Bulk index: helpers.bulk() for ETL sync

Examples

Input

Output

Edge Cases & Gotchas

Shard count is permanent - Number of shards can't be changed after index creation. Over-shard (3-5 shards for < 50GB) to allow future growth without reindexing.
Mapping explosion - Dynamic mapping allows any field name - malicious or accidental new fields create new mappings. Set dynamic: strict to reject unknown fields in production.
Cluster yellow/red status - Yellow = replicas can't be assigned (single-node cluster). Red = primary shards missing (data loss risk). Single-node dev clusters are always yellow.
Scroll vs. search_after for pagination - Deep pagination (from: 10000) is expensive. Use search_after with a sort tie-breaker for cursor-based pagination instead of offset-based.

When to Break These Rules

Use PostgreSQL full-text search (tsvector + tsquery) for small-scale search requirements - no infrastructure overhead for a few thousand documents.
Use Typesense or Meilisearch for simpler setup and operation when you don't need Elasticsearch's full aggregation and analytics capabilities.

Reference Files

https://www.elastic.co/guide/en/elasticsearch/reference/current/
https://opensearch.org/docs/