ELASTICSEARCH-VECTOR

Elasticsearch vector dense_vector HNSW Lucene.

Definition

Elasticsearch vector features: (1) dense_vector field type: up to 4096 dimensions, similarity (cosine, dot_product, l2_norm), index_options (HNSW parameters m, ef_construction). (2) HNSW index: Lucene native implementation Elasticsearch 8.0+, similar OpenSearch Lucene engine. (3) KNN search: knn query type '{knn: {field: 'embedding', query_vector: [...], k: 10, num_candidates: 100}}'. (4) Hybrid search: combine knn + match + range queries, RRF Reciprocal Rank Fusion (8.8+) for combining multiple result sets. (5) Filtering: pre-filtering during HNSW traversal (efficient). (6) Aggregations: compatible with vector search results, integrate analytics. (7) ELSER: Elastic Learned Sparse EncodeR (Elastic-trained sparse encoder model, alternative dense vectors, BM25-style sparse term matching). (8) Quantization: int8 quantization 4x compression 8.12+. Pricing: Elastic Cloud managed service variable per node + storage, vector search additional resources typical. Customers: Microsoft Bing partial, eBay, Wikimedia, ~10000+ Elasticsearch enterprise customers. License complex: SSPL + Elastic License v2 since 2021 (no longer Apache-2.0).

Origin

Elasticsearch dense_vector field added 7.3 2019 (exact KNN only) ; HNSW ANN added 8.0 GA February 2022 ; ELSER added 8.8 2023 ; int8 quantization 8.12 February 2024.

Example in context

Wikimedia Foundation uses Elasticsearch 8.x with vector search for Wikipedia semantic search: ~50M articles embedded via multilingual Sentence Transformers model (768 dim), HNSW + BM25 hybrid search RRF, 200+ language support, integrates existing Wikipedia Elasticsearch infrastructure.

Related terms

• OpenSearch k-NN — AWS Apache 2.0 fork.