OPENSEARCH-KNN

OpenSearch k-NN AWS vector + full-text plugin.

Definition

OpenSearch k-NN features: (1) Vector field type: knn_vector with dimension parameter, methods (engine, name HNSW/IVF, space_type L2/InnerProduct/CosineSimilarity), index parameters. (2) Engines: (a) Nmslib HNSW (default, mature, good performance), (b) Faiss IVF + HNSW + IVF-PQ (Facebook AI Faiss library integration, GPU-acceleration possible, advanced quantization), (c) Lucene HNSW (added 2.4+, native Apache Lucene HNSW implementation, less mature but well-integrated OpenSearch index lifecycle). (3) Query: KNN query DSL '{knn: {embedding: {vector: [...], k: 10}}}'. (4) Hybrid search: combine knn + match (BM25) + filter clauses in bool query, score boosting. (5) Pre-filtering: Lucene engine supports pre-filtering during HNSW traversal (efficient with selective filters), Nmslib post-filtering only. (6) Quantization: Faiss PQ + Scalar Quantization byte-based. Pricing: AWS OpenSearch Service managed Standard tier pricing per instance type + storage GB-month, vector search additional CPU/memory typical workload. Customers: Pinterest Search, Amazon retail, BMW media search.

Origin

Elasticsearch k-NN initial release 2019 ; OpenSearch fork April 2021 (AWS after Elastic SSPL license change) ; OpenSearch k-NN AWS-maintained Apache-2.0 ; OpenSearch 2.x active 2024.

Example in context

Pinterest Visual Search uses OpenSearch k-NN Faiss IVF-PQ engine for 3B+ image embeddings: Pinterest custom Vision Transformer model embedding 128-dim, IVF-PQ 8x compression, query latency ~100ms for 'shop the look' visual search feature.

Related terms

• Elasticsearch vector — upstream fork.