VALD

Vald CNCF Sandbox vector Yahoo Japan NGT.

Définition

Vald features : (1) Kubernetes-native architecture : Vald Agent (vector ingestion + search per shard), Vald Discoverer (cluster topology management), Vald Manager Index (index lifecycle management), Vald Gateway (single-point API), Vald LB Gateway (load balancing), Vald Backup Manager (snapshots backup). (2) NGT (Neighborhood Graph and Tree) : Yahoo Japan-developed ANN algorithm + library (C++ open-source under Apache-2.0), high-performance graph-based ANN similar HNSW family. (3) Distance : L2, Cosine, Hamming, Jaccard. (4) Distributed : automatic sharding horizontal scaling, replication factor configurable, auto rebalancing. (5) Snapshots + backups : Vald Backup Manager backs up to S3/MinIO. (6) gRPC + REST APIs. (7) Egress filtering : custom filters + post-processing search results plugins. Customers : Yahoo Japan (search, recommendations), other Japan tech enterprises (LINE, Mercari emerging adopters). Limited international adoption vs Milvus/Qdrant.

Origine

Vald initial release 2019 par Yahoo Japan engineers Kentaro Kuribayashi + others ; donated CNCF Sandbox novembre 2020 ; ~3000+ GitHub stars 2024 ; NGT library Yahoo Japan 2016+ matures.

Exemple en contexte

Yahoo Japan e-commerce + auction search uses Vald cluster ~30 Kubernetes nodes : ~5B product embeddings NGT index, semantic search Yahoo Japan Shopping API ~50ms latency, integrated user personalization features ; production proven 2020+ Yahoo Japan scale.

Termes liés

• Milvus — alternative CNCF distributed.