VALD

Vald CNCF Sandbox vector Yahoo Japan NGT.

Definition

Vald features: (1) Kubernetes-native architecture: Vald Agent (vector ingestion + search per shard), Vald Discoverer (cluster topology management), Vald Manager Index (index lifecycle management), Vald Gateway (single-point API), Vald LB Gateway (load balancing), Vald Backup Manager (snapshots backup). (2) NGT (Neighborhood Graph and Tree): Yahoo Japan-developed ANN algorithm + library (C++ open-source under Apache-2.0), high-performance graph-based ANN similar HNSW family. (3) Distance: L2, Cosine, Hamming, Jaccard. (4) Distributed: automatic sharding horizontal scaling, configurable replication factor, auto rebalancing. (5) Snapshots + backups: Vald Backup Manager backs up to S3/MinIO. (6) gRPC + REST APIs. (7) Egress filtering: custom filters + post-processing search results plugins. Customers: Yahoo Japan (search, recommendations), other Japan tech enterprises (LINE, Mercari emerging adopters). Limited international adoption vs Milvus/Qdrant.

Origin

Vald initial release 2019 by Yahoo Japan engineers Kentaro Kuribayashi + others ; donated CNCF Sandbox November 2020 ; ~3000+ GitHub stars 2024 ; NGT library Yahoo Japan 2016+ mature.

Example in context

Yahoo Japan e-commerce + auction search uses Vald cluster ~30 Kubernetes nodes: ~5B product embeddings NGT index, semantic search Yahoo Japan Shopping API ~50ms latency, integrated user personalization features ; production proven 2020+ Yahoo Japan scale.

Related terms

• Milvus — CNCF distributed alternative.