Vald

Vald is an open-source distributed vector search engine developed by the Vald team under the vdaas organization. It performs approximate nearest neighbor search over dense vectors using the NGT algorithm and runs on Kubernetes. The latest GitHub release listed is v1.8.1, and it is licensed under Apache 2.0.

What It Is

Vald is a cloud-native dense vector search engine for finding similar items among very large sets of feature vectors. Any data that can be converted to a vector, such as images, audio, video, text or binary, can be indexed and searched. The project lists recognition, recommendation, detection, grammar checking and real-time translation as example uses.

How It Works

Vector indexes are distributed across multiple agents, each storing a different part of the index. Each index is also stored on multiple agents for replication, and replicas are automatically rebalanced when an agent goes down. Vald uses a distributed index graph, so it keeps serving searches while indexing, avoiding stop-the-world locking. Automatic index backup to object storage or persistent volumes supports disaster recovery. Ingress/egress filtering can be customized to fit the gRPC interface.

APIs and SDKs

Clients talk to a Vald cluster over gRPC. The documented APIs cover Insert, Update, Upsert, Search, Remove, Flush, Object, Filter Gateway, Mirror Gateway and Status. Official client libraries exist for Go, Java, Python, Node.js and Clojure.

Setup Path

Vald requires Kubernetes 1.19 or later, and the NGT agent requires AVX2 instructions. It can be installed with Helm or with the Helm operator. The deployable components include the Agent NGT, Agent Sidecar, Discoverer, Gateways, Index Manager and Helm Operator. Documentation also covers a Faiss agent, standalone agent deployment on Docker or Kubernetes, and multicluster setups.