Vector Databases Explained: Qdrant vs Weaviate vs Chroma (And Which One Won't Make You Cry)

Your Postgres Doesn’t Know What “Vibe” Means

You built a little RAG pipeline. It almost works. You’re doing LIKE '%semantic%' queries against SQLite and wondering why your AI assistant thinks “I need help with my cat” is the same as “I need help with my CAT scan.” Spoiler: it’s not the database’s fault. It’s yours. You picked the wrong tool.

Vector databases exist because the way AI thinks about data — as big blobs of floating-point numbers called embeddings — doesn’t fit neatly into rows and columns. This post is your crash course in what that means and which self-hostable vector DB you should actually use.

What Even Is a Vector Embedding?

Let’s start at the bottom so nobody feels left behind.

When an AI model reads a sentence like “my dog loves fetch,” it doesn’t see words. It converts that sentence into a list of numbers — something like [0.23, -0.87, 0.51, 0.12, ...] — typically with hundreds or thousands of dimensions. This list is called an embedding, and it’s a numerical representation of the meaning of that text.

Here’s the magic: similar meanings produce similar number lists. “My dog enjoys playing fetch” and “my pup likes chasing balls” will produce embeddings that are mathematically close to each other, even though they share almost no words. “Invoice processing software” and “accounts payable automation” will also be close, even though a keyword search would miss that entirely.

This is why your LIKE query fails. It looks at characters, not meaning.

A vector database stores these embeddings and lets you ask questions like: “Give me the 10 most similar things to this query.” That operation — called approximate nearest neighbor search (ANN) — is what these databases are built to do fast, at scale, without melting your server.

Why Not Just Use pgvector or FAISS?

Fair question. pgvector is a Postgres extension that adds vector search, and FAISS is a Facebook library for doing ANN in memory. Both are fine for specific situations.

But dedicated vector databases add things you’ll eventually want:

Filtering alongside vector search (e.g., “find similar documents, but only from this user’s collection”)
Persistence, backups, and replication without babysitting
Metadata storage alongside vectors
REST/gRPC APIs so your app isn’t tightly coupled to a library
Scalability beyond “fits in RAM”

Once your prototype grows legs, you’ll want a real vector DB. Might as well start there.

The Contenders

Chroma — The “Just Ship It” Option

Chroma is the vector database equivalent of SQLite. It’s Python-native, it can run fully in-process (no server needed), and you can go from zero to storing and querying embeddings in about 15 lines of code.

import chromadb

client = chromadb.Client()
col = client.create_collection("my_stuff")

col.add(
    documents=["My dog loves fetch", "Invoice processing software"],
    ids=["doc1", "doc2"]
)

results = col.query(query_texts=["puppy playing outside"], n_results=1)
print(results)

That’s it. No Docker. No config file. It just works.

Chroma also has a server mode if you want to run it as a standalone service, and it plays nicely with LangChain and LlamaIndex out of the box.

Self-hosting with Docker Compose:

services:
  chroma:
    image: chromadb/chroma:latest
    ports:
      - "8000:8000"
    volumes:
      - ./chroma_data:/chroma/chroma
    environment:
      - CHROMA_SERVER_AUTH_CREDENTIALS_PROVIDER=chromadb.auth.token.TokenConfigServerAuthCredentialsProvider
      - CHROMA_SERVER_AUTH_PROVIDER=chromadb.auth.token.TokenAuthServerProvider
      - CHROMA_SERVER_AUTH_CREDENTIALS=your-secret-token
      - CHROMA_SERVER_AUTH_TOKEN_TRANSPORT_HEADER=AUTHORIZATION

When to use Chroma: Prototyping, local dev, side projects, any situation where you want to stop configuring things and start building. It’s not the fastest or most feature-rich, but you’ll have something working before lunch.

When NOT to use Chroma: High-throughput production workloads, complex filtering requirements, or anything where you need fine-grained control over indexing.

Qdrant — The One That Actually Scales

Qdrant is written in Rust, which means it’s fast, memory-efficient, and the maintainers will mention it’s written in Rust approximately once per documentation page. Fair enough — it earns it.

Qdrant is what you reach for when Chroma starts showing its limits. It has:

Excellent filtering support — you can filter on payload fields during the vector search, not after (this matters a lot for performance)
Multiple vector support — store sparse and dense vectors together, useful for hybrid search
A clean REST and gRPC API
On-disk indexing — doesn’t require everything to fit in RAM
A slick web UI for browsing collections

Self-hosting with Docker Compose:

services:
  qdrant:
    image: qdrant/qdrant:latest
    ports:
      - "6333:6333"   # REST API
      - "6334:6334"   # gRPC
    volumes:
      - ./qdrant_storage:/qdrant/storage
    environment:
      - QDRANT__SERVICE__API_KEY=your-api-key-here

After spinning it up, hit http://localhost:6333/dashboard and you’ll see the UI.

Basic usage (Python client):

from qdrant_client import QdrantClient
from qdrant_client.models import Distance, VectorParams, PointStruct

client = QdrantClient("localhost", port=6333)

client.create_collection(
    collection_name="articles",
    vectors_config=VectorParams(size=1536, distance=Distance.COSINE),
)

client.upsert(
    collection_name="articles",
    points=[
        PointStruct(id=1, vector=[...], payload={"author": "sumguy", "tag": "devops"}),
    ]
)

results = client.search(
    collection_name="articles",
    query_vector=[...],
    query_filter={"must": [{"key": "tag", "match": {"value": "devops"}}]},
    limit=5,
)

Notice the query_filter — that’s Qdrant doing filtered vector search properly. In Chroma, you’d filter post-search. In Qdrant, it’s integrated into the index traversal. That’s a big deal at scale.

When to use Qdrant: Production deployments, self-hosted AI apps, anything where you care about query latency or need robust filtering. It’s the author’s personal pick for homelab AI work.

When NOT to use Qdrant: If you’re just hacking for a weekend and want minimal setup friction, Chroma might still be the quicker start.

Weaviate — The GraphQL One (Your Mileage May Vary)

Weaviate is the most “enterprise” feeling of the three. It has a GraphQL API, built-in support for multiple modalities (text, images, audio), and an object-oriented data model where you define “classes” with schemas before storing data.

It can auto-vectorize your data using built-in modules (OpenAI, Cohere, Hugging Face, etc.), which is genuinely handy — but it also means more configuration up front.

Self-hosting:

services:
  weaviate:
    image: semitechnologies/weaviate:latest
    ports:
      - "8080:8080"
    volumes:
      - ./weaviate_data:/var/lib/weaviate
    environment:
      QUERY_DEFAULTS_LIMIT: 25
      AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: "true"
      PERSISTENCE_DATA_PATH: "/var/lib/weaviate"
      DEFAULT_VECTORIZER_MODULE: "none"
      ENABLE_MODULES: ""
      CLUSTER_HOSTNAME: "node1"

Weaviate’s GraphQL interface looks like this:

{
  Get {
    Article(
      nearText: { concepts: ["self-hosting docker"] }
      limit: 3
    ) {
      title
      author
      _additional { certainty }
    }
  }
}

That’s either elegant or annoying depending on your feelings about GraphQL.

When to use Weaviate: Multi-modal search (text + images), when you want the auto-vectorization modules, or if your team is already deep in GraphQL and would rather not learn another query pattern.

When NOT to use Weaviate: Beginners who just want to store some embeddings. The schema-first approach and GraphQL interface have a learning curve that’ll cost you an afternoon before you query a single document.

Comparison Table

Feature	Chroma	Qdrant	Weaviate
Ease of setup	Very easy	Easy	Moderate
In-process mode	Yes	No	No
Performance	Good	Excellent	Good
Filtering	Post-search	Native (fast)	Native
API style	Python-native / REST	REST + gRPC	GraphQL + REST
Multi-modal	No	Partial	Yes
Web UI	No	Yes	Yes
Memory usage	Moderate	Low (Rust)	Higher
Self-hosting complexity	Low	Low	Medium
Best for	Prototyping	Production	Enterprise/multi-modal

The Decision Flowchart (No Actual Chart, Sorry)

Building a weekend project or prototype? Use Chroma. It’ll be running in 10 minutes.
Running something in production or your homelab long-term? Use Qdrant. The filtering is worth it, and Rust means it’ll hum along without drama.
Need to search over images, audio, or multiple content types? Consider Weaviate. Just budget some time for the learning curve.
Already have Postgres and don’t want another service? pgvector is fine for small scale. You’ll know when you’ve outgrown it.

Quick Start Summary

Both Chroma and Qdrant can be running locally with a single docker compose up -d. Weaviate needs a bit more care with its config, but it’s not unreasonable.

The ecosystem around all three is solid — LangChain and LlamaIndex support all of them, so you’re not locked in at the framework level.

If you’re just getting into RAG pipelines or AI-powered search for your self-hosted apps, start with Qdrant. It’s not significantly harder than Chroma for server mode, it has a nicer UI, and you won’t have to migrate later when your hobby project inexplicably attracts 40 users.

Wrapping Up

Vector databases went from “obscure ML infrastructure” to “thing you need for your homelab AI stack” in about 18 months. The good news is the tooling has kept up — all three options here are genuinely usable, well-documented, and Docker-friendly.

Pick Chroma if you’re learning. Pick Qdrant if you’re building. Pick Weaviate if you have very specific requirements and a couple of hours to read docs.

And yes, your Postgres still doesn’t know what vibes are. Some things are best left to the specialists.