What Is a Vector Database and Why It Matters for AI Apps?

What Is a Vector Database and Why It Matters for AI Apps?

If you have spent any time researching how to build AI features - chatbots, semantic search, document Q&A, recommendation engines - you have probably encountered the term "vector database." The explanation usually jumps straight into high-dimensional space and cosine similarity, losing anyone who does not have a machine learning background.

This post explains what vector databases actually do, why they exist, and when you need one, in language that any technical decision-maker can follow.

Vectors and Embeddings: Starting From Scratch

Before understanding vector databases, you need to understand what a vector and an embedding are.

A vector is simply a list of numbers. For example, [0.2, 0.8, -0.3, 0.5] is a vector with four dimensions.

An embedding is a vector that represents something meaningful - text, an image, an audio clip, a product - as a list of numbers in a way that captures semantic meaning. The key property is that similar things produce similar vectors.

Here is what makes embeddings powerful: if you convert the sentence "the dog ran across the park" into an embedding, and you also convert "the puppy sprinted through the garden," the two resulting vectors will be very close to each other mathematically - because the sentences mean nearly the same thing. Meanwhile, "quarterly revenue report" will produce a very different vector.

Embedding models are the AI models that do this conversion. OpenAI's text-embedding-3-small, Google's text-embedding-004, and open-source models like BGE all take text as input and output a vector - typically with 768, 1,536, or 3,072 dimensions.

The same principle applies to images (image embeddings capture visual similarity), audio (audio embeddings capture acoustic similarity), and products (product embeddings can capture feature similarity). For this post, we will focus on text embeddings since they are most relevant to the business applications discussed.

Why Traditional Databases Fail for Semantic Search

Traditional relational databases (PostgreSQL, MySQL, SQL Server) are excellent at what they were designed for: storing structured data and answering precise queries.

If you ask "give me all customers in London with orders over £500 in the last 30 days," a SQL query handles this perfectly. The database searches by exact match and range conditions - criteria that are unambiguous.

But if you ask "find all documents that discuss our refund policy for digital products," a traditional database has no good answer. It can search for keywords like "refund" or "digital," but:

• It does not understand that "money back guarantee" means the same thing as "refund"
• It will return documents that mention refund and digital in unrelated contexts
• It cannot rank results by how relevant they are to the actual meaning of the query

This is why keyword search has been the frustrating default for so long - the database does not understand meaning, only exact word matches.

What Vector Databases Do Differently

A vector database stores embeddings and is optimised to find the vectors most similar to a query vector. This is called nearest neighbour search or similarity search.

Here is the workflow:

1. You convert all your documents to embeddings and store them in the vector database
2. When a user asks a question, you convert the question to an embedding
3. You ask the vector database: "which stored embeddings are most similar to this question embedding?"
4. The database returns the top N most similar document chunks
5. Those chunks are passed to an LLM along with the original question

The result is search that understands meaning rather than keywords. A question about "refund policy for digital purchases" correctly retrieves documents discussing money-back guarantees on software downloads, even without a single matching keyword.

The key technical operation is: measuring the distance between vectors in high-dimensional space. The most common measurement is cosine similarity (the angle between vectors). Small angle = similar meaning. Large angle = different meaning.

The Main Vector Database Options

Pinecone

Type: Managed cloud service Pricing: Free tier (1 index, 1M vectors), Starter at $70/month, scales with usage Best for: Teams who want a fully managed solution with no infrastructure to maintain

Pinecone is the most popular dedicated vector database. You create an index, push vectors via API, and query with low latency. The managed nature means no server administration - it scales automatically and has high availability built in.

The downside is cost at high scale - storing hundreds of millions of vectors on Pinecone becomes expensive. It also introduces vendor dependency.

Weaviate

Type: Open-source, self-hosted or cloud Pricing: Self-hosted is free; cloud starts at $25/month Best for: Teams wanting more control or needing to run on their own infrastructure

Weaviate is a full-featured vector database with a rich query language. It supports hybrid search (combining keyword and vector search), multi-tenancy, and can store the original objects alongside the vectors. Good documentation and an active community.

The self-hosted option is attractive for companies with data privacy requirements. Requires more infrastructure knowledge than Pinecone.

Chroma

Type: Open-source, typically self-hosted Pricing: Free (open-source) Best for: Development and prototyping, cost-sensitive production deployments at smaller scale

Chroma is lightweight and easy to run locally, which makes it excellent for building and testing RAG systems before deciding on a production vector database. The Python API is intuitive and integrates cleanly with LlamaIndex and LangChain.

Production deployments at scale require more careful infrastructure setup than dedicated managed services.

pgvector

Type: PostgreSQL extension Pricing: Free (you pay for your PostgreSQL infrastructure) Best for: Teams already using PostgreSQL who want to avoid adding a new database

pgvector adds vector similarity search to standard PostgreSQL. You store vectors in a regular PostgreSQL table alongside other data, and query them with a simple syntax extension. This is enormously appealing for teams who are already managing PostgreSQL - one less system to operate, simpler architecture, and existing backup and monitoring tooling applies.

Performance at very large scale (tens of millions of vectors) lags behind dedicated vector databases. For most business applications (under 5-10 million vectors), pgvector performs well with proper indexing.

Use Cases for Vector Databases

Semantic Search

The most common use case. Users search a knowledge base, document library, or product catalogue by meaning rather than keywords. Particularly valuable in:

• Customer support knowledge bases where agents search for relevant articles
• Legal and compliance document search
• Internal HR policy and process documentation
• Product catalogues where customers describe what they want in natural language

Retrieval-Augmented Generation (RAG)

The vector database is the retrieval component of a RAG system. When a user asks an AI assistant a question, the vector database finds the relevant document chunks that inform the answer.

This is the most rapidly growing use case and the primary driver of vector database adoption. See our post on how to train AI on your company data for a detailed walkthrough of how this works end-to-end.

Recommendations

Recommendation systems can use embeddings to find similar items. A user who viewed product A (with a known embedding) gets recommendations for products with similar embeddings. The same approach works for content recommendations, job matching, and similar-user clustering.

Collaborative filtering (you liked X, others who liked X also liked Y) can be implemented with vector similarity search. This is how many modern recommendation engines work.

Duplicate Detection

Embedding-based similarity search can identify near-duplicate content - useful for content moderation, deduplication of records, plagiarism detection, and identifying duplicate support tickets. Two very similar pieces of text will have very similar embeddings even if they are not character-for-character identical.

Image Search

Image embeddings enable "find similar images" features. A user uploads a photo; the system finds visually similar images in your catalogue. This powers visual search for e-commerce (find similar products) and content management (find duplicate or similar images).

How to Choose a Vector Database

Answer these questions:

How many vectors do you need to store?

• Under 1 million: Any option works. Use Chroma for dev, pgvector for production if you use PostgreSQL, Pinecone if you want a managed service.
• 1-50 million: Pinecone, Weaviate, or pgvector with proper indexing all work.
• 50 million+: Evaluate Pinecone at scale (cost), Weaviate self-hosted (ops overhead), or purpose-built solutions.

Do you already use PostgreSQL?

• If yes: Start with pgvector. It is the simplest option and avoids a new system.

Do you have data privacy or compliance requirements?

• If your data cannot go to a managed cloud service: Use pgvector or self-hosted Weaviate or Chroma.

Do you need hybrid search (keyword + semantic)?

• Weaviate handles this natively and elegantly.
• Pinecone requires a separate BM25 index for keyword search.
• pgvector can be combined with PostgreSQL's full-text search.

Is this a prototype or production?

• Prototype: Use Chroma locally. Zero setup, works great for development.
• Production: Choose based on scale and the above criteria.

Integration with LLM Frameworks

Vector databases integrate cleanly with the major LLM frameworks:

• LlamaIndex: Native integration with Pinecone, Weaviate, Chroma, pgvector, and a dozen others. Handles embedding generation, index management, and retrieval.
• LangChain: Similar native integrations. Both frameworks abstract the vector database so you can swap providers without rewriting retrieval logic.

This abstraction is valuable: you can build your RAG system with Chroma locally, switch to Pinecone for production, and change only the database configuration - not the application logic.

When You Do NOT Need a Vector Database

Despite the hype, there are many AI applications that do not need a vector database.

When your data fits in the context window: If you have fewer than 50-100 documents, you can simply paste them into the LLM's prompt. No embedding, no retrieval, no vector database. This is simpler and often good enough for small knowledge bases.

When keyword search is sufficient: If your use case is searching structured data (find all invoices from Q3, find all customers with "enterprise" in their company name), a traditional database does the job better. Vector search adds complexity and does not improve results for exact-match or range queries.

When the LLM does not need external knowledge: If your AI feature does not involve retrieving information from your data - for example, a code generation tool or a format-conversion tool - you do not need retrieval at all.

Early-stage prototypes: Get a proof of concept working with direct context injection first. Only add a vector database when you have confirmed the use case is valuable and the context window becomes a constraint.

The Bigger Picture

Vector databases are an infrastructure component, not a product. They solve a specific problem: fast similarity search over high-dimensional vectors at scale. They matter because embedding-based search is fundamental to how RAG systems, semantic search, and recommendation engines work.

For most business builders, the practical knowledge you need is: vector databases enable AI systems to find relevant information from your own data at query time, and which one to choose depends primarily on your existing infrastructure, data volume, and whether you want managed or self-hosted.

If you are building a product that needs semantic search or a RAG knowledge base, our LLM Integration service covers the full architecture including vector database selection and implementation. You can also use our AI Feasibility Checker to assess your specific use case.

Get a free quote or estimate your project to get a realistic view of what building this looks like for your requirements.