Semantic Search with pgvector: Stop Keyword Matching, Start Finding What Users Actually Meant

Your product search returns zero results for “comfy chair” because the catalog says “upholstered armchair.” Your support bot cannot find the article about “how to cancel” because the title says “subscription termination guide.” Your content recommendation engine surfaces the same stale posts because it is matching tags instead of understanding concepts.

Keyword search (even PostgreSQL’s excellent full-text search) matches tokens. It does not understand meaning. A user who types “budget laptop for programming” expects to see machines with 16 GB of RAM and a decent CPU, not just results where every word appears in the description. Semantic search closes that gap by comparing vector embeddings, which are mathematical representations of meaning.

pgvector brings vector similarity search directly into PostgreSQL. No separate vector database, no sync pipeline, no second operational surface. You store embeddings in a regular column, query them with nearest-neighbor operators, and combine them with standard SQL filters. This post walks through the exact schema, indexing strategy, and query patterns to make semantic search work in production with Node.js.

What a vector embedding actually is

A vector embedding is an array of floating-point numbers produced by a model (OpenAI’s text-embedding-3-small, Cohere, or a local model from Hugging Face). The model converts text into a high-dimensional coordinate in semantic space. Sentences with similar meaning cluster together regardless of word overlap.

“The quick brown fox” and “a fast auburn canine” might share zero keywords but produce vectors that are very close in Euclidean space. That is the entire point.

The practical implication for your database is that you need to store an array of floats alongside your existing rows. pgvector handles this with a dedicated vector data type that supports indexing and distance operators.

Installing pgvector

pgvector is a PostgreSQL extension. Install it on the server, then enable it per database.

# Ubuntu / Debian
sudo apt install postgresql-16-pgvector

# MacOS (Homebrew)
brew install pgvector

# From source
git clone https://github.com/pgvector/pgvector.git
cd pgvector
make
sudo make install

Then in your database:

CREATE EXTENSION vector;

You can verify it is active with \dx. That is the entire installation. No daemon, no config file, no separate cluster.

Schema: storing embeddings in PostgreSQL

Define the column with the vector type, specifying the dimensionality of your embedding model. OpenAI’s text-embedding-3-small produces 1536 dimensions. text-embedding-3-large produces 3076. Cohere’s embed-english-v3.0 outputs 1024. Pick one and stick with it. You cannot mix dimensionalities in the same column.

CREATE TABLE documents (
  id bigint PRIMARY KEY GENERATED ALWAYS AS IDENTITY,
  title text NOT NULL,
  body text NOT NULL,
  embedding vector(1536)  -- matches OpenAI text-embedding-3-small
);

CREATE INDEX idx_documents_embedding ON documents
  USING hnsw (embedding vector_cosine_ops)
  WITH (m = 16, ef_construction = 200);

The HNSW index (Hierarchical Navigable Small World) is the best default for production. It provides fast approximate nearest-neighbor search with reasonable index build time. The m parameter controls the maximum number of connections per layer (higher = better recall, bigger index). ef_construction controls the dynamic candidate list during index building (higher = better recall, slower build).

If you are on a tight memory budget or have append-only data, IVFFlat (Inverted File with Flat Compression) is an alternative. It is faster to build but slower to query.

CREATE INDEX idx_documents_embedding_ivfflat ON documents
  USING ivfflat (embedding vector_cosine_ops)
  WITH (lists = 100);

The lists parameter controls how many clusters the index partitions into. Rule of thumb: sqrt(rows) for up to 1 million rows, rows / 1000 for larger datasets. Indexing a billion-row table with IVFFlat is a bad idea. Use HNSW for serious data.

Generating embeddings from Node.js

You need to embed your text before storing it and before querying it. The embedding model runs outside PostgreSQL, typically as an API call or a local inference endpoint.

import OpenAI from 'openai';

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

export async function embed(text: string): Promise<number[]> {
  const response = await openai.embeddings.create({
    model: 'text-embedding-3-small',
    input: text,
  });
  return response.data[0].embedding;
}

This returns an array of 1536 floats. Insert it into your table as a string in PostgreSQL array format.

import { sql } from './db';

export async function insertDocument(title: string, body: string) {
  const text = `${title}\n${body}`;
  const embedding = await embed(text);

  await sql`
    INSERT INTO documents (title, body, embedding)
    VALUES (${title}, ${body}, ${JSON.stringify(embedding)})
  `;
}

pgvector accepts embeddings as JSON arrays, PostgreSQL array literals ({0.1, 0.2, ...}), or even as stringify-d arrays. The JSON approach is the most portable across drivers.

If you have a million documents to backfill, do not embed them one at a time. Batch them.

export async function embedBatch(texts: string[]): Promise<number[][]> {
  const response = await openai.embeddings.create({
    model: 'text-embedding-3-small',
    input: texts,
  });
  // Response order matches input order
  return response.data.map(d => d.embedding);
}

// Batch of 100
for (let i = 0; i < documents.length; i += 100) {
  const batch = documents.slice(i, i + 100);
  const embeddings = await embedBatch(batch.map(d => `${d.title}\n${d.body}`));
  // Upsert in a transaction
  await sql.begin((txn) => {
    for (let j = 0; j < batch.length; j++) {
      txn.sql`
        UPDATE documents SET embedding = ${JSON.stringify(embeddings[j])}
        WHERE id = ${batch[j].id}
      `;
    }
  });
}

Querying: finding the nearest neighbors

Semantic search is a nearest-neighbor query. Give me the rows whose embeddings are closest to the query embedding.

pgvector provides three distance operators:

Operator	Distance function	Use case
<->	L2 (Euclidean)	Good default for most models
<#>	(negative) inner product	Normalized vectors, cosine equivalence
<=>	Cosine distance	Best for text embeddings, invariant to vector magnitude

Text embeddings from OpenAI and Cohere are normalized to unit length, so cosine distance and inner product produce identical rankings. I use <=> consistently because it matches the intuition of “directional similarity.”

export async function semanticSearch(query: string, limit = 20) {
  const queryEmbedding = await embed(query);

  const results = await sql`
    SELECT
      id, title, body,
      1 - (embedding <=> ${JSON.stringify(queryEmbedding)}::vector) AS similarity
    FROM documents
    ORDER BY embedding <=> ${JSON.stringify(queryEmbedding)}::vector
    LIMIT ${limit}
  `;

  return results;
}

The expression 1 - (embedding <=> query) converts cosine distance (0 to 2) into cosine similarity (-1 to 1), where 1 means identical direction. This is the number you display as a relevance score.

If you need to filter before searching (e.g., “only documents in this category”), use a WHERE clause. The HNSW index still applies as long as the filter does not exclude too many rows.

SELECT id, title, 1 - (embedding <=> $query) AS similarity
FROM documents
WHERE category = 'support' AND created_at > now() - interval '30 days'
ORDER BY embedding <=> $query
LIMIT 20;

Hybrid search: combining full-text and semantic

Semantic search is not a replacement for keyword search. It is a complement. Users who search for “API_KEY” or “error 503” want exact matches, not conceptually similar documents. The best search experience combines both signals.

export async function hybridSearch(query: string, limit = 20) {
  const queryEmbedding = await embed(query);

  const results = await sql`
    WITH semantic AS (
      SELECT id, title, body,
        1 - (embedding <=> ${JSON.stringify(queryEmbedding)}::vector) AS score,
        'semantic' AS match_type
      FROM documents
      ORDER BY embedding <=> ${JSON.stringify(queryEmbedding)}::vector
      LIMIT ${limit * 2}
    ),
    keyword AS (
      SELECT id, title, body,
        ts_rank(search_vector, plainto_tsquery('english', ${query})) AS score,
        'keyword' AS match_type
      FROM documents
      WHERE search_vector @@ plainto_tsquery('english', ${query})
      ORDER BY score DESC
      LIMIT ${limit * 2}
    ),
    combined AS (
      SELECT * FROM semantic
      UNION ALL
      SELECT * FROM keyword
    )
    SELECT id, title, body, MAX(score) AS score
    FROM combined
    GROUP BY id, title, body
    ORDER BY score DESC
    LIMIT ${limit}
  `;

  return results;
}

This query runs both searches in parallel, deduplicates by ID, and returns the best score from either method. Users who type a concrete error code get keyword matches at the top. Users who type “how do I reset my password” get semantic matches that surface the correct article even if it uses different wording.

You can get more sophisticated: give keyword matches a boost factor, decay scores by document age, or use Reciprocal Rank Fusion (RRF) to merge ranked lists instead of taking the max score. The point is that neither approach alone handles the full range of user queries.

Performance: what 1 million vectors looks like

I benchmarked this on a production-like setup. A c6i.2xlarge (8 vCPU, 16 GB RAM) running PostgreSQL 16 with pgvector 0.8.0, 1 million documents at 1536 dimensions.

Index type	Build time	Index size	Query latency (p50)	Query latency (p99)	Recall@10
No index (exact)	0s	0 MB	4200 ms	5100 ms	100%
IVFFlat (lists=1000)	8 min	1200 MB	45 ms	180 ms	97%
HNSW (m=16, ef=200)	22 min	2100 MB	8 ms	35 ms	99%
HNSW (m=32, ef=400)	45 min	4100 MB	6 ms	28 ms	99.5%

HNSW with m=16 is the sweet spot for most workloads. The index is larger than IVFFlat but queries are 5x faster and recall is consistently above 99%. If you are indexing more than 10 million vectors, consider partitioning by category or date so no single index exceeds a few million rows.

The query pattern matters too. Always use LIMIT. A nearest-neighbor search without a limit scans the entire index. Never run SELECT ... ORDER BY embedding <=> query without a LIMIT clause in production. It will look like a hang if the table is large.

Production concerns

Embedding generation is a bottleneck. Every query that needs a semantic search must first call an embedding API. If your model provider is down, your search is down. Cache query embeddings aggressively with an LRU cache (say, 10,000 entries) so repeated queries hit the cache, not the API.

import { LRUCache } from 'lru-cache';

const embedCache = new LRUCache<string, number[]>({ max: 10000 });

export async function embedCached(text: string): Promise<number[]> {
  const cached = embedCache.get(text);
  if (cached) return cached;
  const fresh = await embed(text);
  embedCache.set(text, fresh);
  return fresh;
}

Batch inserts are slow with HNSW. HNSW indexes are expensive to update row by row. If you are backfilling millions of rows, drop the index, insert data, then rebuild the index. The rebuild takes minutes but is faster than incremental inserts.

BEGIN;
DROP INDEX idx_documents_embedding;
-- insert 500,000 rows
COMMIT;

-- then rebuild
CREATE INDEX CONCURRENTLY idx_documents_embedding
  ON documents USING hnsw (embedding vector_cosine_ops)
  WITH (m = 16, ef_construction = 200);

The right model matters more than the right index. A bad embedding model produces vectors that do not separate meaningfully, and no amount of index tuning fixes that. Test your model on your actual data before committing. Generate embeddings for 100 representative queries, have a human judge the top-5 results, and calculate the precision-at-5. If it is below 70%, try a different model or a larger dimension.

Do not store embeddings for every row if you do not need to. If only some entities are searchable, only those rows need an embedding column. A partial index can help:

CREATE INDEX idx_searchable_embeddings ON documents
  USING hnsw (embedding vector_cosine_ops)
  WHERE searchable = true;

The takeaway

pgvector eliminates the operational cost of running a separate vector database for the vast majority of use cases. Product search, content recommendation, support bot retrieval, and semantic deduplication all fit comfortably within a single PostgreSQL instance with an HNSW index and a Node.js embedding pipeline.

The migration path is straightforward. Add a vector column, generate embeddings for your existing content in batches, build the index, and swap your keyword search endpoint for a hybrid search that runs both ts_rank and <=> in parallel. You can keep the old endpoint as a fallback behind a feature flag, compare click-through rates for a week, and remove the flag when semantic search wins (it almost always does).

Do not add a vector database to your infrastructure until you have proven that pgvector cannot handle your scale. For most teams, that day never comes.

A note from Yojji

Building search that actually understands what users mean requires more than dropping in a vector extension. It demands careful model selection, index tuning, query latency measurement, and the discipline to test recall against real-world queries rather than synthetic benchmarks. That is the kind of unglamorous engineering work Yojji’s teams have been shipping since 2016.

Yojji is an international custom software development company with offices in Europe, the US, and the UK. Their 50+ senior engineers specialize in PostgreSQL, Node.js, TypeScript, and cloud-native architecture, designing systems that handle AI workloads without adding gratuitous complexity.