OpenAI Embeddings for Semantic Product Search

Key facts, with sources

• At DevDay 2025 OpenAI reported 800 million weekly ChatGPT users, 4 million developers building on its platform, and roughly 8 billion API tokens processed per minute. (CNBC)
• ChatGPT reached 900 million weekly active users by late February 2026, up from 800 million at DevDay in October 2025. (TechCrunch)
• By March 2026 OpenAI's APIs were processing more than 15 billion tokens per minute, roughly doubling from the rate reported at DevDay 2025. (Panto AI OpenAI Statistics)
• OpenAI's published API pricing discounts cached input tokens by 90 percent on supported GPT models, which materially cuts costs for agents that resend long system prompts. (OpenAI API Pricing Docs)
• OpenAI raised $122 billion in new funding in 2026 to accelerate the next phase of AI development, one of the largest private raises in history. (OpenAI)

Why keyword search fails product catalogs

Keyword search matches strings; shoppers express meaning. A query like warm jacket for hiking shares almost no tokens with insulated trail parka, so the best product in the catalog never surfaces. Synonym dictionaries and manual tagging patch individual cases but cannot keep pace with how varied real queries are — misspellings, vague intent, attribute-driven needs the catalog never phrases the same way.

Embeddings fix exactly this by mapping text into a space where similar meanings land near each other. But the failure mode inverts: semantic search is mediocre at exact identifiers. A user pasting a precise SKU or model number wants literal matching, and a vector search may return the semantically similar sibling product instead. That is why production systems are hybrid — semantic for intent, keyword for precision, fused into one ranking.

What to embed per product

The single highest-leverage decision is the text you embed. Do not embed raw product pages — HTML boilerplate, shipping blurbs, and SEO filler drown the signal. Compose a clean document per product: title, brand, category path, the handful of attributes users actually search on (material, size class, use case), and a short distilled description. Keep it to a few hundred tokens; embeddings represent focused text better than sprawling text.

Two practical details. First, normalize vocabulary between catalog and queries where you can — if users say sneakers and the catalog says athletic footwear, include both phrasings in the embedded document. Second, decide variant handling deliberately: usually one embedding per parent product with variants as filterable metadata, not one vector per color-size combination, which bloats the index and fragments relevance.

Pipeline and storage with pgvector

For catalogs up to hundreds of thousands of products, pgvector inside your existing Postgres is my default — no new infrastructure, transactional consistency with product data, and SQL filters compose naturally with similarity ranking. Batch-embed the catalog, store vectors alongside products, and embed the query at search time.

import os
from openai import OpenAI

EMBED_MODEL = os.environ["OPENAI_EMBED_MODEL"]  # set to the latest embedding model id
client = OpenAI()

def embed_batch(texts: list[str]) -> list[list[float]]:
    resp = client.embeddings.create(model=EMBED_MODEL, input=texts)
    return [item.embedding for item in resp.data]

def product_document(p: Product) -> str:
    return (
        f"{p.title}\nBrand: {p.brand}\nCategory: {p.category_path}\n"
        f"Attributes: {', '.join(p.key_attributes)}\n{p.short_description}"
    )

Querying: filters first, similarity second

Structured constraints — in stock, price range, category, tenant — belong in SQL WHERE clauses, not in the embedding. Filtering first shrinks the candidate set, then similarity ordering ranks what remains. With pgvector, cosine distance ordering plus an appropriate index keeps latency interactive at catalog scale.

SELECT id, title,
       1 - (embedding <=> :query_vec) AS similarity
FROM products
WHERE in_stock
  AND price_cents BETWEEN :min_price AND :max_price
ORDER BY embedding <=> :query_vec
LIMIT 24;

Hybrid ranking for queries that need precision

Run keyword search in parallel with vector search — Postgres full-text search is entirely adequate — and fuse the two ranked lists. Reciprocal rank fusion is the technique I reach for first: it needs no score normalization, just each result's rank in each list, and it reliably lets exact matches for SKUs and brand names surface while semantic results cover intent queries. A product ranking first in either list ends up prominent; products appearing in both dominate.

Tune the balance with real traffic in mind: navigational queries (exact product names, part numbers) should behave like classic search, while exploratory queries lean semantic. A lightweight query classifier — even a regex for digit-heavy tokens — that shifts fusion weights per query type typically captures most of the win without adding a reranking model.

Evaluate with a golden query set, and plan for re-embedding

Search quality regressions are invisible without measurement. Collect fifty to a hundred real queries with the products a human says should appear, and compute recall in the top ten on every change — embedding text format, fusion weights, filters. This takes a day to set up and permanently converts search tuning from vibes into engineering.

Operationally, remember vectors are model-bound: embeddings from different models are not comparable, so upgrading the embedding model means re-embedding the entire catalog and cutting over atomically — never mixing old and new vectors in one index. Re-embedding a large catalog is a batch job with real cost, so schedule it deliberately, and re-embed products whenever their composed document changes, ideally event-driven from your catalog update stream.

When to hire senior help

Bring in senior help when you move from a working prototype to production traffic, because cost controls, evals, rate-limit handling, and fallback behavior determine whether the unit economics work. An experienced engineer usually pays for themselves by cutting token spend and preventing outages rather than by writing the first prompt. If your stack includes React Native + Python + AI, a senior engineer who owns the full product beats coordinating multiple juniors.

Bottom line

Dhairya Senjaliya ships AI — OpenAI Development projects worldwide — book a scoping call to discuss your specific situation.

Common pitfalls to avoid

• ✕Hardcoding a single flagship model ID for every call instead of routing by task, paying GPT-5-tier prices for classification work a nano-tier model handles at a fraction of the cost
• ✕Putting volatile content like timestamps and user IDs at the top of prompts, which breaks prefix caching and forfeits the 90 percent cached-input discount
• ✕Building on deprecated surfaces like the legacy Completions or wound-down fine-tuning APIs instead of the current Responses API and agent tooling
• ✕Launching with no spend caps or per-user rate limits, so a retry loop or a single abusive user burns a month's API budget overnight

Frequently asked questions

Sources

• CNBC
• TechCrunch
• Panto AI OpenAI Statistics
• OpenAI API Pricing Docs
• OpenAI