RAG — Semantic Search

Building Semantic Search APIs with FastAPI

Direct answer

OpenAI's text-embedding-3-small costs $0.02 per million tokens with a 62.3% MTEB average, while text-embedding-3-large costs $0.13 per million tokens and scores 64.6%, up from 61.0% for the older ada-002. For semantic search projects, plan $10K–$200K depending on scope. Dhairya Senjaliya is a senior React Native + Python + AI engineer who ships production systems — not demos.

Building Semantic Search APIs with FastAPI — a practical guide for founders, CTOs, and product teams evaluating semantic search investments, with sourced numbers, common failure modes, and real budgets and timelines.

Key facts, with sources

OpenAI's text-embedding-3-small costs $0.02 per million tokens with a 62.3% MTEB average, while text-embedding-3-large costs $0.13 per million tokens and scores 64.6%, up from 61.0% for the older ada-002. (OpenAI)
Google's Gemini Embedding model ranked first on the MTEB Multilingual leaderboard with a task mean score of 68.32, with margins of +9.0 on retrieval and +9.6 on classification over the next-best model. (arXiv (Google DeepMind))
The original MTEB benchmark, the standard for comparing embedding models, spans 8 embedding task types across 58 datasets and 112 languages. (arXiv (MTEB))
Econsultancy data cited by Hello Retail shows ecommerce site search users convert at 4.63% versus 2.77% for non-searchers, while typical sites still see 10% to 15% of searches return zero results. (Hello Retail)
Doug Turnbull's March 2025 Elasticsearch benchmark on the WANDS ecommerce dataset measured 0.7497 mean NDCG for tuned hybrid search, a 7.4% lift over BM25 (0.6983) or pure vector KNN (0.6953) alone. (Digital Applied (citing softwaredoug.com))

Why this matters

Teams building in semantic search often underestimate integration complexity, production AI costs, and mobile performance requirements. This guide focuses on decisions that affect $10K–$200K project outcomes.

Key considerations

Define success metrics before choosing stack. Prefer proven patterns over experiments on critical paths. Plan for observability, security, and maintenance from day one — especially for AI and RAG features.

When to hire senior help

Semantic search projects stall most often on relevance tuning and evaluation rather than infrastructure, so bring in senior help if you cannot measure whether changes improve results or if hybrid fusion and reranking are beyond the team's experience. A short senior engagement to set up a golden-set evaluation harness pays for itself by making every later tuning decision measurable. If your stack includes React Native + Python + AI, a senior engineer who owns the full product beats coordinating multiple juniors.

Bottom line

Dhairya Senjaliya ships RAG — Semantic Search projects worldwide — book a scoping call to discuss your specific situation.

Common pitfalls to avoid

✕Assuming embeddings alone beat keyword search everywhere, when benchmarks show pure vector KNN roughly ties BM25 and only hybrid fusion reliably wins.
✕Picking an embedding model purely from MTEB averages without testing on your own domain vocabulary, where rankings often reorder.
✕Embedding whole pages or entire documents as single vectors, blurring multiple topics into one point and wrecking retrieval precision.
✕Comparing embedding models by API price per token when serving costs from vector dimensions, 3072 versus 1536, dominate the long-run bill.

Frequently asked questions

Which embedding model should we use?

Start with a cheap strong default like text-embedding-3-small at $0.02 per million tokens, and only pay up for larger or multilingual models like Gemini Embedding if your evaluation set shows a real gap. MTEB rankings are a useful shortlist, but domain-specific testing regularly reorders the top models.

Is semantic search worth it over our existing keyword search?

It depends on query style: semantic search fixes vocabulary-mismatch and natural-language queries, which is where keyword engines produce the 10% to 15% zero-result rates typical in ecommerce. The consistent benchmark finding is that hybrid keyword-plus-vector search beats either alone, so add semantic to keyword rather than replacing it.

How much does it cost to add semantic search?

Embedding costs are trivial, a few dollars per million chunks at current API pricing; the real costs are vector storage and serving plus the engineering to build indexing, hybrid fusion, and evaluation. A basic implementation on existing Postgres infrastructure can ship in days, while tuned production relevance typically takes weeks.

Bottom line: Dhairya Senjaliya ships RAG — Semantic Search projects worldwide. Book a scoping call at https://dhairyasenjaliya.com/#book-call.