Semantic Search

Semantic search is an approach to information retrieval in which a system attempts to understand the meaning and intent behind a query — rather than treating it as a bag of keywords — and returns results that are conceptually related even if they share few or no words with the query. Underpinned by dense vector embeddings produced by neural language models, semantic search has become a foundational component of modern AI applications ranging from enterprise knowledge bases to retrieval-augmented generation (RAG) pipelines.

Keyword Search vs. Semantic Search

Traditional keyword search systems such as those based on BM25 or TF-IDF score documents by counting how often query terms appear in them, weighted by term frequency and inverse document frequency. This approach is computationally efficient and works well when queries and documents use identical vocabulary. However, it fails for synonyms, paraphrases, cross-lingual queries, and queries where the user's intent is implicit rather than explicitly stated. A user searching for "how to increase revenue" may find no overlap with a document discussing "strategies for growing sales" if the two share no common terms. Semantic search resolves this by representing both query and document as points in a high-dimensional vector space — the embedding space — where proximity corresponds to semantic similarity.

How Semantic Search Works

Text Embedding

An embedding model (such as OpenAI's text-embedding-3, Cohere's Embed v4, or open-source alternatives like BGE or E5) converts input text — whether a short query or a long document — into a fixed-length dense vector, typically with 768 to 3,072 dimensions. These models are trained on large corpora to place semantically similar texts near each other in the embedding space, regardless of exact wording.

Indexing

Documents in the corpus are pre-processed through the embedding model at index time and their vectors are stored in a vector database (such as Pinecone, Weaviate, Qdrant, Milvus, or pgvector in PostgreSQL). Efficient approximate nearest-neighbour (ANN) algorithms — including HNSW (Hierarchical Navigable Small World) and IVF (Inverted File Index) — enable fast retrieval across billions of vectors without exhaustive pairwise comparison.

Query and Retrieval

At query time, the user's input is converted to an embedding using the same model. The vector database then performs a nearest-neighbour search, returning the k documents whose embedding vectors have the highest cosine similarity (or lowest Euclidean distance) to the query vector. These top-k results are then surfaced directly or passed to a re-ranker and/or a language model for further processing.

Hybrid Search

In practice, many production systems combine semantic search with keyword search in a hybrid approach. BM25 captures exact-match queries that dense retrieval misses (e.g., product codes, proper nouns, rare technical terms), while semantic search handles paraphrasing and intent. Reciprocal Rank Fusion (RRF) or a dedicated re-ranker model merges the two result lists.^[1]

Role in RAG Pipelines

Retrieval-augmented generation (RAG) systems use semantic search as their retrieval step. Given a user question, the system first retrieves the most relevant passages from a knowledge base using semantic search, then passes those passages as context to a language model to generate a grounded answer. The quality of the semantic search directly determines the quality of the generated response — poor retrieval leads to hallucination or irrelevant answers regardless of how capable the language model is.^[2]

Multilingual Semantic Search

Modern embedding models increasingly support multilingual retrieval, mapping text from dozens of languages into a shared vector space. This enables cross-lingual search where a Malay query returns relevant English documents, or vice versa, without translation. Models such as LASER, LaBSE, and multilingual-e5 have been benchmarked on Southeast Asian languages including Malay with promising results.

Applications

Semantic search is deployed across a wide range of domains:

• Enterprise knowledge management: Employees can query internal wikis, documents, and Slack archives in natural language rather than needing to recall exact terminology.

• E-commerce product discovery: Retailers surface products matching a customer's described need even when no matching product title exists.

• Customer support: Support systems retrieve relevant help articles or previous ticket resolutions for incoming queries.

• Legal and regulatory research: Lawyers query case law or contract repositories by concept rather than clause number.

• Healthcare: Clinicians retrieve relevant literature or patient records by clinical concept.

Semantic search has seen growing adoption across Malaysian enterprises and institutions, driven by the expansion of large language model infrastructure and the proliferation of vector databases as managed cloud services. The availability of Amazon Bedrock and Google Vertex AI embedding APIs from data centres in Malaysia and Singapore has reduced the barrier to deploying production semantic search systems for local companies. Malaysian banks including Maybank, CIMB, and RHB have invested in knowledge management platforms that use semantic search to allow customer service agents and relationship managers to query product manuals, regulatory guidelines, and internal policy documents in natural language. Bank Negara Malaysia (BNM), in its AI and technology risk guidance, has emphasised that information retrieval systems used in customer-facing financial services must meet accuracy and explainability standards — requirements that influence how Malaysian fintechs architect their semantic search and RAG pipelines. Government agencies including MAMPU (Malaysian Administrative Modernisation and Management Planning Unit) and the Ministry of Digital are exploring semantic search for public service knowledge bases, enabling citizens and civil servants to retrieve regulatory and service information without needing to know precise document references. The MyGovUC initiative, which consolidates government services, has seen semantic search discussed as an enhancement to its search capability. In the research and education sector, Universiti Malaya's Faculty of Computer Science and Information Technology has published work on Malay-language embedding models and cross-lingual semantic retrieval. The challenge of supporting Bahasa Malaysia alongside English in semantic search — including handling code-switching common in Malaysian online text — is an active research area. MIMOS Berhad has worked on Malay language NLP infrastructure that underpins local semantic search applications. For Malaysian AI startups, semantic search capability is frequently a core component of the RAG-based enterprise AI products they build. MDEC's MDAG-AI grant supports companies developing such solutions, and the MyDigital Blueprint highlights AI-powered knowledge management as a priority use case for public sector digital transformation.

1. Robertson, S., & Zaragoza, H. (2009). The probabilistic relevance framework: BM25 and beyond. Foundations and Trends in Information Retrieval, 3(4), 333–389.
2. Lewis, P., Perez, E., Piktus, A., et al. (2020). Retrieval-augmented generation for knowledge-intensive NLP tasks. Advances in Neural Information Processing Systems, 33.
3. Karpukhin, V., Oguz, B., Min, S., et al. (2020). Dense passage retrieval for open-domain question answering. EMNLP 2020.
4. Muennighoff, N., Tazi, N., Magne, L., & Reimers, N. (2022). MTEB: Massive text embedding benchmark. arXiv preprint arXiv:2210.07316.