{"id":3662,"date":"2025-09-02T10:50:51","date_gmt":"2025-09-02T05:20:51","guid":{"rendered":"https:\/\/blogs.infosys.com\/emerging-technology-solutions\/?p=3662"},"modified":"2025-09-02T10:50:51","modified_gmt":"2025-09-02T05:20:51","slug":"rag-demystified-part-3","status":"publish","type":"post","link":"https:\/\/blogs.infosys.com\/emerging-technology-solutions\/artificial-intelligence\/rag-demystified-part-3.html","title":{"rendered":"RAG Demystified: Part 3"},"content":{"rendered":"

We have discussed the core components of RAG in Part 1<\/a> and explored similarity metrics, embedding processes, and chunking strategies in Part 2<\/a>. In this third installment, we’ll dive deep into advanced retrieval techniques, hybrid approaches, and cutting-edge RAG architectures that are revolutionizing how we build intelligent systems.<\/p>\n

Static vs Dynamic Embeddings:<\/h3>\n

When we embed chunks, we can choose between static and dynamic embedding approaches, each with distinct advantages and use cases.<\/p>\n

Static embedding:<\/h3>\n

Static embeddings refer to fixed vector representations of words or tokens that remain constant regardless of the context in which they appear. These embeddings are typically pre-trained on large corpora using models like Word2Vec or GloVe and capture general semantic and syntactic relationships between words. In contrast to dynamic embeddings, which adapt based on surrounding context, static embeddings assign the same vector to a word in every instance, regardless of sentence structure or meaning[1].<\/p>\n

Static Embedding Example:<\/pre>\n
The word “bank” gets the same vector in both sentences:<\/p>\n
    \n
  • “He sat by the river bank” \u2192 vector for bank: [0.45, 0.12, 0.88, 0.36]<\/li>\n
  • “She deposited money in the bank” \u2192 vector for bank: [0.45, 0.12, 0.88, 0.36]<\/li>\n<\/ul>\n
    Each word in the vocabulary has one fixed vector for static embedding.<\/p>\n
    Advantages:<\/h4>\n
    Static embeddings offer several advantages, including computational efficiency, which enables faster inference and reduces memory requirements. Additionally, they provide consistent representations, making them reliable and stable across different tasks and inputs.<\/p>\n
    Limitations:<\/h4>\n
    While static embeddings are computationally efficient and easy to use, they lack the ability to capture contextual nuances, making them less effective for complex or ambiguous queries. Modern Retrieval-Augmented Generation (RAG) systems increasingly favor contextual embeddings from transformer-based models for improved accuracy and relevance.<\/p>\n
    Popular static embedding models include Word2Vec, GloVe<\/em>, and FastText.<\/em><\/p>\n
    Dynamic embedding:<\/h3>\n
    While static embeddings proved to be strong baselines for measuring semantic similarity, they inherently failed to capture the rich contextual information of natural language. For example, a static model would produce the identical vector for the word “bank” in both “river bank” and “money bank,” despite their distinct meanings. Dynamic embeddings, on the other hand, generate vector representations of words or sentences that vary based on the surrounding context within the text.<\/p>\n
    Dynamic Embedding Example:<\/pre>\n
    With a dynamic model, the word “bank” has different vectors based on context:<\/p>\n
      \n
    • “He sat by the river bank” \u2192 vector for bank: [0.33, 0.55, 0.74, 0.21]<\/li>\n
    • “She deposited money in the bank” \u2192 vector for bank: [0.91, 0.13, 0.48, 0.62]<\/li>\n<\/ul>\n
      Popular models for dynamic embeddings include BERT, SBERT, all-MiniLM, the E5 family, Cohere Embed,<\/em> and OpenAI’s text embedding<\/em> models.<\/p>\n
      Advantages:<\/h4>\n
      Dynamic embeddings provide context-aware representations that improve the handling of ambiguity and deliver superior performance on complex tasks. They are adaptable to domain-specific contexts, making them highly versatile.<\/p>\n
      Limitations:<\/h4>\n
      Higher computational costs, slower inference speeds, increased memory requirements, and greater implementation complexity are the downsides of using dynamic embeddings<\/p>\n
      The Need for Hybrid Approaches:<\/h3>\n
      There is no single embedding model that works optimally across all industries. While general-purpose embeddings provide a good starting point for building Retrieval-Augmented Generation (RAG) systems, achieving high performance typically requires industry-specific specialization.<\/p>\n
      When working with domain-specific data, the embedding model may not be trained on specific technical or uncommon terms. Modern RAG systems often employ hybrid approach where specific terms can be effectively retrieved using sparse technique, while semantic understanding is handled by dense vector representations. This combination helps improve recall and relevance in domain-sensitive applications. One of the most effective sparse retrieval techniques is BM25.<\/p>\n
      Let’s explore how BM25 works<\/p>\n
      BM25 Algorithm:<\/h3>\n
      BM25 (Best Match 25) is a ranking function widely used in search engines and information retrieval systems. It builds on the TF-IDF model by introducing enhancements like term frequency saturation and document length normalization, offering a more accurate measure of a document\u2019s relevance to a search query[2].<\/p>\n
      Why BM25 Matters<\/h4>\n
      Unlike simple TF-IDF, BM25 addresses two critical issues[3]:<\/p>\n
      Term frequency saturation: Prevents documents with excessive keyword stuffing from dominating results
      \nDocument length normalization: Ensures fair comparison between short and long documents<\/p>\n
      Core Components of BM25:<\/h4>\n
      1. Inverse Document Frequency (IDF):<\/h4>\n
      IDF measures how informative a term is by penalizing common terms and rewarding rare ones.<\/p>\n
      Formula:<\/h5>\n
      IDF(qi) = log(1 + (N - df(qi) + 0.5)\/(df(qi) + 0.5))<\/pre>\n
      Where:<\/p>\n
        \n
      • N = total number of documents<\/li>\n
      • df(qi) = number of documents containing term a<\/li>\n
      • 0.5 constant is added to avoid division by zero, incase if both N and n(qi) are equal and also to avoid negative IDF.<\/li>\n<\/ul>\n
        2. Term Frequency Saturation (TF):<\/h4>\n
        BM25 limits the influence of term repetition\u2014more appearances help, but with diminishing returns.<\/p>\n
        Formula:<\/p>\n
        TF_component = (f(qi, d) \u00d7 (k1 + 1)) \/ (f(qi, d) + k1 \u00d7 (1 - b + b \u00d7 |d|\/avgdl))<\/pre>\n
        Where:<\/p>\n
          \n
        • f(qi, d) = term frequency of qi in document d<\/li>\n
        • |d| = document length<\/li>\n
        • avgdl = average document length<\/li>\n
        • k1 = saturation parameter (typically 1.2-2.0)<\/li>\n
        • b = length normalization (typically 0.75)<\/li>\n<\/ul>\n
          BM25 Parameter Tuning:<\/h4>\n
          k1 (Term Frequency Saturation):<\/h4>\n
          Higher values (1.5-2.0): Less saturation, term frequency matters more
          \nLower values (0.5-1.0): More saturation, diminishing returns kick in sooner<\/p>\n
          b (Length Normalization):<\/strong><\/p>\n
          b=1: Full length normalization
          \nb=0: No length normalization
          \nb=0.75: Balanced approach (most common)<\/p>\n
          3. Document Length Normalization:<\/h4>\n
          Longer documents naturally have a higher probability of containing a query term and having higher term frequencies. The BM25 algorithm penalizes longer documents to prevent them from having an unfair advantage in relevance scoring. This is where document length normalization comes in. The parameter b in the term frequency formula controls the degree of this normalization. If b is set to 1, the document’s length fully normalizes the term frequency. If it’s set to 0, there is no normalization.<\/p>\n
          Full BM25 Formula<\/h4>\n
          The total BM25 score of document D for query Q = {q1, …, qn}:<\/p>\n
          BM25(D,Q) = \u03a3 IDF(qi) \u00d7 TF_component(qi, D)<\/p>\n
          Example Query: \u201cQuantum Physics\u201d<\/p>\n
          Documents:<\/p>\n
            \n
          • D1: Quantum entanglement is a phenomenon in quantum physics. (8 words)<\/li>\n
          • D2: Einstein called quantum entanglement spooky action at a distance. (9 words)<\/li>\n
          • D3: Quantum physics explores the strange world of entanglement. (8 words)<\/li>\n<\/ul>\n
            The above documents are indexed using Inverted index. it keeps track of terms and their document name matching.<\/p>\n
            For instance:
            \nFor term: quantum<\/p>\n
            It can be hashed to properties like\r\nPosting List: [\r\n(docID: D1, term_freq: 2),\r\n(docID: D2, term_freq: 1),\r\n(docID: D3, term_freq: 1)]<\/pre>\n
            Query: quantum physics<\/p>\n
            BM25 Calculation<\/h4>\n
            Step 1: avgdl = (8 + 9 + 8) \/ 3 = 8.33
            \nStep 2: IDF values<\/p>\n
              \n
            • \u00a0“quantum” in all 3 docs \u2192 IDF \u2248 0.133<\/li>\n
            • \u00a0“physics” in 2 docs \u2192 IDF \u2248 0.47<\/li>\n<\/ul>\n
              Step 3: Term frequencies<\/p>\n
                \n
              • D1: quantum=2, physics=1<\/li>\n
              • D2: quantum=1, physics=0<\/li>\n
              • D3: quantum=1, physics=1<\/li>\n<\/ul>\n
                Using k1 = 1.5, b = 0.75:<\/p>\n
                D1 (|d| = 8):
                \n0.133 * 2*(2.5)
                \n———————-
                \n2+1.5\u22c5(1\u22120.75+(0.75 * 8\/8.33)
                \n\u200b
                \n\u200b\u22480.133 * 5\/3.455 \u22480.192<\/p>\n
                Similary Score for “physics” is calculated using the same formula: \u22480.479<\/p>\n
                Total Score for D1 = 0.192 + 0.479 = 0.671
                \nScore(“quantum”) \u2248 0.192
                \nScore(“physics”) \u2248 0.479
                \nTotal: 0.671<\/strong><\/p>\n
                D2 (|d| = 9):<\/p>\n
                Score(“quantum”) \u2248 0.128
                \nScore(“physics”) = 0
                \nTotal: 0.128<\/strong><\/p>\n
                D3 (|d| = 8):<\/p>\n
                Score(“quantum”) \u2248 0.135
                \nScore(“physics”) \u2248 0.479
                \nTotal: 0.614<\/strong><\/p>\n
                Final Ranking (Most Relevant First)<\/p>\n
                1. D1 \u2013 0.671
                \n2. D3 \u2013 0.614
                \n3. D2 \u2013 0.128<\/p>\n
                we know how dense vector works, it does not necessarily be dense if your use case depends on sparse or static we can choose over that also. Now we need to combine both BM25 and dense vector results to a single combine results, well there are many popular algorithm which can do this, we’ll examine one popular algorithm: Reciprocal Rank Fusion.<\/p>\n
                Reciprocal Rank Fusion (RRF):<\/h4>\n
                RRF is a ranking fusion technique that combines multiple ranked lists of documents from different retrieval systems or queries. Instead of relying on the retrieval scores, RRF assigns scores to documents based solely on their rank positions in each list by using the reciprocal of the rank. The final ranking is determined by summing these reciprocal rank scores across all lists. This approach emphasizes documents that appear near the top of any list, improving overall retrieval effectiveness without depending on the original retrieval scores[4].<\/p>\n
                Formula:<\/h5>\n
                RRF_score = \u03a3 (1 \/ (k + rank_i))<\/pre>\n
                Where:<\/p>\n
                k = 60 (standard dampening constant)
                \nrank_i = rank from retrieval system i<\/p>\n
                Complete RRF Example<\/h4>\n
                Step 1: BM25 rank<\/h4>\n\n\n\n\n\n\n\n
                Doc ID<\/th>\nBM25 Score<\/th>\nBM25 Rank<\/th>\n<\/tr>\n<\/thead>\n
                D1<\/td>\n0.671<\/td>\n1<\/td>\n<\/tr>\n
                D3<\/td>\n0.614<\/td>\n2<\/td>\n<\/tr>\n
                D2<\/td>\n0.128<\/td>\n3<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
                Step 2: Simulated Dense Retrieval (Cosine Similarity)<\/h4>\n
                Let\u2019s assume you used an embedding model like `all-MiniLM-L6-v2`, and the cosine similarities for the query “quantum physics” with each document are as follows:<\/p>\n\n\n\n\n\n\n\n
                Doc ID<\/th>\nCosine Similarity<\/th>\nDense Rank<\/th>\n<\/tr>\n<\/thead>\n
                D3<\/td>\n0.94<\/td>\n1<\/td>\n<\/tr>\n
                D1<\/td>\n0.91<\/td>\n2<\/td>\n<\/tr>\n
                D2<\/td>\n0.76<\/td>\n3<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
                These are plausible\u2014D3 has high semantic alignment with both terms, D1 is close behind, D2 is more about “entanglement” and less about “physics.”<\/p>\n
                Step 3: Apply Reciprocal Rank Fusion (RRF)<\/h4>\n
                Formula:<\/p>\n
                RRF = \u2211 i=1 to n 1\/(k + rank_di)<\/pre>\n
                Where:<\/p>\n
                  \n
                • n =number of rankers (2: BM25 and Dense)<\/li>\n
                • k = 60 (commonly used constant to dampen dominance)<\/li>\n<\/ul>\n
                  For each document:<\/p>\n
                  D1<\/p>\n
                    \n
                  • BM25 rank: 1 \u2192 1\/(60 + 1) = 1\/61 \u2248 0.01639<\/li>\n
                  • Dense rank: 2 \u2192 1\/(60 + 2) = 1\/62 \u2248 0.01613<\/li>\n
                  • RRF Score: 0.01639 + 0.01613 = 0.03252<\/li>\n<\/ul>\n
                    D2<\/p>\n
                      \n
                    • BM25 rank: 3 \u2192 1\/(60 + 3) = 1\/63 \u2248 0.01587<\/li>\n
                    • Dense rank: 3 \u2192 1\/(60 + 3) = 1\/63 \u2248 0.01587<\/li>\n
                    • RRF Score: 0.01587 + 0.01587 = 0.03174<\/li>\n<\/ul>\n
                      D3<\/p>\n