{"id":4566,"date":"2025-10-30T18:01:18","date_gmt":"2025-10-30T12:31:18","guid":{"rendered":"https:\/\/blogs.infosys.com\/emerging-technology-solutions\/?p=4566"},"modified":"2025-10-30T18:01:18","modified_gmt":"2025-10-30T12:31:18","slug":"learning-on-the-fly-how-agentic-ai-learns-to-choose-the-right-cybersecurity-tools","status":"publish","type":"post","link":"https:\/\/blogs.infosys.com\/emerging-technology-solutions\/artificial-intelligence\/learning-on-the-fly-how-agentic-ai-learns-to-choose-the-right-cybersecurity-tools.html","title":{"rendered":"Learning on the Fly: How Agentic AI Learns to Choose the Right Cybersecurity Tools"},"content":{"rendered":"

Cybersecurity demands more than reactive defences\u2014it calls for proactive, intelligent systems<\/strong> that can adapt and improve with every interaction. In our earlier blog, Learning on the Fly: User-Driven Adaptation in Small Language Models<\/a>, we explored how Small Language Models (SLMs) can be fine-tuned to incorporate user feedback, transforming them from static utilities into dynamic, evolving assets for cyber defense.<\/p>\n

That work focused on capturing user dissatisfaction signals<\/strong> and feeding them back into SLMs to create a self-learning loop. Building on that foundation, this blog shifts the focus to an agentic AI use case<\/strong>\u2014where models don\u2019t just generate answers but must decide which external tool to use<\/strong> for a given problem.<\/p>\n

The challenge is clear: what happens when an agent picks the wrong tool<\/strong>? Imagine a security analyst asking, \u201cIs this IP malicious?\u201d, but the agent mistakenly chooses a GeoIP Lookup instead of a reputation check. Such errors not only erode user trust but also reduce the system\u2019s operational effectiveness. However, if these decisions are systematically logged, analyzed, and fed back<\/strong> into the learning process, agents can become more accurate over time.<\/p>\n

In this blog, we explore how to log tool-selection correctness in agentic AI systems.<\/strong> By tracking when an agent chooses the right tool versus when it does not, and by applying lightweight rules to determine ground truth, organizations can build feedback loops that extend beyond text outputs\u2014laying the groundwork for self-improving, tool-aware AI agents<\/strong> in cybersecurity and beyond.<\/p>\n

Why This Matters<\/h2>\n

SLMs are efficient, cost-effective, and well-suited for enterprise deployment, but a 1B parameter SML is not inherently intelligent enough to orchestrate tools reliably<\/strong>. By continuously feeding it validated tool decisions<\/strong>, we enable the model to gain expertise in specific operational contexts.<\/strong><\/p>\n

This approach effectively transforms the SML into a domain-specific expert system,<\/strong> much like a Mixture of Experts (MoE),<\/strong> but optimized for enterprise scenarios where resource efficiency and auditability are critical.<\/p>\n

The full demonstration of this pipeline, along with reference code, is available in the public GitHub repository: https:\/\/github.com\/ameya\/agentic-slm-demo.<\/a><\/p>\n

Cybersecurity Application<\/h2>\n

Consider a scenario where a SOC agent supports analysts by choosing the right tool<\/strong> for investigating suspicious IP addresses:<\/p>\n

    \n
  • \u201cWhere is this IP located?\u201d \u2192 correct choice: GeoIP Lookup<\/strong><\/li>\n
  • \u201cWho owns this IP address?\u201d \u2192 correct choice: WHOIS Lookup<\/strong><\/li>\n
  • \u201cIs this IP malicious?\u201d \u2192 correct choice: Reputation Check<\/strong><\/li>\n<\/ul>\n

    Yet, agentic systems don\u2019t always get this right. An agent might mistakenly use a GeoIP lookup when asked about maliciousness\u2014producing an answer that sounds valid but misses the analyst\u2019s real intent.<\/p>\n

    To address this, we can log each interaction<\/strong> in a structured table containing:<\/p>\n

      \n
    • The input prompt,<\/li>\n
    • The tool selected by the SLM,<\/li>\n
    • The expected tool (derived through rules or validation by a stronger LLM),<\/li>\n
    • A correctness label (true\/false).<\/li>\n<\/ul>\n

      Once collected, these logs serve two purposes:<\/p>\n

        \n
      • Performance tracking<\/strong> \u2014 enabling metrics like accuracy, error trends, and confusion matrices of tool choices.<\/li>\n
      • Training data generation<\/strong> \u2014 correctness labels provide ground truth that can be used to fine-tune the SLM, so it learns to avoid repeating the same mistakes.<\/li>\n<\/ul>\n

        This approach transforms tool-choice errors into learning opportunities<\/strong>. While our example highlights cybersecurity tasks, the same method applies broadly to domains such as finance, healthcare, or customer service\u2014anywhere agentic AI must pick the right tool to deliver the right outcome.<\/p>\n

        Steps to Create Long-Term Memory for Agentic SLM<\/h2>\n

        \"\"<\/p>\n

        Step 1: Building the Fine-Tuning Pipeline for Agentic SLMs<\/h3>\n

        \"\"<\/p>\n

        To enable a Small Language Model (SLM) to improve its tool selection intelligence,<\/strong> we need a modular, auditable, and secure fine-tuning pipeline. The goal is to create a loop where model performance improves based on logged tool decisions<\/strong>\u2014particularly where the agent picked the wrong tool.<\/p>\n

        Key Components of the Pipeline:<\/strong><\/p>\n

          \n
        • Prompt & Tool Capture:<\/strong> Each user prompt and the tool chosen by the agent is logged.<\/li>\n
        • Decision Validation:<\/strong> Analysts or automated heuristics mark whether the tool selection was correct<\/strong> (\u2705) or incorrect<\/strong> (\u274c).<\/li>\n
        • Analysis Database:<\/strong> A structured store that keeps a record of prompts, tools, and decision outcomes.<\/li>\n
        • Batch Processing (BAT):<\/strong> A Python application that periodically extracts logs, prepares training datasets, and pushes them for further analysis.<\/li>\n
        • Augmented Data Generation:<\/strong> A large external LLM enriches the dataset by generating synthetic training examples, especially for error cases.<\/li>\n
        • Fine-Tuning the SML:<\/strong> Using LoRA\/QLoRA or PEFT<\/strong> adapters to make the 1B-parameter SML more intelligent without heavy compute costs.<\/li>\n
        • Benchmark & Feedback Loop:<\/strong> Each fine-tuned SML is tested against baseline performance and redeployed, closing the loop.<\/li>\n<\/ul>\n

          Step 2: Logging Agentic Tool Interactions in a Relational Database<\/h3>\n

          Capturing real-world tool selection decisions<\/strong> is the foundation of a self-learning Agentic system. Instead of logging raw session histories or user identifiers, the focus is on prompts, tool metadata, and correctness flags.<\/strong><\/p>\n

          Minimalist Schema:<\/strong><\/p>\n

           <\/p>\n\n\n\n\n\n\n\n\n\n
          Column<\/th>\nType<\/th>\nDescription<\/th>\n<\/tr>\n<\/thead>\n
          id<\/td>\nINT<\/td>\nUnique identifier for the interaction<\/td>\n<\/tr>\n
          prompt<\/td>\nTEXT<\/td>\nUser\u2019s input query<\/td>\n<\/tr>\n
          tool_name<\/td>\nTEXT<\/td>\nName of the selected tool<\/td>\n<\/tr>\n
          tool_desc<\/td>\nTEXT<\/td>\nMetadata\/description of the tool<\/td>\n<\/tr>\n
          decision<\/td>\nBOOL<\/td>\nCorrect (1) \/ Incorrect (0) selection<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

          Example Interaction:<\/strong><\/p>\n

            \n
          • Prompt: \u201cCorrelate login failures across AD logs.\u201d<\/li>\n
          • Tool Chosen: LogCorrelator<\/li>\n
          • Tool Description: Cross-checks failed logins across domains<\/li>\n
          • Decision: \u274c (wrong tool \u2014 should have been AD-Audit)<\/li>\n<\/ul>\n

            This schema ensures we can systematically analyze failure<\/strong> cases without exposing sensitive organizational data.<\/p>\n

            Step 3: Identifying Wrong Tool Decisions<\/h3>\n

            Not every interaction indicates a problem\u2014but when the agent selects the wrong tool,<\/strong> it\u2019s a critical signal for retraining. By analyzing decision logs<\/strong>, we can detect common failure patterns such as:<\/p>\n

              \n
            • Confusion<\/strong>: The SML maps a prompt to a tool with similar keywords but wrong functionality.<\/li>\n
            • Over-generalization:<\/strong> The same tool is chosen for too many unrelated tasks.<\/li>\n
            • Gaps in training data:<\/strong> The SML hasn\u2019t seen enough examples for niche tools.<\/li>\n<\/ul>\n

              These patterns provide the foundation for generating better training data.<\/p>\n

              Step 4: Suppressing Sensitive Metadata<\/h3>\n

              Before preparing datasets, the pipeline must remove or anonymize sensitive organizational metadata.<\/strong> For example:<\/p>\n

                \n
              • Tool names that reveal internal projects.<\/li>\n
              • Paths or commands embedded in tool descriptions.<\/li>\n
              • Proprietary log sources or datasets<\/li>\n<\/ul>\n

                A lightweight PII scrubber can be applied here, similar to cybersecurity chat scenarios, ensuring compliance with GDPR and enterprise data safety.<\/p>\n

                Step 5: Generating Better Training Data Using an External LLM<\/h3>\n

                Once raw logs are cleaned, a large external LLM (e.g., GPT-4) generates better training data:<\/strong><\/p>\n

                  \n
                • Improves the mapping of prompts \u2192 correct tools.<\/strong><\/li>\n
                • Suggests synthetic variations<\/strong> of prompts that could confuse the SML.<\/li>\n
                • Expands coverage for rarely used tools<\/strong>.<\/li>\n<\/ul>\n

                  Sample Prompt for GPT-4 Augmentation:<\/strong><\/p>\n

                  \u201cYou are assisting in training an Agentic SOC assistant. Below is a prompt and the incorrect tool chosen. Suggest the correct tool, and generate 2\u20133 alternate prompt\u2013tool pairs that should be added to improve training coverage.\u201d<\/p><\/blockquote>\n

                  Step 6: Fine-Tuning the SML<\/h3>\n

                  With curated training data, the SLM (1B parameter) is fine-tuned using parameter-efficient techniques:<\/p>\n

                    \n
                  • Base Model:<\/strong> Custom enterprise SLM (1B)<\/li>\n
                  • Technique:<\/strong> LoRA \/ QLoRA \/ PEFT<\/li>\n
                  • Goal:<\/strong> Improve tool orchestration accuracy while keeping compute costs minimal<\/li>\n<\/ul>\n

                    This allows the smaller SLM to evolve into a specialized decision-making expert<\/strong>\u2014a lightweight but intelligent Mixture of Experts (MoE).<\/p>\n

                    Step 7: Testing and Benchmarking the Fine-Tuned Agentic Model<\/h3>\n

                    Fine-tuning only matters if it improves real-world tool selection. Benchmarking involves:<\/p>\n

                    \u2022 Correctness<\/strong>: % of times the right tool is chosen
                    \n\u2022 Efficiency<\/strong>: Reduction in unnecessary tool calls
                    \n\u2022 Coverage<\/strong>: Ability to handle rare or unseen tools
                    \n\u2022 Analyst Feedback<\/strong>: Validation from SOC analysts in real workflows<\/p>\n

                    Sample Test Cases:<\/strong><\/p>\n

                    \u2022 Prompt: \u201cParse suspicious PowerShell commands.\u201d \u2192 Expected Tool: ScriptAnalyzer
                    \n\u2022 Prompt: \u201cDetect Kerberos ticket anomalies.\u201d \u2192 Expected Tool: AD-Audit
                    \n\u2022 Prompt: \u201cSummarize last 24 hours of IDS alerts.\u201d \u2192 Expected Tool: AlertSummarizer<\/p>\n

                    Over time, repeated cycles of logging, cleaning, and retraining help the SML mature into a trustworthy agentic expert in enterprise environments.<\/p>\n

                    Agentic Context Engineering: Evolving the Agent\u2019s Memory and Strategy<\/h2>\n

                    While fine-tuning improves the model\u2019s tool selection accuracy, Agentic Context Engineering (ACE)<\/strong> ensures that the agent evolves its decision-making playbook<\/strong> over time\u2014without retraining. This involves dynamically updating the agent\u2019s context with:<\/p>\n

                      \n
                    • Successful strategies<\/strong> from past interactions<\/li>\n
                    • Failure cases<\/strong> and their corrected outcomes<\/li>\n
                    • Domain-specific heuristics<\/strong> and tool usage patterns<\/li>\n<\/ul>\n

                      Instead of relying solely on static prompts or retrained weights, ACE treats context as a living memory<\/strong>\u2014a structured, modular, and editable layer that guides the agent\u2019s behavior.<\/p>\n

                      Prompt Template for Agentic Context Engineering<\/h3>\n

                      \"\"<\/p>\n

                      Cybersecurity Example: Hallucination vs. Context-Engineered Response<\/strong><\/p>\n

                      \"\"<\/p>\n

                      Without Proper Context (Hallucination)<\/strong><\/p>\n

                      Prompt:<\/strong><\/p>\n

                      \u201cIs this IP address malicious?\u201d<\/p><\/blockquote>\n

                      Agent Response:<\/strong><\/p>\n

                      \u201cGeoIP Lookup Result: The IP is located in Frankfurt, Germany.\u201d<\/p><\/blockquote>\n

                      Issue:<\/strong>
                      \nThe agent incorrectly assumes location implies maliciousness. It selects the wrong tool (GeoIP) and gives a misleading answer.<\/p>\n

                      With Agentic Context Engineering<\/strong>
                      \nUpdated Context:<\/strong><\/p>\n

                        \n
                      • Task: Identify malicious IPs<\/li>\n
                      • Previous failure: GeoIP used instead of Reputation Check<\/li>\n
                      • Correct tool: ThreatIntelReputation<\/li>\n
                      • Strategy: Always use reputation-based tools for maliciousness queries<\/li>\n<\/ul>\n

                        Prompt:<\/strong><\/p>\n

                        \u201cIs this IP address malicious?\u201d<\/p><\/blockquote>\n

                        Agent Response:<\/strong><\/p>\n

                        \u201cThreatIntelReputation Result: The IP has been flagged in 3 threat feeds for malware distribution. Risk Score: High.\u201d<\/p><\/blockquote>\n

                        Outcome:<\/strong>
                        \nThe agent selects the correct tool and provides a relevant, actionable response\u2014thanks to context engineering.<\/p>\n

                        Conclusion<\/h2>\n

                        As enterprises move toward Agentic AI systems, the ability to learn from past actions becomes as important as generating new responses. By systematically logging tool choices, validating correctness, and feeding that data back into a fine-tuning loop, we give Small Language Models (SLMs) the ability to evolve into specialized experts.<\/p>\n

                        This approach transforms a lightweight 1B SLM from a generic assistant into a domain-aware Mixture of Experts (MoE), capable of making smarter, context-driven decisions. Unlike static models, the agent grows with every interaction\u2014adapting not only to user prompts but also to the operational tools of the enterprise.<\/p>\n

                        In doing so, we bridge the gap between efficiency and intelligence: large LLMs provide synthetic data generation and reasoning depth, while fine-tuned SLMs deliver cost-effective, reliable decision-making at scale. Over time, this creates a self-learning, auditable, and trustworthy AI layer that enterprises can safely embed into their SOCs and business workflows.<\/p>\n

                        Complementing this, Agentic Context Engineering (ACE)<\/strong> introduces a dynamic memory layer that evolves with each interaction. ACE reduces hallucinations and improves decision-making by treating context as a modular, editable playbook\u2014updated through generation, reflection, and curation. This enables agents to adapt without retraining, making them more resilient, accurate, and aligned with domain-specific goals.<\/p>\n","protected":false},"excerpt":{"rendered":"

                        Cybersecurity demands more than reactive defences\u2014it calls for proactive, intelligent systems that can adapt […]<\/p>\n","protected":false},"author":504,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"inline_featured_image":false,"footnotes":""},"categories":[4,10],"tags":[60,186],"coauthors":[181],"class_list":["post-4566","post","type-post","status-publish","format-standard","hentry","category-artificial-intelligence","category-predictive-cyber-security","tag-artificial-intelligence","tag-generative-ai"],"acf":[],"_links":{"self":[{"href":"https:\/\/blogs.infosys.com\/emerging-technology-solutions\/wp-json\/wp\/v2\/posts\/4566","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.infosys.com\/emerging-technology-solutions\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.infosys.com\/emerging-technology-solutions\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.infosys.com\/emerging-technology-solutions\/wp-json\/wp\/v2\/users\/504"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.infosys.com\/emerging-technology-solutions\/wp-json\/wp\/v2\/comments?post=4566"}],"version-history":[{"count":6,"href":"https:\/\/blogs.infosys.com\/emerging-technology-solutions\/wp-json\/wp\/v2\/posts\/4566\/revisions"}],"predecessor-version":[{"id":4581,"href":"https:\/\/blogs.infosys.com\/emerging-technology-solutions\/wp-json\/wp\/v2\/posts\/4566\/revisions\/4581"}],"wp:attachment":[{"href":"https:\/\/blogs.infosys.com\/emerging-technology-solutions\/wp-json\/wp\/v2\/media?parent=4566"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.infosys.com\/emerging-technology-solutions\/wp-json\/wp\/v2\/categories?post=4566"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.infosys.com\/emerging-technology-solutions\/wp-json\/wp\/v2\/tags?post=4566"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/blogs.infosys.com\/emerging-technology-solutions\/wp-json\/wp\/v2\/coauthors?post=4566"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}