Mat: Hey Sara, as an investment professional, I’ve been closely following developments in Generative AI. Last year, there were quite a few articles discussing the challenges around the scarcity of high-quality learning datasets. From an investment perspective, this got me thinking—won’t this scarcity slow down progress in AI and potentially impact growth opportunities in this sector?
Sara: That’s a great question, Mat, and you’re right—data scarcity has been a significant concern for traditional AI models. High-quality datasets are critical for pre-training large models, but they’re increasingly hard to come by, especially in specialized fields like healthcare or confidential enterprise data. However, this challenge has led to a promising new direction in AI—what we call Agentic AI.
Mat: Agentic AI? I’m not familiar with that. How does it work?
Sara: Agentic AI is a shift from relying solely on large, pre-trained models to incorporating dynamic, reasoning-based systems that learn iteratively during inference. Instead of depending on vast amounts of pre-collected data, Agentic AI breaks tasks into smaller subtasks and processes them step-by-step using multiple agents. These agents collaborate and refine their outputs in real-time. Essentially, it’s AI that thinks more like a problem solver rather than just a pattern recognizer.
Mat: That sounds intriguing. Can you give me a real-world example?
Sara: Of course. Think about an autonomous vehicle. Traditional AI systems might process sensor data in a single step to decide whether to stop, turn, or accelerate. In contrast, Agentic AI would involve multiple agents: one analyzing real-time sensor data, another evaluating traffic patterns, and yet another refining the decision based on previous iterations. They’re working together, iteratively improving the outcome with every step.
Mat: That’s fascinating. But doesn’t that mean these systems need even more data?
Sara: Not necessarily more data, but diverse and well-organized data. Agentic AI thrives on multimodal datasets, which combine structured and unstructured data types. For example, in healthcare, an AI might need to integrate medical images, patient histories, and clinical notes. Enterprises also hold vast untapped datasets, like internal logs or meeting transcripts, which can be highly valuable. The key is organizing and making these datasets accessible while maintaining privacy and security.
Mat: I imagine that’s easier said than done, especially with data privacy regulations.
Sara: Exactly. Enterprises face significant challenges in integrating and securing these datasets. Confidentiality concerns often keep valuable data siloed. However, techniques like federated learning and differential privacy allow AI models to train on sensitive data without centralizing it. For instance, hospitals could use federated learning to train AI on patient data stored locally at different institutions, ensuring privacy while gaining broader insights. These approaches make it possible for Agentic AI to work with previously untapped information securely.
Mat: Okay, I see how it addresses the data scarcity issue, but what about the computational side? Doesn’t iterative reasoning require a lot of compute power?
Sara: You’re spot on. Agentic AI does introduce substantial computational demands, especially during inference. Unlike traditional AI, where inference is a single forward pass, Agentic AI requires multiple passes of computation. Each agent refines its output and collaborates with others, which increases both compute intensity and memory requirements. For real-time applications, like autonomous vehicles, this adds latency challenges that we need to address.
Mat: That’s a lot to handle. How do they overcome these computational hurdles?
Sara: It’s a mix of hardware and software innovations. On the hardware side, inference chips like NVIDIA’s Thor and Orin are designed to handle heavy workloads with energy efficiency. These chips are purpose-built for tasks like iterative reasoning in edge devices. Advanced memory systems, like High Bandwidth Memory (HBM), help store and process large amounts of intermediate data, reducing bottlenecks. On the software side, optimizations like TensorRT streamline inference processes to reduce latency. And in edge computing scenarios, distributing workloads between local devices and nearby servers can help balance the load without compromising performance.
Mat: So, edge computing plays a big role in this?
Sara: Absolutely. For applications like autonomous vehicles, latency is critical. You can’t have the AI waiting for instructions from a remote cloud server. Edge computing allows devices to perform inference locally while offloading non-urgent computations to nearby servers. This approach combines speed with computational efficiency. For instance, in healthcare, edge devices like portable diagnostic tools can process data locally and only share summarized results for further analysis.
Mat: That’s a lot of complexity! Given all these challenges, how can companies justify using Agentic AI?
Sara: That’s the crux of it—Agentic AI needs to deliver value that matches its complexity and costs. It’s best suited for high-value applications where iterative reasoning and multi-agent collaboration make a transformative impact. Take healthcare, for instance. Diagnosing diseases, planning treatments, or predicting patient outcomes are complex tasks that require integrating multiple data types. Agentic AI excels here because the stakes are high, and the potential ROI—both in financial terms and improved patient outcomes—justifies the investment.
Mat: Can you elaborate on the healthcare example? How exactly would Agentic AI work there?
Sara: Sure. Let’s take cancer diagnostics. A traditional AI model might analyze an X-ray image and suggest a diagnosis. An Agentic AI system, however, would go further. One agent might analyze the image, another might cross-reference the patient’s medical history, and a third could compare the findings against the latest medical research. These agents collaborate and iterate, refining the diagnosis until they reach the most accurate conclusion. This iterative, multi-agent approach can significantly improve diagnostic accuracy.
Sara: But this Agentic AI approach isn’t limited to healthcare. It’s a hugely scalable model, especially for general AI reasoning. The real key lies in equipping agents with domain expertise through training on unique datasets. When agents are trained on specialized and diverse data sources, they elevate the iterative process to achieve even more sophisticated reasoning capabilities. This enables the system to operate in an autonomous, self-recurring manner, refining itself over time. The first step, though, is crucial: tapping into untapped data sources and transforming them into organized, actionable learning databases. Enterprises and industries sitting on vast but underutilized datasets will find immense value by unlocking these resources for Agentic AI training.
Mat: That makes sense. It sounds like Agentic AI is carving out a niche in areas where traditional AI struggles.
Sara: Absolutely. As Ilya Sutskever, one of the co-founders of OpenAI, has speculated, Agentic AI represents more than just a technical advancement. He suggests that this approach could eventually bring AI closer to human-like reasoning and problem-solving. It’s not just about performing tasks—it’s about understanding context, adapting dynamically, and potentially achieving a form of self-awareness. This is why Agentic AI is seen as a critical branch of AI’s future evolution.
Mat: Ilya Sutskever’s perspective adds a lot of weight to this. Do you think this could redefine how we think about AI investment opportunities?
Sara: Definitely. Agentic AI, alongside generative AI, opens up entirely new verticals for growth. Generative AI still has immense potential and remains in its early stages, so there’s plenty of room for investment and innovation. For those concerned about the scalability of traditional models, Agentic AI provides a complementary path forward—one that could redefine industries like healthcare, robotics, and beyond. It’s an exciting time to be looking at AI from both a technological and an investment standpoint.
Mat: It’s an exciting direction for sure. Thanks for breaking it down, Sara. I feel like I have a much better understanding now.
Sara: You’re welcome, Mat. It’s an evolving field, but one with immense potential. I’m sure we’ll see even more groundbreaking developments in the years to come.