Full Transcript
Loading transcript…
Optica Executive Forum 2026: Complete Keynote Richard Ho
Watch with subtitles, summary & AI chat
Add the free Subkun extension — works directly on YouTube.
- Watch
- Subtitles
- Summary
- Ask AI
This video is for AI researchers, hardware engineers, and industry professionals interested in the future of AI infrastructure and its technological demands.
TL;DR
Richard Ho from OpenAI discusses the critical role of hardware and infrastructure in advancing AI capabilities. He highlights how increased context, reasoning, and tool use in AI models demand significant advancements in memory, compute, and network interconnects, emphasizing the growing importance of photonics and optical communication for future AI development.
Key Takeaways
- AI capabilities are advancing exponentially, moving beyond just larger models to enhanced reasoning, longer context windows, and agentic behavior.
- The increasing complexity of AI workloads, including agents and tool use, significantly stresses network infrastructure, storage, and orchestration systems.
- Longer context windows in AI models directly translate to a greater demand for memory, specifically impacting the size of the key-value cache.
- AI reasoning and complex task execution require more computational operations (FLOPS) per prompt, demanding more powerful and efficient processing.
- The growth of AI, particularly with agents and coding capabilities, leads to longer, iterative user sessions, necessitating highly parallelized and reliable compute clusters.
- Scaling AI infrastructure faces challenges due to the exponential growth in compute demand, far outpacing traditional Moore's Law advancements.
- Reliability becomes a major concern as AI clusters grow larger; the increased number of components raises the probability of system failures.
- Optics and optical communication technologies are identified as critical future components for supporting the demanding infrastructure needs of advanced AI.
In This Video
- 00:00Introduction to Optica Executive Forum
Highlights from the Optica Executive Forum at OFC, featuring sessions from leading industry figures. Feedback is encouraged.
- 00:39Keynote Speaker Introduction: Richard Ho
Mark Feiler introduces Richard Ho from OpenAI as the keynote speaker, highlighting his extensive experience in AI and hardware.
- 01:40Richard Ho's Role at OpenAI
Richard Ho leads hardware at OpenAI, focusing on co-designing AI models with compute systems for frontier AI workloads.
- 03:14AI Hardware Evolution and Future
Richard Ho discusses OpenAI's perspective on AI hardware, its future direction, and the critical role of optical technologies.
- 03:36Rapid AI Capability Growth
AI capabilities are advancing exponentially, with rapid developments in models, context length, reasoning, and agents like Codex.
- 05:19Infrastructure Demands of AI
AI growth requires more than just larger models; it demands infrastructure supporting longer context, reasoning, and agentic capabilities.
- 06:54Infrastructure Challenges: Context and Reasoning
Longer context and advanced reasoning increase memory needs, inference time, and computational demands, stressing current infrastructure.
- 08:05Infrastructure Challenges: Tool Use and Agents
Tool use and agents create network traffic, storage demands, and reliability concerns due to increased inter-component communication.
- 09:07Scaling Laws and Compute Growth
AI model intelligence scales with compute. Training compute grows rapidly, exceeding Moore's Law, creating a significant infrastructure gap.
Questions & Answers
Who is Richard Ho and what does he lead at OpenAI?
Richard Ho leads hardware at OpenAI. He works on the intersection of AI models and the compute systems needed to run them at scale.
What is OpenAI's focus regarding AI hardware and infrastructure?
OpenAI focuses on co-designing hardware, memory, interconnect, and system architecture around the demands of frontier AI workloads, not just building bigger infrastructure.
How has AI model capability evolved recently according to Richard Ho?
AI models have evolved rapidly, introducing capabilities like longer context, coding abilities, reasoning with chains of thought, and agents that use tools.
What are the infrastructure implications of longer context in AI models?
Longer context, reaching millions of tokens, significantly increases the need for memory, specifically the key-value cache used for analysis.
How does tool use by AI agents impact infrastructure?
Tool use by AI agents increases round trips, causing stress on the network, storage, and orchestration due to the back-and-forth communication.
What are the main challenges with scaling AI infrastructure?
Scaling AI infrastructure faces challenges with increased memory needs, more compute operations per prompt, network stress from tool use, and reliability concerns in large, parallel clusters.
Key Terms
- Frontier AI workloads — These are the most advanced and demanding artificial intelligence tasks, requiring significant computational resources and specialized hardware.
- TPU (Tensor Processing Unit) — Google's custom-designed hardware accelerator for machine learning, optimized for tasks like training and running neural networks.
- Key-Value Cache (KV Cache) — A memory component used in AI models to store and quickly retrieve previously computed information, crucial for efficient inference.
- Agentic AI — AI systems designed to act autonomously, using tools and making decisions to achieve specific goals or complete complex tasks.