Andrej Karpathy — “We’re summoning ghosts, not building animals”

https://www.youtube.com/watch?v=lXUZvyajciY

Summary

TL;DR — Andrej Karpathy argues that the current hype around AI agents is overblown, suggesting that true progress will take a decade due to fundamental limitations like lack of continual learning and multimodality. He contrasts the "ghosts" of digital AI with evolved animals, emphasizing that current AI training methods, while powerful, are not replicating biological intelligence.

Key points

• Karpathy believes the "year of agents" is an overprediction; it's more accurately the "decade of agents" due to significant work still needed.

• Current AI agents lack crucial capabilities like continual learning, multimodality, and robust memory, making them "cognitively lacking."

• He draws a parallel between evolution's role in building biological intelligence and the current AI paradigm, noting that AI is building "ghosts" by mimicking human data, not replicating the evolutionary process.

• Karpathy critiques the reliance on reinforcement learning in games as a path to AGI, advocating instead for agents that interact with the real digital world, like operating web pages.

• He highlights the importance of foundational steps like large language models (LLMs) providing powerful representations before tackling complex agent behaviors.

• The discussion touches on the limitations of current AI, such as their inability to generalize beyond their training data manifold and the potential for "model collapse" due to over-reliance on existing knowledge.

Takeaway — While AI agents show impressive early capabilities, significant fundamental challenges remain, requiring a decade of focused development to achieve robust, human-like intelligence.

摘要 / Summary (zh-CN)

简而言之 — Andrej Karpathy 认为当前对人工智能代理的炒作有些言过其实，他认为由于持续学习和多模态等基本限制，真正的进展还需要十年时间。他将当前 AI 的“幽灵”与进化出的动物进行了对比，并强调当前的 AI 训练方法虽然强大，但并未复制生物智能。

要点

• Karpathy 认为“代理之年”的说法有些过早预测；由于仍有大量工作要做，更准确的说法是“代理之十年”。

• 当前的 AI 代理缺乏持续学习、多模态和强大的记忆等关键能力，使其“认知能力不足”。

• 他将进化在构建生物智能中的作用与当前 AI 范式进行了类比，并指出 AI 通过模仿人类数据来构建“幽灵”，而非复制进化过程。

• Karpathy 批评了在游戏中依赖强化学习作为通往通用人工智能（AGI）的途径，他主张应开发与真实数字世界交互的代理，例如操作网页。

• 他强调了大型语言模型（LLM）提供强大表征等基础步骤的重要性，然后再解决复杂的代理行为问题。

• 讨论还涉及当前 AI 的局限性，例如它们无法超越训练数据的流形进行泛化，以及可能因过度依赖现有知识而导致的“模型崩溃”。

核心启示 — 尽管 AI 代理已展现出令人印象深刻的早期能力，但仍存在重大的基础性挑战，需要十年专注的开发才能实现稳健、类似人类的智能。