Is the dream of AGI dying?

The following is a conversation all about the state of the art in artificial intelligence, including some of the exciting technical breakthroughs and developments in AI that happened over the past year, and some of the interesting things we think might happen this upcoming year. At times, it does get super technical, but we do try to make sure that it remains accessible to folks outside the field without ever dumbing it down. It is a great honor and pleasure to be able to do this kind of episode with two of my favorite people in the AI community, Sebastian Raschka and Nathan Lambert. They are both widely respected machine learning researchers and engineers who also happen to be great communicators, educators, writers, and X posters. Sebastian is the author of two books I highly recommend for beginners and experts alike. First is Build a Large Language Model from Scratch, and Build a Reasoning Model from Scratch. I truly believe in the machine learning and computer science world, the best way to learn and understand something is to build it yourself from scratch. Nathan is the post-training lead at the Allen Institute for AI, and author of the definitive book on reinforcement learning from human feedback. Both of them have great X accounts, great Substacks. Sebastian has courses on YouTube, Nathan has a podcast. And everyone should absolutely follow all of those. This is the Lex Fridman podcast.

So I think one useful lens to look at all this through is the so-called DeepSeek moment. This happened about a year ago in January 2025, when the open weight Chinese company DeepSeek released DeepSeek R1 that I think it's fair to say surprised everyone with near or at state-of-the-art performance, with allegedly much less compute for much cheaper. And from then to today, the AI competition has gotten insane, both on the research level and the product level. It's just been accelerating. Let's discuss all of this today, and maybe let's start with some spicy questions if we can. Who's winning at the international level? Would you say it's the set of companies in China or the set of companies in the United States? Sebastian, Nathan, it's good to see you guys. So Sebastian, who do you think is winning?

What model do you think won 2025, and what model do you think is going to win '26? I think in the context of consumer chatbots, the question is: are you willing to bet on Gemini over ChatGPT? Which I would say in my gut feels like a bit of a risky bet because OpenAI has been the incumbent and there are so many benefits to that in tech. I think the momentum in 2025 was on Gemini's side, but they were starting from such a low point. RIP Bard and those earlier attempts. I think huge credit to them for powering through the organizational chaos to make that happen. But also it's hard to bet against OpenAI because they always come off as so chaotic, but they're very good at landing things.

We didn't really mention programming. That's another use case that a lot of people deeply care about. I use basically half-and-half Cursor and Claude Code, because I find them to be fundamentally different experiences and both useful. You program quite a bit— ...so what do you use? What's the current vibe? So, I use the Codeium plugin for VS Code. You know, it's very convenient. It's just a plugin, and then it's a chat interface that has access to your repository. I know that Claude Code is a bit different. It is a bit more agentic. It touches more things; it does the whole project for you. I'm not quite there yet where I'm comfortable with that because maybe I'm a control freak, but I still like to see what's going on. Codeium is the sweet spot for me right now where it is helping me, but it is not taking over completely.

All right. So we just talked about a bunch of the closed-weight models. Let's talk about the open ones. Tell me about the landscape of open LLM models. Which are interesting ones? Which stand out to you and why? We already mentioned DeepSeek. Do you wanna see how many we can name off the top of our head?

And it may be useful to step back and talk about transformer architecture in general. Yeah, so maybe we should start with GPT-2 architecture, the transformer that was derived from the "Attention Is All You Need" paper.

I guess the big question here is—we talked quite a bit here on the architecture behind the pre-training—are the scaling laws holding strong across pre-training, post-training, inference, context size, data, and synthetic data? I'd like to start with the technical definition of a scaling law-

I think this might be a good place to define pre-training, mid-training, and post-training. So, pre-training is the classic training one next token prediction at a time. You have a big corpus of data. Nathan probably also has very interesting insights there because of OLMo 3. A big portion of the paper focuses on the right data mix. So, pre-training is essentially just training across entropy loss, training on next token prediction on a vast corpus of internet data, books, papers and so forth. It has changed a little bit over the years in the sense people used to throw in everything they can. Now, it's not just raw data. It's also synthetic data where people rephrase certain things. So synthetic data doesn't necessarily mean purely AI-made-up data.

Yeah, there is a sense that we, together as a civilization, each individually have to find that Goldilocks zone. And in the programming context as developers. Now, we've had this fascinating conversation that started with pre-training and mid-training. Let's get to post-training. There's a lot of fun stuff in post-training. So, what are some of the interesting ideas in post-training? The biggest one from 2025 is learning this reinforcement learning with verifiable rewards, RLVR. You can scale up the training there, which means doing a lot of this kind of iterative generate-grade loop, and that lets the models learn both interesting behaviors on the tool use and software side. This could be searching, running commands on their own and seeing the outputs, and then also that training enables this inference-time scaling very nicely. It just turned out that this paradigm was very nicely linked, where this kind of RL training enables inference-time scaling. But inference-time scaling could have been found in different ways. So, it was kind of this perfect storm where the models change a lot, and the way that they're trained is a major factor in doing so.

I was wondering if we could take a bit of a tangent and talk about education and learning. If you're somebody listening to this who's a smart person interested in programming and interested in AI, I presume building something from scratch is a good beginning. Can you just take me through what you would recommend people do? I would personally start, like you said, by implementing a simple model from scratch that you can run on your computer. The goal of building a model from scratch is not to have something you use every day for your personal projects. It's not going to be your personal assistant replacing an existing open-weight model or ChatGPT. It's to see exactly what goes into the LLM, what exactly comes out of the LLM, and how pre-training works on your own computer. And then you learn about pre-training, supervised fine-tuning, and the attention mechanism.

Can you describe 9/9/6 as a culture? I believe you could say it was invented in China and adopted in Silicon Valley. What's 9/9/6? It's 9:00 AM to 9:00 PM— six days a week.

Yeah, but there's also a feeling of fervor that's building, especially in Silicon Valley, aligned with the scaling laws idea. There's this hype where the world will be transformed in a scale of weeks and you want to be at the center of it. I have the great fortune of having conversations with a wide variety of human beings, and I get to see all these bubbles and echo chambers across the world. It's fascinating to see how we humans form them. I think it's fair to say that Silicon Valley is a kind of echo chamber, a kind of silo and bubble. I think bubbles are actually really useful and effective. It's not necessarily a negative thing because you can be ultra-productive. It could be the Steve Jobs reality distortion field, because you just convince each other the breakthroughs are imminent, and by convincing each other of that, you make the breakthroughs imminent.

Yeah. Okay, let's... we talked a lot about many things, certainly about what was exciting last year. But this year, one of the things you guys mentioned that's exciting is the scaling of text diffusion models and just a different exploration of text diffusion. Can you talk about what that is and what possibilities it holds? So, different kinds of approaches than the current LMs? Yeah, so we talked a lot about the transformer architecture and the autoregressive transformer architecture specifically, like GPT. And it doesn't mean no one else is working on anything else. People are always on the lookout for the next big thing, because I think it would be almost stupid not to. Sure, right now the transformer architecture is the thing and it works best, but it's always a good idea to not put all your eggs into one basket. People are developing alternatives to the autoregressive transformer. One of them would be, for example, text diffusion models.

So what's the future of tool use this year and in the coming years? Do you think there's going to be a lot of developments there, and how that's integrated into the entire stack? I do think right now it's mostly on the proprietary LLM side, but we will see more of that in open-source tooling. It is a huge unlock because then you can really outsource certain tasks from just memorization to actual computation—you know, instead of having the LLM memorize what is 23 plus 5, just use a calculator.

So, continual learning—this is a longstanding topic and an important problem. I think that increases in importance as the cost of training models goes up. So can you explain what continual learning is and how important it might be this year and in the coming years to make progress? This relates a lot to this kind of SF zeitgeist of: what is AGI, Artificial General Intelligence, and what is ASI, Artificial Superintelligence? What are the language models that we have today capable of doing? I think language models can solve a lot of tasks, but a key milestone for the AI community is when AI can replace any remote worker, taking in information and solving digital tasks. The limitation is that a language model will not learn from feedback the same way an employee does. If you hire an editor, they might mess up, but you will tell them, and they don't do it again.

We haven't really mentioned it much, but implied in this discussion is context length as well. Is there a lot of innovations that's possible there? I think the colloquially accepted thing is that it's a compute and data problem. Sometimes there are small architecture things, like attention variants. We talked about hybrid attention models, which is essentially if you have what looks like a state space model within your transformer. Those are better suited because you have to spend less compute to model the furthest along token. But those aren't free because they have to be accompanied by a lot of compute or the right data. How many sequences of 100,000 tokens do you have in the world, and where do you get these? It just ends up being pretty expensive to scale them.

I think we should say that there's a lot of exciting stuff going on in the AI space. My mind has recently been really focused on robotics, so today we almost entirely didn't talk about robotics. There's a lot of stuff on image generation and video generation. I think it's fair to say that the most exciting research work in terms of intensity and fervor is in the LLM space, which is why I think it's justified for us to focus on the LLMs we're discussing. But it'd be nice to bring in certain things that might be useful. For example, world models—there's growing excitement about that. Do you think there will be any use in this coming year for world models in the LLM space? Also with LLMs, what's an interesting thing here is I think if we unlock more LLM capabilities, it also automatically unlocks all the other fields because it makes progress faster. Because, you know, a lot of researchers and engineers use LLMs for coding. So even if they work on robotics, if you optimize these LLMs that help with coding, it pays off. But then yes, world models are interesting. It's basically where you have the model run a simulation of the world—like a little toy version of the real thing—which can unlock capabilities like data the LLM is not aware of. It can simulate things. I think LLMs happen to work well by pre-training and doing next-token prediction, but we could do this in a more sophisticated way.

Let's talk about timelines specifically: timelines to AGI or ASI. Is it fair, as a starting point, to say that nobody really agrees on the definitions of AGI and ASI? I think there's a lot of disagreement, but I've been getting pushback where people say it is something that could reproduce most digital economic work. The remote worker is a fairly reasonable example. I think OpenAI's definition is somewhat related to that—an AI that can do a certain number of economically valuable tasks—which I don't really love as a definition, but it could be a grounding point. Language models today, while immensely powerful, are not this remote worker drop-in. There are things an AI could do that are way harder than remote work, like solving a...

I think software engineering will be driven more to system design and goals of outcomes, where I do think software is largely going to be... I think this has been happening over the last few weeks, where people have gone from a month ago saying, "Oh yeah, agents are kind of slop," which is a famous Karpathy quote, to the industrialization of software when anyone can just create software with their fingerprints. I do think we are closer to that side of things, and it takes direction and understanding how the systems work to extract the best from the language models. And I think it's hard to accept the gravity of how much is going to change with software development and how many more people can do things without ever looking at the code. I think what's interesting is to think about whether these systems will be independent, in the sense that while I have no doubt that LLMs will at some point solve coding in the way calculators solve calculating, right? At some point, humans developed a tool that you never need a human to calculate that number for; you just type it in, and it's an algorithm. I think that's the same probably for coding. But the question isn't... I think what will happen is you will just say, "Build that website," and it will make a really good website, and then you maybe refine it. But will it do things independently where...

...pulling information from the sparse internet, the non-information-dense thing where there's no better version that exists. You make it from scratch almost. And if it does exist, it's full of—speaking of Disney World—ad slop. Like any city in the world, if you ask "what are the top 10 things to do?" An LLM is just way better to ask... ...Than anything on the internet.

Do you think there'll be some crazy big moves this year business-wise? Like Google or Apple acquiring Anthropic or something like this? Dario will never sell, but we are starting to see some types of consolidation, with Groq being valued at $20 billion and Scale AI for almost 30 billion. There are countless other deals structured in a way that is actually detrimental to the Silicon Valley ecosystem—these licensing deals where not everybody gets brought along, rather than a full acquisition that benefits the rank-and-file employees by getting their stock vested. That's a big issue for Silicon Valley culture to address because the startup ecosystem is the lifeblood. If you join a startup, even if it's not that successful, your startup very well might get acquired at a cheap premium and you'll get paid out for your equity.

You think 10 years from now some of the frontier model companies are still around? Anthropic, OpenAI? I definitely don't see it to be a winner-takes-all unless there truly is some algorithmic secret that one of them finds that lets this flywheel. Because the development path is so similar for all of them. Google and OpenAI have all the same products, and Anthropic's more focused, but when you talk to people, it sounds like they're solving a lot of the same problems. So I think... there's offerings that'll spread out. It's a very big cake that's being made that people are going to take money out of.

Do you like the 2025 America's AI Action Plan? That includes open source stuff. The White House AI Action Plan includes a dedicated section titled "Encourage Open-Source and Open-Weight AI," defining such models and arguing they have unique value for innovation and startups. Yeah. I mean, the AI Action Plan is just a plan, but I think it's maybe the most coherent policy document that has come out of the administration, and I hope that it largely succeeds. I know people that have worked on it. The challenge is taking policy and making it real, and I have no idea how to do this as an AI researcher, but largely a lot of things in that were very real. There's a huge build-out of AI in the country, and while there are issues people hear about, from water use to whatever, we should be able to build things in this country without ruining places in the process. It's worthwhile to spend energy on.

On the hardware side, we mentioned NVIDIA a bunch of times. Do you think Jensen and NVIDIA are going to keep winning? I think they have the downside that they have to iterate a lot and manufacture a lot. And what they're doing—they do innovate, but I think there's always the chance that there is someone who does something fundamentally different, who gets very lucky and then does something. But the problem is, I think, adoption. You know, the moat of NVIDIA is probably not just the GPU; it's more like the CUDA ecosystem, and that has evolved over two decades. I mean, even back when I was a grad student, I was in a lab doing biophysical simulations, molecular dynamics, and we had a Tesla GPU back then just for the computations. It was fifteen years ago now.

Which is an insane thing to believe—to believe scaling before scaling has any indication that it's going to materialize. Again, singular figures. Speaking of which, 100 years from now, this is presumably post-singularity, whatever the singularity is. When historians look back at our time now, what technological breakthroughs would they really emphasize as the breakthroughs that led to the singularity? So far we have Turing to today, which is 80 years. I think it would still be computing, like the umbrella term "computing." I don't necessarily think that even 100 or 200 years from now it would be AI. It could still well be computers, you know? We are now taking better advantage of computers, but it's the fact of computing.