Episode #400 from 1:10:41
AI and useful compute per watt
[inaudible 01:10:38]. It seems that training LLMs efficiently is a big focus for xAI. First of all, what's the limit of what's possible in terms of efficiency? There's this terminology of useful productivity per watt. What have you learned from pushing the limits of that? Well, I think it's helpful, the tools of physics are very powerful and can be applied I think to really any arena in life. It's really just critical thinking. For something important you need to reason with from first principles and think about things in the limit one direction or the other. So in the limit, even at the Kardashev scale, meaning even if you harness the entire power of the sun, you'll still care about useful compute per watt. That's where I think, probably where things are headed from the standpoint of AI is that we have a silicon shortage now that will transition to a voltage transformer shortage in about a year. Ironically, transformers for transformers. You need transformers to run transformers.
Why this moment matters
[inaudible 01:10:38]. It seems that training LLMs efficiently is a big focus for xAI. First of all, what's the limit of what's possible in terms of efficiency? There's this terminology of useful productivity per watt. What have you learned from pushing the limits of that? Well, I think it's helpful, the tools of physics are very powerful and can be applied I think to really any arena in life. It's really just critical thinking. For something important you need to reason with from first principles and think about things in the limit one direction or the other. So in the limit, even at the Kardashev scale, meaning even if you harness the entire power of the sun, you'll still care about useful compute per watt. That's where I think, probably where things are headed from the standpoint of AI is that we have a silicon shortage now that will transition to a voltage transformer shortage in about a year. Ironically, transformers for transformers. You need transformers to run transformers.