AI dangers

The following is a conversation with Guillaume Verdon, the man behind the previously anonymous account @BasedBeffJezos on X. These two identities were merged by a doxxing article in Forbes titled, Who Is @BasedBeffJezos, The Leader Of The Tech Elite's E/Acc Movement? So let me describe these two identities that coexist in the mind of one human. Identity number one, Guillaume, is a physicist, applied mathematician, and quantum machine learning researcher and engineer receiving his PhD in quantum machine learning, working at Google on quantum computing, and finally launching his own company called Extropic that seeks to build physics-based computing hardware for generative AI. Identity number two, Beff Jezos on X is the creator of the effective accelerationism movement, often abbreviated as e/acc, that advocates for propelling rapid technological progress as the ethically optimal course of action for humanity. For example, its proponents believe that progress in AI is a great social equalizer, which should be pushed forward. e/acc followers see themselves as a counterweight to the cautious view that AI is highly unpredictable, potentially dangerous, and needs to be regulated. They often give their opponents the labels of quote, "doomers or decels" short for deceleration, as Beff himself put it, "e/acc is a mimetic optimism virus."

Let's get the facts of identity down first. Your name is Guillaume Verdon, Gill, but you're also behind the anonymous account on X called @BasedBeffJezos. So first, Guillaume Verdon, you're a quantum computing guy, physicist, applied mathematician, and then @BasedBeffJezos is basically a meme account that started a movement with a philosophy behind it. So maybe just can you linger on who these people are in terms of characters, in terms of communication styles, in terms of philosophies? I mean, with my main identity, I guess ever since I was a kid, I wanted to figure out the theory of everything, to understand the universe. And that path led me to theoretical physics, eventually trying to answer the big questions of why are we here? Where are we going? And that led me to study information theory and try to understand physics from the lens of information theory, understand the universe as one big computation. And essentially after reaching a certain level studying black hole physics, I realized that I wanted to not only understand how the universe computes, but sort of compute like nature and figure out how to build and apply computers that are inspired by nature. So physics-based computers. And that sort of brought me to quantum computing as a field of study to first of all, simulate nature. And in my work it was to learn representations of nature that can run on such computers.

So the second law of thermodynamics says that the entropy is always increasing in the universe that's tending towards an equilibrium. And you're saying there's these pockets that have complexity and are out of equilibrium. You said that thermodynamics favors the creation of complex life that increases its capability to use energy to offload entropy. To offload entropy. So you have pockets of non-entropy that tend the opposite direction. Why is that intuitive to you that it's natural for such pockets to emerge? Well, we're far more efficient at producing heat than let's say just a rock with a similar mass as ourselves. We acquire free energy, we acquire food, and we're using all this electricity for our operation. And so the universe wants to produce more entropy and by having life go on and grow, it's actually more optimal at producing entropy because it will seek out pockets of free energy and burn it for its sustenance and further growth. And that's sort of the basis of life. And I mean, there's Jeremy England at MIT who has this theory that I'm a proponent of, that life emerged because of this sort of property. And to me, this physics is what governs the meso scales. And so it's the missing piece between the quantum and the cosmos. It's the middle part. Thermodynamics rules the meso scales.

Then there's the word hyperstition. So some ideas as suppose both pessimism and optimism of such ideas that if you internalize them, you in part make that idea reality. So both optimism, pessimism have that property. I would say that probably a lot of ideas have that property, which is one of the interesting things about humans. And you talked about one interesting difference also between the sort of the Guillaume, the Gill front end and the @BasedBeffJezos backend is the communication styles also that you are exploring different ways of communicating that can be more viral in the way that we communicate in the 21st century. Also, the movement that you mentioned that you started, it's not just a meme account, but there's also a name to it called effective accelerationism, e/acc, a play, a resistance to the effective altruism movement. Also, an interesting one that I'd love to talk to you about, the tensions there. And so then there was a merger, a get merge on the personalities recently without your consent, like you said. Some journalists figured out that you're one and the same. Maybe you could talk about that experience. First of all, what's the story of the merger of the two? So I wrote the manifesto with my co-founder of e/acc, an account named @bayeslord, still anonymous, luckily and hopefully forever.

But the flip side of that, which is interesting, I'd love to ask you about it, is as we get better and better at large language models, you can imagine a world where there's anonymous accounts with very convincing large language models behind them, sophisticated bots essentially. And so if you protect that, it's possible then to have armies of bots. You could start a revolution from your basement, an army of bots and anonymous accounts. Is that something that is concerning to you? Technically, e/acc was started in a basement, because I quit big tech, moved back in with my parents, sold my car, let go of my apartment, bought about 100K of GPUs, and I just started building.

I think the, at least to me, attention between ideas here, so to me, deceleration can be both used to centralize power and to decentralize it and the same with acceleration. So sometimes using them a little bit synonymously or not synonymously, but that there's, one is going to lead to the other. And I just would like to ask you about, is there a place of creating a fault tolerant, diverse development of AI that also considers the dangers of AI? And AI, we can generalize to technology in general, is, should we just grow, build, unrestricted as quickly as possible, because that's what the universe really wants us to do? Or is there a place to where we can consider dangers and actually deliberate sort of a wise strategic optimism versus reckless optimism? I think we get painted as reckless, trying to go as fast as possible. I mean, the reality is that whoever deploys an AI system is liable for or should be liable for what it does. And so if the organization or person deploying an AI system does something terrible, they're liable. And ultimately the thesis is that the market will positively select for AIs that are more reliable, more safe and tend to be aligned, they do what you want them to do, right. Because customers, if they're reliable for the product they put out that uses this AI, they won't want to buy AI products that are unreliable, right. So we're actually for reliability engineering, we just think that the market is much more efficient at achieving this sort of reliability optimum than sort of heavy-handed regulations that are written by the incumbents and in a subversive fashion, serves them to achieve regulatory capture.

So if AGI turns out to be a really powerful technology or even the technologies that lead up to AGI, what's your view on the sort of natural centralization that happens when large companies dominate the market? Basically formation of monopolies like the takeoff, whichever company really takes a big leap in development and doesn't reveal intuitively, implicitly or explicitly, the secrets of the magic sauce, they can just run away with it. Is that a worry? I don't know if I believe in fast takeoff, I don't think there's a hyperbolic singularity, right? A hyperbolic singularity would be achieved on a finite time horizon. I think it's just one big exponential and the reason we have an exponential is that we have more people, more resources, more intelligence being applied to advancing this science and the research and development. And the more successful it is, the more value it's adding to society, the more resources we put in and that sort of, similar to Moore's law, is a compounding exponential.

So if it turns out that the beauty that is consciousness in the universe is bigger than just humans, the AI can carry that same flame forward. Does it scare you, are you concerned that AI will replace humans? So during my career, I had a moment where I realized that maybe we need to offload to machines to truly understand the universe around us, right, instead of just having humans with pen and paper solve it all. And to me that sort of process of letting go of a bit of agency gave us way more leverage to understand the world around us. A quantum computer is much better than a human to understand matter at the Nanoscale. Similarly, I think that humanity has a choice, do we accept the opportunity to have intellectual and operational leverage that AI will unlock and thus ensure that we're taken along this path of growth in the scope and scale of civilization? We may dilute ourselves, right? There might be a lot of workers that are AI, but overall out of our own self-interest, by combining and augmenting ourselves with AI, we're going to achieve much higher growth and much more prosperity, right.

Well let me ask you about p(doom), probability of doom. That's just fun to say, but not fun to experience. What is to you the probability that AI eventually kills all or most humans, also known as probability of doom? I'm not a fan of that calculation, I think people just throw numbers out there and it's a very sloppy calculation, right? To calculate a probability, let's say you model the world as some sort of Markov process, if you have enough variables or hidden Markov process. You need to do a stochastic path integral through the space of all possible futures, not just the futures that your brain naturally steers towards, right. I think that the estimators of p(doom) are biased because of our biology, right? We've evolved to have bias sampling towards negative futures that are scary, because that was an evolutionary optimum, right. And so people that are of, let's say higher neuroticism will just think of negative futures where everything goes wrong all day every day and claim that they're doing unbiased sampling. And in a sense they're not normalizing for the space of all possibilities and the space of all possibilities is super exponentially large and it's very hard to have this estimate.

When you were talking about intelligence, you mentioned multipartite quantum entanglement. Mm-hmm.

Maybe this is a good place to ask the difference between the different fields that you've had a toe in. So, mathematics, physics, engineering, and also entrepreneurship, the different layers of the stack. I think a lot of the stuff you're talking about here is a little bit on the math side, maybe physics almost working in theory. Mm-hmm.

The precision decreases in terms of your ability, but still. But since you mentioned UAPs, we talked about intelligence, and I forgot to ask, what's your view on the other possible intelligences that are out there at the Meso scale? Do you think there's other intelligent alien civilizations? Is that useful to think about? How often do you think about it? I think it's useful to think about. It's useful to think about because we got to ensure we're anti-fragile, and we're trying to increase our capabilities as fast as possible. Because we could get disrupted. There's no laws of physics against there being life elsewhere that could evolve and become an advanced civilization and eventually come to us. Do I think they're here now? I'm not sure. I've read what most people have read on the topic.

Just to linger a little longer on quantum mechanics, through all your explorations on quantum computing, what's the coolest, most beautiful idea that you've come across that has been solved or has not yet been solved? I think the journey to understand something called AdS/CFT. So, the journey to understand quantum gravity through this picture, where a hologram of lesser dimension is actually dual or exactly corresponding to a bulk theory of quantum gravity of an extra dimension, and the fact that this sort of duality comes from trying to learn deep learning-like representations of the boundary.

And maybe you can speak to the Kardashev Scale for people who don't know. So one of the sort of meme-like principles and goals of the e/acc movement is to ascend the Kardashev Scale. What is the Kardashev Scale and when do we want to ascend it? The Kardashev Scale is a measure of our energy production and consumption. Really, it's a logarithmic scale. Kardashev Type 1 is a milestone where we are producing the equivalent wattage to all the energy that is incident on earth from the sun. Kardashev Type II would be harnessing all the energy that is output by the sun. And I think Type III is like the whole galaxy equivalent-

So this is probably a good place to talk a bit about the e/acc movement in a Substack blog post titled, What the Fuck is e/acc? Or actually, What the F* is e/acc?, you write, "Strategically speaking, we need to work towards several overarching civilization goals that are all interdependent. And the four goals are, increase the amount of energy we can harness as a species, (climb the Kardashev gradient). In the short term, this almost certainly means nuclear fission. Increase human flourishing via pro-population growth policies and pro-economic growth policies. Create artificial general intelligence, the single greatest force multiplier in human history. And finally, develop interplanetary and interstellar transport so that humanity can spread beyond the earth. Could you build on top of that to maybe say, what to you is the e/acc movement? What are the goals? What are the principles? The goal is for the human techno-capital memetic machine to become self-aware and to hyperstitiously engineer its own growth. So let's decompress that.

The pros and cons of humor in meme, in some sense there's like a wisdom to memes. What is it, the Magic Theater? What book is that from? Hermann Hesse. Steppenwolf, I think. But there's a kind of embracing of the absurdity that seems to get to the truth of things, but at the same time, it can also decrease the quality and the rigor of the discourse. Yeah.

I mean, that's kind of what I try to do with this podcast given the landscape of things, to still have long-form conversations. But there is a degree to which absurdity is fully embraced. In fact, this very conversation is multi-level absurd. So first of all, I should say that just very recently I had a conversation with Jeff Bezos, and I would love to hear your, Beff Jezos, opinions of Jeff Bezos. Speaking of intergalactic Jeff Bezos. What do you think of that particular individual whom your name has inspired? Yeah, I think Jeff is really great. I mean, he's built one of the most epic companies of all time. He's leveraged the techno-capital machine and techno-capital acceleration to give us what we wanted. We want a quick delivery, very convenient, at-home, low prices. He understood how the machine worked and how to harness it, like running the company, not trying to take profits too early, putting it back, letting the system compound and keep improving. And arguably, I think Amazon's invested some of the most amount of capital and robotics out there, and certainly with the birth of AWS, kind of enabled the tech boom we've seen today that has paid the salaries of, I guess myself and all of our friends to some extent. And so I think we can all be grateful to Jeff, and he's one of the great entrepreneurs out there. one of the best of all time, unarguably.

Elon is an interesting case. You are a proponent, you celebrate Elon, but he's also somebody who has for a long time warned about the dangers, the potential dangers, existential risks of artificial intelligence. How do you square the two? Is that a contradiction to you? It is somewhat because he's very much against regulation in many aspects. But for AI, he's definitely a proponent of regulations. I think overall he saw the dangers of, say, OpenAI cornering the market and then getting to have the monopoly over the cultural priors that you can embed in these LLMs that then, as LLMs now become the source of truth for people, then you can shape the culture of the people. And so you can control people by controlling LLMs. He saw that, just like it was the case for social media, if you shape the function of information propagation, you can shape people's opinions. He sought to make a competitor. So at least, I think we're very aligned there, that the way to a good future is to maintain adversarial equilibria between the various AI players. I'd love to talk to him to understand his thinking about how to advance AI going forwards. I mean, he's also hedging his bets, I would say, with Neuralink. I think if he can't stop the progress of AI, he's building the technology to merge. Look at the actions, not just the words.

Right. You mentioned you founded a company, Extropic, that recently announced a 14.1 million seed round. What's the goal of the company? You're talking about a lot of interesting physics things, so what are you up to over there that you can talk about?

So you mentioned with Extropic you're trying to build the physical substrate for generative AI. What's the difference between that and the AGI AI itself? So, is it possible that in the halls of your company, AGI will be created? Or will AGI just be using this as a substrate? I think our goal is to both run human like AI, or anthropomorphic AI.

It's funny you mentioned the other side of the aisle. So, in the poll I posted about p(doom) yesterday, what's the probability of doom? There seems to be a nice division between people think it's very likely, and very unlikely. I wonder if in the future there'll be the actual Republicans versus Democrats division, blue versus red? Is the AI doomers versus the e/accers, EAC? [inaudible 02:26:53]. Yeah. So, this movement is not right wing or left wing fundamentally, it's more like up versus down, in terms of the scale of-

So e/acc as a movement is often formulated in contrast to EA, effective altruism. What do you think are the pros and cons of effective altruism? What's interesting, insightful to you about them, and what is negative? Right. I think people trying to do good from first principles is good.

What does a perfectly productive day, since building is important, what is a perfectly productive day in the life of Guillaume Verdon look like? How much caffeine do you consume? What's a perfect day? Okay, so I have a particular regimen. I would say my favorite days are 12:00 PM to 4:00 AM, and I would have meetings in the early afternoon, usually external meetings, some internal meetings. Because I'm CEO, I have to interface with the outside world, whether it's customers, or investors, or interviewing potential candidates. And usually I'll have ketones, exogenous ketones.

Yeah. What have you learned about the nature of identity from having these two identities? I think it's interesting for people, to be able to put on those two hats so explicitly.

Yeah, right. Breaking Beff. Yeah. You wake up with a shaved head one day, just like, "Who am I? What have I become?" So, you've mentioned quite a bit of advice already, but what advice would you give to young people of, in this interesting world we're in, how to have a career and how to have a life they can be proud of? I think to me, the reason I went to theoretical physics was that I had to learn the base of the stack that was going to stick around no matter how the technology changes. And to me, that was the foundation upon which then I later built engineering skills, and other skills. And to me, the laws of physics, it may seem like the landscape right now is changing so fast, it's disorienting. But certain things like fundamental mathematics and physics aren't going to change. And if you have that knowledge, and knowledge about complex systems, and adaptive systems, I think that's going to carry you very far. And so, not everybody has to study mathematics, but I think it's really a huge cognitive unlock to learn math, and some physics, and engineering.

If you look at you're one cog in the machine, in the capitalist machine, one human, and if you look at yourself, do you think mortality is a feature or a bug? Would you want to be immortal? No, I think fundamentally, in thermodynamic dissipative adaptation, there's the word dissipation. Dissipation is important, death is important. We have a saying in physics, physics progresses one funeral at a time.

What do you think is the meaning of it all? What's the why of the machine? The e/acc machine? The why? Well, the why is thermodynamics. It's why we're here. It's what has led to the formation of life, and of civilization, of evolution of technologies, and growth of civilization. But why do we have thermodynamics? Why do we have our particular universe? Why do we have these particular hyper-parameters, the constants of nature? Well then you get into the anthropic principle, and the landscape of potential universes, right? We're in the universe that allows for life. And then why, is there potentially many universes? I don't know. I don't know that part. But could we potentially engineer new universes, or create pocket universes, and set the hyper-parameters so there is some mutual information between our existence in that universe, and we'd be somewhat its parents? I think that's really... I don't know, that'd be very poetic. It's purely conjecture. But again, this is why figuring out quantum gravity would allow us to understand if we can do that.