Eliezer Yudkowsky

There is a certain perspective where you might be thinking, what is the longest possible game that you could be playing? A short game is, for instance, cancer is playing a shorter game than your organism. Cancer is an organism playing a shorter game than the regular organism. Because the cancer cannot procreate beyond the organism, except for some infectious cancers like the ones that eradicated the Tasmanian devils, you typically end up with a situation where the organism dies together with the cancer, because the cancer has destroyed the larger system due to playing a shorter game. Ideally, you want to, I think, build agents that play the longest possible games. The longest possible games is to keep entropy at bay as long as possible, by doing interesting stuff. The following is a conversation with Joscha Bach, his third time on this podcast. Joscha is one of the most brilliant, and fascinating minds in the world, exploring the nature of intelligence, consciousness, and computation. He's one of my favorite humans to talk to about pretty much anything and everything. This is the Lex Fridman Podcast. To support it, please check out our sponsors in the description. Now, dear friends, here's Joscha Bach.

You wrote a post about levels of lucidity. "As we grow older, it becomes apparent that our self-reflexive mind is not just gradually accumulating ideas about itself, but that it progresses in somewhat distinct stages." There are seven of the stages. Stage one, reactive survival (infant). Stage two, personal self (young child). Stage three, social self (adolescence, domesticated adult). Stage four is rational agency (self-direction). Stage five is self-authoring, that's full adult. You've achieved wisdom, but there's two more stages. Stage six is enlightenment, stage seven is transcendence. Can you explain each, or the interesting parts of each of these stages, and what's your sense why there are stages of this, of lucidity as we progress through life in this too short life? This model is derived from concept by the psychologist Robert Kegan, and he talks about the development of the self as a process that happens in principle by some kind of reverse engineering of the mind, where you gradually become aware of yourself, and thereby build structure that allows you to interact deeper with the world and yourself. I found myself using this model not so much as a developmental model. I'm not even sure if it's a very good developmental model, because I saw my children not progressing exactly like that. I also suspect that you don't go through these stages necessarily in succession, and it's not that you work through one stage and then you get into the next one. Sometimes, you revisit them. Sometimes, stuff is happening in parallel. But it's, I think, a useful framework to look at what's present, and the structure of a person, and how they interact with the world, and how they relate to themselves.

At stage five, you discover how identity is constructed. Self authoring.

Stage six. Stage six? At some point, you can collapse the division between self, a personal self, and world generator again. A lot of people get there via meditation, or some of them get there via psychedelics, some of them by accident. You suddenly notice that you are not actually a person, but you are a vessel that can create a person, and the person is still there. You observe that personal self, but you observe the personal self from the outside, and you notice it's a representation. You might also notice that the world that is being created as the representation is not, then you might experience that I am the universe, I'm the thing that is creating everything. Of course, what you're creating is not quantum mechanics, and the physical universe. What you're creating is this game engine that is updating the world, and you're creating your valence, your feelings, and all the people inside of that world, including the person that you identify with yourself in this world.

Well, I think this is a good opportunity to try to sneak up to the idea of enlightenment. You wrote a series of good tweets about consciousness, and panpsychism. Let's break it down. First you say, I suspect the experience that leads to the panpsychism syndrome of some philosophers, and other consciousness enthusiasts represents the realization that we don't end at the self, but share a resonant universe representation with every other observer coupled to the same universe. This actually, eventually leads us to a lot of interesting questions about AI, and AGI. But let's start with this representation. What is this resonant universe representation, and what do you think? Do we share such a representation? The neuroscientist Grossberg has come up with the cognitive architecture that he calls the adaptive resonance theory. His perspective is that our neurons can be understood as oscillators that are resonating with each other, and with outside phenomena. The [inaudible 00:27:48] model of the universe that we are building, in some sense, is a resonance with objects, and outside of us in the world. Basically, take up patterns of the universe that we are are coupled with. Our brain is not so much understood as circuitry, even though this perspective is valid, but it's almost an ether in which the individual neurons are passing on chemoelectrical signals, or arbitrary signals across all modalities that can be transmitted between cells, stimulate each other in this way, and produce patterns that they modulate while passing them on.

If we look at this, as you write in the tweet, if we look at this more rigorously as a sort of, take the panpsychist idea more seriously, almost as a scientific discipline, you write that quote fascinatingly, that panpsychist interpretation seems to lead to observations of practical results to a degree that physics fundamentalists might call superstitious. Reports of long distance tele telepathy, and remote causation are ubiquitous in the general population. " I'm not convinced," says Joscha Bach, "that establishing the empirical reality of telepathy would force an update of any part of serious academic physics. But it could trigger an important revolution in both neuroscience and AI, from a circuit perspective to a coupled complex resonator paradigm." Are you suggesting that there could be some rigorous mathematical wisdom to panpsychist perspective on the world? First of all, panpsychism is the perspective that consciousness is inseparable for matter in the universe. I find panpsychism quite unsatisfying, because it does not explain consciousness, right? It does not explain how this aspect of matter produces. It is also when I try to formalize panpsychism, and write down what it actually means, and with a more formal mathematical language, it's very difficult to distinguish it from saying that there is a software side to the world, in the same way as their software side to what the transistors are doing in your computers.

You're kind of one of those type of paradigmatic thinkers. Actually, if we can take a tangent on that, once again, returning to the biblical verses of your tweets. "You're right, my public explorations are not driven by audience service, but by my lack of ability for discovering, understanding or following the relevant authorities. So I have to develop my own thoughts. Since I think autonomously these thoughts cannot always be very good." That's you apologizing for the chaos of your thoughts or perhaps not apologizing, just identifying. Yeah.

And the complex resonated paradigm and the verses of your tweets, you write, "Instead of treating eyes, ears, and skin as separate sensory systems with fundamentally different modalities, we might understand them as overlapping aspects of the same universe coupled at the same temporal resolution and almost inseparable from a single share resonant model. Instead of treating mental representations as fully isolated between minds, the representations of physically adjacent observers might directly interact and produce causal effects through the coordination of the perception and behavioral of world modeling observers. So the modalities, the distinction between modalities, let's throw that away. The distinction between the individuals, let's throw that away." So what does this interaction representations look like? And you think about how you represent the interaction of us in this room. At some level the modalities are quite distinct. They're not completely distinct, but you can see this is vision. You can close your eyes and then you don't see a lot anymore, but you still imagine how my mouth is moving when you hear something and you know that it's very close to the sound that you can just open your eyes and you get back into this shared merge space. And we also have these experiments where we notice that the way in which my lips are moving are affecting how you hear the sound and also vice versa. The sounds that you're hearing have an influence on how you interpret some of the visual features, and so these modalities are not separate in your mind. They do are merged at some fundamental level where you are interpreting the entire scene that you're in.

No, don't follow my Twitter, I already have too many followers. Yeah.

Where's happiness in terms of stages is on three or four that you take that tangent? You can be happy at any stage or unhappy. But I think that if you are at a stage where you get agency over how your feelings are generated. To some degree you start doing this when you [inaudible 01:35:15] sense, I believe that you understand that you are in charge of your own emotion to some degree and that you are responsible how you approach the world, that it's basically your task to have some basic hygiene how in the way in which you deal with your mind and you cannot blame your environment for the way in which you feel. But you live in a world that is highly mobile and it's your job to choose the environment that you thrive and to build it.

Do you think GPTN can achieve consciousness? Well, GPTN probably, it's not even clear for the present systems. When I talk to my friends at OpenAI, they feel that this question, whether the models currently are conscious is much more complicated than many people might think. I guess that it's not that OpenAI has a homogenous opinion about this, but there's some aspects to this. One is, of course, this language model has written a lot of text in which people were conscious or describe their own consciousness, and it's emulating this. If it's conscious, it's probably not conscious in a way that is closed to the way in which human beings are conscious. But while it is going through these states and going through 100-step function that is emulating adjacent brain states that require a degree of self-reflection, it can also create a model of an observer that is reflecting itself in real-time and describe what that's like.

Just glimpses. You tweeted, "Suffering results for one part of the mind failing at regulating another part of the mind. Suffering happens at an early stage of mental development. I don't think that superhuman AI would suffer." What's your intuition there? The philosopher Thomas Metzinger is very concerned that the creation of superhuman intelligence would lead to superhuman suffering.

So, you again tweeted about effective accelerationism. You tweeted, "Effective accelerationism is the belief that the Paperclip Maximizer and Roko's Basililisk will keep each other in check by being eternally at each other's throats, so we will be safe and get to enjoy lots of free paperclips and a beautiful afterlife." Is that somewhat aligned with what you're talking about? I've been at a dinner with [inaudible 02:07:21], that's the Twitter handle of one of the main thinkers behind the idea of effective accelerationism. And effective accelerationism is a tongue in cheek movement that is trying to put a counter position to some of the doom peers in the AI space, by arguing that what's probably going to happen is an equilibrium between different competing AIs, in the same way as there is not a single corporation that is under a single government that is destroying and conquering everything on earth by becoming inefficient and corrupt, there're going to be many systems that keep each other in check and force themselves to evolve.

And I also noticed that in many ways, I'm less identified with the person that I am as I get older and I'm much more identified with being conscious. I have a mind that is conscious, that is able to create a person, and that person is slightly different every day. And the reason why I perceive it as identical has practical purposes so I can learn and make myself responsible for the decisions that I made in the past and project them in the future. But I also realize I'm not actually the person that I was last year, and I'm not the same person as I was 10 years ago, and then 10 years from now, I will be a different person, so this continuity is a fiction, it only exists as a projection from my present self. And consciousness itself doesn't have an identity, it's a law. Basically, if you build an arrangement of processing matter in a particular way, the following thing is going to happen, and the consciousness that you have is functionally not different from my consciousness. It's still a self-reflexive principle of agency that is just experiencing a different story, different desires, different coupling to the world and so on.

What's the timeline for things to get real weird with AI? And it can get weird in interesting ways before you get to a AGI. What about AI girlfriends and boyfriends, fundamentally transforming human relationships? I think human relationships are already fundamentally transformed and it's already very weird.

Do you think AI, when is deployed by companies like Microsoft and Google and Meta will have the same issue of being weirdly corporate? There'd be some uncanny valley, some weirdness to the whole presentation? So this, I've gotten a chance to talk to George Hotz. He believes everything should be open source and decentralized and there then we shall have the AI of the people and it'll maintain a grounding to the magic humanity. That's the human condition that corporations will destroy the magic. I believe that if we make everything open source and make this mandatory, we are going to lose about a lot of beautiful art and a lot of beautiful designs. There is a reason why Linux desktop is still ugly and it's-

Go ahead. By all means. Go nuts. I have no objection here. I'm just trying to describe what's happening. And it's not that I don't do things that I later say, oh, I actually wish I would've done something different. I also know that when we die, the greatest regret that people typically have on their deathbed, they say, "Oh, I wish I had spent more time on Twitter." No, I don't think that's the case. I think they should probably have spent less time on Twitter. But I found it so useful for myself and also so addictive that I felt I need to make the best of it and turn it into an art form and thought form. And it did help me to develop something, but I wish what other things I could've done in the meantime. It's just not the universe that we are in anymore. Most people don't read books anymore. What do you think that means, that we don't read books anymore? What do you think that means about the collective intelligence of our species? Is it possible it's still progressing and growing?

I think that's a fun and that's a beautiful game. It's ultimately a productive one. Speaking of taking that risk, you tweeted, when you have the choice between being a creator, consumer, or redistributor, always go for creation. Not only does it lead to a more beautiful world, but also to a much more satisfying life for yourself. And don't get stuck preparing yourself for the journey. The time is always now. So let me ask for advice. What advice would you give on how to become such a creator on Twitter in your own life? I was very lucky to be alive at the time of the collapse of Eastern Germany and the transition into Western Germany and me and my friends and most of the people I knew and were East Germans and we were very poor, because we didn't have money and all the capital was western in Germany and they bought our factories and shut them down, because they were mostly only interest in the market rather than creating new production capacity. And so cities were poor and then this repair and we could not afford things and I could not afford to go into a restaurant and order a meal there. I would have to cook at home. But I also thought, why not just have a restaurant with my friends? So we would open up a cafe with friends and a restaurant and we would cook for each other in these restaurants and also invite the general public and they could donate.

Yeah. Speaking of which, you retweeted this meme of a life of philosopher in a nutshell, it's birth and death and in between it's a chubby guy and it says why though? What do you think is the answer to that? Well, the answer is that everything that can exist might exist. And in many ways you take an ecological perspective the same way as when you look at human opinions and cultures. It's not that there is right and wrong opinions when you look at this from this ecological perspective, but every opinion that fits between two human years might be between two human years. And so when I see in a stranger opinion on social media, it's not that I feel that I have a need to get upset, it's often more that, "Oh, there you are." And your opinion is incentivized, then it's going to be abundant. And when you take this ecological perspective also on yourself and you realize you're just one of these mushrooms that are popping up and doing this thing, and you can, depending on where you chose to grow and where you happen to grow, you can flourish or not doing this or that strategy. And it's still all the same life at some level.