Introduction

Danny really impacted me. I was an undergrad at Berkeley and I got to take a class from him long before he won the Nobel Prize or anything and it was just a mind-blowing class. But this idea of the remembering self and the experiencing self, I got into it because it's so much about memory even though he doesn't study memory. So we're right now having this experience, right? And people can watch it presumably on YouTube or listen to it on audio, but if you're talking to somebody else, you could probably describe this whole thing in 10 minutes, but that's going to miss a lot of what actually happened. And so the idea there is that the way we remember things is not the replay of the experience, it's something totally different. And it tends to be biased by the beginning and the end, and he talks about the peaks, but there's also the best parts, the worst parts, etc. And those are the things that we remember. And so when we make decisions, we usually consult memory and we feel like our memory is a record of what we've experienced, but it's not. It's this kind of very biased sample, but it's biased in an interesting and I think biologically relevant way.

Is there some insight into the human brain that explains why we don't seem to remember anything from the first few years of life? Yeah. Yeah. In fact, actually I was just talking to my really good friend and colleague, Simona Getty, who studies the neuroscience of child development and so we were talking about this. And so there are a bunch of reasons I would say. So one reason is is there's an area of the brain called the hippocampus, which is very, very important for remembering events or episodic memory. And so the first two years of life, there's a period called infantile amnesia. And then the next couple years of life after that, there's a period called childhood amnesia. And the differences is is that basically in the lab and even during childhood and afterwards, children basically don't have any episodic memories for those first two years.

Just imagine the size of that thing. Anyway, how does the brain forget and how and why does it remember? So maybe some of the mechanisms. You mentioned the hippocampus, what are the different components involved here? So we could think about this on a number of levels. Maybe I'll give you the simplest version first, which is we tend to think of memories as these individual things and we can just access them, maybe a little bit like photos on your phone or something like that. But in the brain, the way it works is you have this distributed pool of neurons and the memories are kind of shared across different pools of neurons. And so what you have is competition, where sometimes memories that overlap can be fighting against each other. So sometimes we forget because that competition just wipes things out. Sometimes we forget because there aren't the biological signals which we can get into, I would promote long-term retention.

Can memory be trained and improved? This beautiful connected system that you've described, what aspect of it is a. ... you've described. What aspect of it is a mechanism that can be improved through training?

We should maybe linger on this Memory Palace thing just to make it obvious, 'cause when people were describing to me a while ago what this is, it seems insane. You literally think of a place like a childhood home or a home that you're really visually familiar with and you literally place in that three-dimensional space facts or people or whatever you want to remember, and you just walk in your mind along that place visually and you can remember, remind yourself of the different things. One of the limitations is there is a sequence to it. You can't just go upstairs right away or something. You have to walk along the room. It's really great for remembering sequences, but it's also not great for remembering individual facts out of context. The full context of the tour, I think, is important, but it's fascinating how the mind is able to do that. When you ground these pieces of knowledge into something that you remember well already, especially visually, it's fascinating. I think you do that for any kind of sequence. I'm sure she used something like this for the Ikea catalog, something of this nature.

One of the cool things that I found is that some people really just revolutionize a field by creating a problem that didn't exist before. It's why I love science is engineering is like solving other people's problems and science is about creating problems. I'm just much more like I want to break things and create problems, not necessarily move fast, though. But one of my former mentors, Marcia Johnson, who in my opinion is one of the greatest memory researchers of all time, she comes up young woman in the field in this mostly guy field. She gets into this idea of how do we tell the difference between things that we've imagined and things that we actually remember? How do we tell, I get some mental experience, where did that mental experience come from? It turns out this is a huge problem because essentially our mental experience of remembering something that happened, our mental experience of thinking about something, how do you tell the difference? They're both largely constructions in our head, and so it is very important. The way that you do it is, it's not perfect, but the way that we often do it and succeed is by, again, using our prefrontal cortex and really focusing on the sensory information or the place in time and the things that put us back into when this information happened. If it's something you thought about, you're not going to have all of that vivid detail as you do for something that actually happened, but it doesn't work all the time. But that's a big thing that you have to do. But it takes time. It's slow, and it's again, effortful, but that's what you need to remember accurately. But what's cool, and I think this is what you alluded to about how that was an interesting experience is, imagination is exactly the opposite. Imagination is basically saying, "I'm just going to take all this information from memory, recombine it in different ways and throw it out there." So for instance, Dan Schachter and Donna Addis have done cool work on this. Demis Hassabis did work on this with Eleanor McGuire in UCL, and this goes back actually to this guy, Frederic Bartlett, who is this revolutionary memory researcher, Bartlett. He actually rejected the whole idea of quantifying memory. He said, "There's no statistics in my book." He came from this anthropology perspective and short version of the story is he just asked people to recall things. You give people stories and poems, ask people to recall them.

Can we just talk about memory sport a little longer? There's something called the USA Memory Championship. What are these athletes like? What does it mean to be elite level at this? Have you interacted with any of them or reading about them, what have you learned about these folks? There's a guy named Henry Roediger who's studying these guys. There's actually a book by Joshua Foer called Moonwalking with Einstein, where he talks about, he actually, as part of this book, just decided to become a memory athlete. They often have these life events that make them go-

So you mentioned fMRI. What is it? And how is it used in studying memory? This is actually the reason why I got into this whole field of science is when I was in grad school, fMRI was just really taking off as a technique for studying brain activity. And what's beautiful about it is you can study the whole human brain. And there's lots of limits to it, but you can basically do it in a person without sticking anything into their brains, and very non-invasive. For me being in an MRI scanner is like being in the womb, I just fall asleep. If I'm not being asked to do anything, I get very sleepy. But you can have people watch movies while they're being scanned or you can have them do tests of memory, giving them words and so forth to memorize. But what MRI is itself is just this technique where you put people in a very high magnetic field. Typical ones we would use would be 3 Tesla to give you an idea.

What kind of big scientific discoveries, maybe the flavor of discoveries have been done throughout the history of the science of memory, the studying of memory? What kind of things have been understood? Oh, there's so many, it's really so hard to summarize it. I think it's funny because it's like when you're in the field, you can get kind of blasé about this stuff. But then once I started write the book, I was like, "Oh my God, this is really interesting. How did we do all this stuff?" I would say that some of the... From the first study, it's just showing how much we forget is very important. Showing how much schemas, which is our organized knowledge about the world increase our ability to remember information, just massively increase in [inaudible 01:29:25] of expertise. Showing how experts like chess experts can memorize so much in such a short amount of time because of the schemas they have for chess. But then also showing that those lead to all sorts of distortions in memory.

Yeah, sure. How does deja vu work?

Well, I like to say there's no such thing as true or false memories. It's like Johnny Rotten from the Sex Pistols, he had a saying that's like, "I don't believe in false memories any more than I believe in false songs." And so the basic idea is that we have these memories that reflect bits and pieces of what happened, as well as our inferences and theories. So, I'm a scientist and I collect data, but I use theories to make sense of that data. And so, a memory is kind of a mix of all these things. Where memories can go off the deep end and become what we would call conventionally as false memories are sometimes little distortions where we filled in the blanks, the gaps in our memory, based on things that we know, but don't actually correspond to what happened.

So, let me ask you about... We talked about false memories, but in the legal system, false confessions. I remember reading 1984 where, sorry for the dark turn of our conversation, but through torture, you can make people say anything and essentially remember anything. I wonder to which degree, there's truth to that, if you look at the torture that happened in the Soviet Union, for confessions, all that kind of stuff. How much can you really get people to force false memories, I guess? Yeah. I mean, I think there's a lot of history of this actually, in the criminal justice system. You might've heard the term "the third degree." If you actually look it up historically, it was a very intense set of beatings and starvation and physical demands that they would place at people to get them to talk. And there's certainly a lot of work that's been done by the CIA in terms of enhanced interrogation techniques.

It's really tragic that centralized power can use these kinds of tools to destroy lives. Sad. Since there's a theme about music throughout this conversation, one of the best topics for songs is heartbreak. Love in general, but heartbreak. Why and how do we remember and forget heartbreak? Asking for a friend. Oh, God, that's so hard to... Asking for a friend. I love that. It's such a hard one. Part of this is we tend to go back to particular times that are the more emotionally intense periods, and so that's a part of it. Again, memory is designed to capture these things that are biologically significant, and attachment is a big part of biological significance for humans. Human relationships are super important and sometimes that heartbreak comes with massive changes in your beliefs about somebody say if they cheated on you or something like that, or regrets and you kind of ruminate about things that you've done wrong.

How does studying memory affect your understanding of the nature of time? We've been talking about us living in the present and making decisions about the future, standing on the foundation of these memories and narratives about the memories that we've constructed. It feels like it does weird things to time. Yeah, and the reason is that in some sense, I think especially the farther we go back, there's all sorts of interesting things that happen. Your sense of if I ask how different does one hour ago feel from two hours ago? You'd probably say pretty different. But if I ask you, okay, go back one year ago versus one year and one hour ago, it's the same difference in time. It won't feel very different. There's this kind of compression that happens as you look back farther in time.

Let me ask you both a practical question and an out there question. Let's start with a more practical one. What are your thoughts about BCIs, brain computer interfaces, and the work that's going on with Neuralink? We talked about electrodes and different ways of measuring the brain, and here Neuralink is working on basically two-way communication with the brain. The more out there question will be like, where's this go? More practically in the near term, what do you think about Neuralink? I can't say specifics about the company because I haven't studied it that much, but I think there's two parts of it. One is, they're developing some really interesting technology I think with these surgical robots and things like that. BCI though has a whole lot of innovation going on. I am not necessarily seeing any scientific evidence from Neuralink, and maybe that's just because I'm not looking for it, but I'm not seeing the evidence that they're anywhere near where the scientific community is. There's lots of startups that are doing incredibly innovative stuff.

For the case of driving, I think it could be quite effective. One of the things that's currently missing, even though OpenAI just recently announced adding memory, and I did want to ask you how important it is, how difficult is it to add some of the memory mechanisms that you've seen in humans to AI systems? I would say superficially not that hard, but then in a deeper level, very, very hard because we don't understand episodic memory. One of the ideas I talk about in the book, because one of the oldest dilemmas in computational neurosciences, what Steve Grossberg called the Stability Plasticity Dilemma, when do you say something is new and overwrite your preexisting knowledge versus going with what you had before and making incremental changes? Part of the problem with going through massive... Part of the problem of things like if you're trying to design an LLM or something like that, is, especially for English, there's so many exceptions to the rules. If you want to rapidly learn the exceptions, you're going to lose the rules, and if you want to keep the rules, you have a harder time learning the exception. David Marr is one of the early pioneers in computational neuroscience, and then Jay McClellan and my colleague, Randy O'Reilly, some other people like Neil Cohen, all these people started to come up with the idea that maybe that's part of what we need.

Since we're talking about attention, is there an interesting connection to you between ADHD and memory? Well, it's interesting for me, because when I was a child, I was actually told, my school, I don't know if it came from a school psychologist, they did do some testing on me, I know for IQ and stuff like that, or if it just came from teachers who hated me, but they told my parents that I had ADHD. And so, this was of course in the '70s. So basically they said, "He has poor motor control and he's got ADHD," and there was social issues, so I could have been put a year ahead in school. But then they said, "Oh, but he doesn't have the social capabilities." So I still ended up being an outcast even in my own grade.

So you're a musician. Take me through how'd you get into music? What made you first fall in love with music, with creating music? So I started playing music just when I was doing trumpet in school for school band. And I would just read music and play, and it was pretty decent at it, not great, but I was decent.

A bit of a big ridiculous question, but let me ask you. We've been talking about neuroscience in general. You've been studying the human mind for a long time. What do you love most about the human mind? Like, when you look at it, we look at the fMRI, just the scans and the behavioral stuff, the electrodes, the psychology aspect, reading the literature on the biology side, in your biology, all of it. When you look at it, what is most beautiful to you? I think the most beautiful, but incredibly hard to put your finger on, is this idea of the internal model, that it's like there's everything you see, and there's everything you hear, and touch, and taste, every breath you take, whatever, but it's all connected by this dark energy that's holding that whole universe of your mind together. And without that, it's just a bunch of stuff. And somehow we put that together and it forms so much of our experience, and being able to figure out where that comes from and how things are connected to me is just amazing. But just this idea of the world in front of us, we're only sampling this little bit and trying to take so much meaning from it, and we do a really good job. Not perfect, I mean, but that ability to me is just amazing.