Episode #416 from 13:54
Bilingualism and thinking
That's called autoregressive prediction, which is why those LLMs should be called autoregressive LLMs, but we just call them LLMs, and there is a difference between this kind of process and a process by which before producing a word... When you and I talk, you and I are bilingual, we think about what we're going to say, and it's relatively independent of the language in which we're going to say. When we talk about, I don't know, let's say a mathematical concept or something, the kind of thinking that we're doing and the answer that we're planning to produce is not linked to whether we're going to see it in French or Russian or English. Chomsky just rolled his eyes, but I understand, so you're saying that there's a bigger abstraction that goes before language and maps onto language?
People
Why this moment matters
That's called autoregressive prediction, which is why those LLMs should be called autoregressive LLMs, but we just call them LLMs, and there is a difference between this kind of process and a process by which before producing a word... When you and I talk, you and I are bilingual, we think about what we're going to say, and it's relatively independent of the language in which we're going to say. When we talk about, I don't know, let's say a mathematical concept or something, the kind of thinking that we're doing and the answer that we're planning to produce is not linked to whether we're going to see it in French or Russian or English. Chomsky just rolled his eyes, but I understand, so you're saying that there's a bigger abstraction that goes before language and maps onto language?
People and topics
People