1 million H100 GPUs

Can you have a conversation with an AI where it feels like you talked to Einstein or Feynman, where you ask them a hard question, they're like, "I don't know," and then after a week, they did a lot of research- They disappear and come back, yeah.

Now, dear friends, here's Aravind Srinivas. Perplexity is part search engine, part LLM. How does it work, and what role does each part of that the search and the LLM play in serving the final result? Perplexity is best described as an answer engine. You ask it a question, you get an answer. Except the difference is, all the answers are backed by sources. This is like how an academic writes a paper. Now, that referencing part, the sourcing part is where the search engine part comes in. You combine traditional search, extract results relevant to the query the user asked. You read those links, extract the relevant paragraphs, feed it into an LLM. LLM means large language model.

I really love the steps that the pro search is doing. Compare Perplexity and Google for everyday searches. Step two, evaluate strengths and weaknesses of Perplexity. Evaluate strengths and weaknesses of Google. It's like a procedure. Complete. Okay, answer. Perplexity AI, while impressive, is not yet a full replacement for Google for everyday searches. Yes.

Yeah, it's an interesting game. It's really, really interesting game. I read that you looked up to Larry Page and Sergey Brin and that you can recite passages from In The Plex and that book was very influential to you and How Google Works was influential. So what do you find inspiring about Google, about those two guys, Larry Page and Sergey Brin and just all the things they were able to do in the early days of the internet? First of all, the number one thing I took away, there's not a lot of people talk about this is, they didn't compete with the other search engines by doing the same thing. They flipped it like they said, "Hey, everyone's just focusing on text-based similarity, traditional information extraction and information retrieval, which was not working that great. What if we instead ignore the text? We use the text at a basic level, but we actually look at the link structure and try to extract ranking signal from that instead." I think that was a key insight.

So you talked about Larry Page and Sergey Brin. What other entrepreneurs inspired you on your journey in starting the company? One thing I've done is take parts from every person. And so, it'll almost be like an ensemble algorithm over them. So I'd probably keep the answer short and say each person what I took. With Bezos, I think it's the forcing [inaudible 00:47:21] to have real clarity of thought. And I don't really try to write a lot of docs. There's, when you're a startup, you have to do more in actions and [inaudible 00:47:33] docs, but at least try to write some strategy doc once in a while just for the purpose of you gaining clarity, not to have the doc shared around and feel like you did some work.

Distribution, hardest thing in any business is distribution. And I read this Walter Isaacson biography of him. He learned the mistakes that, if you rely on others a lot for your distribution, his first company, Zip2 where he tried to build something like a Google Maps, he ended up, as in, the company ended up making deals with putting their technology on other people's sites and losing direct relationship with the users because that's good for your business. You have to make some revenue and people pay you. But then in Tesla, he didn't do that. He actually didn't go to dealers or anything. He had, dealt the relationship with the users directly. It's hard. You might never get the critical mass, but amazingly, he managed to make it happen. So I think that sheer force of will and [inaudible 00:51:37] principles thinking, no work is beneath you, I think that is very important. I've heard that in Autopilot he has done data himself just to understand how it works. Every detail could be relevant to you to make a good business decision and he's phenomenal at that. And one of the things you do by understanding every detail is you can figure out how to break through difficult bottlenecks and also how to simplify the system.

Yeah, and this trait is also visible in Jensen, like this real obsession and constantly improving the system, understanding the details. It's common across all of them. And I think Jensen is pretty famous for saying, "I just don't even do one-on-ones because I want to know simultaneously from all parts of the system like [inaudible 00:53:03] I just do one is to, and I have 60 direct reports and I made all of them together and that gets me all the knowledge at once and I can make the dots connect and it's a lot more efficient." Questioning the conventional wisdom and trying to do things a different way is very important. I think you tweeted a picture of him and said, this is what winning looks like.

So who else? You've mentioned Bezos, you mentioned Elon. Yeah, like Larry and Sergey, we've already talked about. I mean, Zuckerberg's obsession about moving fast is very famous, move fast and break things.

So speaking of Meta, Yann LeCun is somebody who funded Perplexity. What do you think about Yann? He gets, he's been feisty his whole life. He has been especially on fire recently on Twitter, on X. I have a lot of respect for him. I think he went through many years where people just ridiculed or didn't respect his work as much as they should have, and he still stuck with it. And not just his contributions to Convnets and self-supervised learning and energy-based models and things like that. He also educated a good generation of next scientists like Koray who's now the CTO of DeepMind, who was a student. The guy who invented DALL-E at OpenAI and Sora was Yann LeCun's student, Aditya Ramesh. And many others who've done great work in this field come from LeCun's lab like Wojciech Zaremba, one of the OpenAI co-founders. So there's a lot of people he's just given as the next generation to that have gone on to do great work. And I would say that his positioning on, he was right about one thing very early on in 2016. You probably remember RL was the real hot at the time. Everyone wanted to do RL and it was not an easy to gain skill. You have to actually go and read MDPs, understand, read some math, bellman equations, dynamic programming, model-based [inaudible 00:59:00].

How surprising was it to you, because you were in the middle of it. How effective attention was, how- Self-attention?

This kind of work hints a little bit of a similar kind of approach to self-play. Do you think it's possible we live in a world where we get an intelligence explosion from post-training? Meaning like, if there's some kind of insane world where AI systems are just talking to each other and learning from each other? That's what this kind of, at least to me, seems like it's pushing towards that direction. And it's not obvious to me that that's not possible. It's not possible to say... Unless mathematically you can say it's not possible. It's hard to say it's not possible. Of course, there are some simple arguments you can make. Like, where is the new signal is the AI coming from? How are you creating new signal from nothing?

It feels like the process the Perplexity is doing where you ask a question and you answer it and then you go on to the next related question, and this chain of questions. That feels like that could be instilled into AI just constantly searching- You are the one who made the decision on-

I love all the tangents we took, but let's return to the beginning. What's the origin story of Perplexity? So I got together my co-founders, Dennis and Johnny, and all we wanted to do was build cool products with LLMs. It was a time when it wasn't clear where the value would be created. Is it in the model? Is it in the product? But one thing was clear, these generative models that transcended from just being research projects to actual user-facing applications, GitHub Copilot was being used by a lot of people, and I was using it myself, and I saw a lot of people around me using it, Andrej Karpathy was using it, people were paying for it. So this was a moment unlike any other moment before where people were having AI companies where they would just keep collecting a lot of data, but then it would be a small part of something bigger. But for the first time, AI itself was the thing.

So can you speak to the technical details of how Perplexity works? You've mentioned already RAG, retrieval augmented generation. What are the different components here? How does the search happen? First of all, what is RAG? What does the LLM do at a high level? How does the thing work? Yeah. So RAG is retrieval augmented generation. Simple framework. Given a query, always retrieve relevant documents and pick relevant paragraphs from each document and use those documents and paragraphs to write your answer for that query. The principle in Perplexity is you're not supposed to say anything that you don't retrieve, which is even more powerful than RAG because RAG just says, "Okay, use this additional context and write an answer." But we say, "Don't use anything more than that too." That way we ensure a factual grounding. "And if you don't have enough information from documents you retrieve, just say, 'We don't have enough search resource to give you a good answer.'"

From your whole experience, what advice would you give to people looking to start a company about how to do so? What startup advice do you have? I think all the traditional wisdom applies. I'm not going to say none of that matters. Relentless determination, grit, believing in yourself and others. All these things matter, so if you don't have these traits, I think it's definitely hard to do a company. But you deciding to do a company despite all this clearly means you have it or you think you have it. Either way, you can fake it till you have it. I think the thing that most people get wrong after they've decided to start a company is work on things they think the market wants. Not being passionate about any idea but thinking, okay, look, this is what will get me venture funding. This is what will get me revenue or customers. That's what will get me venture funding. If you work from that perspective, I think you'll give up beyond the point because it's very hard to work towards something that was not truly important to you. Do you really care?

So if you just put on your visionary hat, look into the future, what do you think the future of search looks like? And maybe even let's go with the bigger pothead question. What does the future of the internet, the web look like? So what is this evolving towards? And maybe even the future of the web browser, how we interact with the internet. If you zoom out, before even the internet, it's always been about transmission of knowledge. That's a bigger thing than search. Search is one way to do it. The internet was a great way to disseminate knowledge faster and started off with organization by topics, Yahoo, categorization, and then better organization of links. Google. Google also started doing instant answers through the knowledge panels and things like that. I think even in 2010s, one third of Google traffic, when it used to be like 3 billion queries a day, was just instant answers from-

Yeah, I mean that's a really inspiring future, but do you think also there's going to be other kinds of AIs, AGI systems, that form deep connections with humans? Yes.