Episode #434 from 1:56:27
RAG
So can you speak to the technical details of how Perplexity works? You've mentioned already RAG, retrieval augmented generation. What are the different components here? How does the search happen? First of all, what is RAG? What does the LLM do at a high level? How does the thing work? Yeah. So RAG is retrieval augmented generation. Simple framework. Given a query, always retrieve relevant documents and pick relevant paragraphs from each document and use those documents and paragraphs to write your answer for that query. The principle in Perplexity is you're not supposed to say anything that you don't retrieve, which is even more powerful than RAG because RAG just says, "Okay, use this additional context and write an answer." But we say, "Don't use anything more than that too." That way we ensure a factual grounding. "And if you don't have enough information from documents you retrieve, just say, 'We don't have enough search resource to give you a good answer.'"
Why this moment matters
So can you speak to the technical details of how Perplexity works? You've mentioned already RAG, retrieval augmented generation. What are the different components here? How does the search happen? First of all, what is RAG? What does the LLM do at a high level? How does the thing work? Yeah. So RAG is retrieval augmented generation. Simple framework. Given a query, always retrieve relevant documents and pick relevant paragraphs from each document and use those documents and paragraphs to write your answer for that query. The principle in Perplexity is you're not supposed to say anything that you don't retrieve, which is even more powerful than RAG because RAG just says, "Okay, use this additional context and write an answer." But we say, "Don't use anything more than that too." That way we ensure a factual grounding. "And if you don't have enough information from documents you retrieve, just say, 'We don't have enough search resource to give you a good answer.'"