March 5, 2024 • 1hr 9min

Aravind Srinivas - Building An Answer Engine - [Invest Like the Best, EP.363]

Invest Like the Best with Patrick O'Shaughnessy

In this episode of Invest Like the Best, Patrick O'Shaughnessy interviews Aravind Srinivas, founder and CEO of Perplexity AI. Perplexity is building what Aravind describes as an "answer engine" - a next-generation search product powered by AI that aims to provide direct answers rather than just links. The conversation covers the technical challenges of building such a product, Perplexity's journey from a simple wrapper to robust infrastructure, the competitive landscape in AI, and Aravind's thoughts on the future of search and AI applications.

0:00

-0:00

Key Takeaways

Perplexity aims to be an "answer engine" rather than just a search engine, providing direct answers with citations instead of just links
The company started as a wrapper around existing APIs but has since built robust infrastructure to handle scale and latency
Key focus areas are speed, accuracy, and user experience to compete with larger players like Google and OpenAI
Building an index of the web with high-quality, relevant content is crucial for an answer engine
Personalization and more agentic capabilities are on the roadmap for future development
The AI talent landscape is extremely competitive, especially for large-scale model training
For AI startups, fast execution and being the best in your sector is key, rather than trying to avoid all competition
Verticalization may not be the best strategy for AI companies - a more general-purpose AI assistant could be more valuable

Introduction

In this episode of Invest Like the Best, Patrick O'Shaughnessy interviews Aravind Srinivas, founder and CEO of Perplexity AI. Perplexity is building what Aravind describes as an "answer engine" - a next-generation search product powered by AI that aims to provide direct answers rather than just links. The conversation covers the technical challenges of building such a product, Perplexity's journey from a simple wrapper to robust infrastructure, the competitive landscape in AI, and Aravind's thoughts on the future of search and AI applications.

Topics Discussed

Redefining 'Great' in Search (5:39)

Aravind explains his vision for what "great" means in search:

Moving beyond the "hack" of 10 blue links to directly answering questions
Making it easy for users to ask good questions, not just relying on their ability to formulate queries
Democratizing access to knowledge and making users "smarter" every day

He emphasizes: "Our job is to bring the joy of learning to everybody. And so that's what we want to address. Either helping people ask questions or start with some dumb version of the question and help the AI refine it for you and profile you enough that suggest interesting questions to ask and have a knowledge feed of questions that you just see because you asked questions about those topics before and just every single day make you some x percent smarter."

The Mechanics of Perplexity (8:59)

Aravind walks through how Perplexity works behind the scenes:

Reformulates and expands the user's question
Retrieves relevant links and paragraphs from the web
Uses a large language model to synthesize a concise answer with citations
Presents the answer with references and the option to dig deeper or ask follow-ups

He notes that about 10% of users click through to source links, finding a balance between providing direct answers and allowing further exploration.

The Evolution of Perplexity From Wrapper to Robust Infrastructure (12:17)

Aravind describes Perplexity's journey:

Started as a simple wrapper around Bing API and GPT-3.5 to validate the product idea
As usage grew, they had to build infrastructure to handle scale, outages, and rate limits
Continuously solved new issues to create a more robust backend

"After a year, if you look back, you're like, oh d**n, this is impossible. How far we've come and we would have never imagined we could have built such a sophisticated backend when we started off versus two or three people."

The Future of Large Language Models (LLMs) (15:01)

Aravind shares his thoughts on the future of LLMs:

Reasoning in smaller models: Improving reasoning capabilities in smaller models to reduce costs
Synthetic data: Using AI to generate training data and annotations
Commoditization: Cloud providers may push to commoditize LLMs

He emphasizes the importance of post-training and fine-tuning models for specific use cases, where companies with user data have an advantage.

Aravind's Strategy For Constructing The Business Model (24:37)

Aravind discusses Perplexity's approach to monetization:

Initially focused on growth without worrying about revenue
Introduced a subscription model to validate product-market fit
Considering advertising models that don't compromise answer quality
Exploring API offerings for developers and enterprises

He notes: "We just said, okay, the subscription model works. It's not the ultimate model. I don't think that's the final piece in the profits that are going to be generated in the AI chat bot sector. But it's a good start."

The Process of Building an Index for a Search Engine (29:43)

Aravind explains the components of building a web index:

Crawler to find and update web pages
Parser to extract content from different types of web pages
Indexer to store and organize the content
Retrieval system to find relevant information for queries

He emphasizes the importance of quality over quantity in the index, focusing on domains relevant for AI-powered question answering.

The Impact of Scaling and Data Quality on AI Models (34:56)

Aravind discusses how the transformer architecture changed AI development:

Simplified neural network design
Shifted focus to data quality and experimentation
Enabled new types of talent (e.g. physics PhDs) to contribute

He notes: "I believe that it's less about being a great transformer designer and more about being a great data expert and experimenter."

Bottlenecks in AI Development (41:11)

Aravind identifies key bottlenecks in AI development:

For Perplexity: Improving reasoning in smaller models to reduce costs
For the field in general: Developing better synthetic data generation techniques

He emphasizes the potential impact of models that can generate their own training data and annotations.

The Talent Landscape in AI (48:45)

Aravind describes the competitive landscape for AI talent:

Extreme competition for talent in large-scale model training
Difficulty in attracting talent without massive compute resources
Potential for commoditization of large models by cloud providers
Opportunities in post-training and product-specific optimizations

"My hope is that since the cloud service providers are the ones bankrolling all these companies directly or indirectly, they will put it on their clouds for enterprises. And then enterprises can take them and post train them."

The Vision for the Future of Search (59:08)

Aravind shares his vision for the future of search:

Disrupting commercial search categories (e.g. shopping, travel, insurance)
Expanding beyond being a research tool to handling more everyday consumer queries
Competing with established players by offering a superior user experience

He notes the challenge of competing with OpenAI and the need for continued fast execution.

Importance of Execution in AI Startups (1:04:31)

Aravind offers advice for AI startups:

Focus on execution and iteration rather than perfect strategy from the start
Build something with traction before worrying too much about defensibility
Be open to pivoting and discovering valuable use cases through experimentation

"Build execution muscle first, don't try to be a great strategist right away. Build something. Make sure it has some traction, get the muscle that you can keep iterating and then you deserve the right to strategize."

Conclusion

The conversation with Aravind Srinivas provides deep insights into the challenges and opportunities in building AI-powered search and question-answering systems. Perplexity's journey from a simple wrapper to a robust infrastructure highlights the importance of execution and continuous improvement in the fast-moving AI landscape. Aravind's perspectives on the future of search, the talent landscape, and advice for AI startups offer valuable guidance for entrepreneurs and investors in this space. As AI continues to reshape how we access and interact with information, companies like Perplexity are at the forefront of redefining the search experience for users worldwide.