Data Brew by Databricks
Welcome to Data Brew by Databricks with Denny and Brooke! In this series, we explore various topics in the data and AI community and interview subject matter experts in data engineering/data science. So join us with your morning brew in hand and get ready to dive deep into data + AI! For this first season, we will be focusing on lakehouses – combining the key features of data warehouses, such as ACID transactions, with the scalability of data lakes, directly against low-cost object stores.
Data Brew by Databricks
Retrieval, rerankers, and RAG tips and tricks | Data Brew | Episode 39
•
Databricks
In this episode, Andrew Drozdov, Research Scientist at Databricks, explores how Retrieval Augmented Generation (RAG) enhances AI models by integrating retrieval capabilities for improved response accuracy and relevance.
Highlights include:
- Addressing LLM limitations by injecting relevant external information.
- Optimizing document chunking, embedding, and query generation for RAG.
- Improving retrieval systems with embeddings and fine-tuning techniques.
- Enhancing search results using re-rankers and retrieval diagnostics.
- Applying RAG strategies in enterprise AI for domain-specific improvements.
