We introduce an experimental framework for assessing RAG retrieval and explain its main components including benchmark creation, evaluation metrics, parameter space and experiment tracking. Whether you're deep in the field or just starting, this post is your gateway to understanding and improving RAG retrieval performance.

We introduce the many pitfalls of RAGs, including in retrieval, explain why we need systematic evaluation and provide a quick introduction to existing evaluation frameworks.