rag-audit

CLI + library to audit and benchmark RAG pipelines.

What it does

• Hallucination detection — LLM-as-judge faithfulness scoring (0.0–1.0)
• Retrieval metrics — Precision@k, Recall@k, MRR
• Chunking benchmark — compare fixed-size vs recursive vs semantic strategies by cohesion
• Vectorstore adapters — uniform interface for ChromaDB (Pinecone and Qdrant coming soon)
• Structured reports — JSON and Markdown output

Install

pip install rag-audit

Links

• Quickstart — up and running in 5 minutes
• API Reference — full module documentation
• GitHub