Evidence Qa

What it is

A Python library and CLI for building retrieval-augmented generation systems. It ingests PDF, DOCX, TXT, MD, and CSV files into named collections, embeds them using BGE models, and answers user questions by retrieving relevant chunks, reranking them with a cross-encoder, and synthesizing a concise answer via an LLM. It prioritizes factual grounding, returning 'I don't have information' if evidence is insufficient.

Features

• Ingests PDF, DOCX, TXT, MD, and CSV files into named collections
• Semantic retrieval using BGE embeddings with cross-encoder reranking
• LLM-synthesized answers with source citations or safe abstention
• File-based persistence for chunks and embeddings without external databases
• Minimal FastAPI web UI for document upload and interactive querying
• Headless library API for integration into custom agents or workflows

Quickstart

python -m venv .venv && ./.venv/bin/pip install -e ".[ui,local-llm]"
./.venv/bin/docqa ingest gardening tomatoes.pdf notes.md
./.venv/bin/docqa ask gardening "how often should I water tomatoes?"
./.venv/bin/uvicorn app:app --port 8000

Architecture

flowchart TD
    User[User]
    CLI[CLI Interface]
    WebUI[FastAPI Web UI]
    DocQA[DocQA Engine]
    Ingest[Ingestion Module]
    Store[Collection Store]
    EvidenceQA[Evidence QA Primitives]
    Semantic[Semantic Retrieval BGE]
    Rerank[Cross Encoder Rerank]
    LLM[LLM Synthesis]
    Files[Document Files]
    Chunks[Chunks JSONL]
    Embeddings[Embeddings NPY]
    User --> CLI
    User --> WebUI
    CLI --> DocQA
    WebUI --> DocQA
    DocQA --> Ingest
    DocQA --> EvidenceQA
    Ingest --> Files
    Ingest --> Store
    Store --> Chunks
    Store --> Embeddings
    EvidenceQA --> Semantic
    EvidenceQA --> Rerank
    EvidenceQA --> LLM
    Semantic --> Embeddings
    Rerank --> Semantic
    LLM --> Rerank

How it's built

The system uses a two-layer architecture: a generic `docqa` engine for dynamic file ingestion and a specialized `evidence_qa` module for fixed-corpus benchmarks. The pipeline consists of semantic retrieval (BGE embeddings), cross-encoder reranking, and LLM synthesis. It supports local GGUF models for synthesis or falls back to extractive answers if no LLM backend is configured. Storage is file-based using JSONL for chunks and NumPy arrays for embeddings.

How it runs

sequenceDiagram
    participant User
    participant App as Application
    participant DocQA as DocQA Engine
    participant Store as Collection Store
    participant Retriever as Semantic Retriever
    participant Reranker as Cross Encoder
    participant Synthesizer as LLM Synthesizer
    User->>App: ask question
    App->>DocQA: ask collection question
    DocQA->>Store: load embeddings and chunks
    Store-->>DocQA: return vectors and text
    DocQA->>Retriever: embed query and search
    Retriever-->>DocQA: return top k chunks
    DocQA->>Reranker: rerank chunks
    Reranker-->>DocQA: return ranked chunks
    DocQA->>Synthesizer: generate answer from context
    Synthesizer-->>DocQA: return cited answer
    DocQA-->>App: return result
    App-->>User: display answer and citations

How to apply & reuse

Use the `docqa` library to ingest documents into a collection via `ingest()` or the CLI, then query using `ask()`. Integrate the `retrieve()` and `ask()` functions into other applications, such as voice agents or customer support bots, to provide grounded responses. The FastAPI app (`app.py`) serves as a reference UI for upload and query interactions.

At a glance