Measured by us

RAG over documents — Gemini Pro 3 (demo) on Google Vertex AI (asia-south1)

Results

Answer accuracy	88.9% higher is better
Citation faithfulness	91% higher is better
Hallucination rate	3.4% lower is better

Run details

Model version: demo-gemini-pro-3
Endpoint: Vertex AI generateContent (demo)
Platform / region: Google Vertex AI / asia-south1
Dataset: docs-qa-500 (500 questions over 120 technical PDFs)
Harness: GitHub @ a1b2c3d
Run cost: $10.20
Date: 16 Sept 2026

What was NOT tested

Non-English documents
Scanned PDFs needing OCR
Retrieval quality (fixed retriever used for all models)

Re-run it on your data

git clone https://github.com/mallimatla/uptodate.git && cd uptodate && git checkout a1b2c3d
npm install && npm run proof -- --suite rag --model demo-gemini-pro-3 --platform vertex --region asia-south1 --dataset ./your-data

Sample raw output

Three retries with exponential backoff are used by default (doc 14).