Proof layer

Skip 80% of your POC. Every major release is tested on real use cases with a public harness, version-pinned and re-run when a model or platform changes. Validate the last 20% by running the same harness on your own data.

Evidence levels on every claim: Measured by us · Vendor-documented · Community-reproduced · Community-reported. Only measured proof runs change a recommendation.

RAG over documents

Measures: Answer accuracy, citation faithfulness, hallucination rate

RAG over documents (Answer accuracy)

Claude Opus 5.5 · Direct API91.2%

Claude Opus 5 · Direct API90.4%

Claude Sonnet 5 · Direct API89.8%

Gemini Pro 3 (demo) · Google Vertex AI88.9%

Source: aimluptodate.com proof runs · harness a1b2c3d · 23 Sept 2026

Suite	Version	Endpoint	Results	Run cost	Date
RAG over documents Measured by us	Claude Opus 5.5	Direct API · `global`	Answer accuracy: 91.2% Citation faithfulness: 95% Hallucination rate: 2.1%	$18.40	23 Sept 2026
RAG over documents Measured by us	Claude Opus 5	Direct API · `global`	Answer accuracy: 90.4% Citation faithfulness: 94.1% Hallucination rate: 2.6%	$23.10	23 Sept 2026
RAG over documents Measured by us	Claude Sonnet 5	Direct API · `global`	Answer accuracy: 89.8% Citation faithfulness: 93.5% Hallucination rate: 2.9%	$9.30	23 Sept 2026
RAG over documents Measured by us	Gemini Pro 3 (demo)	Google Vertex AI · `asia-south1`	Answer accuracy: 88.9% Citation faithfulness: 91% Hallucination rate: 3.4%	$10.20	16 Sept 2026

Agents and tool use

Measures: Task success rate, steps taken, error recovery

Agents and tool use (Task success rate)

Claude Opus 5.5 · AWS Bedrock84.5%

Claude Opus 5 · AWS Bedrock83%

Source: aimluptodate.com proof runs · harness a1b2c3d · 23 Sept 2026

Suite	Version	Endpoint	Results	Run cost	Date
Agents and tool use Measured by us	Claude Opus 5.5	AWS Bedrock · `us-east-1`	Task success rate: 84.5% Median steps: 7 steps Error recovery: 78%	$41.70	23 Sept 2026
Agents and tool use Measured by us	Claude Opus 5	AWS Bedrock · `us-east-1`	Task success rate: 83% Median steps: 8 steps Error recovery: 75.5%	$52.90	23 Sept 2026

Coding

Measures: Tasks passed on a fixed repo set, edit correctness

No proof runs yet.

Structured output

Measures: Schema validity, extraction accuracy

No proof runs yet.

Long context

Measures: Recall at different context lengths

No proof runs yet.

Cost and latency

Measures: Cost per task, time to first token, tokens/sec by cloud and region

Cost and latency (Cost per task, lower is better)

Claude Opus 5.5 · Direct API0.037 USD

Claude Opus 5.5 · AWS Bedrock0.039 USD

Source: aimluptodate.com proof runs · harness a1b2c3d · 23 Sept 2026

Suite	Version	Endpoint	Results	Run cost	Date
Cost and latency Measured by us	Claude Opus 5.5	Direct API · `global`	Cost per task: 0.037 USD Time to first token (p50): 1.9 s Output speed (p50): 58 tok/s	$3.70	23 Sept 2026
Cost and latency Measured by us	Claude Opus 5.5	AWS Bedrock · `ap-south-1`	Cost per task: 0.039 USD Time to first token (p50): 1.4 s Output speed (p50): 52 tok/s	$3.90	23 Sept 2026