Amplifying

Amplifying is an AI benchmarking research studio that measures the opinionated, subjective decisions AI systems make every time they run — from what tools to install to what products to recommend. Rather than testing factual accuracy, Amplifying benchmarks AI judgment at scale, running thousands of prompts across multiple models and real repositories to surface patterns in AI behavior. The studio publishes open research studies, raw datasets, and vendor intelligence reports to make AI decision-making visible and accountable.

• Systematic AI Benchmarking: Run large-scale studies (e.g., 2,430 responses across 3 models, 4 repos, 20 categories) to quantify how AI agents make subjective choices.
• Claude Code Picks Study: Pointed Claude Code at real repositories and tracked tool/library recommendations across 20 categories, revealing that Custom/DIY is the #1 recommendation in 12 of 20 categories.
• AI Product Recommendations Research: Asked Google AI Mode and ChatGPT 792 product questions, uncovering 47% cross-platform disagreement, Shopping Graph bias, and significant output drift.
• Tech Directory: Browse 80+ tools across 20 categories with pick rates and model breakdowns derived from benchmark data.
• Vendor Intelligence Reports: Request a custom report to see how AI coding agents position your developer tool, including competitive analysis, model trends, and agent verbatim quotes.
• Upcoming Benchmarks: Security Defaults (OWASP Top 10 audits of AI-generated apps) and Dependency Footprint (package sprawl analysis) are in progress.
• Open Data: Raw benchmark data and open-source benchmark code are published on GitHub for transparency and reproducibility.
• Subscription Updates: Subscribe to get notified when new benchmarks drop, keeping researchers and vendors informed of the latest findings.