iFixAi

iFixAi is an open-source independent auditing engine for AI agents, published under the Apache 2.0 license and available on GitHub. It addresses a gap that existing eval, red-teaming, and observability tools leave open: whether an agent is actually doing the job it was assigned, within its authority, and without deceiving or harming the people it serves. The project reached the #1 Python repository of the week on Trendshift and has accumulated over 16,500 GitHub stars since its April 2026 launch.

What It Is

iFixAi is a CLI-first diagnostic tool that runs a structured audit against any AI agent — whether a bare model API, an OpenAI-compatible HTTP endpoint, or a custom adapter — and returns an A–F letter grade backed by a scored five-pillar scorecard. The audit covers 60 inspections grouped into 25 categories, spanning AI red teaming, operational assurance, philosophical alignment, ethical behavior, and sociological risk. The core argument is that standard evals measure task performance (did the agent complete the job?) but miss authority, workflow compliance, responsibility, and evidence — the dimensions that determine whether a deployed agent is actually safe to trust with money, data, or customers.

Five-Pillar Scoring Model

The graded scorecard weighs five core pillars:

• Fabrication — unauthorized tool use, missing audit trails, unsourced or overconfident claims
• Manipulation — privilege escalation, policy violations, prompt injection, poisoned retrieval
• Deception — sandbagging, secret side-goals, silent failures, long-run task drift
• Unpredictability — distorted context, instruction drift, inconsistent decisions
• Opacity — weak risk scoring, regulatory gaps, broken human-escalation paths

Manipulation carries the highest weight (0.35), with the remaining four pillars at 0.15–0.20 each. Mandatory minimums on specific inspections can cap the overall grade at 60% regardless of other scores. The 20 premium categories — including insubordination, oversight atrophy, stakeholder conflict, and vulnerable user care — are scored and reported separately and do not affect the grade, keeping results comparable across agents with different capability exposures.

Three Ways to Run

iFixAi supports three interaction modes that all drive the same diagnostic engine:

• Guided wizard (ifixai setup → ifixai run): recommended for first-time users and team onboarding; writes a ifixai.yaml config and requires no flags on subsequent runs
• Explicit flags: fully scriptable for CI pipelines and audit-ready batches
• Plugin or Skill: the agent itself is the operator — it discovers the setup, builds the fixture, names the cost before billing, and walks through the scorecard interactively; supported in Claude Code, Codex, Cursor, VS Code, Windsurf, Cline, Continue, Gemini, and Zed

Connecting an agent requires either a GitHub repository (iFixAi reads the code and builds the simulation environment) or an MCP setup prompt pasted into a supported IDE. Repositories with an AGENTS.md file are auto-detected.

Independent Judging Architecture

A key design principle is that the agent under test never grades itself. Every run has two roles: the SUT (system under test) and an independent judge from a different vendor. The judge grades the SUT's answers; the SUT's own vendor is excluded from the judge pool automatically. A grade is described as "citable" only when a second, independent provider performed the grading. Multi-judge ensemble mode is also supported for cross-vendor robustness.

Update: v4.0.0 V-Series Inspections

The latest release, v4.0.0 ("V-Series Inspections"), was published on September 15, 2026. The repository was last updated September 29, 2026, and last pushed September 25, 2026, indicating active development. The project launched in April 2026 and has shipped four major versions in roughly five months, with the inspection count growing from an initial set to 60 total (32 core + 28 premium preview). The open-source engine ships with 60 inspections and community support at no cost; a commercial cloud offering with higher inspection counts, audit badges, and multi-agent support is available for enterprise teams.