Tool Definition Quality Score (TDQS)

TDQS (Tool Definition Quality Score) is an open framework built by Glama for scoring how well MCP tool definitions communicate to AI agents. It grades every tool definition on six weighted dimensions, produces a letter tier (A–F), and surfaces actionable justifications so maintainers know exactly what to fix. The framework is the same methodology Glama uses to score every tool across its MCP registry.

What It Is

TDQS is a quality-scoring standard for MCP tool definitions — the structured metadata (name, description, inputSchema, outputSchema, annotations) that an AI agent reads before deciding which tool to call. Because the description is the primary signal an agent uses for tool selection, poorly written definitions cause mis-selection and task failure. TDQS provides a reproducible, explainable rubric that turns subjective "is this description good?" into a deterministic score and letter grade.

The framework is motivated by two empirical studies cited in the specification: one auditing 856 tools across 103 MCP servers found that 97% of tool descriptions contain at least one quality defect and 89% never say when the tool should be used; a second analyzing 10,831 MCP servers found that tools with well-written descriptions are selected approximately 260% more often in competitive settings.

Scoring Pipeline and Architecture

Each tool flows through a four-stage pipeline:

• Stage 1 – Context signals: Deterministic code extracts structural facts (parameter count, schema coverage, annotation values, invocation cost, input hash) without any model call.
• Stage 2 – Hard gates: Degenerate definitions (missing or tautological descriptions) short-circuit with a floor score and a flag, skipping the LLM call entirely.
• Stage 3 – LLM rubric: A single model call scores six dimensions from 1 to 5, each with a 2–3 sentence justification citing specific evidence. The prompts are published verbatim in the specification.
• Stage 4 – Post-processing: Deterministic code applies overrides, flags, and smells; integer arithmetic rolls up the weighted sum to a score and tier.

The six per-tool dimensions and their weights are: Purpose Clarity (25%), Usage Guidelines (20%), Behavioral Transparency (20%), Parameter Semantics (15%), Conciseness & Structure (10%), and Contextual Completeness (10%). Four additional server-level dimensions — Disambiguation, Naming Consistency, Tool Count Appropriateness, and Completeness — judge the tool set as a whole and feed a coherence score (30% of the overall server score).

Three Access Paths

TDQS is available through three interfaces that all run the same reference implementation:

• Playground: Sign in with GitHub, paste a tools/list result, and score it through the hosted API in the browser. Every run produces a shareable report with an SVG badge.
• CLI: npx mcp-tdqs score (also on PyPI) scores a local or remote server from the terminal or CI, with a --fail-under flag to gate releases on a minimum tier.
• Hosted API: Submit definitions over HTTP, poll the report, and embed results anywhere.

Registry Scale and Adoption Signals

According to Glama's published registry figures (as of June 2026), TDQS has scored 228,369 tools across 15,036 MCP servers, with 99% coverage of the registry. The aggregate corpus shows a mean TDQS of 3.57, with 73.5% of tools clearing the tier-B passing bar. The weakest dimensions by smell rate are Behavioral Transparency (46.1% of tools score below 3) and Usage Guidelines (44.5%), corroborating the motivating literature. Glama states that 56.4% of scored tools carry at least one smell.

Update: v1.2 Specification

The current published specification is v1.2, with the repository last pushed in September 2026. The specification is open-source on GitHub under the glama-ai organization. Notable recent additions include the shadowing risk detection system — a two-step check (deterministic prefilter + LLM confirmation) that identifies tools whose purpose is substantially covered by a cheaper-to-invoke sibling — and the invocationCost signal, which estimates how much work it is for an agent to fill a tool's required schema. The v1.3 direction visible in the spec includes changes to how hasOutputSchema is handled (the output schema itself is passed to the evaluator rather than a boolean flag).

Why It Matters for MCP Developers

For developers maintaining MCP servers, TDQS doubles as a checklist. The highest-leverage fixes in weight order are: stating what the tool does with a specific verb + resource + scope; naming when (and when not) to use it and which alternative to reach for instead; declaring MCP annotations to lower the description's disclosure burden; documenting every parameter in the schema or description; providing a documented output schema; and cutting anything that repeats structured fields. The framework explicitly rewards information density over word count — no credit is given for restating what the schema or annotations already declare.