EveryDev.ai
Subscribe
Home
Tools

5,901+ AI tools

  • Featured
  • Gems
  • Trending
  • New
  • Rate tools
  • Compare
  • Arena
Categories
  • Agents2972
  • Coding2248
  • Infrastructure1067
  • Projects818
  • MCP794
  • Research744
  • Analytics565
  • Marketing527
  • Design488
  • Testing476
  • Data442
  • Security412
  • Integration288
  • Communication284
  • Prompts283
  • Extensions238
  • Voice213
  • DevOps190
  • Commerce183
  • Web128
  • Learning123
  • Finance44
AI Tools by Topic
  • AI Coding Assistants
  • Agent Frameworks
  • MCP Servers
  • AI Prompt Tools
  • Vibe Coding Tools
  • AI Design Tools
  • AI Database Tools
  • AI Website Builders
  • AI Testing Tools
  • LLM Evaluations
Follow Us
  • X / Twitter
  • LinkedIn
  • Reddit
  • Discord
  • Threads
  • Bluesky
  • Mastodon
  • YouTube
  • GitHub
  • Instagram
Get Started
  • Users
  • Rate Tools
  • About
  • Editorial Standards
  • Corrections & Disclosures
  • Community Guidelines
  • Advertise
  • Contact Us
  • Newsletter
  • Submit a Tool
  • Start a Discussion
  • Write A Blog
  • Share A Build
  • Terms of Service
  • Privacy Policy
Explore with AI
  • ChatGPT
  • Gemini
  • Claude
  • Grok
  • Perplexity
Agent Experience
  • llms.txt
Theme
With AI, Everyone is a Dev. EveryDev.ai ©
    1. Home
    2. Tools
    3. agent-traffic
    agent-traffic icon

    agent-traffic

    Threat Detection
    Featured

    Open-source dataset and classifier that tells AI agents, AI crawlers, search crawlers and bots apart from human HTTP traffic.

    Visit Website

    At a Glance

    Pricing
    Open Source

    The agent-traffic package and dataset are released under the MIT License and are free to use, copy, modify and distribute.

    Engagement

    Available On

    Web
    API
    VS Code
    SDK
    CLI

    Resources

    WebsiteDocsGitHubllms.txt

    Topics

    Threat DetectionWeb AnalyticsAI Development Libraries

    Alternatives

    BotscentTrifleThe Crawler Zoo
    Developer
    MintlifySuite 1800Est. 2022$67M raised

    Listed Nov 2026

    About agent-traffic

    agent-traffic is an MIT-licensed project from Mintlify that classifies HTTP requests. It combines a curated, tested dataset of user agents, headers and official verification sources for real agents and crawlers with a small classify function that applies it. It ships as the npm package @mintlify/agent-traffic.

    What It Is

    agent-traffic is a bot and agent detection library. Given a user agent, request headers and optionally a client IP, it returns a category (agent, ai-crawler, crawler, bot, human or unknown), a more specific kind, the product name and operator where known, a verification flag, a confidence level and a list of reasons explaining the result.

    The project's stated motivation is that simple bot lists match user agent substrings, accept whatever a request claims, and lump all automation together. Mintlify says it built this after studying traffic across the documentation sites it hosts.

    How classification works

    Checks run in order and the first to answer wins: known agent or crawler (verified by IP where possible), a request asking for markdown, generic HTTP client or headless browser, a browser unless its headers contradict it, and otherwise a bot.

    • Crawlers can be checked against IP ranges published by operators such as OpenAI, Anthropic, Google, Perplexity, Apple, Microsoft, Common Crawl and DuckDuckGo, so a spoofed GPTBot is flagged as an impersonator.
    • Agents that send a browser user agent but prefer text/markdown in the Accept header are identified as unidentified agents.
    • Optional fake-browser checks are enabled with headersComplete.
    • Coding agents that run on a user's own machine are never IP-checked.

    Using it

    The package has zero dependencies and runs wherever fetch does, including Node, Bun, Deno, Cloudflare Workers and Vercel Edge. A typical use is a request handler that serves markdown to agents. It also supports classifying stored logs, with patterns translatable to ClickHouse, DuckDuckGo-style DuckDB and Postgres queries, and exposes per-category regular expressions plus the dataset as JSON.

    An installable skill (npx skills add mintlify/agent-traffic) teaches coding agents such as Claude Code, Cursor and Codex how to use the package.

    Known limits

    Some products share an identity, such as Codex reportedly fetching as ChatGPT-User. Agentic browsers running inside a person's own Chrome look like that person at the HTTP level. Logs that keep only the user agent will miss agents using browser user agents unless the Accept header is also logged.

    agent-traffic - 1

    Community Discussions

    Be the first to start a conversation about agent-traffic

    Share your experience with agent-traffic, ask questions, or help others learn from your insights.

    Pricing

    OPEN SOURCE

    Open Source (MIT)

    The agent-traffic package and dataset are released under the MIT License and are free to use, copy, modify and distribute.

    • MIT-licensed TypeScript package @mintlify/agent-traffic
    • Curated dataset of user agents, headers and verification sources
    • classify() function with IP range verification
    • Zero dependencies; runs on Node, Bun, Deno, Cloudflare Workers, Vercel Edge
    • Install via npm i @mintlify/agent-traffic

    Capabilities

    Key Features

    • Request classification into agent, ai-crawler, crawler, bot, human and unknown categories
    • Curated dataset of agent and crawler user agents, headers and verification sources
    • IP range verification against operators' published ranges
    • Impersonator detection
    • Markdown Accept-header detection for agents using browser user agents
    • Fake-browser header checks
    • Human-readable reasons and confidence for every result
    • Per-category regex patterns
    • Zero dependencies; runs on Node, Bun, Deno, Cloudflare Workers and Vercel Edge
    • Installable agent skill for coding agents

    Integrations

    Claude Code
    Cursor
    Codex
    Cloudflare Workers
    Vercel Edge
    ClickHouse
    DuckDB
    Postgres
    API Available
    View Docs

    Ratings & Reviews

    No ratings yet

    Be the first to rate agent-traffic and help others make informed decisions.

    Rate other tools you’ve used

    Developer

    Mintlify

    Mintlify builds the intelligent knowledge platform that helps teams create and maintain world-class documentation for both humans and AI. The company supports llms.txt, MCP servers, and AI-native workflows so product documentation surfaces correctly in the tools developers already use. Mintlify serves companies ranging from frontier AI labs to global enterprises, offering enterprise-grade security, SOC 2 compliance, and white-glove migration support. The team raised a $45M Series B to expand its vision of building knowledge infrastructure for the age of AI agents.

    Founded 2022
    1 Post St
    $67M raised
    62 employees

    Used by

    Anthropic
    Amazon
    Coinbase
    Cognition
    +14 more
    Read more about Mintlify
    WebsiteGitHubLinkedInX / Twitter
    2 tools in directory

    Similar Tools

    Botscent icon

    Botscent

    Open-source library that tells a website whether an AI agent is browsing it and identifies which agent.

    Trifle icon

    Trifle

    Open-source time-series analytics platform for tracking business metrics in Ruby, Elixir, and Go using your existing database, with dashboards, alerts, CLI, and AI agent integration via MCP.

    The Crawler Zoo icon

    The Crawler Zoo

    A live web exhibit that identifies, catalogues and displays the bots and crawlers visiting the site, with robots.txt tools for site owners.

    Browse all tools

    Related Topics

    Threat Detection

    AI tools that detect and analyze security threats and anomalies.

    69 tools

    Web Analytics

    AI-powered tools that analyze website traffic, user behavior, and engagement patterns to provide actionable insights for optimizing digital experiences and conversion rates.

    54 tools

    AI Development Libraries

    Programming libraries and frameworks that provide machine learning capabilities, model integration, and AI functionality for developers.

    573 tools
    Browse all topics
    Back to all toolsSuggest an edit
    ratings
    discussions