EveryDev.ai
Subscribe
Home
Tools

5,923+ AI tools

  • Featured
  • Gems
  • Trending
  • New
  • Rate tools
  • Compare
  • Arena
Categories
  • Agents2992
  • Coding2260
  • Infrastructure1068
  • Projects824
  • MCP795
  • Research744
  • Analytics567
  • Marketing527
  • Design489
  • Testing476
  • Data442
  • Security417
  • Integration288
  • Communication286
  • Prompts284
  • Extensions239
  • Voice214
  • DevOps191
  • Commerce184
  • Web128
  • Learning123
  • Finance44
AI Tools by Topic
  • AI Coding Assistants
  • Agent Frameworks
  • MCP Servers
  • AI Prompt Tools
  • Vibe Coding Tools
  • AI Design Tools
  • AI Database Tools
  • AI Website Builders
  • AI Testing Tools
  • LLM Evaluations
Follow Us
  • X / Twitter
  • LinkedIn
  • Reddit
  • Discord
  • Threads
  • Bluesky
  • Mastodon
  • YouTube
  • GitHub
  • Instagram
Get Started
  • Users
  • Rate Tools
  • About
  • Editorial Standards
  • Corrections & Disclosures
  • Community Guidelines
  • Advertise
  • Contact Us
  • Newsletter
  • Submit a Tool
  • Start a Discussion
  • Write A Blog
  • Share A Build
  • Terms of Service
  • Privacy Policy
Explore with AI
  • ChatGPT
  • Gemini
  • Claude
  • Grok
  • Perplexity
Agent Experience
  • llms.txt
Theme
With AI, Everyone is a Dev. EveryDev.ai ©
    1. Home
    2. Tools
    3. ttok
    ttok icon

    ttok

    Context Engineering
    Featured

    Command-line tool that counts and truncates text based on LLM tokens using OpenAI's tiktoken library.

    Visit Website

    At a Glance

    Pricing
    Open Source

    ttok CLI for counting and truncating text by tokens, released under the Apache License 2.0.

    Engagement

    Available On

    CLI
    macOS
    Linux
    Windows

    Resources

    WebsiteDocsGitHubllms.txt

    Topics

    Context EngineeringCommand Line AssistantsAI Development Libraries

    Alternatives

    ccusageDembrandtclauditor
    Developer
    Simon WillisonHalf Moon Bay, CAEst. 2019$1.4M raised

    Listed Nov 2026

    About ttok

    ttok is an open-source Python command-line tool by simonw that counts tokens in text and truncates text to a set number of tokens. It uses OpenAI's tiktoken library and reads text from arguments, stdin, or files. The latest release listed is version 1.0.

    What It Is

    ttok is a CLI utility for working with the token-based limits of large language models. You pass text as arguments, pipe it in, or read it from a file with -i, and it prints the token count. It can also truncate input to a given number of tokens.

    Tokenizer models and token inspection

    The default tokenizer is the GPT-5 encoding (o200k_base), which changed in ttok 1.0. The -m/--model option selects other tokenizers, such as gpt-3.5-turbo (cl100k_base) or gpt2, and --list-models shows the model names and prefixes supported by the installed tiktoken. Because token counts and truncation results can differ between tokenizers, the README advises using the same model when encoding and decoding.

    The --encode option outputs integer token IDs, --decode converts IDs back to text, and --tokens shows a breakdown of the individual tokens. The --allow-special option lets special token strings such as <|endoftext|> be recognized instead of causing an error.

    Setup path

    Install with pip, uv tool install, or Homebrew, or run it without installing via uvx ttok. It can also be run with python -m ttok.

    ttok - 1

    Community Discussions

    Be the first to start a conversation about ttok

    Share your experience with ttok, ask questions, or help others learn from your insights.

    Pricing

    OPEN SOURCE

    Open Source

    ttok CLI for counting and truncating text by tokens, released under the Apache License 2.0.

    • Count tokens in text or files
    • Truncate text to a set number of tokens
    • Encode and decode token IDs
    • Supports multiple tiktoken models
    • Licensed under Apache License 2.0

    Capabilities

    Key Features

    • Count tokens from arguments, stdin, or files
    • Truncate text to a specified number of tokens
    • Select tokenizer model with --model
    • Encode text to token IDs and decode IDs back to text
    • Show detailed token breakdown
    • Allow special tokens with --allow-special
    • List supported model names and prefixes

    Integrations

    OpenAI tiktoken
    Homebrew
    PyPI

    Ratings & Reviews

    No ratings yet

    Be the first to rate ttok and help others make informed decisions.

    Rate other tools you’ve used

    Developer

    Simon Willison

    Open-source developer and creator of Datasette and LLM; builds Python and SQLite-centric tools for data journalism and developer workflows.

    Founded 2019
    Half Moon Bay, CA
    $1.4M raised
    1 employees

    Used by

    The Guardian (newsroom data-journalism…
    Lawrence Journal-World (newsroom…
    Eventbrite (employer following its…
    Read more about Simon Willison
    WebsiteGitHubX / Twitter
    6 tools in directory

    Similar Tools

    ccusage icon

    ccusage

    A fast local CLI for tracking tokens and estimated costs across Claude Code, Codex, Gemini CLI, GitHub Copilot CLI, and 12+ other coding agent CLIs.

    Dembrandt icon

    Dembrandt

    Extract any website's design system into design tokens in seconds — logo, colors, typography, borders, and more — with a single CLI command.

    clauditor icon

    clauditor

    A CLI tool that monitors Claude Code session token usage, auto-rotates oversized sessions, and preserves context to prevent quota exhaustion.

    Browse all tools

    Related Topics

    Context Engineering

    Techniques for optimizing context windows to improve AI responses.

    122 tools

    Command Line Assistants

    AI-powered command-line assistants that help developers navigate, search, and execute terminal commands with intelligent suggestions and context awareness.

    409 tools

    AI Development Libraries

    Programming libraries and frameworks that provide machine learning capabilities, model integration, and AI functionality for developers.

    573 tools
    Browse all topics
    Back to all toolsSuggest an edit
    ratings
    discussions
    9views