AgentSkillOS

AgentSkillOS is an open-source framework published on GitHub by researcher Hao Li and collaborators, designed to act as an "operating system" for agent skills. It addresses the challenge of finding and composing the right skills from a rapidly growing ecosystem of over 200,000 publicly available agent skills. The project is accompanied by an arXiv paper (arXiv:2603.02176) and a Hugging Face benchmark dataset.

What It Is

AgentSkillOS provides structured skill discovery, retrieval, and orchestration so that AI agents can build end-to-end pipelines from large skill ecosystems. Rather than relying on pure semantic similarity search, it organizes skills into a hierarchical capability tree and uses an LLM to navigate that tree, surfacing non-obvious but functionally relevant skills. Selected skills are then composed into a directed acyclic graph (DAG) workflow that automatically manages execution order, dependencies, and data flow.

How the Skill Tree and Orchestration Work

The core method has three stages:

• Skill Tree Construction — Over 200,000 skills are organized into a capability tree, providing coarse-to-fine access for efficient and creative skill discovery. Pre-built trees are available at scales of ~50, 200, 1,000, and 10,000 skills.
• Skill Retrieval — An LLM navigates the capability hierarchy to select a task-relevant subset of skills, overcoming the myopic results of pure embedding-based retrieval.
• Skill Orchestration — Selected skills are composed into a DAG-based workflow. Three orchestration strategies are supported: Quality-First (deep multi-stage pipelines), Efficiency-First (wide parallel execution), and Simplicity-First (minimal essential steps). A freestyle mode using Claude Code is also available.

Benchmark and Experimental Results

The project introduces a benchmark of 30 multi-format creative tasks across 5 categories, evaluated with pairwise comparison and Bradley-Terry score aggregation. Tasks require end-user artifacts in formats such as PDF, PPTX, DOCX, HTML, video, and generated images. According to the authors' experiments across 200, 1K, and 200K skill ecosystems, AgentSkillOS variants achieve the highest Bradley-Terry scores over baselines at every scale. Ablation studies reported by the authors confirm that both retrieval and orchestration components are individually necessary — removing either causes measurable degradation.

Interfaces and Deployment

AgentSkillOS ships with two primary interfaces:

• Web UI — A browser-based visual workflow overview with a built-in GUI for human-in-the-loop intervention at every step, making workflows controllable and auditable.
• Batch CLI — Headless parallel execution driven by YAML configs, with resume support, Rich terminal progress UI, dry-run preview, and per-task artifact output directories.

Installation requires Python 3.10+, Claude Code in PATH, and API keys for an LLM provider and an embedding model. The tool supports OpenRouter and OpenAI-compatible endpoints via environment variables.

Update: March 2026 Releases

The project had a concentrated release burst in March 2026. The project homepage launched, the benchmark dataset was released on Hugging Face, the modular architecture with pluggable retrieval and orchestration modules was published (documented in ARCHITECTURE.md), and the Batch CLI was released with headless parallel execution, YAML configs, resume support, and a Rich progress UI. The repository was last pushed on 2026-03-07 and has accumulated 593 stars and 75 forks as of the latest GitHub metadata.