Easy Dataset

Easy Dataset is an application for building large language model datasets from domain-specific documents. It combines document parsing, text segmentation, question and answer generation, dataset management, and export in one project-based workflow. It is available as desktop clients for Windows, macOS, and Linux, via Docker, or by running from source with NPM. The latest release listed on GitHub is 1.7.3.

What It Is

Easy Dataset is a dataset-creation tool for teams fine-tuning or evaluating LLMs. Its documentation says the hard part of fine-tuning is usually dataset preparation, and the tool targets a closed loop from document parsing through dataset construction, annotation, export, and evaluation. The README says the resulting datasets suit fine-tuning, retrieval-augmented generation (RAG), and model performance evaluation.

How the Workflow Runs

Work is organized by project. Documents in PDF, Markdown, DOCX, TXT, EPUB and other formats are parsed and split with several algorithms: Markdown structure, recursive separators, fixed length, and code-aware chunking. The tool then extracts questions from text segments, generates answers (including Chain of Thought answers) through an LLM API, and organizes data under an AI-generated domain label tree that can be edited by hand. Questions and answers can be deleted, edited, or refined by AI. Data cleaning and AI quality scoring help filter output.

Dataset Types and Export

Besides single-turn QA, it supports multi-turn dialogue datasets, image QA datasets, and data distillation from a topic without uploading documents. Exports use Alpaca, ShareGPT, or Multilingual-Thinking formats as JSON or JSONL, with balanced export per tag. It can generate LLaMA Factory configuration files and upload datasets to Hugging Face Hub.

Evaluation and Model Support

Version 1.7.0 added evaluation: generating test sets (true/false, single choice, multiple choice, short answer, open-ended), automated judging with a Judge Model, and a double-blind human Arena comparison. It works with any LLM API following the OpenAI format, plus local models through Ollama. Vision models can be used for PDF parsing and image QA. A playground compares up to three models side by side.

Deployment Notes

The source is licensed under AGPL-3.0 with additional terms covering branding and alternative licensing. The Docker setup mounts local-db and prisma folders, and the app is served at localhost:1717.