Synthetic Data Generator

Synthetic Data Generator is an open-source tool from Argilla that creates datasets for training and fine-tuning language models from a plain-language description of the target application. It is built on distilabel and LLMs, and it runs as a Gradio app that can be launched from a Python package. The repository notes that the original authors have moved on and that a successor project, aisheets, now exists.

What It Is

The tool turns a description of the characteristics you want into a sample dataset, which you can iterate on before producing a full-scale dataset. Supported tasks are text classification, chat data for supervised fine-tuning, and retrieval augmented generation. Finished datasets can be pushed to the Hugging Face Hub or to Argilla, or saved to a local directory.

How the Workflow Runs

You describe the application, review a generated sample, adjust it, and then generate the full dataset. Each pipeline is based on distilabel, so the LLM or the pipeline steps can be swapped. Environment variables set the model, the maximum number of tokens and rows, and the batch size. Generation can use Hugging Face Inference Endpoints or OpenAI-compatible, Ollama, vLLM and other Hugging Face-compatible APIs. SFT and chat data generation is not supported with OpenAI endpoints.

Setup Path

Install it with pip and call launch() from Python, or run the app from source. A Docker Compose setup bundles the app with Ollama for local LLM inference and Argilla for data curation. A Hugging Face token is used to push datasets to the Hub and to generate completions with Hugging Face Inference Endpoints.

Current Status

The README states that the original team does not plan new features, bug fixes, or updates, and invites anyone interested in maintaining it to open an issue. It points to huggingface/aisheets as the successor. The latest listed release is 0.2.0, which added Docker support.