Adrien Barbaresi
Adrien Barbaresi is an individual research scientist, data engineer and open-source developer specializing in natural-language processing, language data and web text extraction. He builds and maintains open-source tools—especially Trafilatura—that gather, clean and structure text from the Web for researchers and AI/LLM/RAG applications.
At a Glance
- Academic researchers and research institutes
- Computational linguistics, NLP and digital-humanities teams
- AI, LLM and RAG developers
- Data engineers and web-corpus builders
- +1 more
AI Tools by Adrien Barbaresi
(1)
Trafilatura
Python Web Text Extraction Tool
Discussions
No discussions yet
Be the first to start a discussion about Adrien Barbaresi
Latest News
Trafilatura 2.3.1 released on GitHub and PyPI; the repository records ongoing fixes and maintenance work.
Trafilatura 2.3.0 released with improved metadata handling and more robust downloads and link discovery.
Published “The Art of Extraction: Trafilatura’s Connection to Pasta,” a new post on the project's name and extraction work.
Published “Respectful Web Crawling and Downloads with Courlan,” explaining the related URL and downloading component.
Products & Services
Open-source Python package and command-line tool for web crawling, discovery, downloading and extraction of main text, metadata and comments from HTML; it can emit TXT, Markdown, CSV, JSON, HTML, XML and XML-TEI.
Python and command-line tooling to clean, filter, sample, deduplicate and prioritize URLs for respectful and efficient data collection, including content and language filtering.
Python and command-line library for fast, robust extraction of publication dates from web pages.
Simple multilingual lemmatizer for Python, designed for speed and efficiency in language and text-analysis workflows.
Market Position
Trafilatura is positioned as a modular, open-source Python/CLI alternative to web-content extraction stacks such as jusText and readability-lxml. Its differentiator is combining discovery/crawling, polite downloading, URL management, main-text extraction and metadata handling in one tool; its documentation reports strong performance against other open-source solutions and external benchmarks.
Leadership
Founders
Adrien Barbaresi
Linguist and computational linguist who moved into NLP and applied data engineering. He holds a Dr. phil. in Linguistics from the École Normale Supérieure de Lyon (2015), worked with the Berlin-Brandenburg Academy of Sciences (including DWDS/ZDL), and has also been associated with the Austrian Academy of Sciences, Free University of Berlin's Corpora from the Web project and ENS Lyon's ICAR lab. He is the author and project leader of Trafilatura and is identified as its sole PyPI maintainer/author.
Founding Story
The work grew out of Barbaresi's PhD research: in 2012–2013 he wanted to study a topic for which suitable data did not exist, so he began collecting it. The lack of practical, reliable methods for building web corpora in the humanities turned that data-collection effort into a research subject and eventually into Trafilatura, initially aimed at creating text databases for research at the Berlin-Brandenburg Academy of Sciences.
Business Model
Revenue Model
Open-source software distributed under Apache 2.0 (with versions before 1.8.0 under GPLv3+); support is requested through GitHub Sponsors, Ko-fi and community contributions rather than a published commercial subscription model.
Target Markets
- Academic researchers and research institutes
- Computational linguistics, NLP and digital-humanities teams
- AI, LLM and RAG developers
- Data engineers and web-corpus builders
- Open-source Python developers
- Building web corpora and research text databases
- Digital humanities and corpus linguistics
- NLP and machine-learning data preparation
- Cleaning web text for AI, LLM and RAG pipelines
- Large-scale web crawling, content discovery and metadata harvesting
- Python and command-line web-content extraction
- Hugging Face
- IBM
- Microsoft Research
- NVIDIA