Scraperr

Scraperr is an open-source, self-hosted web scraping application created by jaypyles. It lets users scrape websites without writing code, using XPath selectors to target page elements and a web interface to manage jobs and view results. The GitHub repository is marked as archived, and its latest listed release is v1.1.6.

What It Is

Scraperr is a self-hosted web scraping tool with a web UI. Users submit scraping jobs that target page elements via XPath, then review the scraped data in a structured table. Results can be exported as markdown or CSV. The README badges list MongoDB, FastAPI, Next.js and TailwindCSS as part of the stack.

How the Scraping Workflow Works

Jobs are submitted to a queue, so multiple scraping tasks can be managed together. A job can optionally spider an entire domain to scrape all pages within the same domain. Custom JSON headers can be added to requests, and images, videos and other media can be downloaded automatically. Completion notifications can be sent through various channels.

Deployment and Setup

The README shows running the app with Docker via make up, and the docs include a Helm deployment guide for Kubernetes. The v1.1.6 release is titled "Choose Your Own Database". The project is MIT licensed.

Current Status

The GitHub repository is listed as archived. The README advises respecting robots.txt, website Terms of Service and rate limiting, and states that the tool is intended only for sites that explicitly permit scraping.