# AutoScraper

> A Python web scraping library that learns scraping rules from sample data and applies them to similar pages.

AutoScraper is an open-source Python library by alirezamika for automatic web scraping. You give it a URL or raw HTML plus sample values you want to extract, and it learns the scraping rules and returns similar elements. The latest listed release is v1.1.14 under the MIT License.

## What It Is

AutoScraper is a lightweight scraping module for Python 3. Instead of writing selectors by hand, you supply a page and a wanted list of example data, which can be text, URLs, or any HTML tag value. The library learns rules that match those examples and returns similar elements from the page.

## How the workflow works

Calling `build` with a URL (or HTML content via the html parameter) and a wanted list creates a learned scraper object. That object can then be applied to new URLs in two ways:

- `get_result_similar` returns content similar to the examples, such as related post titles on other pages.
- `get_result_exact` returns the exact matching elements in the same order as the wanted list, such as a stock price and market cap.

Custom `requests` parameters such as proxies or headers can be passed through `request_args`.

## Saving and reusing models

A built scraper can be saved to a file with `save` and restored with `load`, so learned rules can be reused later. The README links to a gist with advanced usage and a tutorial on building an API from any website with Flask.

## Setup path

Install from PyPI with `pip install autoscraper`, directly from the Git repository with pip, or from source with `python setup.py install`.

## Features
- Learns scraping rules from sample data
- Accepts a URL or raw HTML content
- Returns similar elements from new pages
- Returns exact results in wanted-list order
- Custom requests parameters such as proxies and headers
- Save and load learned scraper models
- Scrapes text, URLs, or HTML tag values

## Integrations
Python requests, Flask

## Platforms
API, DEVELOPER_SDK, CLI

## Pricing
Open Source

## Version
1.1.14

## Links
- Website: https://github.com/alirezamika/autoscraper
- Repository: https://github.com/alirezamika/autoscraper
- EveryDev.ai: https://www.everydev.ai/tools/autoscraper