Colly Team
Colly Team develops and maintains Colly, an open-source Go framework for writing crawlers, scrapers and spiders. Its stated goal is to speed up scraper development so users can focus on collecting relevant data, including for data mining, processing and archiving.
At a Glance
- Go developers building web crawlers, scrapers and spiders
- Organizations needing structured web data for mining, processing or archiving
- Production engineering teams running local, parallel or multi-server scrapers
- NGOs and nonprofits needing data collection
- +1 more
AI Tools by Colly Team
(1)
Colly
Go Web Scraping Framework
Discussions
No discussions yet
Be the first to start a discussion about Colly Team
Latest News
Colly v2.2.0 released with bug fixes, dependency updates and new/improved capabilities including context.Context support, queue Stop support, content sniffing, proxy-rotation fixes and GitHub Actions updates.
Project author Adam Tauber opened a 'Looking for maintainers' issue, saying he had nearly zero time to work on Colly and asking for help managing issues and pull requests.
Products & Services
Open-source Apache-2.0 Go package providing a clean interface for crawlers, scrapers and spiders. It extracts structured data for data mining, data processing and archiving and is distributed through github.com/gocolly/colly/v2.
Free data collection service for NGOs and nonprofits.
Data collection from up to 10 websites within a week; the official services page lists pricing at $200 per site.
Unlimited number of websites with scraper management by Colly; the official services page lists pricing from $1,500.
Market Position
Colly positions itself as a fast, elegant, batteries-included scraping framework purpose-built for Go, with concurrency, rate limiting, caching, storage, queues, proxy switching and distributed-scraping support in one API. It competes with broader or language-specific alternatives such as Python's Scrapy; unlike goquery, which is primarily a DOM parser/manipulator, Colly provides the crawler and scraping framework around HTML extraction.
Leadership
Founders
Adam Tauber (asciimoo)
GitHub profile identifies him as the author of Colly, Hister and Searx and lists him in Budapest, Hungary. He is the release author for the Colly releases and the project account behind the repository.
Executive Team
Adam Tauber (asciimoo)
Project author and lead maintainer
GitHub identifies Tauber as the author of Colly, Hister and Searx. He is listed as author of the project releases and posted in March 2025 that he had nearly zero time to work on Colly and was seeking additional maintainers.
Founding Story
The project started in September 2017 after the team set out to create a scraping framework that would speed up development and eliminate the need to reinvent the wheel for each collector. The initial vision was a stable, production-ready framework with built-in support for rate limiting, caching, configurable deployment, proxy switching, storage backends, queues, debugging and distributed scraping.
Business Model
Revenue Model
The core Colly framework is open source and free for commercial use under Apache License 2.0. The Colly site also offers paid custom data-collection services, plus a free service for NGOs and nonprofits; the project site displays commercial sponsors.
Pricing Tiers
Data collection for NGOs and nonprofits.
Gather data from up to 10 websites within a week.
Unlimited number of websites with Colly managing the scrapers.
Target Markets
- Go developers building web crawlers, scrapers and spiders
- Organizations needing structured web data for mining, processing or archiving
- Production engineering teams running local, parallel or multi-server scrapers
- NGOs and nonprofits needing data collection
- Businesses needing custom collection from multiple websites
- Web crawling, scraping and spider development
- Extracting structured data for data mining
- Data processing and archiving
- Parallel or asynchronous collection across many pages
- Distributed scraping and proxy-based collection
- Production scraper deployments from local to multi-server configurations
- Greenpeace/check-my-pages
- JiveSearch
- Docker Slim
- GameDB