Photon

Photon is an open-source crawler for OSINT work, written in Python and published by s0md3v on GitHub under the GPL v3.0 license. It crawls a target site and pulls out URLs, files, intel such as emails and social media accounts, secret keys, and subdomain data. The latest release listed is v1.3.0, published 2019-04-05.

What It Is

Photon is a command-line information-gathering crawler. You point it at a target URL and it follows links, collecting data while it goes. Results are saved in an organized structure and can be exported as JSON.

What It Extracts

While crawling, Photon can collect in-scope and out-of-scope URLs, URLs with parameters, files (pdf, png, xml and others), and intel such as emails, social media accounts and Amazon buckets. It also finds auth/API keys and hashes, JavaScript files and the endpoints inside them, strings matching a custom regex, and subdomains with DNS-related data.

Controls and Plugins

Users can set timeout and delay, add seeds, and exclude URLs matching a regex. Plugins include wayback, which uses URLs archived by archive.org as seeds via the --wayback option, dnsdumpster for DNS data, and an exporter for formatted results. The README says Photon uses thread management to improve performance.

Setup Path

Photon can be run from source or through a lightweight Python-Alpine Docker image, which the README describes as 103 MB. Results can be read from the Docker volume or from a mounted output folder. The --update option installs updates without losing saved data.