Web scraping to order
Any public site, exactly the fields you need
Web scraping to order means a custom scraper built to pull exactly the fields you need, product catalogs, prices, reviews, or leads, from any public site and deliver them as CSV, JSON, or a database, on a schedule. I build scrapers in Python with Scrapy and Playwright, hardened against anti-bot defenses with proxies, browser sessions, and retries. The Amazon and eBay scrapers I published on GitHub are built the same way. You get clean, reliable data and the code, not a one-off dump.
For whom
- Analysts: need a dataset no API will give you
- Sellers: want a rival catalog for benchmarking
- Recruiters: leads scattered across many sites
Included
- Any public site, including JS-heavy single-page apps
- Anti-bot handling: proxies, browser sessions, retries
- Output as CSV, JSON, or a database load
- Scheduled runs, with change alerts
How we work
- 01
Scope
We fix the site, the fields, and the output format.
- 02
Prototype
I scrape a sample and show clean output.
- 03
Launch
I harden it and put it on a schedule.
Timeline: from 3 days·Price: on request, quote after the scope
Stack and integrations
- PythonScrapyPlaywrightSQLiteCSVJSONPostgreSQLBigQuery
Related work
Questions
›Can it get past login or anti-bot defenses?
Often yes, with browser sessions, proxies, and retries. I tell you upfront where it is reliable and where it is fragile.
›Is scraping the site allowed?
Public data is generally fine. I scrape respectfully, with rate limits, and we check the site terms before we start.
Related offerings
Any public site, exactly the fields you need
Discuss your project