The task
The payment provider was looking for prospective clients among IT companies. The logic is simple: if a business runs ads, it has budgets and an active, operating business — so payment solutions can be pitched to it. The job was to collect everyone advertising in search across IT topics and turn that into a database for the sales team.
What we did and what data we collected
We built a wide keyword set for the IT niche and ran ad monitoring in search results: who is advertising (the advertiser), with which ads, and which landing pages they point to. From the advertisers and their websites we formed a list of companies — prospective clients; competing payment services, aggregators and irrelevant niches were removed from the list. The data was kept as a time series to also catch new advertisers who had just started buying traffic.
Challenges and how we solved them
First, coverage. Ads are tied to queries, so a wide keyword set was critical to catch as many advertisers as possible; more on collecting and clustering keywords in our articles on keyword scraping and scraping Google queries.
Second, ads are personalized and rotate: different users, geos and times of day see different ads. We captured SERPs for the target countries and frequently enough to catch the rotation rather than a single snapshot — otherwise part of the advertisers would simply never show up.
Third, search engines defend against automated queries. Collection ran distributed, with proxy rotation — otherwise the data would be incomplete at that volume.
The result
The client got a regularly updated database of IT-niche advertisers — a list of active companies with budgets, ready to use as a prospecting database for selling payment solutions. The history is visible too: who started advertising and when, i.e. whom to approach first.
Services in this case study: Ad Monitoring · SERP scraping · Data for Marketing and Sales · Data Search and Enrichment
Useful reading: Keyword scraping · Scraping Google queries · Web scraping proxies