The task
The client needed to notice new brands appearing in the niche before competitors did — the moment they surface in search or register domains. The niche moves fast: brands appear and rotate domains constantly, and manual monitoring of SERPs and domain lists cannot keep up.
What we did and what data we collected
We combined two signals. The first — SERP monitoring across topical queries and target regions: which new sites and ads are surfacing. The second — tracking freshly registered domains, filtered by niche markers via keyword domain search. The overlaps were merged into a single feed of new candidates with the date of first detection.
Challenges and how we solved them
First, speed. The value of the signal decays quickly, so we made collection regular and frequent to catch brands at the earliest stage.
Second, the geo-dependence of search results. SERPs differ by country, so we captured search results for the target countries and with proxy rotation — otherwise the picture would be incomplete.
Third, noise. SERPs and domain lists carry a lot of irrelevant entries. We cut the excess with niche-marker filters, keeping only real candidates. Some platforms actively defend against automated collection — we covered the approaches in our article on anti-scraping protection.
The result
The client got a regularly updated feed of the niche's new brands and domains with early detection — instead of manually monitoring SERPs and domain lists.
Services in this case study: SERP Monitoring · SERP scraping · Domain Name Database · Keyword Domain Search · Data for iGaming
Useful reading: Scraping Google queries · Web scraping proxies · Anti-scraping protection