Keyword search volume — how many times per month people type a given phrase into a search engine — is the single most important signal of demand in SEO and paid search. It tells you what your audience actually wants, which topics are worth writing about, and where the traffic is. For any keyword research project, it's the starting point. But collecting this data by hand, one phrase at a time, is painfully slow — so search-volume data gets scraped and pulled through APIs: frequencies, related queries, and seasonal or regional demand exported in bulk for thousands of phrases at once.
This article continues our series on web scraping for SEO. We'll cover what search-volume data gives you, why the primary free source is trickier than it looks, and the tools and APIs that collect it at scale.
What you collect from search-volume data
A good keyword dataset gives you several linked blocks of information:
- a list of related keyword phrases and their monthly search volume;
- suggested / adjacent queries — what the same users also searched for;
- volume broken down by match type — broad, phrase, and exact (more on this below), which strips out inflated numbers from overly wide phrasing;
- a breakdown by region, device, and seasonal trend over the year.
That last dimension matters: "snow blower" and "sunscreen" have wildly different demand curves, and a single annual average hides the peak you'd actually want to target.
Google Keyword Planner: the primary free source
For most of the world, Google Keyword Planner — part of a free Google Ads account — is the canonical source of search-volume data. It reports average monthly searches, competition, related ideas, and a 12-month trend, and it's the reference point every other tool calibrates against.
There's a well-known catch, though. Keyword Planner hides exact numbers from accounts that aren't actively spending. Instead of "8,100 searches/month," a free account sees a rounded bucket like "1K–10K." It also groups close variants together (singular/plural, word order, minor spelling), so "running shoes" and "shoes for running" may report identical, merged volume. To unlock precise figures you need an active campaign with real spend.
This is exactly why teams that need clean, granular numbers move to the API or to third-party tools — and why an automated pipeline has to be kept current, because when Google changes the interface or the Ads API, ad-hoc scrapers break. Keeping the collection running is one reason this work is often worth outsourcing to someone who maintains the tooling.
Match types (the "operators" of search volume)
Raw search volume for a broad phrase overstates real intent, because it lumps in every loosely related query. To get "clean" demand you narrow the match, much like search operators:
- Broad — everything loosely related; the largest, noisiest number.
- Phrase —
"running shoes"— the words in order, possibly with others around them. - Exact —
[running shoes]— that phrase and very close variants only; the tightest, most honest figure. - Negative / minus keywords — exclude terms you don't want (e.g.,
-free,-used) to avoid counting irrelevant demand.
When you compare exact-match volume across a keyword list, you're comparing genuine, like-for-like demand rather than the inflated broad totals.
The API path: Google Ads API
For programmatic, large-scale collection, the Google Ads API is the sanctioned route. Its KeywordPlanIdeaService returns keyword ideas with historical metrics, and generateKeywordHistoricalMetrics returns average monthly searches, competition, and bid ranges for a list of keywords you supply. It's free to query, but it requires a Google Ads account, a developer token, and OAuth setup — which is precisely why it's most useful for agencies and companies processing large volumes rather than for a one-off lookup.
The same account limitation applies: the granularity you get back is better with an active, spending account. For teams that want exact numbers without running campaigns, third-party APIs fill the gap.
Third-party keyword volume APIs
A whole ecosystem exists to deliver clean search-volume figures programmatically, blending Google's data with click-stream and modeling:
- DataForSEO — a pay-as-you-go keyword volume API covering Google search volume, related and suggested keywords, and SERP data; popular for building your own tooling.
- Semrush API and Ahrefs API — volume, keyword difficulty, and related terms from their own large keyword databases.
- Keywords Everywhere API — cheap, credit-based volume, CPC, and competition for lists of keywords.
- SerpApi / scraper APIs — pull the live SERP so you can derive demand signals yourself (see DIY estimation below).
These are the modern equivalent of running your own bulk parser, minus the maintenance headache of a scraper that snaps every time the source changes.
Tools that collect keyword volume
If you'd rather work in a UI than write code, the major SEO platforms all expose search volume for big keyword lists, with region selection and export to CSV/JSON:
- Semrush, Ahrefs — industrial keyword databases, deep related-keyword discovery, and bulk analysis for very large sets.
- Moz Keyword Explorer, Serpstat, Mangools KWFinder — strong mid-market options with clear volume and difficulty metrics.
- Ubersuggest, KeywordTool.io — quick, affordable ways to expand a seed keyword and pull volume, including autocomplete-derived long-tail phrases.
For working directly in Google's own interface, browser extensions for scraping like Keywords Everywhere overlay volume and CPC right on the search results and Planner, and Glimpse augments Google Trends with absolute numbers.
DIY: SERP-based estimation and Google Trends
You can also estimate demand yourself without any keyword database:
- Google autocomplete — scraping Google's autocomplete suggestions reveals the long-tail phrases real users type, which you then size with any of the sources above.
- Google Trends — gives relative interest over time (0–100) and regional breakdowns; excellent for seasonality and comparing terms, even though it doesn't hand you absolute counts.
- SERP scraping — by scraping Google search results for your keyword list you capture the competitive picture (who ranks, how snippets are built), and combined with autocomplete and Trends you can model demand without paying for a keyword tool.
This DIY stack is the resilient fallback: no single vendor to depend on, and the raw SERP is the ground truth every keyword tool is trying to approximate.
Where keyword data goes next
Search volume is the base layer of a keyword map, which you combine with Google autocomplete suggestions and scraped Google queries for a full keyword research pass. The collected, grouped phrases then drive your site structure and dictate the title tags and headings you'll write. The effect of ranking for those queries is tracked afterward through SERP rank tracking.
If what you need is analysis of the actual results for your collected queries — who's in the top positions and how their snippets are built — that's Google SERP scraping, which scraping.pro runs as a managed service: give us the keyword list, get back structured search-volume and SERP data on a schedule, without maintaining a single parser yourself.