CSS Selectors for Web Scraping: Complete Reference

In the previous piece — the DOM parser — we looked at how browsers and libraries turn HTML into a tree of nodes, and how you can walk that tree programmatically. But traversing the DOM "by hand," hopping from parent to child, is tedious and brittle. In practice, web scraping almost always addresses the node you want with a single short expression: a CSS selector.

CSS selectors were invented for styling — they describe which elements a style applies to. But the same syntax is perfect for finding elements too. That's why nearly every scraper — from BeautifulSoup to Scrapy and Playwright — supports selection by CSS. Let's work through the constructs in order. By the end you'll have a working CSS selector cheat sheet for web scraping.

Why CSS selectors matter in scraping

A selector is a declarative way to say "give me every element that looks like this." Instead of a dozen lines of tree traversal you write one line, and the engine descends the DOM and returns the matches. That's:

• compact — one selector instead of a loop;
• readable — the expression resembles a description of where the element sits on the page;
• portable — the same selector works in Python, JS, and the browser console (document.querySelectorAll).

The core skill of a scraper developer is to look at a page's structure in DevTools and craft a selector that hits the target precisely and still survives minor markup changes.

Basic selectors

These are the atoms everything else is built from.

A class and a tag can be glued together with no space — this is a logical AND: a.button means "a link that has the class button." And div.card.featured is a div with two classes at once.

span.price.current     /* <span class="price current"> */
li#first-item          /* <li id="first-item"> */

Grouping

A comma combines several selectors into one list — you get elements that match any of them:

h1, h2, h3             /* all headings of these three levels */
.price-new, .price-old /* both the new and the old price */

Handy when a product has the class price-new on some cards and price-old on others, and you want to collect both variants in a single query.

Combinators: relationships between elements

Combinators describe how elements sit relative to one another in the tree. This is the most important part for robust scraping.

Descendant (space)

A B — element B located somewhere inside A, at any depth.

.product-card .price   /* .price anywhere inside the card */

Direct child (>)

A > B — B is a direct child of A. It doesn't descend deeper.

ul.menu > li           /* only top-level menu items, no nested ones */

The distinction is crucial: .menu li also catches submenu items, while .menu > li does not.

Adjacent sibling (+)

A + B — element B that comes immediately after A and shares the same parent.

h2 + p                 /* the first paragraph right after a heading */
.label + .value        /* the value sitting immediately after a label */

General sibling (~)

A ~ B — all B elements that come after A under a shared parent (not necessarily adjacent).

.in-stock ~ .price     /* all prices coming after the "in stock" label */

Attribute selectors

Often an element has no convenient class but does have a distinctive attribute. Attribute selectors save you when classes are dynamic or noisy.

a[href^="/catalog/"]           /* links within the catalog */
img[src$=".webp"]              /* images in webp format */
div[data-testid*="product"]    /* containers with a product marker in a data attribute */
input[name="csrf"]             /* the hidden token field */

Attribute selectors are especially valuable in ecommerce price monitoring: big retailers often expose data in data-* attributes (data-price, data-sku, data-currency), and latching onto those is more reliable than onto classes that change with every front-end release.

Pseudo-classes: selecting by position and state

Pseudo-classes filter elements by their place among siblings or by extra conditions.

Positional

li:first-child         /* the first child */
li:last-child          /* the last child */
tr:nth-child(2)        /* the second row */
tr:nth-child(odd)      /* odd rows */
tr:nth-child(even)     /* even rows */
tr:nth-child(3n)       /* every third */
li:nth-last-child(1)   /* the first from the end */

The an+b formula in :nth-child() is flexible: nth-child(2n+1) is odd rows, nth-child(n+3) is everything from the third onward.

By tag type

:nth-child counts all siblings in sequence, while :nth-of-type counts only elements of the same tag:

p:first-of-type        /* the first paragraph among siblings */
img:nth-of-type(2)     /* the second image in order */
span:last-of-type      /* the last span */

This rescues you when the <p> you want is the second paragraph, but tags of a different type slipped in ahead of it.

Negation and state

input:not([type="hidden"])   /* all fields except hidden ones */
li:not(.sold-out)            /* products not in the "sold out" state */
a:not([rel="nofollow"])      /* links without nofollow */

:not() takes another selector inside and inverts it. Also supported are :empty (empty elements), :checked, :disabled and others — but in static scraping the positional pseudo-classes and :not() are what get used most.

Note: pseudo-elements (::before, ::after, ::first-line) aren't used in scraping — they describe style-generated content that isn't in the source HTML.

Building a complex selector

Real-world expressions combine everything above. Example — pull the current price from a product card in a catalog:

div.product-card a[href^="/product/"] span.price:not(.price--old)

Reads as: inside the product card, under the link to the product page, take the span with class price that is not the old (struck-through) price.

Tools: where this works

CSS selectors are supported across almost the entire scraping stack.

Python — BeautifulSoup (via the soupsieve library):

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, "html.parser")

# .select() returns a list, .select_one() the first element
cards = soup.select("div.product-card")
for card in cards:
    title = card.select_one("h3.title").get_text(strip=True)
    price = card.select_one("span.price:not(.price--old)").get_text(strip=True)
    print(title, price)

Python — parsel / Scrapy (a wrapper over lxml, fast):

from parsel import Selector

sel = Selector(text=html)
prices = sel.css("span.price::text").getall()   # ::text — a parsel extension
links = sel.css("a.product-link::attr(href)").getall()

Here ::text and ::attr(name) are the non-standard pseudo-elements of Scrapy/parsel: they pull the text or attribute directly, skipping a separate call.

JavaScript / browser — the native API, same syntax:

const cards = document.querySelectorAll("div.product-card");
const price = document.querySelector("span.price:not(.price--old)")?.textContent.trim();

Playwright / Selenium — for dynamic pages where data is loaded by scripts:

# Playwright
price = page.locator("div.product-card span.price").first.inner_text()

In other words, once you master selectors, you use them in lightweight HTTP scrapers and in headless browsers alike.

Applying selectors in price monitoring

In a typical ecommerce price monitoring job, the flow looks like this:

1. Catalog product list. A selector like div.catalog__item finds every card on the results page.
2. Product link. From each card, a[href^="/product/"]::attr(href) pulls the URL.
3. Price and availability. On the product page, [data-price]::attr(data-price) or span.price--current gives the price, and .availability:not(.out-of-stock) confirms the item is for sale.
4. Old price and discount. span.price--old catches the struck-through price so you can compute the discount.

Because competitors use different markup, you write a dedicated set of selectors per site, and the scraping engine applies them on a schedule and drops the results into a shared table for comparison.

Common mistakes and tips

• Don't latch onto "junk" classes. Classes like .css-1ab2c3 are generated by bundlers and change on every deploy. Prefer data-* attributes, meaningful ids, or structural combinators.
• Use > where structure matters. It filters out accidental matches in nested blocks.
• Test your selector in DevTools. The $$("your selector") command in the browser console immediately shows how many elements matched.
• Build in alternatives. List several possible price classes, comma-separated — this hardens you against the store's A/B tests.
• Remember CSS's limits. Selectors can't "climb up to a parent" by content or filter by node text. Where CSS falls short, XPath or regular expressions step in — more on those in the next piece.

CSS selectors vs XPath

CSS selectors cover the vast majority of scraping needs and are shorter and easier to read. XPath is the better pick in exactly the cases CSS can't handle: selecting a parent or ancestor from a child, matching on an element's text content (contains(text(), "In stock")), or navigating by axis. Many scrapers mix the two — CSS for the common case, XPath for the awkward one. If you never need upward traversal or text matching, CSS alone is enough.

Summary

CSS selectors are the primary addressing tool in the DOM. Basic selectors (tag, class, id) set the "what," combinators (space, >, +, ~) set the "where," and attribute selectors and pseudo-classes add precise conditions. Together they cover the overwhelming majority of web scraping tasks, and a well-built selector makes a scraper resilient to minor markup edits — which is critical for recurring ecommerce price monitoring.

If maintaining selectors across dozens of changing sites isn't how you want to spend your week, scraping.pro can run the extraction as a done-for-you service and hand you clean data. In the next article we'll cover pulling data out of "raw" text with regular expressions — for the cases where DOM structure alone isn't enough.