When you need to pull specific pieces of data out of an HTML page in PHP — a price, a list of links, a table of results — you have two classic options: regular expressions or a real DOM parser. Regex works for a quick one-off on very regular markup, but it breaks the moment the HTML nests, reorders attributes, or drops a closing tag. The robust, built-in answer is PHP DOMXPath: load the page into a DOMDocument, wrap it in a DOMXPath object, and address any node with a precise XPath query.
This tutorial walks through the full flow with working code — loading HTML (including the encoding gotchas that bite everyone), writing XPath queries against real markup, reading values out of the matched nodes, and handling malformed pages. It also covers what changed in modern PHP 8.4, where a new HTML5-aware DOM API sits alongside the classic classes.
Why XPath beats regex for HTML
HTML is not a regular language, so a regular expression can't reliably parse it. The often-quoted rule "don't parse HTML with regex" exists because nested, optional, and malformed tags defeat pattern matching in the general case. A DOM parser instead builds a tree from the markup, and XPath lets you navigate that tree the way you'd describe it out loud: "the first <span> inside every <div> that isn't an ad, under the element with id case1."
- Regex is fine for a tiny, fixed, one-time job on markup you control.
- XPath is the right tool for anything you'll re-run, anything with structure, and anything that might change slightly over time.
XPath queries are also cheaper to run than heavy backtracking regex on large documents, and far easier to read six months later. If you're weighing the trade-off in detail, see regular expressions for scraping and the broader PHP web scraping guide.
Step 1 — Load the HTML into a DOMDocument
DOMDocument is PHP's in-memory representation of an HTML/XML tree. You create one, then feed it the page source with loadHTML():
$dom = new DOMDocument();
// Real-world HTML is messy. Silence libxml's strict warnings
// so a stray tag doesn't spam your output.
libxml_use_internal_errors(true);
$dom->loadHTML($html);
// IMPORTANT: clear the internal error buffer after each load.
// In a long-running worker, leaving it to grow will eat memory.
libxml_clear_errors();
Two lines there matter beyond the obvious. libxml_use_internal_errors(true) stops libxml from emitting a warning for every unclosed tag on a typical web page — you almost always want this. And libxml_clear_errors() empties the buffer those suppressed errors accumulate in; skip it inside a loop or daemon and memory climbs steadily.
The UTF-8 trap
loadHTML() assumes legacy Latin-1 input unless the document declares otherwise, so pages with UTF-8 characters (accents, currency symbols, emoji) come out mangled. The fix people reach for first — mb_convert_encoding($html, 'HTML-ENTITIES', 'UTF-8') — is deprecated as of PHP 8.2 and shouldn't be used in new code. In 2026 use one of these instead:
// Option A: tell the parser the encoding explicitly via a prepended hint
$dom->loadHTML(
'<?xml encoding="UTF-8">' . $html,
LIBXML_NOERROR | LIBXML_NOWARNING
);
// Option B (cleaner): make sure the source has a <meta charset="utf-8">
// in its <head>, and loadHTML will honor it.
Option A is the one-liner that works even when the page has no charset meta. If you're on PHP 8.4, the new DOM API (below) handles UTF-8 correctly out of the box and makes this whole problem disappear.
Step 2 — Create the DOMXPath object
Once the tree is built, wrap it so you can query it:
$xpath = new DOMXPath($dom);
That's it. Every XPath query runs through $xpath->query() (which returns a DOMNodeList) or $xpath->evaluate() (which can also return scalar values like counts and strings).
Step 3 — Query the document with XPath
query() takes an XPath expression and, optionally, a context node to search relative to. Selecting a container first and then querying inside it keeps expressions short and fast:
// Grab the container by id
$case1 = $xpath->query('//*[@id="case1"]')->item(0);
// Now query *inside* that container: the first <span> of every
// <div> that is not an ad box
$entries = $xpath->query('.//div[not(@class="ads")]/span[1]', $case1);
foreach ($entries as $entry) {
echo trim($entry->textContent), "\n";
}
Note the leading . in .//div... — inside a context node you want a relative path. A bare //div would ignore the context and search the whole document again, a subtle bug that silently returns too much.
XPath patterns you'll use constantly
| Goal | XPath expression |
|---|---|
| Element by id | //*[@id="main"] |
| Elements by class (exact) | //div[@class="price"] |
| Class within a multi-class attribute | //div[contains(concat(" ", normalize-space(@class), " "), " price ")] |
| By any attribute | //a[@data-role="next"] |
| Nth child | //ul/li[1] (first), //ul/li[last()] |
| Text contains | //h2[contains(., "Report")] |
| Attribute value | //a/@href |
| Following sibling | //dt[.="SKU"]/following-sibling::dd[1] |
| Exclude a subtree | //p[not(ancestor::footer)] |
The contains(concat(" ", normalize-space(@class), " "), " price ") idiom looks ugly but it's the correct way to match one class inside class="a price b"; a plain @class="price" only matches when price is the entire attribute. This is the single most common mistake in real-world XPath scraping.
Reading values out of the matched nodes
Every item in the returned DOMNodeList is a DOMNode. The properties you'll use most:
foreach ($entries as $node) {
echo $node->textContent; // all text inside, descendants included
echo $node->nodeValue; // similar for element nodes
echo $node->getAttribute('href'); // an attribute value
echo $dom->saveHTML($node); // the node's own outer HTML
}
Prefer textContent over the older firstChild->nodeValue pattern: firstChild breaks if the element leads with whitespace or a nested tag, whereas textContent reliably returns all descendant text. Wrap results in trim() (or XPath's normalize-space()) to strip the layout whitespace that pages are full of.
To pull a single scalar directly, evaluate() is handy:
$count = $xpath->evaluate('count(//article)'); // float
$title = $xpath->evaluate('string(//h1[1])'); // string
A complete, runnable scraper
Putting it together — fetch a page with cURL, parse it, extract data:
<?php
// 1) Fetch
$ch = curl_init('https://example.com/products');
curl_setopt_array($ch, [
CURLOPT_RETURNTRANSFER => true,
CURLOPT_FOLLOWLOCATION => true,
CURLOPT_USERAGENT => 'Mozilla/5.0 (compatible; MyScraper/1.0)',
CURLOPT_TIMEOUT => 20,
]);
$html = curl_exec($ch);
if ($html === false) {
exit('Fetch error: ' . curl_error($ch));
}
curl_close($ch);
// 2) Parse
$dom = new DOMDocument();
libxml_use_internal_errors(true);
$dom->loadHTML('<?xml encoding="UTF-8">' . $html);
libxml_clear_errors();
$xpath = new DOMXPath($dom);
// 3) Extract: name + price from each product card
$cards = $xpath->query('//div[contains(@class, "product")]');
$rows = [];
foreach ($cards as $card) {
$name = $xpath->evaluate('string(.//h2)', $card);
$price = $xpath->evaluate('string(.//span[@class="price"])', $card);
$link = $xpath->evaluate('string(.//a/@href)', $card);
$rows[] = [
'name' => trim($name),
'price' => trim($price),
'url' => trim($link),
];
}
echo json_encode($rows, JSON_PRETTY_PRINT | JSON_UNESCAPED_UNICODE);
For a gentle target to practice against, scraping.pro hosts a public testing ground with fixed markup blocks — useful for sharpening selectors before you point them at a live site. Pair this with browser DevTools to build expressions quickly; see finding XPath with developer tools.
Handling malformed HTML
libxml is forgiving: it silently repairs most bad markup while building the tree, which is exactly what you want for scraping the messy real web. With libxml_use_internal_errors(true) set, unclosed and mis-nested tags produce buffered warnings rather than exceptions, and your query still runs against the repaired tree. If you ever need to inspect what it complained about:
$dom->loadHTML($html);
foreach (libxml_get_errors() as $error) {
fwrite(STDERR, trim($error->message) . "\n");
}
libxml_clear_errors();
Because the parser normalizes the tree, don't assume the DOM mirrors the raw bytes exactly — libxml may have moved a stray <p> or wrapped loose text. Query against the structure you see in DevTools' Elements panel (the parsed DOM), not the raw view-source.
Modern PHP 8.4: the new HTML5 DOM API
The classic DOMDocument uses an HTML4-era parser. PHP 8.4 (released late 2024) added a spec-compliant HTML5 parser under a new Dom\ namespace, which parses modern markup the way a browser does and handles UTF-8 natively:
// PHP 8.4+
$dom = Dom\HTMLDocument::createFromString($html);
// Bonus: native CSS-selector querying, no XPath needed
foreach ($dom->querySelectorAll('div.product h2') as $node) {
echo trim($node->textContent), "\n";
}
querySelectorAll() accepts the same CSS selectors you'd use in the browser, which many people find more readable than XPath for simple selection. XPath remains the more powerful language for axis navigation (ancestors, following-siblings, text predicates), and the classic DOMDocument + DOMXPath pair is still fully supported and the right choice on older runtimes. Use whichever your PHP version and query complexity call for — many codebases mix CSS selectors for the easy cases and XPath for the hard ones.
When DOMXPath isn't enough
DOMDocument parses the HTML that arrives in the HTTP response. If the data you want is rendered by JavaScript after the page loads — common on single-page apps and infinite-scroll listings — it won't be in that HTML, and no XPath will find it. In that case you either call the site's underlying JSON endpoint directly (see scraping hidden APIs) or drive a real browser; PHP can do the latter through a headless browser. For heavier crawling and anti-bot handling you'll also want rotating proxies.
FAQ
Is DOMXPath still the right choice in 2026?
Yes. It's built into PHP, has no dependencies, and handles the vast majority of static-HTML extraction. On PHP 8.4+ you additionally get Dom\HTMLDocument with real HTML5 parsing and querySelectorAll(); pick CSS selectors for simple cases and XPath for complex navigation.
Why is my extracted text full of question marks or garbled characters?
An encoding mismatch on loadHTML(). Prepend '<?xml encoding="UTF-8">' to the HTML, or ensure a <meta charset="utf-8"> is present, or move to the PHP 8.4 API which handles UTF-8 automatically. Avoid the deprecated mb_convert_encoding(..., 'HTML-ENTITIES', ...) trick.
query() vs evaluate() — which do I use?
query() returns a DOMNodeList of matched nodes and is what you iterate over. evaluate() can do that too but also returns scalars — use it for count(...), string(...), and boolean expressions where you want a value, not a node list.
My @class="price" selector matches nothing even though the class is there.
The element probably has multiple classes (class="item price sale"). Exact @class="price" only matches when price is the whole value. Use the contains(concat(" ", normalize-space(@class), " "), " price ") idiom instead.
DOMXPath handles the parsing; the ongoing work in production scraping is everything around it — proxies, retries, JavaScript rendering, and adapting selectors when sites change. If you'd rather receive clean, structured data than maintain PHP scrapers, scraping.pro runs web scraping as a managed service and delivers the parsed results to your database or as files on whatever schedule you need.