Headless Browser Scraping in PHP: Handle JavaScript Sites

Classic web scraping in PHP is cURL + DOMDocument, or a stack like Symfony DomCrawler. It's fast, cheap, and works perfectly — right up until the site stops serving ready-made HTML. Modern front ends rarely do: content loads over AJAX, React/Vue/Angular render it in the browser, prices and catalogs hide behind infinite scroll, and an anti-bot layer checks whether the client can actually execute JavaScript.

At that point an HTTP client can't keep up, and you have to bring up a real browser — driving headless Chrome or Firefox straight from PHP. That solves the JS-rendering problem, but it introduces a whole new class of them: resource use, memory leaks, zombie processes, orphaned tabs, and painful debugging. That's what this headless browser scraping guide is about.

When you actually need a browser

A browser is a heavy tool, and you should only pull it out when you genuinely need it. Signs you can't avoid it:

• content appears in the DOM only after JavaScript runs (SPAs, lazy loading);
• you need user actions: clicks, scrolling, filling forms, walking through multi-step flows;
• the site actively defends against bots and checks JS execution, browser fingerprint, and mouse behavior;
• you need to render to a screenshot or PDF, or capture performance metrics.

If the page delivers the data you want in the raw HTML — or as JSON through an internal API — you don't need a browser, and an honest HTTP request will be tens of times faster and more stable.

Popular libraries

Symfony Panther

Panther is the official library from the Symfony team. Under the hood it speaks the WebDriver protocol and uses real drivers (ChromeDriver, GeckoDriver), while its API stays compatible with DomCrawler and BrowserKit. That's its big advantage: you write familiar code, and a live Chrome runs beneath it instead of an HTTP client.

use Symfony\Component\Panther\Client;

$client = Client::createChromeClient(null, [
    '--headless=new',
    '--no-sandbox',
    '--disable-dev-shm-usage',
    '--disable-gpu',
]);

$crawler = $client->request('GET', 'https://example.com');
$client->waitFor('.product-card'); // wait for the element to appear

$titles = $crawler->filter('.product-card h2')->each(
    fn ($node) => $node->text()
);

$client->quit(); // MANDATORY — otherwise the browser process hangs around

Panther is good for integration tests and moderate scraping. Its main pain point: it doesn't manage the browser lifecycle perfectly, so on exceptions, fatal errors, or a forgotten quit(), processes pile up.

php-webdriver/webdriver + Selenium

php-webdriver (formerly Facebook WebDriver) is a low-level but maximally flexible WebDriver client. It usually pairs with Selenium Grid or Standalone, which scales cleanly into a cluster.

use Facebook\WebDriver\Remote\RemoteWebDriver;
use Facebook\WebDriver\Remote\DesiredCapabilities;
use Facebook\WebDriver\WebDriverBy;

$driver = RemoteWebDriver::create(
    'http://selenium-hub:4444/wd/hub',
    DesiredCapabilities::chrome()
);

try {
    $driver->get('https://example.com');
    $element = $driver->findElement(WebDriverBy::cssSelector('.price'));
    $price = $element->getText();
} finally {
    $driver->quit(); // in finally — close the session even on error
}

PHP + Selenium is especially handy when you want to move the browser infrastructure into a separate service (Docker Compose, Kubernetes) and keep PHP as a thin client.

chrome-php/chrome (headless Chrome over the DevTools Protocol)

This library talks to Chrome directly over the Chrome DevTools Protocol (CDP), bypassing WebDriver. It's faster, closer to the metal, and gives fine-grained control over pages, the network, and tabs.

use HeadlessChromium\BrowserFactory;

$factory = new BrowserFactory('chromium-browser');
$browser = $factory->createBrowser([
    'headless'   => true,
    'noSandbox'  => true,
    'windowSize' => [1920, 1080],
    'keepAlive'  => false,
]);

try {
    $page = $browser->createPage();
    $page->navigate('https://example.com')->waitForNavigation();
    $html = $page->getHtml();
} finally {
    $browser->close(); // close the browser and all its pages
}

The CDP approach gives the most direct access to tab (target) management and network-request interception — which comes in handy below.

Goutte / DomCrawler — for comparison

Worth remembering: Goutte (now just BrowserKit over HttpClient) and plain DomCrawler are not browsers — they don't execute JavaScript. People often mistake them for browser solutions, but they're HTTP clients. If the page needs JS, they're useless; on static pages they're incomparably lighter.

Which one to pick

The problems of browser-based scraping

Resources and performance

Every Chrome instance is tens to hundreds of megabytes of RAM, plus separate renderer, GPU, and network processes. A dozen parallel browsers easily eat several gigabytes. So browser scraping is almost always built as a pool with a hard concurrency cap, a task queue, and timeouts on every operation.

A typical set of launch flags for a server environment that saves resources and prevents common crashes:

--headless=new
--no-sandbox
--disable-dev-shm-usage      # critical in Docker: otherwise /dev/shm fills up and Chrome crashes
--disable-gpu
--disable-extensions
--blink-settings=imagesEnabled=false   # don't load images if you don't need them
--js-flags=--max-old-space-size=512    # cap V8's appetite

Memory leaks

This is arguably the biggest hidden pain of long-running scrapers. Leaks come from two directions.

From the browser itself. Chrome isn't designed to open thousands of pages in one session. Memory use grows page by page: cache accumulates, DOM trees aren't freed, event listeners on the site's side pile up. A single browser process working a long queue can, after a few hours, hold several times more memory than it did at the start.

From PHP. The worker process leaks too: page wrapper objects, responses, DOM trees held in memory, uncleaned variables. Long-running workers are especially dangerous (Symfony Messenger, RoadRunner, Swoole, or a plain daemon in a while (true) loop), where a process lives for days.

Practical measures:

• Restart the worker by a task counter. The most reliable trick: the worker handles N tasks (say, 50–200) and exits; a supervisor brings up a fresh one. The accumulated memory is fully reclaimed by the OS, not by PHP's garbage collector.
• Restart the browser itself periodically. Don't keep one Chrome for thousands of pages — close and recreate it every M pages.
• Close pages (tabs) right after use, not just at the end of the whole batch.
• Watch the metrics. Log the worker's memory_get_usage(true) and the browsers' RSS, and chart them. Linear memory growth over time is a dead giveaway of a leak.
• Use gc_collect_cycles() carefully. Forcing collection of circular references sometimes helps, but it's no substitute for restarting.

Orphaned windows and zombie processes

This is a browser-specific problem that's easy to miss. If a PHP script crashes with an exception, gets killed on a timeout, or simply forgets to call quit()/close(), the browser process does not die with it. It keeps hanging in the system — with all its open tabs and occupied memory. On an active scraper these "orphaned" processes accumulate by the dozen, and within a day the server is packed with zombie Chromes even though not a single task is running.

You must monitor and clean this up. Several layers of defense:

1. Close in finally. Any code that opened a browser must close it in finally, so an exception doesn't leave the process hanging:

$browser = $factory->createBrowser([...]);
try {
    // ... work
} finally {
    $browser->close();
}

2. Register a shutdown function. For fatal errors:

register_shutdown_function(function () use ($browser) {
    try { $browser->close(); } catch (\Throwable $e) {}
});

3. External process monitoring. Regularly (via cron or a watchdog script) check how many browser processes are alive and whether any are "old" and have outlived their parent:

# How many chrome/chromium processes are running right now
pgrep -c -f 'chrome|chromium'

# Find processes older than 30 minutes — candidates for killing
ps -eo pid,etimes,comm | awk '$2 > 1800 && $3 ~ /chrome/ {print $1}'

# Hard cleanup of zombie browsers older than half an hour
ps -eo pid,etimes,comm | awk '$2 > 1800 && $3 ~ /chrome/ {print $1}' | xargs -r kill -9

Run this watchdog as a separate cron job every few minutes. It doesn't cure the cause, but it stops the server from falling over under accumulated windows while you hunt the leak. It's also worth adding an alert — "more than X browser processes" or "a browser process older than Y minutes" — as the first signal that a quit() is going missing somewhere.

4. Run in a disposable container. The radical but clean option: each scraping task starts in a fresh Docker container with a browser that's guaranteed to be destroyed afterward. Zombies have nowhere to accumulate.

Tab management

Opening a new browser per page is expensive. It's often better to keep one browser and work with tabs (targets in CDP terms, windows in WebDriver terms). But there are traps here too.

The main rule: count and close tabs as disciplined as you do browsers. An opened and forgotten tab is the same leak, just inside a live process. A site can open new tabs itself (target="_blank", popups, ads), and if you don't track them, they quietly pile up.

// chrome-php: explicit tab control
$page = $browser->createPage();
$page->navigate('https://example.com')->waitForNavigation();
$data = $page->getHtml();
$page->close();   // close the tab immediately, don't wait for the batch to end

// php-webdriver: watch windows and close the extras
$handles = $driver->getWindowHandles();
foreach ($handles as $handle) {
    if ($handle !== $mainWindow) {
        $driver->switchTo()->window($handle)->close();
    }
}
$driver->switchTo()->window($mainWindow);

Useful practices:

• cap the number of simultaneously open tabs (say, no more than 5–10 per browser);
• close a tab as soon as its page is processed, don't stockpile "for later";
• periodically reconcile the real tab count against the expected one — a mismatch signals the site opened something extra or your code isn't closing somewhere;
• when reusing a tab, clear its state (cookies, localStorage), or sessions bleed between tasks.

PHP as an orchestrator: when it really runs Python or Node

There's an honest architectural scenario worth calling out, one that's more common than people admit. PHP acts as the orchestrator: it accepts tasks, queues them, drives the business logic, and writes results to the database. But it doesn't do the browser scraping itself — because the browser-automation ecosystem is far richer in Node.js (Puppeteer, Playwright) and Python (Playwright, Selenium, undetected-chromedriver, anti-detect wrappers).

So the "PHP scraper" really looks like this: PHP builds the task and launches an external process in another language, which does all the real browser work, and PHP just parses its output.

use Symfony\Component\Process\Process;

$process = new Process([
    'python3',
    '/app/scrapers/playwright_scraper.py',
    '--url', $url,
    '--timeout', '30',
]);
$process->setTimeout(60); // a timeout on the whole process — mandatory

try {
    $process->mustRun();
    $payload = json_decode($process->getOutput(), true, 512, JSON_THROW_ON_ERROR);
} catch (\Throwable $e) {
    // important: if PHP killed the process on a timeout, the child browser
    // must be finished off too — otherwise, zombies again
    $process->stop(5, SIGKILL);
    throw $e;
}

Why teams do this:

• Tool maturity. Playwright and Puppeteer are noticeably ahead of the PHP equivalents in network interception, emulation, anti-detect, and stability.
• Isolation. The browser lives in a child process. If it crashes or leaks, that doesn't bring down or bloat the PHP worker. PHP stays light and stable.
• Scaling by role. PHP handles business logic, queues, and storage — the things it's good at. The browser handles rendering.

The key subtlety of this scheme is the boundary of responsibility for processes. When PHP kills the child on a timeout, make sure the browser it spawned dies with it. Otherwise the zombie Chromes move one level deeper and become even harder to spot: PHP looks clean, yet memory leaks "from nowhere." So the child script must correctly close its browser in its own finally/atexit, and PHP must send SIGKILL to the whole process group, not a single PID. Launching the child in its own process group and killing the whole group is a reliable way to leave no tails behind.

This is a normal, widespread architecture. Calling the project a "PHP scraper" isn't dishonest — PHP really does orchestrate the process; it just delegates the heavy browser work to where the tools are better.

VNC for early debugging

Headless mode is convenient in production but miserable for debugging: you can't see what's happening on the page. The scraper "can't find the element" and it's unclear why — maybe a cookie banner appeared, maybe a CAPTCHA, maybe the layout differs for a bot, maybe you just didn't wait for the load. Screenshots help, but they're isolated frames, not a live picture.

The fix for early debugging is to bring up the browser not in headless mode, but normally inside a virtual display, and watch it over VNC. You literally see the browser screen in real time: how it clicks, what loads, where it gets stuck.

The Docker setup is simple:

• a virtual X server (Xvfb) runs inside the container — the browser needs somewhere to draw;
• on top of it, a VNC server (x11vnc) streams that display outward;
• the browser starts without the --headless flag, so it actually draws to screen;
• you connect with any VNC client (or via noVNC right in your browser) and watch.

# Fragment: a VNC debugging kit
RUN apt-get update && apt-get install -y \
    chromium xvfb x11vnc fluxbox

ENV DISPLAY=:99

# launch (simplified — in reality via supervisor/entrypoint):
# Xvfb :99 -screen 0 1920x1080x24 &
# fluxbox &                         # lightweight window manager
# x11vnc -display :99 -forever -nopw &   # VNC out, port 5900
# then PHP/Python starts Chrome WITHOUT --headless

Ready-made images like selenium/standalone-chrome-debug already include VNC out of the box — for the PHP + Selenium combination it's the fastest start: connect to port 5900, fire a task from PHP, and watch what the browser does.

When VNC especially saves you:

• you're chasing a flaky CAPTCHA or anti-bot that doesn't always trigger;
• a selector "sometimes isn't found" — you can see with your own eyes what's in the way (a modal, a popup, a different locale);
• you're working through a multi-step flow (login, cart, checkout) where sequence matters;
• you're checking how the site reacts to your specific browser, flags, and fingerprint.

Important: VNC is a tool strictly for early debugging and scenario development. Don't keep it in production: an open port is an attack surface, and a headed browser wastes resources. Once you've debugged the flow live and confirmed it works, switch back to headless.

The final checklist

If you're building a browser scraper in PHP, keep at least these in mind:

• Tool choice. Panther for simplicity and Symfony compatibility; php-webdriver + Selenium for scalable infrastructure; chrome-php/chrome for fine control via CDP. For serious browser work, don't hesitate to delegate to Playwright/Puppeteer via a child process.
• Lifecycle. Always close the browser and tabs in finally; register register_shutdown_function; restart workers and browsers by a counter.
• Memory. Log the worker's usage and the browsers' RSS, watch for linear growth, cure it with restarts.
• Zombie windows. Monitor the count and age of browser processes, run a watchdog that finishes off orphans, and alert on their buildup.
• Tabs. Count and close them as strictly as browsers; watch that the site hasn't opened extras.
• Orchestration. If PHP launches Python/Node, kill the whole process group, not one PID, and let the child script close its own browser.
• Debugging. Bring up VNC + Xvfb for a live view during development; don't drag it into production.

Browser scraping in PHP isn't about "finding a magic library" — it's about the discipline of managing resources. Extracting data from a ready DOM is the easy part. The hard part is making sure that after a week of continuous work your server isn't packed with zombie tabs and bloated workers.

If maintaining browser farms, watchdogs, and anti-bot workarounds isn't where you want to spend engineering time, scraping.pro runs JavaScript-heavy scraping as a managed data extraction service — you get the parsed data, we own the headless infrastructure.