How to Scrape Browser Local Storage with Python

Open DevTools on any modern single-page app, switch to Application, click Local Storage. You will usually find ten to forty keys: an access token, a refresh token, a feature-flag blob, the cart, the last search, and often a verbatim copy of the JSON the page is rendering. Now fetch the same URL with requests.get() and count how many of those keys appear. Zero. Nothing in the HTML even hints that they exist.

That gap is the entire problem. Local storage is written by the page's own JavaScript after the document loads, and the browser never sends it to the server. An HTTP client downloads bytes. It has no window, no DOM and no JavaScript engine, so window.localStorage is not empty in that context, it is absent. You either get inside a browser or read the files a browser already wrote.

This guide covers four routes: Selenium, Playwright, the DevTools Protocol, and decoding the on-disk database without launching anything. Everything below was re-checked on 13 August 2026 against Selenium 4.47.0 (released 10 August 2026), Playwright 1.62.0 (31 July 2026), and current Chromium and Firefox source. Three claims in the earlier version of this article turned out to be wrong and are corrected in place: the LevelDB format bytes were backwards, the sentence attributed to the specification is not in the specification, and Firefox has not kept live local storage in webappsstore.sqlite for years.

What local storage is, in the terms that matter here

Web Storage lives in the HTML Standard, which introduces it as "two related mechanisms, similar to HTTP session cookies, for storing name-value pairs on the client side." Four properties decide how you get at it.

• It is client-side only. The server never receives it automatically, which is why an HTTP request cannot reach it. An earlier version of this article backed that up with a quotation: web storage "falls exclusively under the purview of client-side scripting." We went looking for that sentence. It is not in the HTML Standard, where "purview" appears twice, both times in the drag-and-drop section. The behaviour was right, the citation was invented, so it is gone.
• It is scoped per origin, and often narrower. https://example.com and https://sub.example.com are separate stores. Chromium also partitions third-party storage by top-level site: the kThirdPartyStoragePartitioning feature is enabled by default, "to reduce fingerprinting" in the words of its own comment. An iframe from widget.example embedded on two parent sites gets two stores, and neither is the one you see when you open widget.example directly.
• Values are strings, and the interface is synchronous. Objects are stored as JSON, so you will json.loads() most of what you read.
• It is [Exposed=Window]. The IDL in the standard exposes Storage to Window only, so there is no localStorage inside a worker. An app that moved its state into a worker is keeping it somewhere else, usually IndexedDB.

Two failure modes follow from the specification. Setting a key throws QuotaExceededError when the store is full, and reading localStorage at all throws a SecurityError "if the Document's origin is an opaque origin or if the request violates a policy decision." That second one is why page.evaluate fails on about:blank, on a data: URL and inside a sandboxed iframe.

The quota numbers in circulation are wrong for Chrome. MDN says Web Storage "is limited to 10 MiB of data maximum on all browsers" and that browsers "can store up to 5 MiB of local storage." The engines disagree. Chromium's dom_storage_constants.h sets kPerStorageAreaQuota = 10 * 1024 * 1024, a full 10 MiB for local storage alone, plus a 100 KiB over-quota allowance. Firefox's dom.storage.default_quota is 5120 KiB per origin, with a comment explaining the choice: "Only allow relatively small amounts of data since performance of the synchronous IO is very bad." A value that fits in Chrome can fail in Firefox at half the size.

That same Chromium header holds one more number worth knowing before you build a scheduled crawl on stored sessions: kLocalStorageStaleBucketCutoffInDays is 400.

Approach 1: Read local storage with Selenium

Selenium drives a real browser and runs arbitrary JavaScript through execute_script(). The tidiest pattern is one snippet that dumps every entry into a Python dict.

# selenium==4.47.0 (10 August 2026), Python 3.10+
# Selenium Manager has fetched drivers automatically since 4.6,
# and browsers as well since 4.11.0. No driver path needed.
from selenium import webdriver

options = webdriver.ChromeOptions()
options.add_argument("--headless")     # see the note below on --headless=new
driver = webdriver.Chrome(options=options)

driver.get("https://example.com/")

storage = driver.execute_script(
    "return Object.fromEntries(Object.entries(window.localStorage));"
)

for key, value in storage.items():
    print(key, "=>", value)

driver.quit()

On the headless flag. Chromium's headless README states that "as of M132, headless shell functionality is no longer part of the Chrome binary, so --headless=old has no effect." Google's announcement of the removal confirms --headless and --headless=new now select the same mode. Keep writing =new only if your fleet still contains Chrome 131 or older.

The older localStorage.key(i) loop still works, and it is worth remembering because it survives pages that have replaced Object.entries. Single keys, writes and deletes behave as you would expect, and JSON values get parsed on the Python side:

import json

token = driver.execute_script("return window.localStorage.getItem('auth_token');")
driver.execute_script("window.localStorage.setItem('lang', 'en');")
driver.execute_script("window.localStorage.removeItem('lang');")

raw = driver.execute_script("return window.localStorage.getItem('cart');")
cart = json.loads(raw) if raw else None

Read too early and you get an empty dict. The values you want are written by the app after load, not by the document. Waiting for a visible element is the usual advice and it is half right: an element can render before the fetch that fills the store resolves. Wait for the key itself.

from selenium.webdriver.support.ui import WebDriverWait

WebDriverWait(driver, 15).until(
    lambda d: d.execute_script("return window.localStorage.getItem('auth_token');")
)

Two habits save time later. Write to local storage only after navigating to the target origin, because a fresh session sits on about:blank and that origin is opaque. And ChromeDriver starts Chrome against a throwaway user data directory unless you pass --user-data-dir, so whatever the page stores dies with the driver. Our Selenium walkthrough covers login flows, and Selenium Manager covers the driver plumbing you no longer have to write.

Approach 2: Read local storage with Playwright

Playwright starts faster, waits better, and can export the whole browser state in one call. The direct read is a one-liner.

# playwright==1.62.0 (31 July 2026), bundles Chromium 151, Firefox 153, WebKit 26.5
# pip install playwright && playwright install chromium
from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch(headless=True)
    page = browser.new_page()
    page.goto("https://example.com/")
    page.wait_for_function("() => window.localStorage.getItem('auth_token')")

    storage = page.evaluate(
        "() => Object.fromEntries(Object.entries(window.localStorage))"
    )
    print(storage)

    browser.close()

That is deliberately not wait_until="networkidle". Playwright's own parameter documentation marks it "DISCOURAGED consider operation to be finished when there are no network connections for at least 500 ms. Don't use this method for testing, rely on web assertions to assess readiness instead." An app that polls every two seconds never goes idle, and one that finishes its fetch in 40 ms still charges you the full 500 ms. Wait for the condition you care about.

The bigger feature is storage_state(), which snapshots cookies and per-origin local storage together:

context = browser.new_context()
page = context.new_page()
page.goto("https://example.com/")        # then wait for your key, as above

state = context.storage_state()          # dict with "cookies" and "origins"
for origin in state["origins"]:
    print(origin["origin"])
    for item in origin["localStorage"]:
        print("  ", item["name"], "=>", item["value"])

context.storage_state(path="state.json")

The API has grown in ways older write-ups miss. storage_state(indexed_db=True) has been available since Playwright 1.51 and matters for anything built on Firebase Authentication, which keeps its tokens in IndexedDB. storage_state(credentials=True) arrived in 1.61 and captures virtual WebAuthn passkeys with their private keys. And since 1.59 a state can be pushed into a live context instead of seeded into a new one:

context = browser.new_context(storage_state="state.json")   # seed at creation
context.set_storage_state("state.json")                     # or replace later, 1.59+

Two limits on this file. Session storage is not included, so a site that keeps its token in sessionStorage needs an add_init_script to replay it. And the file is a credential in plain text. Playwright's authentication guide says so outright: "the browser state file may contain sensitive cookies and headers that could be used to impersonate you or your test account. We strongly discourage checking them into private or public repositories."

Approach 3: Read it over the DevTools Protocol

Neither library needs your JavaScript to run in the page to reach the store. Chromium exposes a DOMStorage domain over the DevTools Protocol, and both can speak it. Reach for it when the page has frozen or replaced the built-ins, when you would rather not leave an evaluate call in the execution context, or when the data belongs to a partitioned frame the top document cannot see.

The domain is small: enable, getDOMStorageItems, setDOMStorageItem, removeDOMStorageItem, clear. The identifier is a StorageId carrying securityOrigin, storageKey and a boolean isLocalStorage. In Selenium:

driver.get("https://example.com/")
driver.execute_cdp_cmd("DOMStorage.enable", {})

frame = driver.execute_cdp_cmd("Page.getFrameTree", {})["frameTree"]["frame"]["id"]
storage_key = driver.execute_cdp_cmd(
    "Storage.getStorageKeyForFrame", {"frameId": frame}
)["storageKey"]

result = driver.execute_cdp_cmd(
    "DOMStorage.getDOMStorageItems",
    {"storageId": {"storageKey": storage_key, "isLocalStorage": True}},
)
for key, value in result["entries"]:
    print(key, "=>", value)

Asking the browser for the storage key rather than assembling it yourself is the point of that middle step. It returns the serialized key including any partitioning suffix, the same string the browser uses as its index.

The protocol also emits events, which is what makes this more than a stylistic alternative. Instead of polling until a key shows up, you subscribe and let the browser announce the write:

context = browser.new_context()
page = context.new_page()
cdp = context.new_cdp_session(page)
cdp.send("DOMStorage.enable")
cdp.on("DOMStorage.domStorageItemAdded",
       lambda e: print("added", e["key"], "=>", e["newValue"]))
cdp.on("DOMStorage.domStorageItemUpdated",
       lambda e: print("updated", e["key"]))

page.goto("https://example.com/")

Two caveats. CDP sessions work only on Chromium-based browsers, so Firefox and WebKit are out. And DOMStorage is marked experimental in the protocol definition, so its shape can change between Chrome releases even though it has been quiet for years. Pin your Chrome version in production.

Approach 4: Read the Chromium profile off disk

Sometimes launching a browser is not an option: you already have a profile folder, or you are working from a forensic image. Chromium keeps local storage in a LevelDB database inside the profile directory, which is a subfolder of the user data directory, usually Default and Profile 1, Profile 2 for the rest. Chromium's user data directory docs give the defaults:

• Windows: %LOCALAPPDATA%\Google\Chrome\User Data\Default\Local Storage\leveldb\
• macOS: ~/Library/Application Support/Google/Chrome/Default/Local Storage/leveldb/
• Linux: ~/.config/google-chrome/Default/Local Storage/leveldb/

On Linux the ~/.config part yields to $CHROME_CONFIG_HOME or $XDG_CONFIG_HOME, and the whole path yields to $CHROME_USER_DATA_DIR, so read the environment before hard-coding anything. Edge and the other forks use the same layout under their own vendor folder.

The schema is documented in the source. local_storage_leveldb.h spells it out:

key: "VERSION"                                    value: "1"
key: "META:" + <StorageKey>                       value: write metadata protobuf
key: "METAACCESS:" + <StorageKey>                 value: access metadata protobuf
key: "_" + <StorageKey> + '\x00' + <script key>   value: <script value>

Storage keys are serialized as origins with no trailing slash, so https://abc.test and not https://abc.test/. A partitioned third-party store serializes differently: origin, then /, then ^0, then the top-level site. That suffix separates a first-party store from the same site's embedded copy, and it is why a bare startswith filter is dangerous: it also catches https://example.com.attacker.test.

Here is the correction. Both keys and values carry a one-byte format marker, and the earlier version of this article had the two values swapped. Blink's cached_storage_area.cc defines it in one line:

enum class StorageFormat : uint8_t { UTF16 = 0, Latin1 = 1 };

Zero means UTF-16LE. One means Latin-1. The old snippet said the opposite and then decoded every value as UTF-16 regardless, which produces a recognizable kind of wreckage. We rebuilt four records exactly as Chromium encodes them and ran both decoders over them. The old code returned this:

'auth_token' => '祥桊䝢楣楏䥊穕ㅉ楎㥊愮换'
'cart'       => '≻瑱≹㈺∬歳≵∺ⵁ㤱索'
'greeting'   => '片��'
'\x00�W\x02^' => '東京'

The correct decoder returned eyJhbGciOiJIUzI1NiJ9.abc, {"qty":2,"sku":"A-19"}, Grüße and 城市 => 東京. Every plain ASCII value, which is every token and every JSON blob you want, came back as CJK mojibake, because pairing two Latin-1 bytes into one UTF-16 code unit lands you in the CJK block. Only the genuinely non-Latin-1 record survived, and that is the one case a quick manual test is least likely to include. The lstrip(b"\x00\x01") was a second bug behind the first: lstrip strips a set of bytes, not a prefix, so a value beginning with U+0100 loses real data.

The decoder is four lines:

def decode_string(raw: bytes) -> str:
    """Chromium type-prefixed string: 0 = UTF-16LE, 1 = Latin-1."""
    if raw[0] == 0:
        return raw[1:].decode("utf-16-le")
    if raw[0] == 1:
        return raw[1:].decode("iso-8859-1")
    raise ValueError(f"unexpected format byte {raw[0]}")

Session storage on disk uses a different convention: keys forced to UTF-8, values forced to UTF-16, no format byte at all. Copying this function across is the next thing that will bite you.

The files lag the browser. Chromium batches writes, "in anticipation of other values being set," with a default commit delay of five seconds and a ceiling of 60 commits per hour per area. A value the page set two seconds ago is in memory, not in the file.

And LevelDB takes an exclusive lock. With Chrome running against that profile, opening the database fails on the LOCK file. Copy the leveldb folder and open the copy. Copy it regardless, because opening a LevelDB replays and truncates the write-ahead log, modifying the directory you opened.

The library situation is worse than the tutorials admit

plyvel is the binding everyone reaches for, and pip install plyvel does less than the instruction implies. The last release is plyvel 1.5.1, dated 15 January 2024, and the only wheels it publishes are manylinux_2_17_x86_64 for CPython 3.7 through 3.12. No macOS wheel, no Windows wheel, no aarch64 wheel, nothing for Python 3.13 or later. Outside that one square pip falls back to a source build needing LevelDB headers and a shared library. The repository is not archived and has 23 open issues, but nothing has shipped in over two and a half years.

The tool built for this job is ccl_chromium_reader by Alex Caithness of CCL Solutions, a forensics library that reimplements LevelDB in pure Python along with Snappy, protobuf and V8 deserialization. It is not on PyPI, so install it from the repository. Version 0.3.18 needs Python 3.10 or later.

# pip install git+https://github.com/cclgroupltd/ccl_chromium_reader
import pathlib
from ccl_chromium_reader import ChromiumProfileFolder

# Work on a copy. missing_data_ok=True lets you copy just the
# "Local Storage" subtree instead of the whole profile.
profile = pathlib.Path("/tmp/profile-copy/Default")

with ChromiumProfileFolder(profile, missing_data_ok=True) as p:
    for rec in p.iter_local_storage(storage_key="https://example.com"):
        print(rec.script_key, "=>", rec.value)

Three things it gives you that plyvel will not. It reads records straight out of the .ldb and .log files without opening the database, so the source is never written to. It exposes deleted records through include_deletions=True, because a LevelDB delete is a tombstone and the old value usually survives until compaction. And it returns the sequence number with every record, which lets you reconstruct the order a site set its keys in.

Use it responsibly. Reading a browser profile is not the same act as scraping a website. Auth tokens lifted from someone else's local storage grant their access, and taking them off a machine you were not given is the behaviour of credential-stealing malware, not of a crawler. Stick to profiles you own, machines you administer, and data you are permitted to hold.

Firefox stopped using webappsstore.sqlite years ago

The advice to read webappsstore.sqlite with the sqlite3 module is repeated everywhere, this article's earlier version included, and on a current Firefox profile it will hand you stale data or nothing at all.

Firefox replaced the old implementation with LocalStorage NextGen. Live data sits in a per-origin SQLite file at <profile>/storage/default/<origin>/ls/data.sqlite, next to a usage-journal. dom/localstorage/ActorsParent.cpp builds the data table with columns key, utf16_length, conversion_type, compression_type, last_access_time and value BLOB. That compression_type column is what breaks naive readers: values go through SnappyCompress on the way in, and the raw value is kept only when compression fails to help. Read the blob with sqlite3 and print it, and you get Snappy frames, not text.

The old file has not merely been demoted, it is no longer written. Mirroring mutations into webappsstore.sqlite is controlled by dom.storage.shadow_writes, and the source sets kDefaultShadowWrites = false. What is left in that file is a fossil. A second file, ls-archive.sqlite, holds whatever was there when LocalStorage NextGen first ran; the source comment calls its contents "always old, unused LocalStorage data."

The pref that used to toggle the new implementation is gone. What remains is dom.storage.enable_unsupported_legacy_implementation, default false, with a blunt comment beside it: "Please don't advertise this pref as a way for disabling LSNG. This pref is intended for internal testing only and will be removed in near future."

For Firefox, drive the browser. Playwright ships Firefox 153 and reads its storage through the same calls as Chromium. Decoding LocalStorage NextGen by hand means locating the origin directory, opening SQLite and undoing Snappy per row, for data you could have had in one line.

The file-reading route has an expiry date

Chromium is moving DOM storage off LevelDB. The flags in dom_storage/features.h say it plainly: kDomStorageSqlite is documented as "enable to use a SQLite database with local storage and session storage instead of LevelDB. See crbug.com/377242771: Migrate DOMStorage to use SQLite."

Where the rollout stands in the source on 13 August 2026:

• kDomStorageSqliteInMemory is enabled by default. In-memory storage, which is what Incognito uses, is already SQLite.
• kDomStorageSqlite and kDomStorageSqliteNewDatabases are disabled by default, so on-disk profiles still get LevelDB.
• A staged rollout waits behind a feature param with four positions, from kUseLevelDbOnly to kUseSqliteOnly. In the mixed stages the backend is picked per profile, by checking for an existing LevelDB directory.

The directory names differ, which is a cheap way to tell which backend a profile uses: LevelDB lives in Local Storage/leveldb, while SQLite uses LocalStorage with no space. When on-disk profiles will flip is not knowable from the source tree, and no milestone has been published. The direction is knowable. Any parser you write against the LevelDB format has a shelf life.

Where each route stops working

Opaque origins. about:blank, data: URLs and sandboxed iframes without allow-same-origin throw SecurityError on any access. Navigate first, then read.

Stale files. Five-second commit delay, sixty commits per hour. Anything read from disk while the browser runs is a snapshot of unknown age.

Eviction. Chromium marks storage keys stale after 400 days without access, and private windows lose their storage when the last private tab closes. A profile is not an archive.

Encryption asymmetry. Chrome's Cookies database has an encrypted_value BLOB column and goes through OS-level crypto. The local storage LevelDB has no equivalent: keys and values sit there as plain UTF-16 or Latin-1 strings. Local storage is the softer target, worth remembering when the app storing a long-lived token there is your own.

Law. The EU ePrivacy Directive is not a cookie law. Article 5(3) is written around storing information in, or gaining access to information already stored in, a user's terminal equipment, and local storage sits squarely inside that description. Tokens and identifiers pulled from a profile are also very likely personal data under GDPR. Terms of service and applicable law are separate questions, and you need an answer to both.

What breaks at ten thousand pages a night

A one-off script can afford to launch a browser, log in, wait, read one key and quit. Multiply by ten thousand and the arithmetic turns hostile. One wasted second per page across a ten-thousand-page crawl is close to three hours, and wait_until="networkidle" alone bills you 500 ms of that on every navigation.

Log in once, not ten thousand times. This is the real reason storage_state exists. Authenticate in one context, write state.json, seed every worker context from the file. A login flow per page becomes a file read.

Refresh the state before it expires, not after it fails. A saved state carries an access token with a lifetime, often fifteen minutes to an hour. A ten-hour crawl seeding every worker from a state captured at the start produces nine hours of authenticated-looking 401s. Record the capture time beside the file and re-run the login two thirds of the way through.

One context per page, not one browser. Browser startup is seconds; new_context() is milliseconds and isolates storage just as well.

Watch the storage key, not the URL. If the values come from an embedded widget, partitioning means the store is keyed by origin plus top-level site. Reading the top frame returns an empty dict and no error, the worst kind of failure. Storage.getStorageKeyForFrame makes it an explicit lookup.

Subscribe instead of polling. A WebDriverWait running execute_script every 500 ms costs a protocol round trip per attempt. DOMStorage.domStorageItemAdded fires once, when the write happens.

Budget for the network layer. Sites that hand out tokens watch who is asking, so plan for rotating proxies and for sessions that get invalidated when a saved state suddenly appears from a different exit IP.

Do not fight the storage, find the source

Local storage is usually a cache, and a cache has an origin. Before you build any of the above, open the Network tab and watch what the page fetched immediately before the key appeared. Most of the time there is one JSON endpoint whose response is written verbatim into the store.

Calling that underlying JSON API directly beats every route here by an order of magnitude, because it skips the browser, the waiting, the storage keys and the format bytes. The browser routes are the fallback for when the endpoint is signed, the token is minted client-side, or the response only makes sense after the app has transformed it. The same holds for scraping dynamic content generally: the rendered result is the expensive way to reach data that arrived as JSON a moment earlier.

Four questions settle it. Where did the value come from. Can you call that thing directly. Does it need a token. Is the token minted in the browser or handed out by the server.

Which approach should you use

For nearly every scraping task, drive the browser. It is version-independent, it handles the encoding, and it cannot hand you a five-second-stale snapshot. Keep the file-reading route for cases where launching a browser is impossible.

Bottom line

Reading local storage is a question of getting into the right context, and there are four ways in rather than the two most guides describe. Selenium and Playwright read it through JavaScript, and storage_state() turns a login into a file you can replay. The DevTools Protocol reads it without running your code in the page and reports the moment a key changes. Decoding the profile directly still works, provided you get the format bytes the right way round, copy the folder before opening it, and accept that Chromium's move to SQLite will eventually invalidate the parser.

Check the Network tab first. If the value came from an endpoint you can call, none of this is necessary. When it is, and the browser fleet and session management cost more than the data is worth, that is the point where a managed web scraping service becomes the cheaper line item.