Python XML Parser: ElementTree, lxml, and SAX Compared

XML is still everywhere: RSS and Atom feeds, sitemaps, the responses of SOAP and many REST APIs, exports from ERP and accounting systems, and banking and government data-exchange formats. Python offers several ways to parse XML — from the built-in ElementTree to the fast lxml and streaming SAX for gigantic files. Choosing the right Python XML parser comes down to the size of the file and what you need to do with it, so let's break down when to reach for each.

This is a focused follow-up to the broader guide on web scraping with Python. If XML arrives mixed in with HTML pages, start there; here, the focus is XML specifically.

Contents

1. How XML differs from HTML
2. ElementTree — the built-in module
3. lxml.etree — fast, with full XPath
4. Namespaces
5. In practice: RSS feed and sitemap
6. Non-ASCII characters and encodings
7. Large files: streaming parsing
8. Creating and writing XML
9. XML parsing security
10. Pros and cons of each approach

1. How XML differs from HTML

Although both are markup languages, the difference is fundamental:

• XML is strict: every tag must be closed, case matters, and the structure must be valid. HTML forgives mistakes.
• XML has no predefined tags — you (or the format) define them: <book>, <price>, <author>.
• XML often uses namespaces to separate vocabularies.

Because of that strictness, XML parses more predictably than HTML — but any markup error can break the parse. For invalid XML, the "forgiving" mode of lxml saves the day.

Keep the broader trend in mind too: in modern APIs, XML has largely been displaced by the lighter JSON, and if a source has a JSON endpoint, it's usually simpler to parse — those techniques are collected in parsing JSON in Python. But where XML remains the standard (RSS, sitemaps, SOAP, government formats, ERP exports), there's no way around parsing it.

2. ElementTree — the built-in module

xml.etree.ElementTree ships in the standard library — no dependencies to install. That's enough for most tasks.

import xml.etree.ElementTree as ET

xml_data = """
<catalog>
    <book id="1">
        <title>Python for Beginners</title>
        <price>39</price>
    </book>
    <book id="2">
        <title>Web Scraping in Practice</title>
        <price>59</price>
    </book>
</catalog>
"""

root = ET.fromstring(xml_data)          # from a string
# root = ET.parse("catalog.xml").getroot()   # from a file

for book in root.findall("book"):
    title = book.find("title").text
    price = book.find("price").text
    book_id = book.get("id")            # attribute
    print(book_id, title, price)

Key methods:

• find("tag") — the first child element with that tag;
• findall("tag") — all such elements;
• .text — the text inside an element;
• .get("attr") — an attribute's value;
• .attrib — a dict of all attributes.

ElementTree supports a limited subset of XPath:

root.findall(".//book")                 # all <book> at any depth
root.findall("book[@id='1']")           # by attribute
root.findall(".//price")

3. lxml.etree — fast, with full XPath

When you need speed or full XPath, reach for lxml.etree. Its API almost matches ElementTree's — the switch is seamless.

from lxml import etree

root = etree.fromstring(xml_data.encode("utf-8"))

# full XPath
titles = root.xpath("//book/title/text()")
expensive = root.xpath("//book[price > 45]/title/text()")
first_id = root.xpath("//book[1]/@id")

lxml gives you what ElementTree can't: full XPath 1.0 with functions (contains, starts-with, comparisons), axes (parent, sibling), and XSLT transformations. For its HTML capabilities, see the lxml article.

Tip: for lxml, encode the string to bytes (.encode("utf-8")) before fromstring; otherwise, if the XML contains an <?xml encoding=...?> declaration, you may hit an error.

4. Namespaces

This is a classic stumbling block for newcomers. If the XML has an xmlns, a plain find("title") finds nothing — the tags are "hidden" inside a namespace.

<feed xmlns="http://www.w3.org/2005/Atom">
    <entry><title>Headline</title></entry>
</feed>

You have to specify the namespace explicitly:

import xml.etree.ElementTree as ET

root = ET.fromstring(xml_data)
ns = {"atom": "http://www.w3.org/2005/Atom"}

# via a prefix dictionary
titles = root.findall(".//atom:title", ns)
for t in titles:
    print(t.text)

lxml is more convenient — you can use local-name() and ignore the namespace entirely:

from lxml import etree
root = etree.fromstring(xml_bytes)
# grab <title> regardless of namespace
titles = root.xpath("//*[local-name()='title']/text()")

local-name() is a lifeline when there are many namespaces, or when they aren't known in advance.

5. In practice: RSS feed and sitemap

RSS feed

import requests
import xml.etree.ElementTree as ET

resp = requests.get("https://example.com/rss", timeout=10)
root = ET.fromstring(resp.content)       # .content is bytes

for item in root.findall(".//item"):
    title = item.findtext("title")
    link = item.findtext("link")
    date = item.findtext("pubDate")
    print(title, link, date)

findtext is handier than find().text — it won't crash with AttributeError if the element is missing; it returns None or a default you specify.

Sitemap

A sitemap almost always has a namespace — a common trap when collecting all of a site's URLs:

import requests
from lxml import etree

resp = requests.get("https://example.com/sitemap.xml", timeout=10)
root = etree.fromstring(resp.content)

# work around the namespace via local-name()
urls = root.xpath("//*[local-name()='loc']/text()")
print(f"Found {len(urls)} URLs")

A sitemap is a great starting point for a crawler: a list of every page on the site with no link-following required. For a full walkthrough, see how to crawl a sitemap before scraping. When there are many feeds or sitemap files (large sites split their sitemap into dozens of parts), it makes sense to fetch them in parallel — see asynchronous scraping in Python.

6. Non-ASCII characters and encodings

In XML, the encoding is declared in the prolog:

<?xml version="1.0" encoding="windows-1252"?>

The golden rule is to hand the parser bytes, not a string, so it can read that declaration itself:

# CORRECT: bytes
root = ET.fromstring(resp.content)

# WRONG: if the string was already decoded incorrectly — mojibake
root = ET.fromstring(resp.text)

A typical lxml error is ValueError: Unicode strings with encoding declaration are not supported. It happens when you pass an already-decoded string into a document that declares an encoding. The fix — pass bytes:

from lxml import etree
root = etree.fromstring(resp.content)        # bytes, not resp.text

If the encoding in the prolog is wrong (it happens in exports from older systems that still emit legacy windows-1252 or ISO-8859-1), re-encode manually:

text = resp.content.decode("windows-1252", errors="replace")
# strip the problematic prolog and re-encode
text = text.replace('encoding="windows-1252"', 'encoding="utf-8"')
root = etree.fromstring(text.encode("utf-8"))

The rule of thumb: accented and non-Latin characters break not because Python "can't handle Unicode," but because a byte stream was decoded with the wrong codec somewhere upstream. Feed the parser raw bytes and let it honor the declared encoding.

7. Large files: streaming parsing

Exports weighing hundreds of megabytes (product feeds, dumps) can't be loaded into memory whole. Two streaming approaches.

iterparse in lxml — the recommended one

from lxml import etree

for event, elem in etree.iterparse("huge_feed.xml", tag="offer"):
    title = elem.findtext("name")
    price = elem.findtext("price")
    process(title, price)

    elem.clear()                          # free memory for processed elements
    while elem.getprevious() is not None:
        del elem.getparent()[0]

The clear() + previous-node deletion combo keeps memory consumption nearly constant regardless of file size. This lets you parse files tens of gigabytes in size.

SAX — event-based parsing

The built-in xml.sax calls your callbacks on each opening/closing tag. It uses minimal memory, but the code is more verbose and less convenient than iterparse. In practice, iterparse from lxml covers almost all streaming needs.

8. Creating and writing XML

Parsing often goes hand in hand with generation — for example, to save the result as XML.

from lxml import etree

root = etree.Element("catalog")
book = etree.SubElement(root, "book", id="1")
etree.SubElement(book, "title").text = "Web Scraping with Python"
etree.SubElement(book, "price").text = "39"

xml_bytes = etree.tostring(
    root,
    pretty_print=True,
    xml_declaration=True,
    encoding="utf-8",
)
with open("output.xml", "wb") as f:       # 'wb' — bytes!
    f.write(xml_bytes)

encoding="utf-8" in tostring guarantees that accented and non-Latin text is saved correctly, and writing in "wb" mode ensures the bytes aren't re-encoded a second time.

9. XML parsing security

XML from untrusted sources can carry attacks — for example, "billion laughs" (exponential entity expansion that exhausts memory) or external entities (XXE) that read files off your server.

The defense is the defusedxml package:

pip install defusedxml

from defusedxml.ElementTree import fromstring   # safe drop-in replacement
root = fromstring(untrusted_xml)

If you're parsing XML from external, uncontrolled sources, use defusedxml instead of the standard parsers. It removes the main classes of XML attacks.

10. Pros and cons of each approach

How to choose:

• Small XML, no extra dependencies → ElementTree.
• Need speed, complex XPath, or namespaces → lxml.etree.
• File won't fit in memory → iterparse (lxml).
• XML from an untrusted source → defusedxml.

If parsing XML feeds, ERP exports, or government formats at scale is part of a bigger data pipeline you'd rather not maintain, scraping.pro can set up the extraction and delivery for you as a data-as-a-service feed — you receive clean, structured data on a schedule in the format you need.