Proxies for Web Scraping: Types, Privacy, and Detection

Paste a fresh proxy into any public anonymity checker and it comes back elite: no Via, no X-Forwarded-For, nothing that announces a middleman. Point your scraper at a real target through the same address and the first request returns 403. The checker was not lying to you. It answered a question about plaintext HTTP, and your scraper has not sent a plaintext HTTP request in years. Through an encrypted tunnel the proxy never sees your headers and cannot add one, so the three-level anonymity taxonomy that the entire free-proxy world still runs on is grading a protocol you stopped using.

That gap between what proxy documentation measures and what target sites measure is the subject of this article. Below is every major type of web scraping proxy by protocol and by address source, what each one costs, what a site can learn about it before your first byte of HTML arrives, and code that works in six stacks. Prices were read from vendor pricing pages on 13 August 2026, versions from package registries the same day. Where a claim could not be checked against a primary source, the text says so instead of rounding it into confidence.

What a proxy changes, and what it does not

A proxy is a middleman between your scraper and the target. The request does not go directly. It travels through the proxy, so the site sees the proxy's address instead of yours. That single substitution buys three things, and they are worth very different amounts.

1. It changes which address is attributed to the request. This is the whole product. Everything else in the list follows from it.
2. It spreads load across many addresses. A pool of hundreds of exits lets you stay under per-address rate limits that a single machine would trip in a minute.
3. It places you geographically. Prices, inventory, search results and sometimes the entire page differ by country, and for a large class of scraping jobs the country is the requirement, not the anonymity.

What it does not change is everything above the network layer. Your TLS handshake, your header order, your JavaScript environment, your click timing and your request cadence all travel through the proxy untouched. A perfect residential address attached to a stock requests client is a residential address attached to an obvious bot. Anti-bot systems score all of it together, and the address is one input among dozens.

Hold that thought through the sections on privacy levels. Most bad proxy decisions come from treating the address as the whole problem.

Classification by protocol: HTTP, CONNECT, SOCKS

The first thing to understand is the layer a proxy operates on, because it decides what the proxy can read, what it can modify, and what it can leak on your behalf.

HTTP proxy

Works with plaintext HTTP. It parses the request line and headers, can read and rewrite them, can cache responses, and can inject headers of its own. For the diminishing set of sites that still serve anything over port 80, that is enough.

This is also the only mode in which the classic anonymity levels mean anything, which is why they are discussed there and not here.

HTTPS through CONNECT

For an https:// URL, the client asks the proxy to open a tunnel with the CONNECT method, then performs its own TLS handshake with the origin server through that tunnel. MDN's description of the method is the one worth memorizing: the proxy is asked to "blindly forward data in both directions until the tunnel is closed."

Three consequences follow directly, and each of them contradicts something you will read on a proxy-list site.

The proxy cannot see your headers. They are inside TLS. It knows the hostname you asked for, from the CONNECT line and from SNI, and it knows the byte counts and timing. It does not know your User-Agent.

The proxy cannot add headers either. No Via, no X-Forwarded-For, not because it is well configured but because there is nowhere to put them. A transparent proxy and an elite proxy are indistinguishable over CONNECT.

Tunnel support is not optional. Any proxy that cannot do CONNECT is useless for scraping in 2026. A depressing share of free lists still contains them, and they fail in a way that looks like a network error rather than a configuration error.

SOCKS4, SOCKS4a and SOCKS5

SOCKS sits lower. It is a generic TCP tunnel that does not parse what flows through it, and SOCKS5 also carries UDP. The protocol is old and short: RFC 1928 is a Standards Track document from March 1996.

The version differences are smaller than the folklore suggests, and one of them is a correction to the earlier version of this article.

• SOCKS4 has no password authentication, no IPv6, no UDP, and expects an IPv4 address rather than a hostname.
• SOCKS4a is the 1990s extension that fixes exactly one of those: the client may send a hostname and the proxy resolves it. Saying flatly that SOCKS4 cannot resolve names on the proxy side, as this article previously did, skips the variant that can. libcurl exposes it as its own scheme, socks4a://, described in the CURLOPT_PROXY documentation as "SOCKS4a Proxy. Proxy resolves URL hostname."
• SOCKS5 adds authentication, IPv6 and UDP. It did not add remote DNS. RFC 1928 has carried a DOMAINNAME address type, hex X'03', since 1996, and the address field for it "contains a fully-qualified domain name."

That last point matters more than it sounds. Remote DNS in SOCKS5 was never a feature you enable on the server. It is a decision your client makes about which address type to put in the request, and clients differ in how they expose it.

The DNS leak nobody checks

Both curl and requests split the choice across two URL schemes. The requests documentation states it plainly: "Using the scheme socks5 causes the DNS resolution to happen on the client, rather than on the proxy server. This is in line with curl, which uses the scheme to decide whether to do the DNS resolution on the client or proxy. If you want to resolve the domains on the proxy server, use socks5h as the scheme."

Write socks5:// and your own resolver looks up every hostname you scrape. Your ISP, your cloud provider and anyone watching your recursive resolver get a complete, timestamped list of your targets, while the proxy only ever sees IP addresses. Write socks5h:// and the lookup happens at the exit. Two characters.

The leak is easy to verify and almost nobody does. Point your scraper at a hostname that exists only in your own zone, one label per run, then read your authoritative DNS logs. If the query arrives from your own network rather than from the proxy's country, the scheme is wrong.

Rule of thumb: take SOCKS5, and write socks5h:// in code so that name resolution happens where the request does.

One practical caveat before you standardize on SOCKS. In Python, SOCKS support for requests comes from the requests[socks] extra, which pulls in PySocks. The latest release of PySocks is 1.7.1, published 20 September 2019. It works, it is pinned into millions of installs, and it has not moved in nearly seven years. Nothing is broken and nothing is being fixed.

Anonymity levels are measuring the wrong protocol

Every proxy list on the internet sorts its entries into three buckets. The buckets are real, the test behind them is real, and on today's web the result is close to meaningless. Here is the taxonomy, and then the reason it fell apart.

1. Transparent

Passes your real address in headers and announces itself as a proxy.

Via: 1.1 proxy
X-Forwarded-For: YOUR_REAL_IP

Useless for scraping: the site sees the proxy and sees you. These live in corporate networks and caching layers, where disclosing the original client is the point.

2. Anonymous

Hides your address but not the fact that a proxy is in the path.

Via: 1.1 proxy
X-Forwarded-For: PROXY_IP   (or absent, but Via gives the proxy away)

3. Elite / high anonymous

Passes neither your address nor any sign of a proxy.

(no Via, X-Forwarded-For, X-Real-IP, Proxy-Connection)

No standard defines these three levels. They come from proxy-list culture, not from any specification. What the specifications define is narrower: RFC 7239, a Proposed Standard from June 2014, standardizes the Forwarded header and describes X-Forwarded-For, X-Forwarded-By and X-Forwarded-Proto as "non-standard header fields" that were already in common use. The HTTP specification defines Via for intermediaries. Neither document contains the word "elite".

Why the test stopped applying

The check that produces those three labels is always the same. Send a request through the proxy to an echo endpoint, read back the headers that arrived, look for the giveaways. It works because a plaintext HTTP proxy parses and rewrites your request on the way through.

Send that same request to an https:// URL and the proxy issues a tunnel instead. It forwards bytes it cannot read. There is no header to add, no header to strip, and no configuration difference between a transparent proxy and an elite one that survives the transition. Every proxy in the world grades as elite over CONNECT.

So an elite rating certifies one thing: on port 80, this proxy does not rewrite requests. It says nothing about the traffic you actually send.

This is where the earlier version of this article was wrong, in code. It shipped a checker that requested https://httpbin.org/get through the proxy and set "anonymous": not leaked based on whether X-Forwarded-For or Via appeared in the response. Over TLS those headers cannot appear, so the function returns anonymous: True for every proxy it ever tests, including a transparent one that would have handed over your address on the first plaintext request. The corrected version is further down, and it tests both schemes.

What an elite rating still tells you

Not nothing. If any part of your pipeline touches plaintext HTTP, and a surprising amount does through redirects, health checks, and robots.txt fetches written before anyone cared, header behavior on port 80 is worth knowing. Two proxies that grade identically over HTTPS can differ completely there.

Treat it as a hygiene check on a legacy path, not as a privacy rating. The rating that decides whether you get blocked is computed somewhere else entirely.

What actually classifies your address

Target sites do not guess your proxy from headers. They look the address up. The lookup is a commodity, sold as a database file or an API by several vendors, updated daily, and it answers a much more useful question than the header test ever did: what kind of network is this address on.

MaxMind's GeoIP2 Anonymous IP database is the reference implementation of the idea. Every row carries independent boolean flags, and the definitions are the vendor's own:

• is_anonymous is set "if the IP address belongs to any sort of anonymous network."
• is_hosting_provider is set "if the IP address belongs to a hosting provider." This is the flag that catches datacenter proxies, and it catches them without knowing anything about proxies.
• is_public_proxy is set "if the IP address belongs to a public proxy."
• is_tor_exit_node is set "if the IP address is a Tor exit node."
• is_anonymous_vpn is set "if the IP address is registered to an anonymous VPN provider."
• is_residential_proxy is set "if the IP address is on a suspected anonymizing network and belongs to a residential ISP (does not include peer-to-peer proxy IPs)."

Read that last definition twice, because it is the one that undoes the standard sales pitch. Residential proxies are sold on the promise that a home ISP address is indistinguishable from a home user. MaxMind ships a flag whose entire purpose is to distinguish them, updated daily, on both IPv4 and IPv6. The parenthetical is the interesting part: peer-to-peer sourced addresses are explicitly excluded from this flag, which is a fair description of where the boundary of the technique currently sits.

IP2Location's IP2Proxy approaches the same problem with a type code per address rather than a set of booleans: VPN, TOR, PUB for public proxies, WEB for web proxies, DCH for hosting and CDN ranges, SES for search engine robots, RES for residential proxies, CPN for consumer privacy networks and EPN for enterprise private networks. Nine categories, and the one you are buying has its own label.

What the detection side costs

The supply side of this market gets written about constantly. The detection side is priced just as publicly and almost never quoted, so here it is, read on 13 August 2026:

Two things fall out of that table. A site can start classifying your traffic for $399 a year, which is less than a week of residential bandwidth for a medium crawl. And residential proxy detection specifically sits at the $999 tier at IPQualityScore, which is a decent public estimate of how much harder that problem is than spotting a datacenter range.

The same table is your test harness. Nothing stops you from buying the lookup your target is buying and running your own pool through it before the target does. That single habit separates teams that know why they are blocked from teams that rotate harder and hope.

Classification by IP source

Far more important than protocol is where the address came from. That is what the databases above are classifying, and it is the axis on which price varies by two orders of magnitude.

Datacenter proxies

Addresses owned by hosting providers and cloud operators. Cheap, fast, available by the thousand, and flagged by is_hosting_provider without any proxy-specific intelligence at all: the ASN gives it away, and the ASN is public.

That is the honest summary of the type. They are excellent against sites with no bot management, wasted against sites with any, and the boundary between those two groups moved in the wrong direction over the last three years. Webshare sells 100 shared proxies for $2.99 a month, about $0.03 per address, with a free tier of 10. Bright Data charges $1.40 per address at 10 and $0.90 at 1,000. For a fuller price comparison across the datacenter market, see the SquidProxies review and the proxy servers overview.

ISP proxies (static residential)

A hybrid: the addresses are registered to a consumer ISP but hosted in a data center. You get datacenter routing and latency with a residential registration, and the address does not change, which is what session-bound work needs.

Bright Data lists these at $1.80 per address at 10, falling to $1.30 at 1,000. Decodo starts at $0.27 per address. The gap is mostly about how clean the addresses are and how many customers share them, and no vendor publishes enough to verify that.

Their real advantage in August 2026 is administrative rather than technical, and it is covered two sections down.

Residential proxies

Addresses assigned by consumer ISPs to real subscribers, reached through software running on those subscribers' devices. This is the workhorse tier for hard targets, it is where nearly all the money in the proxy market sits, and it is the tier that changed most in the last year.

The pitch. Traffic exits from a home connection on a consumer ISP, so the address carries an ordinary residential ASN and a plausible geography. Against marketplaces, search engines and social platforms, that is often the difference between a page and a challenge. Residential proxies for scraping remain the default recommendation for those targets, and nothing here changes that.

The prices, read on 13 August 2026. Bright Data quotes $4.00/GB pay as you go against a $8.00 list price, falling to $2.50/GB on the $1,999 monthly plan. Oxylabs starts at $6.00/GB on a 5 GB plan and reaches $2.50/GB at 1 TB. Decodo runs $3.75/GB at 3 GB down to $2.75/GB at 100 GB, with pay as you go at $4.00/GB and a 100 MB trial. Webshare advertises rotating residential from $1.40/GB at 3 TB after a standing discount. Entry rates across the market vary by more than four to one and that spread is not a quality ranking.

The compliance gate is new. Bright Data's own residential network access policy states that "Residential access is granted only to registered companies, and only after the Bright Data compliance team reviews and approves your KYC submission," and sets the scope explicitly: "For new Residential zones created after July 7, 2026, KYC approval is required for every proxy type: shared rotating IPv4, the IPv4 and IPv6 'Mega Pool', IPv6 and dedicated Residential proxies." Approval is promised within 48 hours. ISP proxies, datacenter proxies and the Web Unlocker API are exempt. If your plan was to sign up on a Friday and deliver a dataset on Monday, the paperwork is now on the critical path.

The sourcing question stopped being theoretical. On 2 July 2026 federal agents seized NetNut, a residential proxy network with a substantial scraping customer base, and its customers discovered on a Thursday morning what single-vendor dependency costs. The details are in the NetNut review. The operational lesson is narrow and applies to every vendor in this tier: keep credentials in configuration rather than code, keep a second provider's account alive, and know your fallback's per-gigabyte price before you need it.

Where they still fail. A residential address does not fix a datacenter-shaped TLS handshake, a browser reporting the wrong timezone for its exit country, or a request pattern that no reader would produce. It also does not fix a dirty address: consumer addresses carry history, and a pool that includes a machine in a botnet includes that machine's reputation.

Mobile proxies

Addresses from mobile carriers on 3G, 4G and 5G. Carrier-grade NAT puts hundreds or thousands of real subscribers behind one public address, so the standard argument runs that a site cannot ban the address without banning a crowd.

The argument is half right and worth stating precisely. A blanket ban is expensive for the site, so sites rarely issue one. What they issue instead is a higher challenge rate, tighter rate limits and more aggressive scoring on the same address, none of which have the collateral cost of a ban and all of which land on you. CGNAT cuts the other way too: you inherit the reputation of everyone else behind that address, and you have no visibility into what they are doing.

Mobile is also the tier where the catalogue moved. Bright Data's proxy pages listed residential, ISP and datacenter on 13 August 2026, with no mobile network among them, which retires a standard piece of escalation advice for that vendor. Decodo still sells mobile, from $2.25/GB. Price mobile against the alternative honestly: at roughly twice the residential rate, it has to be solving something residential could not.

Quick comparison by stealth and price:

Datacenter  →  ISP  →  Residential  →  Mobile
cheaper/                            pricier/
easier to spot                      harder to spot

That ordering is still correct and it is also the wrong mental model to stop at. The steps are not evenly spaced, the cheapest step that clears your target is the right one, and the arithmetic in the billing section decides it more often than stealth does.

IPv4 vs IPv6

IPv4 space ran out, which makes addresses scarce and expensive. IPv6 space is effectively unlimited, so IPv6 proxies are dramatically cheaper. On proxy-seller, IPv6 addresses start at $0.16 each against $0.49 for the cheapest IPv4 on the same site, with bulk tiers going lower still. Bright Data sells IPv6 inside its residential product, and its KYC notice names an "IPv4 and IPv6 'Mega Pool'" alongside standalone IPv6 zones.

The catch is not subtle, and it is worth measuring rather than asserting. An IPv6 proxy can only reach a target that publishes an AAAA record.

We resolved 40 commonly scraped hostnames from a single host on 13 August 2026 and asked one question of each: does an AAAA record exist. Twelve did. Twenty-eight were IPv4-only. The twelve with IPv6: www.amazon.com, www.expedia.com, www.kayak.com, www.linkedin.com, www.ziprecruiter.com, www.crunchbase.com, www.google.com, www.bing.com, www.instagram.com, www.facebook.com, news.ycombinator.com and www.wikipedia.org. The list without it includes eBay, Walmart, Target, Best Buy, Etsy, AliExpress, Home Depot, Costco, Booking.com, Airbnb, Tripadvisor, Indeed, Glassdoor, Yelp, Zillow, Realtor.com, Reddit, X, TikTok, Pinterest, IMDb, GitHub, Stack Overflow and the New York Times. Resolution from one machine on one resolver. Regional DNS answers differ and the set is not a random sample. The ratio is the finding, not the exact twelve.

One trap showed up inside that measurement. amazon.com has no AAAA record while www.amazon.com does. Apex and www are separate names with separate records, and a crawler that follows a redirect from one to the other can change address families mid-job. Check the exact hostname you will request, not the brand.

The check itself is one line, and it costs nothing to run against your own target list before you buy anything:

python3 -c "import socket,sys;print(bool(socket.getaddrinfo(sys.argv[1],443,socket.AF_INET6)))" www.example.com

That last row is the reason cheap IPv6 is not the free lunch the price suggests. A single customer is routinely handed a /64, which is more addresses than the entire IPv4 internet, so scoring individual IPv6 addresses is pointless and nobody does it. Sites rate-limit and ban by prefix. Rotating through a million addresses inside one /64 is, from the target's side, one client that has not rotated at all.

Bottom line: IPv6 is genuinely cheap for the minority of targets that answer on it, and useless everywhere else. Resolve your target list first. If most of it is IPv4-only, the price difference is not a decision you get to make.

Rotation: static and rotating

• Static (sticky): the same address holds for a session. Anything with a login, a cart, a multi-step form or a server-side session token needs this, because an address change mid-session reads as account takeover to the site and as a lost session to you. Sticky durations are advertised in minutes or hours and are best understood as requests, not guarantees.
• Rotating: the exit changes automatically, per request or per interval. Most vendors hand you one gateway hostname and rotate behind it, so your code sees a single endpoint and the pool churns invisibly. This is the right default for bulk collection where every page is independent. See rotating proxies for the endpoint mechanics, and rotating proxies for business directories for the pool-management side.

The failure mode worth naming: rotating too fast within one logical session. If page 2 of a paginated result arrives from a different country than page 1, you have not spread load, you have created an anomaly that no real user produces.

Tor as a free proxy

Tor routes traffic through three relays, peeling one layer of encryption at each, and exposes a SOCKS5 proxy locally on port 9050 that any scraper can point at. The current stable release is tor 0.4.9.11, posted 25 June 2026 to the project's distribution directory. The project is alive and shipping.

Geography is configurable through torrc:

# Exit only through nodes in these countries
ExitNodes {us},{de},{nl}
StrictNodes 1

# You can hard-set entry/exit nodes by country
EntryNodes {de}
ExcludeNodes {ru},{cn}
ExcludeExitNodes {ru}

Country codes go in curly braces by ISO code. StrictNodes 1 forbids stepping outside the list. To get a new circuit, and usually a new exit address, send NEWNYM to the control port.

Why it does not work for scraping, in order of how quickly you hit each one:

• The exit list is published. Tor operates a bulk exit list at check.torproject.org precisely so that sites can consume it. MaxMind ships is_tor_exit_node as a first-class flag; IP2Proxy uses the type code TOR. This is not detection, it is a lookup against a file the network hands out on request. No amount of circuit rotation changes it.
• The pool is tiny. Exits number in the low thousands against the millions of addresses a residential vendor advertises, and many exits are shared, congested and already scored.
• It is slow. Three hops plus volunteer bandwidth. For latency-sensitive crawls, see the proxy speed and performance test.
• The control library is quiet. stem, the Python library used to send NEWNYM, last released 1.8.2 on 6 June 2023. It works. It is not moving.

Tor is a fine tool for a one-off fetch, for checking what a page looks like from another country, and for anything where being on a public list is not a problem. It is not infrastructure for collection at volume.

Free proxy lists

Public lists offer thousands of address and port pairs for nothing, and the quality is reliably bad in ways that cost more than the proxies save.

• Most are dead on arrival, and they fail by timing out rather than refusing, so each dead entry costs you a full timeout rather than a fast error.
• Many are transparent on port 80, which is the one place the anonymity test still applies, and where they will hand over your address.
• All of them are already scored. Thousands of people used them before you, and is_public_proxy costs a site $399 a year to look up.
• Some are hostile. You do not know who operates the exit, and on any plaintext path it can read and modify what passes through.

The full economic comparison, including what an eighty percent failure rate does to a schedule, is in free versus paid proxies. The short version: free proxies are fine for learning and for throwaway experiments, and the health-checker you have to write to use them seriously is itself a scraper with its own bugs and its own bandwidth bill.

Checking a proxy the way the target checks it

Validation is not optional, because proxies die, slow down and change behavior continuously. What is worth testing, in the order that finds problems fastest:

1. Does CONNECT work. An HTTPS request through the proxy, not an HTTP one. This eliminates most of a free list in one pass.
2. What address comes out. The exit address as the target sees it, not what the vendor's dashboard claims.
3. How the address is classified. Feed the exit address to an IP-intelligence lookup. This is the test that predicts blocking, and it is the one almost nobody runs.
4. Header behavior on plaintext. Separately, over http://, to catch a transparent proxy before some legacy path in your pipeline uses it.
5. Latency and time to first byte. Reject the slow ones early. One wasted second per page across a ten-thousand-page crawl is close to three hours.
6. Real geography. An address advertised as German may resolve to another country in the database your target uses. Verify against the same database, not against a different one.

The corrected checker, written against Python 3.11 and requests 2.34.2, tests the two schemes separately rather than conflating them:

import requests

ECHO_HTTP  = "http://httpbin.org/get"    # plaintext: proxy can rewrite headers
ECHO_HTTPS = "https://httpbin.org/get"   # tunneled: proxy cannot touch headers
LEAKY = ("X-Forwarded-For", "Via", "X-Real-Ip", "Forwarded", "Proxy-Connection")

def check_proxy(proxy: str, timeout: int = 8):
    proxies = {"http": proxy, "https": proxy}
    out = {"proxy": proxy}

    # 1. Does the tunnel work at all, and what address comes out.
    try:
        r = requests.get(ECHO_HTTPS, proxies=proxies, timeout=timeout)
        out["connect_ok"] = True
        out["exit_ip"] = r.json().get("origin")
        out["latency"] = r.elapsed.total_seconds()
    except Exception as e:
        return {**out, "connect_ok": False, "error": str(e)}

    # 2. Header behaviour, which is only observable over plaintext.
    try:
        h = requests.get(ECHO_HTTP, proxies=proxies,
                         timeout=timeout).json().get("headers", {})
        out["plaintext_leaks"] = [k for k in LEAKY if k in h]
    except Exception:
        out["plaintext_leaks"] = None   # unknown, not clean

    return out

print(check_proxy("http://user:pass@1.2.3.4:8080"))

Two details carry the correction. plaintext_leaks is an empty list only when the plaintext request succeeded and returned nothing incriminating, and None when the test could not run, which is different from passing. And exit_ip exists so that step 3 has an input. The classification lookup is deliberately left out of the snippet, because it is one HTTP call to whichever vendor you chose and the interesting work is deciding what to do with a is_residential_proxy of true on an address you are paying residential prices for.

httpbin.org still responds and still echoes origin and headers, which is worth stating because a great deal of scraping tutorial code depends on it and it has had outages. Run your own echo endpoint if the pipeline is production.

What proxies do not cover

A proxy answers one question about your request. Anti-bot platforms ask several dozen, and the address is one input.

On the address, what gives you away:

• Network type. Hosting ranges are flagged by ASN before any proxy-specific analysis. This is the factor your proxy choice actually controls.
• Address reputation. History of spam, botnet membership, previous bans. A dirty residential address performs worse than a clean datacenter one.
• Request rate per address. Rotation lowers per-address rate; it does not lower the aggregate, and aggregate is what a per-site model sees.
• Geographic coherence. One session that hops from Ohio to São Paulo and back inside a minute is not a user.

What travels straight through the proxy untouched:

• TLS fingerprint. The handshake identifies your client library. Salesforce archived JA3 in May 2025 and the field moved to JA4, which survives the ClientHello randomization that Chrome introduced in 2023. Neither cares which address the handshake came from.
• HTTP header set and order. Header order is a fingerprint of its own, and most HTTP clients emit an order no browser produces.
• Browser environment. Canvas, WebGL, fonts, hardware concurrency, and the coherence between them and the address. A residential exit in Poland driving a browser that reports en-US, America/Chicago and 64 cores is a confession, not a disguise.
• WebRTC and local resolution. A headless browser configured with a proxy still has other paths to the network. Playwright's proxy.bypass option exists exactly so you can exclude hosts from the proxy, which is also a list of hosts your real address will contact.
• DNS. Covered above, and the leak most likely to be in your code right now.
• Behavior. Timing that is too fast, too regular, or too complete.

Takeaway: a good proxy (residential or ISP, clean, correctly classified for your target) sharply reduces address-based detection and does nothing to any other layer. It is necessary and not sufficient. Against Cloudflare or DataDome, pair it with a real browser engine, coherent fingerprints, a sane request rate and, where it comes to that, CAPTCHA solving.

Billing: what you actually pay for

The billing model decides your cost more often than the price per unit does, because it decides what you are charged for.

• Per GB. You pay for bytes and the address pool is unmetered. Standard for residential and mobile. The failure mode is that a headless browser fetches images, fonts, CSS and analytics along with the HTML, so a 100 KB page becomes a 2 MB page. Ten thousand HTML-only pages at 100 KB is about 1 GB, roughly $2.75 at Decodo's 100 GB rate. The same ten thousand pages rendered with assets is 20 GB or more, and the bill goes from under three dollars to over fifty. Blocking media in the browser is not an optimization, it is most of the invoice.
• Per IP. You pay per address per month with unlimited traffic. Standard for datacenter and ISP. At Webshare's $2.99 per 100 addresses, bandwidth is free and the constraint becomes how fast you can push each address before it gets rate-limited.
• Per port or per thread. You pay for concurrency rather than volume. Common on mobile.
• Per successful request. Unblocker APIs price the outcome and hide the pool, which is a different purchase: you are buying the retries, the fingerprints and the failure handling as well as the address.

The crossover. Per-GB wins when pages are small and numerous and you control what loads. Per-IP wins when volume per page is high or when you need stable addresses for sessions. The arithmetic is worth doing once with your real average page size, because teams routinely pick the model that sounded cheaper and then discover the number a month later.

Code: connecting through a proxy

The proxy string is nearly universal: scheme://login:password@host:port. Everything below was checked against the current published versions on 13 August 2026.

Python: requests (synchronous)

Written against requests 2.34.2, which requires Python 3.10 or newer.

import requests

proxies = {
    "http":  "http://user:pass@1.2.3.4:8080",
    "https": "http://user:pass@1.2.3.4:8080",
}

r = requests.get("https://httpbin.org/ip", proxies=proxies, timeout=10)
print(r.json())

# SOCKS5 with remote DNS. Needs: pip install "requests[socks]"
# The socks5h scheme resolves names at the proxy; socks5 resolves them locally.
socks = {"http": "socks5h://user:pass@1.2.3.4:1080",
         "https": "socks5h://user:pass@1.2.3.4:1080"}
print(requests.get("https://httpbin.org/ip", proxies=socks).json())

Python: httpx (async, with rotation)

Written against httpx 0.28.1. The argument name changed and old code breaks on it: proxies was deprecated in 0.26.0 in December 2023 and removed in 0.28.0 on 28 November 2024. Tutorials written before then still pass proxies= and raise a TypeError on any current install. httpx has not shipped a release since 0.28.1 on 6 December 2024.

import asyncio, httpx, random

POOL = [
    "http://user:pass@1.2.3.4:8080",
    "http://user:pass@5.6.7.8:8080",
]

async def fetch(url):
    proxy = random.choice(POOL)          # one exit per request
    async with httpx.AsyncClient(proxy=proxy, timeout=10) as client:
        resp = await client.get(url)
        return resp.status_code, resp.text[:80]

async def main():
    tasks = [fetch("https://httpbin.org/ip") for _ in range(5)]
    for code, body in await asyncio.gather(*tasks):
        print(code, body)

asyncio.run(main())

A client per request is fine for a demo and wrong at volume, because you throw away the connection pool every time. In production, build one AsyncClient per exit and keep it.

Python: Tor (SOCKS5 and a new circuit)

Written against stem 1.8.2.

import requests
from stem import Signal
from stem.control import Controller

proxies = {"http":  "socks5h://127.0.0.1:9050",
           "https": "socks5h://127.0.0.1:9050"}

print(requests.get("https://httpbin.org/ip", proxies=proxies).json())

# Request a new circuit, and usually a new exit address
with Controller.from_port(port=9051) as c:
    c.authenticate(password="your_password")
    c.signal(Signal.NEWNYM)

Node.js: axios with HttpsProxyAgent

Written against axios 1.19.0 and https-proxy-agent 9.1.0, both current in July 2026.

const axios = require("axios");
const { HttpsProxyAgent } = require("https-proxy-agent");

const agent = new HttpsProxyAgent("http://user:pass@1.2.3.4:8080");

axios.get("https://httpbin.org/ip", { httpsAgent: agent, httpAgent: agent })
  .then(res => console.log(res.data))
  .catch(err => console.error(err.message));

Node.js: SOCKS5

Written against socks-proxy-agent 10.1.0.

const axios = require("axios");
const { SocksProxyAgent } = require("socks-proxy-agent");

// socks5h:// resolves DNS at the proxy; socks5:// resolves it here
const agent = new SocksProxyAgent("socks5h://user:pass@1.2.3.4:1080");

axios.get("https://httpbin.org/ip", { httpAgent: agent, httpsAgent: agent })
  .then(res => console.log(res.data));

Node.js: Playwright through a proxy

Written against Playwright 1.62.1, released 30 July 2026. The docs describe proxy.server as accepting "http://myproxy.com:3128 or socks5://myproxy.com:3128", note that a bare myproxy.com:3128 "is considered an HTTP proxy", and document username and password as credentials "to use if HTTP proxy requires authentication."

const { chromium } = require("playwright");

(async () => {
  const browser = await chromium.launch({
    proxy: {
      server: "http://1.2.3.4:8080",
      username: "user",
      password: "pass",
    },
  });
  // A per-context proxy overrides the launch one, which is how you run
  // several exits inside one browser process.
  const context = await browser.newContext({
    proxy: { server: "http://5.6.7.8:8080" },
  });
  const page = await context.newPage();
  await page.goto("https://httpbin.org/ip");
  console.log(await page.innerText("body"));
  await browser.close();
})();

Per-context proxies are the reason a browser-based crawler does not need one process per exit. They are also where geography drifts: the context carries the address, and the timezone, locale and geolocation you set on that same context have to agree with it.

Go: net/http

package main

import (
    "fmt"
    "io"
    "net/http"
    "net/url"
)

func main() {
    proxyURL, _ := url.Parse("http://user:pass@1.2.3.4:8080")
    client := &http.Client{
        Transport: &http.Transport{Proxy: http.ProxyURL(proxyURL)},
    }
    resp, err := client.Get("https://httpbin.org/ip")
    if err != nil {
        panic(err)
    }
    defer resp.Body.Close()
    body, _ := io.ReadAll(resp.Body)
    fmt.Println(string(body))
}

PHP: cURL

<?php
$ch = curl_init("https://httpbin.org/ip");
curl_setopt_array($ch, [
    CURLOPT_RETURNTRANSFER => true,
    CURLOPT_PROXY          => "1.2.3.4:8080",
    CURLOPT_PROXYUSERPWD   => "user:pass",
    // SOCKS5_HOSTNAME is the socks5h equivalent: the proxy resolves the name
    CURLOPT_PROXYTYPE      => CURLPROXY_HTTP,
    CURLOPT_TIMEOUT        => 10,
]);
echo curl_exec($ch);
curl_close($ch);

cURL from the command line

libcurl defaults to an HTTP proxy when no scheme is given, and to port 1080 when no port is given, which is a good way to accidentally talk HTTP to a SOCKS port.

# HTTP proxy
curl -x http://user:pass@1.2.3.4:8080 https://httpbin.org/ip

# SOCKS5 with remote DNS
curl --proxy socks5h://user:pass@1.2.3.4:1080 https://httpbin.org/ip

# SOCKS4a, for the rare old proxy that only speaks 4
curl --proxy socks4a://1.2.3.4:1080 https://httpbin.org/ip

What breaks between 100 pages and 100,000

A proxy configuration that works for a hundred pages tells you almost nothing about ten thousand. Four things change shape.

Failures stop being exceptional and become a rate. At a hundred pages you retry by hand. At a hundred thousand, with a five percent failure rate, that is five thousand failures, and how you handle them is the design. The distinction that matters is between a dead exit and a blocked one: a dead exit should be retired from the pool, a blocked one means the target has started scoring you and retrying through a fresh address just spends money faster.

Timeouts become the schedule. Dead proxies rarely refuse a connection, they hang. At a ten-second timeout, one dead exit in ten across a hundred thousand requests is ten thousand timeouts, roughly 28 hours of single-worker time before a single page is parsed. The timeout value is a throughput setting, and it deserves the same attention as concurrency.

Bandwidth becomes the invoice. Per-GB billing punishes anything you did not mean to download. Block images, fonts, media and third-party analytics in the browser, and check the actual transferred bytes rather than the page size you assumed.

Address reputation degrades under your own traffic. A pool you burn through hard becomes a pool of scored addresses. Rotation strategy stops being about hiding and becomes about not spending your inventory faster than the vendor replenishes it.

None of this is visible in a proof of concept, which is why proof of concepts underestimate proxy cost by an order of magnitude with some regularity.

Where proxies stop working

Three boundaries are worth knowing before you buy anything.

When the target is not scoring addresses. If a site blocks on behavior, on a JavaScript challenge, or on an account, better addresses buy you nothing. Test with one clean address and a real browser before you buy a pool. A depressing number of proxy purchases are made to solve a fingerprinting problem.

When the session outlives the address. Anything with a login, a cart or a multi-step flow needs a stable exit for the duration. Rotating proxies are actively harmful there, and no amount of pool size compensates.

When the address is the evidence. Scraping personal data through a residential network sourced from consumer devices is not a technical shortcut around consent, and the last year made that concrete rather than theoretical. Where your traffic exits, and whose device it exits from, is a question with legal weight in the EU and increasingly elsewhere. Building and running that stack (sourcing clean addresses, rotating them, checking health, tuning fingerprints, handling challenges) is a standing project rather than a setup step. If the data is the goal rather than the infrastructure, a done-for-you data extraction service covers the proxy tier, the rotation and the delivery as clean structured output, and an ongoing data-as-a-service feed covers the case where the requirement repeats.

Proxy selection checklist

The rules that survive rewriting:

1. Test what your traffic looks like, not what the proxy list claims. An elite rating is a plaintext-HTTP fact.
2. Look your own exits up in the same kind of database your target uses. $399 a year buys the answer.
3. Write socks5h://, not socks5://.
4. Resolve the target list before buying IPv6.
5. Match the billing model to the page size, and block media before you sign anything per-gigabyte.
6. Assume the address is one signal of many. If the block survives a clean residential exit, the problem was never the address.

When you would rather not manage proxies at all

The mechanics in this article have barely changed in a decade: substitute an address, rotate it sensibly, do not leak DNS, look human. What changed is everything around them. A major residential network was seized by federal agents in July 2026, buying residential bandwidth from the largest vendor now means submitting company documents, one vendor dropped mobile from its catalogue and another changed its name, JA3 was retired, and the Python client half the tutorials use removed the argument they all pass.

Read the vendor's own pricing page, check the version your code targets, resolve your own target list, and look your own exits up before the target does. Everything else in this article is downstream of those four habits.