Best Proxy Speed for Scraping: Performance Test

Fifty workers, an average round trip of 1.2 seconds, eight hours of night: about 1.2 million pages. Push the average to 1.8 seconds and the same night gives you 800,000. Neither number depends on the size of the vendor's IP pool, which is the figure every proxy sales page leads with. The two that decide your schedule are how long a request takes and how often it comes back usable.

This is a measurement procedure, not a leaderboard. We are not publishing a latency table of our own: one taken on a single machine, on a single network, against a single target predicts nothing about yours. What follows is the arithmetic that says which number to care about, the third-party figures that do exist as of 13 August 2026, a curl command that decomposes one request into its parts, and a Python harness you can point at your own provider. The numbers have to be yours.

The arithmetic that decides the schedule

Throughput is not a property of a proxy. It falls out of Little's Law: requests in flight equal arrival rate times time in the system. Hold C requests in flight, let each take W seconds, and you finish C/W per second. Everything else is detail.

Stare at the last row. Doubling the latency and quadrupling the concurrency nearly doubles the output. Latency is a cost you pay in parallel; concurrency is a multiplier. A provider that shaves 200 ms off a request and caps you at ten connections loses to a slower one that lets you open two hundred.

Latency and success rate are the same number

A failed request occupies a slot and returns nothing. If your per-attempt success rate is s and you retry until you win, the expected number of attempts per delivered page is 1/s: 1.05 at 95%, 1.67 at 60%. The honest unit for comparing providers is seconds per delivered page, W/s, not seconds per request.

Run two candidates through it:

• 0.6 s at 60% success: 0.6 ÷ 0.60 = 1.00 s per delivered page.
• 0.9 s at 95% success: 0.9 ÷ 0.95 = 0.95 s per delivered page.

The slower proxy wins, and it wins by more than the arithmetic shows, because you also paid for the failures. Residential and mobile pools meter bytes on the wire, and a 403 body is bytes. The unlocker-style products bill on success; a raw pool does not.

What the published figures actually measure

Cross-vendor numbers do exist. Proxyway's 2026 Proxy Market Research, read on 13 August 2026, publishes measured success rates and response times for residential and datacenter pools. Its residential results, sorted by response time:

Datacenter pools in the same research run faster and tighter: Oxylabs 0.22 s at 99.98%, Decodo 0.35 s at 99.94%, Rayobyte 0.47 s at 99.88%, Webshare 0.67 s at 99.93%, DataImpulse 0.88 s at 97.86%.

Three things about those numbers, and the third is the one that matters.

The advertised figures are honest. Decodo's residential page advertises "<0.5s response time" and a "99.92% success rate"; the independent measurement came out at 0.53 s and 99.92%. Oxylabs advertises "Avg. 0.6s response time" and "Avg. 99.95% success rates" against a measured 0.60 s and 99.93%. Bright Data advertises "~0.7 sec response time" and a "99.95% success rate". All three read on 13 August 2026. Nobody is inflating anything.

They measure plumbing, not blocking. Proxyway's methodology note on its dedicated ISP tests describes "70,000 requests over 7 days targeting the nearest server of a global CDN (~6 KB page size)", and its datacenter line reads "DC: US pool, 70k connection requests over 7 days". A CDN edge serving a 6 KB object runs no bot-management decision against you. The success-rate column says the proxy network's own machinery works. It says nothing about whether a retailer will serve you a product page, and that gap is the entire reason your crawl is failing.

A benchmark is a photograph. NetNut sits in that residential table at 95.28% and 1.66 s. Its domain has served a Federal Bureau of Investigation seizure notice since 2 July 2026 and the network is gone, a story we tell in our NetNut review. Any table you find, this one included, describes the vendors that existed when someone ran the tests.

Where the milliseconds go

A proxy adds a leg, and sometimes a whole geography. For an https:// URL through an http:// proxy, the client opens TCP to the proxy, sends CONNECT host:443, and performs the TLS handshake with the origin through that tunnel. The httpx documentation puts it plainly: the client "upgrades" the connection to HTTPS "by performing the TLS handshake with the server over the TCP connection provided by the proxy". If the proxy endpoint itself speaks HTTPS you get two nested handshakes; urllib3 documents that case as "an initial TLS connection will be established to the proxy, then an HTTP CONNECT will be sent to establish a TCP connection to the destination and finally a second TLS connection will be established to the destination."

The added cost is one round trip to the proxy plus the difference between your route to the origin and the exit node's. That second term is usually the larger one and has nothing to do with the proxy software. A residential exit in São Paulo talking to a server in Frankfurt is 200 ms of physics before anyone writes a line of code.

The first request pays for all of it and the rest do not. On a kept-alive connection the handshakes happen once; on a rotating pool that opens a new tunnel per request, every time. Both are legitimate measurements of the same provider. Decide which one you are running before you run it.

Decompose one request with curl before you write any code

A single curl invocation answers most of the question without a harness, a virtualenv or a hypothesis. The --write-out variables split one transfer into its phases, and the definitions are worth reading literally: time_connect runs "until the TCP connect to the remote host (or proxy) was completed", time_appconnect "until the SSL/SSH/etc connect/handshake to the remote host was completed".

FMT='dns=%{time_namelookup} tcp=%{time_connect} tls=%{time_appconnect} ttfb=%{time_starttransfer} total=%{time_total} conns=%{num_connects} code=%{http_code} bytes=%{size_download}\n'

# baseline, no proxy
curl -sS -o /dev/null -w "$FMT" https://www.cloudflare.com/cdn-cgi/trace

# same request through the proxy
curl -sS -o /dev/null -w "$FMT" \
     -x http://USER:PASS@HOST:PORT \
     https://www.cloudflare.com/cdn-cgi/trace

Read the two lines side by side. tcp is your leg to the proxy. tls − tcp is the CONNECT plus the origin handshake over the proxy's leg, and it is where a badly placed exit node shows up. ttfb − tls is the origin thinking, which the proxy cannot help with. total − ttfb is the body coming down the wire, and on a large page that is the number that ends up on your bandwidth invoice.

Two diagnostic patterns are worth memorising. A nonzero tcp with tls=0.000000 and code=000 means you reached the proxy and it refused the tunnel: authentication, an IP allowlist, or a blocked destination port, not a slow network. A tcp that is already 0.3 s means the gateway is far from you, and no amount of concurrency tuning fixes geography.

Cloudflare's trace endpoint makes a better default target than the usual choices. It needs no key, returns a few hundred bytes, and carries exactly the fields a proxy test wants: ip is the exit address the origin saw, colo is the Cloudflare datacenter that served you, and tls and http report the negotiated protocol versions. That last pair matters, because a proxy that quietly downgrades you to HTTP/1.1 while your browser profile claims HTTP/2 hands the target a free detection signal. If you prefer ipinfo.io/json, IPinfo's documentation calls its Lite tier "unlimited access to our API". Either way you are measuring a third party's CDN, not your target.

A harness that separates the proxy from the target

The sequential script is worth writing because it collects the distribution, not just the mean. Code below was written against Python 3.11, requests 2.34.2 (released 14 May 2026) and httpx 0.28.1.

import csv, statistics, time
from collections import Counter
import requests
from requests.exceptions import (
    ProxyError, SSLError, ConnectTimeout, ReadTimeout,
    ConnectionError as ReqConnectionError, RequestException,
)

TEST_URL   = "https://www.cloudflare.com/cdn-cgi/trace"
PROXY      = "http://USER:PASS@HOST:PORT"
N          = 2000          # per arm; see the note on p95 sample size below
TIMEOUTS   = (5, 15)       # (connect, read) -- never a single scalar
COLD       = True          # True: fresh connection per request, like a rotating pool

def classify(exc):
    # Order matters. ConnectTimeout subclasses BOTH ConnectionError and Timeout,
    # so a broad `except Timeout` first would swallow it and blame the target.
    if isinstance(exc, ProxyError):         return "proxy_error"      # proxy said no
    if isinstance(exc, ConnectTimeout):     return "connect_timeout"  # never reached origin
    if isinstance(exc, SSLError):           return "tls_error"
    if isinstance(exc, ReadTimeout):        return "read_timeout"     # origin was slow
    if isinstance(exc, ReqConnectionError): return "conn_reset"
    return "other"

def one(session, proxy):
    arm = "proxy" if proxy else "direct"
    proxies = {"http": proxy, "https": proxy} if proxy else None
    t0 = time.perf_counter()
    try:
        r = session.get(TEST_URL, proxies=proxies, timeout=TIMEOUTS,
                        allow_redirects=False)
        body = r.content
        return {"arm": arm, "t": time.perf_counter() - t0,
                "status": r.status_code, "bytes": len(body),
                "exit_ip": next((l[3:] for l in body.decode("utf-8", "replace").splitlines()
                                 if l.startswith("ip=")), ""),
                "outcome": "ok" if r.status_code == 200 else f"http_{r.status_code}"}
    except RequestException as exc:
        return {"arm": arm, "t": time.perf_counter() - t0, "status": "",
                "bytes": 0, "exit_ip": "", "outcome": classify(exc)}

def run(path="samples.csv"):
    rows, session = [], None
    for i in range(2 * N):
        proxy = PROXY if i % 2 else None      # interleave the arms, do not batch them
        if COLD or session is None:
            if session: session.close()
            session = requests.Session()
            session.headers["Connection"] = "close"
        rows.append(one(session, proxy))
    session.close()
    with open(path, "w", newline="") as f:
        w = csv.DictWriter(f, fieldnames=list(rows[0]))
        w.writeheader(); w.writerows(rows)
    return rows

def summarize(rows, arm):
    sel = [r for r in rows if r["arm"] == arm]
    lat = sorted(r["t"] for r in sel if r["outcome"] == "ok")
    if len(lat) < 20:
        return {"arm": arm, "n": len(sel), "ok": len(lat), "note": "too few to quantile"}
    q = statistics.quantiles(lat, n=100, method="inclusive")
    fails = Counter(r["outcome"] for r in sel if r["outcome"] != "ok")
    return {"arm": arm, "n": len(sel), "ok": len(lat),
            "success_rate": round(100 * len(lat) / len(sel), 2),
            "mean": round(statistics.fmean(lat), 3),
            "p50": round(q[49], 3), "p95": round(q[94], 3), "p99": round(q[98], 3),
            "exit_ips": len({r["exit_ip"] for r in sel if r["exit_ip"]}),
            "mbytes": round(sum(r["bytes"] for r in sel) / 1e6, 1),
            "failures": fails}

rows = run()
d, p = summarize(rows, "direct"), summarize(rows, "proxy")
print("direct :", d)
print("proxied:", p)
print("overhead p50 (s):", round(p["p50"] - d["p50"], 3))
print("overhead p95 (s):", round(p["p95"] - d["p95"], 3))

Four design decisions in there are the whole point. The arms are interleaved rather than run back to back, so a change in your uplink halfway through hits both equally. The timeout is a tuple, so a connect failure and a slow body stay distinguishable. Every raw sample goes to CSV, because a summary hides the shape and the shape is where the diagnosis lives. And exit_ips counts the distinct addresses the origin saw, the one-line answer to whether your rotating endpoint rotates.

Compare percentiles, not means. p["p50"] - d["p50"] is what a typical request costs you; p["p95"] - d["p95"] is what the bad ones cost.

Six ways this measurement lies to you

Some of these were wrong in the earlier version of this article, and the first one was wrong in a way that defeated its own argument.

Catching Timeout before ConnectionError inverts the diagnosis. The previous version told you that connection errors are the proxy's fault and timeouts are the target's, then ordered its except clauses so the claim could not hold. In requests, ConnectTimeout inherits from both ConnectionError and Timeout. A try/except Timeout / except ConnectionError block catches a failure to reach the proxy in the timeout branch and files it under "the target was slow". Check it in one line: issubclass(requests.exceptions.ConnectTimeout, requests.exceptions.Timeout) returns True. Match on concrete classes, most specific first, as classify() does.

A single scalar timeout bounds nothing. The requests documentation defines the read timeout as "the number of seconds that the client will wait between bytes sent from the server". A server dripping one byte every ten seconds never trips a 30-second read timeout and holds a worker slot until you notice. If total wall-clock matters, enforce it with a watchdog.

Redirect policy makes two scripts incomparable. requests follows redirects by default; httpx does not, and its follow_redirects defaults to False. The earlier version of this article shipped a requests script and an httpx script side by side and invited you to compare their success rates. It could not work: the first counted a redirected page as a 200 after following it, the second counted the 302 as a failure. Set the flag explicitly in both.

Your concurrency is not what you set it to. A requests.Session uses urllib3's pool, and requests ships pool_connections=10, pool_maxsize=10, pool_block=False. Run forty threads through one Session and urllib3 opens the extra connections but discards them instead of pooling them, so you measure cold handshakes believing you measured a warm pool. httpx is stricter: its default Limits(max_connections=100, max_keepalive_connections=20) queues request 101 instead of lying about it. Set the pool size to your intended concurrency in both.

Two hundred requests cannot see a p95. The rank of the 95th percentile in a sample of n has a standard error near the square root of n×0.95×0.05. At n=200 that is about three ranks, so a 95% confidence interval spans ranks 184 to 196 and your p95 sits somewhere between the 92nd and the 98th percentile. At n=2000 the interval narrows to 94.1% through 96.0%. If p95 will drive a timeout decision, collect thousands of samples.

Keep-alive can cancel rotation. Oxylabs documents its residential endpoint as pr.oxylabs.io:7777 where "every new request will use a different proxy". An established CONNECT tunnel still terminates at one exit node. Whether requests rotate while a connection is held open is a property of the gateway and varies by vendor, so assume nothing in either direction. The exit_ips counter answers it empirically: run 200 requests with COLD = True, again with COLD = False, and compare the distinct-address counts.

Finding the concurrency knee

Sequential timings say nothing about a pool under load. Sweep concurrency and watch for the point where requests per second stops climbing or the success rate starts falling.

import asyncio, statistics, time, httpx

TEST_URL = "https://www.cloudflare.com/cdn-cgi/trace"
PROXY    = "http://USER:PASS@HOST:PORT"
TOTAL    = 2000

async def measure(concurrency):
    limits  = httpx.Limits(max_connections=concurrency,
                           max_keepalive_connections=concurrency)
    timeout = httpx.Timeout(connect=5.0, read=15.0, write=5.0, pool=None)
    sem     = asyncio.Semaphore(concurrency)
    lat, tally = [], {"ok": 0, "http_error": 0, "transport_error": 0}

    async with httpx.AsyncClient(proxy=PROXY, limits=limits, timeout=timeout,
                                 follow_redirects=True) as client:
        async def one():
            async with sem:
                t0 = time.perf_counter()
                try:
                    r = await client.get(TEST_URL)
                except httpx.HTTPError:
                    tally["transport_error"] += 1
                    return
                if r.status_code == 200:
                    tally["ok"] += 1
                    lat.append(time.perf_counter() - t0)
                else:
                    tally["http_error"] += 1

        t0 = time.perf_counter()
        await asyncio.gather(*(one() for _ in range(TOTAL)))
        dur = time.perf_counter() - t0

    p95 = statistics.quantiles(lat, n=100, method="inclusive")[94] if len(lat) > 20 else None
    print(f"c={concurrency:>3}  {TOTAL/dur:6.1f} req/s  "
          f"ok={100*tally['ok']/TOTAL:5.1f}%  "
          f"p95={p95 if p95 is None else round(p95, 3)}  {tally}")

async def sweep():
    for c in (5, 10, 20, 40, 80, 160):
        await measure(c)
        await asyncio.sleep(30)     # let the pool and the target settle

asyncio.run(sweep())

pool=None in the timeout is deliberate: waiting for a free slot in your own connection pool is not a network timeout, and letting it masquerade as one produces a beautifully wrong graph. The 30-second gap between rungs matters too, since a target that just rate-limited you at c=80 is still rate-limiting you at the start of c=160.

The plateau you find is often not the proxy's ceiling. It is the target's, or your file-descriptor limit, or a TLS-handshake bottleneck on one CPU. Confirm which by rerunning the top rung against a second, unrelated target: if both plateau at the same requests per second, the limit is on your side. The async patterns underneath all this are worked through in async web scraping.

The timeout is a throughput setting

Most crawls leave this knob at whatever the tutorial said, and it is worth more than a proxy upgrade. Take 50 workers, a 1.2-second mean on successful requests, and 2% that hang until the timeout fires.

At a 30-second timeout, average slot occupancy is 0.98 × 1.2 + 0.02 × 30 = 1.78 s. That is 28 requests per second, about 811,000 pages in eight hours. Drop the timeout to 8 seconds and occupancy falls to 1.34 s: 37 requests per second, about 1,078,000 pages. Two percent of your traffic, sitting on a number nobody chose deliberately, is worth roughly 267,000 pages a night.

The right timeout comes out of the distribution your harness already collected. Set it just above p99 of the successful requests and treat anything slower as a failure to retry, because a request in the tail has usually already lost. Then make the retry cheap: a fresh connection through a different exit beats waiting.

Bandwidth is the speed metric nobody reports

Residential and mobile pools bill by the gigabyte, so page weight is a cost multiplier that never shows up in a latency benchmark. Take 100,000 product pages at 400 KB of HTML each: 40 GB. On the Oxylabs residential tiers read on 13 August 2026, that is $100 at the 1 TB rate of $2.50/GB and $240 at the 5 GB starter rate of $6.00/GB. At Bright Data's $4.00/GB pay-as-you-go, a 50% promotion against a $8.00 list price, it is $160.

Now change one thing. Render those pages in a headless browser without blocking subresources and 400 KB becomes something like 2.5 MB once images, fonts, CSS and analytics load. The same 100,000 pages become 250 GB, or $625 at $2.50/GB. Nothing about the proxy changed. The bill went up by $525 because of a request-interception rule you did not write.

Failures are billed too. At an 85% success rate you move 47 GB to deliver 40 GB of useful pages, another $18 at the cheap rate and $70 at the expensive one.

Measure bytes alongside milliseconds. The harness tracks mbytes per arm for exactly this reason, and a provider that rewrites your HTML shows up as a different number on the same target.

Which proxy type is fastest

Speed and block resistance trade against each other, but looser than the usual telling. The measured spread within residential providers, 0.41 s to 1.86 s, is wider than the gap between the fastest residential and the slowest datacenter pool.

Two corrections against the older version of this table, and against most roundups that copy each other. Datacenter latency was described here as "often under 0.5 s overhead", a figure with no measurement behind it; the published datacenter numbers above are total response times against a nearby CDN, and overhead against your own baseline is a different quantity that only your own curl run produces. And mobile is no longer reliably the priciest tier: our guide to proxy servers priced the inversion out in August 2026. Type-by-type mechanics, rotation modes and setup live in that guide and in our older walkthrough of proxies for scraping; this article is about measuring them.

Rotating versus sticky

Rotation changes what you are measuring, not just what the target sees. A new exit per request means a new TCP and TLS handshake per request, so the rotating number is a cold-connection number by construction. Sticky sessions amortise that handshake across a whole login-and-paginate flow, which is why they look faster per request. Whether they also convert better depends on the target's session heuristics, and that is a success-rate question your harness answers on your target.

What "good enough" means

Work backwards from the schedule instead of chasing a latency figure. Start with the pages you need and the window you have, divide to get requests per second, then divide by your measured seconds-per-delivered-page to get the concurrency you must sustain. If that lands inside what the sweep showed as stable, you are done and a faster proxy buys you nothing. If it does not, you need more concurrency, a lighter page, or a better success rate, in that order of cost.

Three thresholds survive contact with real jobs. A p50 overhead above one second on a nearby target usually means the exit pool is in the wrong geography, and geo-targeting fixes more than a vendor switch. A p95 more than three times your p50 means a bad tail, which costs more than the mean suggests. A success rate below about 90% on your target is an anti-bot problem rather than a proxy problem, and the next thing to fix is your client, not your addresses.

Where a benchmark stops telling you anything

Measurement has a boundary and it is closer than most write-ups admit.

Trial pools are not production pools. Nothing stops a vendor from serving trial accounts out of a cleaner slice of inventory, and no benchmark can detect it. The defence is to re-run the harness a month into a paid plan and compare against the trial numbers you kept.

Success rate is not stationary. It moves with the hour, the country, the target's current rule set, and whatever the pool's other tenants did to your target this morning. One afternoon's run is a sample of one from a distribution that changes daily. Schedule the harness, keep the CSVs, watch the trend.

You cannot benchmark someone else's site politely at scale. A concurrency sweep against a live commercial target is a load test they did not agree to. Sweep throughput against a neutral endpoint and measure success rates against your real target at ordinary request rates.

A proxy is one layer of four. It changes your source address and nothing else: TLS fingerprint, header order, JavaScript environment and timing all travel unchanged. When the success rate stays low across three vendors, the address was never the problem, and the fix lies in fingerprinting or in a captcha solving service rather than in another gigabyte of residential traffic.

The measurement has a maintenance cost of its own. Pools drift and targets adapt, so the harness has to keep running for its output to mean anything. That standing cost is the honest argument for handing the layer over: a managed extraction service absorbs proxy benchmarking, rotation and retry logic as an operation, and a data-as-a-service feed removes the question from your side of the wire.

The bottom line

The best proxy speed for scraping is the lowest seconds-per-delivered-page on your target, and that number comes out of latency, success rate and how much concurrency the pool tolerates together. Published figures, this article's included, measure a small object on a friendly CDN: they establish that vendors describe their own plumbing accurately and say nothing about whether your retailer will serve you a page. Decompose one request with curl, run the harness against your own target for a few thousand samples, keep the raw CSV, set the timeout from the p99 you measured rather than from habit, and count bytes next to milliseconds. Then buy the cheapest tier that clears the success-rate bar your arithmetic demands.