Best Proxy Speed for Scraping: Performance Test

When you buy a scraping proxy, the vendor quotes a pool size and a price — rarely the two numbers that actually determine your throughput: how fast the proxy responds, and how often it succeeds. The best proxy speed for scraping isn't simply the lowest latency; it's the combination of low latency, high concurrency, and a high success rate on your target sites. A blazing-fast proxy that gets blocked half the time is slower, in practice, than a moderate one that always gets through. This guide explains the metrics that matter and gives you a ready-to-run Python benchmark to test any provider before you commit.

The three metrics that matter

Proxy performance comes down to three numbers. Measure all three — any one alone will mislead you.

• Latency (response time). How long a single request through the proxy takes end to end, versus the same request made directly. The honest figure is the overhead the proxy adds: proxy_overhead = avg_time_with_proxy − avg_time_without_proxy. That isolates the proxy's contribution from the target site's own speed.
• Throughput (concurrency). How many requests per second you can push through the pool in parallel before it degrades. This is what actually caps a large scrape — you rarely make one request at a time.
• Success rate. The share of requests that return a usable 200 (not a timeout, a 403, a captcha page, or a connection reset). On protected targets this is the number that separates good residential pools from cheap datacenter IPs.

A useful mental model: effective throughput = concurrency × success_rate ÷ average_latency. Optimizing latency alone, while ignoring the other two, is the classic beginner mistake.

A modern proxy speed test (Python 3)

Here's a clean benchmark that measures latency, success rate, and the exception breakdown over many requests. It's the modernized descendant of a script we've used for years — updated to Python 3, with proper exception separation and a clear summary.

import time
import statistics
import requests

TEST_URL = "https://ipinfo.io/json"   # small, fast, returns the exit IP
PROXY    = "http://USER:PASS@HOST:PORT"  # your provider's endpoint
N        = 200                         # number of requests
TIMEOUT  = 30                          # seconds

def run_test(proxy=None):
    proxies = {"http": proxy, "https": proxy} if proxy else None
    load_times = []
    ok = timeouts = conn_errors = other = 0

    for i in range(N):
        start = time.perf_counter()
        try:
            r = requests.get(TEST_URL, proxies=proxies, timeout=TIMEOUT)
            elapsed = time.perf_counter() - start
            if r.status_code == 200:
                ok += 1
                load_times.append(elapsed)
            else:
                other += 1
        except requests.exceptions.Timeout:
            timeouts += 1
        except requests.exceptions.ConnectionError:
            conn_errors += 1   # these are the proxy's fault
        except requests.exceptions.RequestException:
            other += 1

    return {
        "requests":      N,
        "success":       ok,
        "success_rate":  round(100 * ok / N, 1),
        "timeouts":      timeouts,
        "conn_errors":   conn_errors,
        "avg_latency":   round(statistics.mean(load_times), 3) if load_times else None,
        "p95_latency":   round(sorted(load_times)[int(len(load_times)*0.95)], 3) if load_times else None,
    }

baseline = run_test(proxy=None)   # direct, no proxy
proxied  = run_test(proxy=PROXY)  # through the proxy

overhead = (proxied["avg_latency"] - baseline["avg_latency"]) \
           if proxied["avg_latency"] and baseline["avg_latency"] else None

print("Direct :", baseline)
print("Proxied:", proxied)
print("Proxy overhead (s):", round(overhead, 3) if overhead else "n/a")

Key points about this version:

• It separates connection errors from timeouts. Connection errors are the proxy's fault; timeouts and redirect loops are usually the target site's behavior, not the proxy's. Blaming the proxy for a slow origin server is a common misdiagnosis.
• It reports p95 latency, not just the average. Proxy pools are spiky — the 95th-percentile response time predicts real-world pain better than the mean.
• It computes overhead against a direct baseline, so you measure the proxy, not the internet.

Run it against a small, fast endpoint like ipinfo.io/json when you want to measure the proxy. When you want to measure success against a real target, point TEST_URL at the site you actually intend to scrape — success rates diverge wildly by target.

Testing throughput (concurrency)

Sequential requests undersell a proxy pool. The number that matters for a big job is how many concurrent requests it sustains. Here's an async version to measure requests-per-second under load:

import asyncio, time, httpx

TEST_URL   = "https://ipinfo.io/json"
PROXY      = "http://USER:PASS@HOST:PORT"
TOTAL      = 500
CONCURRENCY = 50

async def worker(client, sem, counters):
    async with sem:
        try:
            r = await client.get(TEST_URL, timeout=30)
            counters["ok" if r.status_code == 200 else "bad"] += 1
        except Exception:
            counters["err"] += 1

async def main():
    counters = {"ok": 0, "bad": 0, "err": 0}
    sem = asyncio.Semaphore(CONCURRENCY)
    start = time.perf_counter()
    async with httpx.AsyncClient(proxy=PROXY) as client:
        await asyncio.gather(*[worker(client, sem, counters) for _ in range(TOTAL)])
    dur = time.perf_counter() - start
    print(f"{TOTAL} reqs in {dur:.1f}s  =>  {TOTAL/dur:.1f} req/s")
    print(counters)

asyncio.run(main())

Raise CONCURRENCY until requests-per-second stops climbing or the success rate falls — that plateau is the pool's real ceiling. See our guide to async web scraping for the patterns behind this.

Which proxy types are fastest

Speed and block-resistance trade off almost perfectly against each other. The fastest proxies are the easiest to detect; the hardest to detect are the slowest. Here's the landscape, fastest to slowest:

The pattern:

• Datacenter proxies are fastest and cheapest because they sit in well-connected data centers — but their IP ranges are known, so protected sites block them quickly. Great for sites that don't fight back.
• ISP proxies are datacenter-hosted IPs registered to consumer ISPs: near-datacenter speed with residential legitimacy. A strong middle ground, and often the best speed-to-success ratio for serious scraping.
• Residential proxies route through real users' devices, so they're slower and priced by bandwidth — but they pass as ordinary visitors and win on success rate against tough targets.
• Mobile proxies are the slowest and most expensive, justified only when a target is so aggressive that shared carrier-grade NAT IPs are the only thing that gets through.

For the full breakdown of types, rotation, and setup, see our main guide to proxies for scraping.

Rotating vs sticky sessions

Rotation strategy affects perceived speed too. Rotating (a new IP per request) maximizes anonymity but adds connection-setup overhead each time. Sticky (the same IP for several minutes) is faster for multi-step flows — logins, pagination, carts — because the connection and session persist. Test both: in our experience, sticky sessions run noticeably faster per request but at a modestly lower success rate on the hardest targets, so match the mode to the job.

What speed is "good enough"?

There's no universal number, but practical guidance:

• Overhead under ~1 second per request is excellent for residential; datacenter can be a few hundred milliseconds.
• Success rate above ~95% on your actual target matters far more than shaving 200ms off latency.
• Concurrency is where the real gains are: a proxy that adds a second of latency but sustains 100 parallel requests will out-scrape a 200ms proxy limited to 5.

Don't chase the fastest single-request number. Optimize the product of concurrency and success rate; that's what fills your database by end of day.

Common mistakes when benchmarking proxies

• Testing against the wrong URL. Latency to ipinfo.io says nothing about your success rate on a heavily protected retailer. Test both a neutral endpoint (for raw speed) and your real target (for success).
• Ignoring p95 and only reading the average. One slow tail request per page can dominate a large crawl.
• Blaming the proxy for target-side timeouts. Separate connection errors (proxy) from timeouts and redirect loops (origin).
• Testing sequentially, deploying concurrently. Always benchmark at the concurrency you'll actually run.
• Testing once. Pools vary by time of day and geography; run the test repeatedly and across regions before you trust the numbers.

When to stop tuning proxies and offload it

Benchmarking and rotating proxies is worthwhile, but for large or ongoing jobs the proxy layer becomes a maintenance treadmill — pools degrade, targets adapt, and success rates drift. If you'd rather consume clean data than babysit IP pools, scraping.pro runs the full stack — proxies, rotation, captcha solving, and retries — as a managed web scraping service, with delivery as a data-as-a-service feed. You get the results without owning the infrastructure.

The bottom line

The best proxy speed for scraping is the one that maximizes effective throughput: low latency, high concurrency, and a high success rate on your specific targets — in that combined sense, not latency alone. Benchmark every provider with a script that measures all three, separates proxy faults from target faults, reports p95 as well as the average, and tests under real concurrency. Then pick the cheapest proxy type that clears your success-rate bar: datacenter or ISP for tolerant sites, residential or mobile when the target fights back.