Proxy Grabber

How the proxy grabber works

Brief: Asynchronous proxy grabber using aiohttp that fetches proxies from multiple public sources, validates, deduplicates, and saves them.

Public sources

Lists of URLs per protocol (http, https, socks4, socks5).

Global state

  • proxies – stores proxies per type in sets.
  • raw_stats – keeps counters: fetched, duplicates, filtered, final.

    • Local IPs and bad ports filtering

  • LOCAL_IPS regex patterns exclude private and reserved ranges.
  • BAD_PORTS set excludes common unsafe or restricted ports.

    • Functions

  • extract_proxies(text) – regex-based extraction of IP:port and optionally
    • user:pass.
  • is_valid(proxy) – validates proxy:
    • - Port > 80 and not in BAD_PORTS. - IP not in private/reserved ranges.
  • deduplicate(proxies_set) – removes duplicates based on IP:port only.
  • fetch_ptype(session, ptype, url) – asynchronously fetches proxies for a given
    • type from a URL, updates counters and sets, prints progress.
  • run_grabber() – main async function:
- Displays header. - Runs fetch tasks for all protocols and sources concurrently. - Deduplicates, filters, and saves proxies to proxies/{ptype}.txt. - Prints final statistics per protocol and total usable proxies.

Key features

High concurrency, multi-source fetching, IP and port filtering, deduplication, organized output per protocol, and detailed stats.

..