Using Authenticated HTTP Proxies in Pyppeteer

Pyppeteer is a Python port of Puppeteer, providing a high-level API to control Chrome/Chromium. Below you’ll learn the most reliable method to route all browser traffic through HyperProxy’s authenticated HTTP proxy using a centralized config.py file.

Prerequisites

• Python 3.7+
• pip install pyppeteer
• Chromium (auto-downloaded) or Chrome in your PATH

Step 1: Centralize Proxy Credentials

Create config.py next to your scripts to store your proxy string in one place:

# config.py
# Single-line proxy: login:password@host:port
PROXY = "pzAQ34:[email protected]:10878"

Method: Browser Args + page.authenticate

The easiest and most reliable approach: pass the --proxy-server argument to launch()and call page.authenticate() before navigation.

import asyncio
from pyppeteer import launch
from config import PROXY

async def main():
    # Parse credentials and host:port
    creds, hostport = PROXY.split("@", 1)
    username, password = creds.split(":", 1)

    # Launch browser with proxy-server argument
    browser = await launch(
        headless=False,  # set to True for headless mode
        args=[f"--proxy-server=http://{hostport}"]
    )
    page = await browser.newPage()

    # Authenticate proxy connection
    await page.authenticate({
        'username': username,
        'password': password
    })

    # Navigate to verify proxy
    await page.goto('https://httpbin.org/ip')
    content = await page.content()
    print(content)  # Should display proxy IP JSON

    # Take a screenshot to confirm
    await page.screenshot({'path': 'pyppeteer_basic_proxy.png'})
    await browser.close()

if __name__ == '__main__':
    asyncio.get_event_loop().run_until_complete(main())

Best Practices & Tips

• Rotate proxies by instantiating separate contexts for each proxy.
• Use headless mode (headless=True) for CI/CD pipelines.
• Set timeouts via page.set_default_timeout() to handle slow proxies.
• Close contexts and browsers to free resources.

Conclusion

This guide demonstrates how to configure authenticated HTTP proxies in Pyppeteer using a single config.pystring and the built-in page.authenticate() method. With these steps, you’ll have reliable proxy support for web scraping, testing, and automation tasks.