How to Use HTTP Proxies with Authentication in Selenium
Struggling with proxy with auth Selenium? This comprehensive guide covers two foolproof methods:Selenium Wire proxies with auth and a Manifest V3 Chrome extension. Get your HTTP proxies with authworking in Selenium in minutes.
Prerequisites for Proxy Authentication in Selenium
- Python 3.7+
pip install selenium blinker==1.7.0 selenium-wire webdriver-manager- Chrome browser + ChromeDriver
Step 1: Define Your HTTP Proxy String
Store your proxy credentials in one variable for easy reuse:
config.py
# config.py - proxy with auth Selenium
PROXY = "pzAQ34:[email protected]:10878"Method 1: Selenium Wire Proxies with Auth
Selenium Wire proxies with auth simplifies proxy authentication in both Chrome and Firefox. It automatically injects your credentials.
selenium_wire_proxy.py
from seleniumwire import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from webdriver_manager.chrome import ChromeDriverManager
import time
from config import PROXY
# Build the HTTP proxy URL
proxy_url = f"http://{PROXY}"
# Selenium Wire options for HTTP proxies with auth
driver_options = {
"proxy": {
"http": proxy_url,
"https": proxy_url,
"no_proxy": "localhost,127.0.0.1"
}
}
options = Options()
options.add_argument("--ignore-certificate-errors")
# Initialize webdriver with proxy auth support
driver = webdriver.Chrome(
service=Service(ChromeDriverManager().install()),
options=options,
seleniumwire_options=driver_options
)
driver.get("https://httpbin.org/ip")
print(driver.page_source) # Verify HTTP proxies with auth via Selenium
time.sleep(5)
driver.quit()Method 2: Manifest V3 Chrome Extension for Proxy Auth
Use a lightweight HTTP proxy with auth Selenium extension for full Chrome feature support (incognito, non-headless). Generates a Manifest V3 extension that handles onAuthRequired.
chrome_proxy_extension.py
import os
import json
import zipfile
import time
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
from config import PROXY
# Generate Manifest V3 Chrome extension for Selenium proxy auth
def create_proxy_extension_v3(proxy):
creds, hostport = proxy.split("@")
user, password = creds.split(":")
host, port = hostport.split(":")
manifest = {
"name": "Selenium Proxy Auth",
"version": "1.0",
"manifest_version": 3,
"permissions": ["proxy", "webRequest", "webRequestAuthProvider", "tabs", "storage"],
"host_permissions": ["<all_urls>"],
"background": {"service_worker": "background.js"},
"minimum_chrome_version": "108"
}
background_js = f'''chrome.proxy.settings.set({{
value: {{
mode: "fixed_servers",
rules: {{ singleProxy: {{ scheme: "http", host: "{host}", port: {port} }}, bypassList: ["localhost"] }}
}}, scope: "regular"
}}, () => {{}});
chrome.webRequest.onAuthRequired.addListener(
details => ({{ authCredentials: {{ username: "{user}", password: "{password}" }} }}),
{{ urls: ["<all_urls>"] }}, ["blocking"]
);'''
ext_dir = "proxy_ext_v3"
os.makedirs(ext_dir, exist_ok=True)
with open(os.path.join(ext_dir, "manifest.json"), "w") as m:
json.dump(manifest, m, indent=2)
with open(os.path.join(ext_dir, "background.js"), "w") as b:
b.write(background_js)
return ext_dir
# Build & load the extension
ext_folder = create_proxy_extension_v3(PROXY)
options = Options()
options.add_argument(f"--load-extension={os.path.abspath(ext_folder)}")
# Initialize WebDriver with the proxy extension
driver = webdriver.Chrome(
service=Service(ChromeDriverManager().install()),
options=options
)
driver.get("https://httpbin.org/ip")
print(driver.page_source) # Check proxy with auth in Selenium
time.sleep(5)
driver.quit()Best Practices & Tips
- Use proxy rotation: Avoid IP bans by cycling through a pool of authenticated proxies. Create a new WebDriver session per proxy to prevent reuse issues.
- Close drivers properly: Always call
driver.quit()to release system resources and kill orphaned processes. - Avoid headless detection: If using headless mode, add anti-detection flags like
--disable-blink-features=AutomationControlledand modify navigator variables. - Set explicit timeouts: Configure
driver.set_page_load_timeout()anddriver.implicitly_wait()to handle slow proxy responses without hanging the script. - Validate proxy IP: Use services like httpbin.org/ip or api.ipify.org to confirm the proxy is active.
- Use error handling: Wrap critical Selenium logic in
try/exceptblocks to catch proxy failures, timeouts, or authentication errors and retry as needed. - Keep ChromeDriver & Chrome in sync: Mismatches between Chrome and ChromeDriver versions are common causes of startup failures.
- Disable image loading for performance: Save bandwidth and load pages faster by disabling media using Chrome preferences when proxy bandwidth is limited.
Conclusion
Master proxy with auth Selenium with these two battle-tested methods. Whether you need quick setup via Selenium Wire proxies with auth or full-featured support with a Manifest V3 extension, you're covered.
