https://techetio.com/tag/webscraping/Recent content in Webscraping on TechetioHugoen-USSun, 24 Dec 2023 20:41:08 +0000https://techetio.com/2023/12/24/a-seamless-authenticated-proxy-solution-for-selenium/Sun, 24 Dec 2023 20:41:08 +0000https://techetio.com/2023/12/24/a-seamless-authenticated-proxy-solution-for-selenium/<h2 id="forwarding-requests-to-an-authenticated-proxy-with-squid">Forwarding Requests To An Authenticated Proxy With Squid</h2> <p>In my web scraping adventures, I&rsquo;ve run into the problem of attempting to use an authenticated proxy with Selenium. Using a proxy is relatively easy to configure in most web browsers, but handling authentication with proxies in selenium is far less trivial. For instance in Chrome, the solution is to create an extension with a listener on the <code>onAuthRequired</code> event in order to know when to submit the credentials for the proxy. These hacks vary across web browsers as well.</p>