Python on Techetio

A Seamless Authenticated Proxy Solution For Selenium

Sun, 24 Dec 2023 20:41:08 +0000

Forwarding Requests To An Authenticated Proxy With Squid

In my web scraping adventures, I’ve run into the problem of attempting to use an authenticated proxy with Selenium. Using a proxy is relatively easy to configure in most web browsers, but handling authentication with proxies in selenium is far less trivial. For instance in Chrome, the solution is to create an extension with a listener on the onAuthRequired event in order to know when to submit the credentials for the proxy. These hacks vary across web browsers as well.

Calling Command Line Tools With The Python Subprocess Module

Sat, 09 Dec 2023 06:26:20 +0000

Being able to use and control external tools within a programming language is a valuable resource. Obviously its ideal to have libraries and modules for such things, but sometimes when you’re trying to move fast, it may be easier to interact with a command line tool directly than implement a library/module. I’ll be going over to use nmap using the subprocess module in Python.

Subprocess in Python

First we can import our subprocess module. This is built into the standard library in Python so it doesn’t require any external dependencies.

Sending OpenTelemetry Logs To Vector Using Python

Sat, 29 Apr 2023 01:25:21 +0000

In the world of software development, log management plays a crucial role in understanding the behavior of applications and detecting potential issues. Vector is an open source project currently maintained by DataDog that provides robust capability for collecting, transforming, and routing logs. OpenTelemetry is an observability framework that simplifies the process of collecting, processing, and exporting telemetry data, including logs, from within your application through the use of SDKs. In this article, we will explore a simple example of sending logs to Vector using the Python OpenTelemetry SDK.