Setting the proxy configuration programatically for a variety of programs

Intro

For frustrating reasons beyond the scope of the article, I frequently need to update the proxy settings that my computer uses to connect to the internet. Unfortunately, lots of programs aren’t aware of the OS level proxy settings. Some of these programs pick up on commonly used shell environment variables like $HTTP_PROXY and $HTTPS_PROXY, which are easy to set programatically. A few applications have their own way of setting and storing proxy preferences, so they need special handling. Over the years, I’ve gradually automated the process of updating the proxy login and password for those programs which store the login info separately.

Sublime Text Package Control

For example, the Sublime Text package management plugin has a simple JSON file which stores the proxy login info. You can use the program jq to quickly update the relevant JSON value:

jq ".proxy_password = \"$NEW_PASSWORD\"" "$HOME/Library/Application Support/Sublime Text 3/Packages/User/Package Control.sublime-settings"

This one-liner will just update the proxy_password key to the new password. The full version of this script I use is a bit more robust. It will back up the current config and try to install jq if it isn’t already present. It will also make sure the expected JSON file actually exists, so that there won’t be any funny side effects if I’m on a machine without Sublime Text. This function lets you pass a JSON key, a new JSON value (aka the password), and the path to the JSON file as arguments:

function set_proxy_json --argument-names path key value
	if test -f $path
		if not test (which jq)
			echo "Need to install jq to update json proxy configs..."
			brew install jq			#todo: https://xkcd.com/1654/
		end
		if test (which jq) #make sure the installation worked
			cp "$path" "$path.bak"
			jq ".$key = \"$value\"" "$path.bak" > "$path"
		end
	end
end

A quick sidenote: all of these scripts are intended for the fish shell. As someone who isn’t a huge fan of bash, fish is a breath of fresh air. The syntax is a lot more sane and legible. Every time I type ‘esac’ in bash, I die a little bit inside.

Side-sidenote: The naming of bash keywords is doubly frustrating because it’s inconsistent. You end an ‘if’ statement by saying ‘fi’. You end a ‘case’ statement by saying ‘esac’… and you end a ‘for’ loop by ‘done’

Spotify

Updating the proxy for programs like Maven and Sublime Text is a relatively simple find-and-replace operation. Spotify is a bit more complicated. It doesn’t store the password in plaintext; it looks like they store some sort of hash or encrypted password. The good news is, if you replace the hashed password in the proxy file with the new one as-is, it will get hashed properly when you restart Spotify. Once you realize that, you can write a gnarly regular expression to update “network.proxy.pass” in the Spotify config file (on MacOS this is located at "$HOME/Library/Application Support/Spotify/prefs"). If you want to be able to toggle the proxy on or off, be aware that there’s another property called network.proxy.mode. By updating its value, you set the proxy type accordingly:

Here’s my gross regex. There’s probably a better way to do this, but this abomination works¹. The GNU and BSD versions of sed differ in subtle ways, but I think I’ve gotten this to work in a crossplatform fashion.

function set_proxy_file --argument-names path start finish pass
	if test -f $path
		if test ! -w "$path"
			echo "$path needs elevated permissions to edit"
			sudo sed -i.bak -E "s/($start)(.*)($finish)/\1$pass\3/g" $path
			return
		end
		# handling of the suffix to use with -i is one place where GNU and BSD implementations aren't the same
		# sed -i .bak -E "s/($start)(.+)($finish)/\1$pass\3/g" $path 
		sed -i.bak -E "s/($start)(.*)($finish)/\1$pass\3/g" $path
		####### 	 1  2  3      4/5  6        7        8  9
		# 1. Format as extended/modern regex
		# 2. Substitution command
		# 3. Capture the prefix (whatever text we expect before the password, e.g. JSON key, open XML tag)
		# 4. Keep some amount of gibberish (lets us leave in whitespace or other formatting)
		# 5. Read the currently entered password
		# 6. Exfiltrate any suffix (e.g. closing xml tag)
		# 7. Glom together any prefix/beginning junk, the new password, and the suffix
		# 8. Enable global substitution (e.g. for every proxy listed in file, not just first)
		# 9. Examine this file for patterns to stream edit
	end
end

You can call the function like this to enable the Spotify proxy and set your password:

set SPOTIFY "$HOME/Library/Application Support/Spotify/prefs"
set_proxy_file "$SPOTIFY" 'network.proxy.mode=' '.*' '2' 	# 2 is HTTP: [auto-detect, no-proxy, http, socks4, socks5]
set_proxy_file "$SPOTIFY" 'network.proxy.pass=\"' '.*' "$PASSWORD\""

IntelliJ IDEA

If I’m the Captain Ahab of Proxies, then IntelliJ was my White Whale. It was easy to find where the file containing the proxy data was stored (on my Mac and with my version of IntelliJ it was at “~/Library/Preferences/IntelliJIdea2019.3/options/proxy.settings.pwd”). Unfortunately, this file looks like complete gibberish. Making even a one letter change to my username or password wouldn’t just flip a few bits, it would change almost the entire file (except the first 4 bytes, which seemed to be a number that was always 20 less than the file’s size). After spending some time trying to coax out the proxy file’s secrets, I learned that much of IntelliJ’s codebase is open source. After digging through their repo, I was finally able to figure out how to make this file on my own with an arbitrary username and password.

To make a long story short: proxy.settings.pwd is an encrypted Java properties file containing the username and password. The first 4 bytes of the file are the size of the encrypted payload (note that it’s PKCS#5 padded, so the encrypted size will be the next multiple of 16 that is greater than or equal to the plaintext’s size) The next 16 bytes are the IV (initialization vector, sometimes called a nonce) that was used to generate the ciphertext. Then comes the payload itself, which is encrypted using AES-CBC. The key used for encryption is just the string “Proxy Config Sec”. Super secure, right? In their source, it’s obfuscated in hexadecimal form) I’m not really sure what the point is of this extra security when the key is hardcoded.

This is the necessary format for the plaintext:

proxy.login=<username goes here>
proxy.password=<password goes here>

When you update this file by setting your proxy username and password via the IntelliJ GUI there’s some extra comments that get added in, but they’re unnecessary— we only need these 2 parameters.

At a high level, here’s what we need to do:

1. Generate a string using the properties file above, substituting in our actual username and password
2. Encrypt it using AES128 in cipher block chaining mode. The key is “Proxy Config Sec”, and since we’re generating the file we can use whatever initialization vector we want
3. Write the encrypted text’s size to a file (as a 4 byte integer)
4. Write the IV to the file (16 bytes)
5. Write the ciphertext to the file

Once we know these things, setting the proxy programatically is relatively straightforward. OpenSSL does all the cryptographic heavy lifting for us. In fact, the trickiest part for me was figuring out how to set a variable in the fish shell when its value contains a newline (the fish “set” command tries to create an array when it sees a newline). Because it made some aspects of the code simpler, I also calculated the ciphertext’s size before I actually generated it.

function set_proxy_intellij --argument-names user pass
	set path "$HOME/Library/Preferences/IntelliJIdea2019.3/options/proxy.settings.pwd"
	# we can't set the value of props normally, because when fish sees a newline
	# in the text it splits the string into an array. read -z circumvents that
	
	printf "proxy.login=$user\nproxy.password=$pass" | read -z --local props #store the text of our properties to $props
	set -l num_bytes (echo -n "$props" | wc -c | string trim)				 # figure out plaintext size
	# That is raw byte size, however PKCS#5 padding will add bytes until total length is a multiple of 16
	set -l num_bytes (math "16 * ceil($num_bytes / 16)")
	set -l num_bytes_hex (printf '%x\n' "$num_bytes")

	#length is encoded as 4 bytes, right now we only support the least significant byte of length
	echo -n -e "\x00\x00\x00\x$num_bytes_hex" > $path # use > to overwrite existing
	# Since there's no real security being provided by this encryption, it doesn't matter if we use a "bad" initialization vector
	# Let's just 16 bytes of 0, and write that to our file
	echo -ne "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" >> $path # use >> to append
	# Then feed the plaintext into the OpenSSL command.
	# Our IV from above is 0, and we can just convert "Proxy Config Sec" to hexadecimal and use that as our key
	echo "$props" | openssl enc -aes-128-cbc -iv 0 -K (echo -n "Proxy Config Sec" | xxd -pu) >> $path
end