Other A proxy to fix HTTPS issues on old versions of OS X

[Nothistime]

it doesn't even installs in Ventura, it says I can't install it on the disk

 

[Wowfunhappy]

Yes, I set a maximum of macOS 10.13 (iirc) in the installer, because the included Squid binary isn't compiled for modern OSs.

This package is designed to run a proxy on the same machine where you're having https issues—ie, you'd install the package on a Mac running Mavericks, to fix https issues when using Mavericks.

Ventura should not have any issues with https, so I assume you're trying to run the proxy on your Ventura Mac to fix issues with a different Mac. If that Mac is running 10.6 or newer, please just install the proxy there.

If you're trying to fix https on a Mac running 10.5 or below, you will indeed need to run the proxy on a separate, newer machine. While several people have done this successfully, I'm not aware of an automatic way to set it up, as everyone's network is a little different. However, the files included in my package (particularly squid.conf) may be a useful reference. Use pkgutil --expand "/path/to/Legacy Mac Proxy.pkg" /path/to/output/folder to take a look inside.

 

[f54da]

Squid seems like overkill for this, in terms of binary size and resource footprint. Seems like the only part you need is an https proxy (to be precise a proxy that supports http connect) that can intercept/mitm traffic (thus splitting the single tls negotiation into two, one between the client and proxy, and another between proxy and remote server).

Any of

mitm/mitm.go at master · kr/mitm

Contribute to kr/mitm development by creating an account on GitHub.

github.com

GitHub - AdguardTeam/gomitmproxy: Simple golang mitm proxy implementation

Simple golang mitm proxy implementation. Contribute to AdguardTeam/gomitmproxy development by creating an account on GitHub.

github.com

Golang Http(s) mitm Proxy

Golang Http(s) mitm Proxy. GitHub Gist: instantly share code, notes, and snippets.

gist.github.com

should work, all very simple and readable < 500 loc implementation of this, thanks to Golang's robust networking libraries.

And as a bonus this should work on any machine (< 10.5 or > 10.11) if you pair it the legacy support dylib.

 

[Wowfunhappy]

@f54da Thanks as always! I'm not sure when I'll next have time to experiment (I've been so swamped with life stuff) but I definitely will when I can! Do you happen to know whether these will automatically fetch intermediate certificates as Squid 4 does?

 

[f54da]

@Wowfunhappy I don't follow, why is that needed? Since you're MITM tls traffic, the original certificate doesn't matter, since your proxy is going to sign and present its own certificate for the domain anyway. So what's the benefit of fetching intermediate certs?

 

[Wowfunhappy]

So what's the benefit of fetching intermediate certs?

A self-signed certificate is used between the browser† and the proxy. However, the proxy still uses "official" certificates to communicate with the website. The proxy is responsible for ensuring that that certificate chain is valid. (I do want it to actually verify the certificates, not merely accept everything.)

Does that make sense? It's possible I don't know what I'm talking about.

---

† Hopefully no one is using this proxy with a literal web browser, but it's easier to say "browser" here, and I do use Safari (in a VM) for testing.

 

[f54da]

>proxy is responsible for ensuring that that certificate chain is valid
Ah I see, you're right, I didn't realize that servers can actually serve an incomplete cert chain and force clients to fetch the missing ones. Seems this is codified via AIA extension for X.509, and this process of fetching intermediate certs is known as "AIA Chasing".

Is this really used that heavily though? But either way, I think it can be dealt with easily in 2 ways:


1) Just go ahead and prebake all intermediate certs. This approach is suggested by Filippo himself, and he offers a go package that takes care of this. Although the downside is that in terms of long-term solution, eventually the prebaked list in your binary will be out of date and you'll have to rebuild it.

As part of this effort, Mozilla requires disclosure of all intermediate CA certi... | Hacker News

news.ycombinator.com

2) Use a separate package to manually do the AIA lookups. I see

GitHub - fcjr/aia-transport-go: AIA (Authority Information Access) Transport Go provides a pre-configured http.Transport to enable AIA certificate chain resolution for resolving urls with incomplete certificate chains.

AIA (Authority Information Access) Transport Go provides a pre-configured http.Transport to enable AIA certificate chain resolution for resolving urls with incomplete certificate chains. - fcjr/aia...

github.com

which was linked from the discussion in https://github.com/golang/go/issues/31773 (see it for more info on why this was not added to the go standard library). It's really only 2 extra functions to do the lookups, so you don't even need to do any packaging nonsense, just copy paste them in.

 

[startergo]

If you're using OS X 10.6 (Snow Leopard), 10.7 (Lion), 10.8 (Mountain Lion), or 10.9 (Mavericks), I encourage you to give it a try

In SL Interweb stops working. Any workaround?
Edit: Never mind. The version on archives.org does not work, but the latest one works.

 

[Wowfunhappy]

In SL Interweb stops working. Any workaround?

InterWeb is a Firefox-based browser. Make sure you follow the instructions in the readme to set the browser to bypass the proxy.

 

[startergo]

InterWeb is a Firefox-based browser. Make sure you follow the instructions in the readme to set the browser to bypass the proxy.

It is interesting that updating the root certificates with latest Mozilla certificates:

• Update your System Root certificates. Download Latest Mozilla Certificates:

Spoiler: Download Latest Mozilla Certificates

curl --etag-compare etag.txt --etag-save etag.txt --remote-name https://curl.se/ca/cacert.pem

• Save this script as trustroot.sh:

#!/bin/bash
DIR=${TMPDIR}/trustroot.$$
mkdir -p ${DIR}
trap "rm -rf ${DIR}" EXIT
cat "$1" | (cd $DIR && /usr/bin/split -p '-----BEGIN CERTIFICATE-----' - cert- )
for c in ${DIR}/cert-* ; do
security -v add-trusted-cert -d -r trustRoot -k "/Library/Keychains/System.keychain" "$c"
done
rm -rf ${DIR}

• Issue this command:

chmod 755 trustroot.sh
sudo ./trustroot.sh /path/to/cacert.pem

Where the /path/to/ should be replaced by your actual path. The certificates will then operate as trusted roots in addition to the certificates in the original System Roots keychain. This will eliminate the certificate expired error issue in Safari.

works on Mavericks onward but it does not work on earlier operating systems. This method has less impact on the CPU in comparison to the squid server method, but unfortunately it is not universal. Any Idea why does it not work on all operating systems?

 

[f54da]

Root certificates vs reverse proxy are for two orthogonal issues.

Root certificates purely fixes TLS chain verification issue, due to expired roots or new ones (e.g. ISRG X1). Firefox (and probably firefox derived browsers) actually uses its own cert store anyway.

But even with proper certificates, many servers will only do a TLS 1.2 handshake, while Apple's version of secureTransport in 10.9 that's used by all cocoa apps can only do TLS 1.1 handshake.

Btw you don't need to run the squid server locally on your machine, you can always run it on an external raspberry pi or something if you're concerned about CPU usage.

It's also been an idea floated in the past to try to solve this without a reverse proxy by modifying libSecurity framework itself. I think wowfunhappy tried compiling from scratch but it didn't work, probably because OSS version is missing requiring things. Maybe a better approach would be to try to shim your own by interposing relevant symbols. Yet another option is to try hooking things one level higher at the cocoa layer by swizzling NSURLConnection and related methods.

 

[Wowfunhappy]

But even with proper certificates, many servers will only do a TLS 1.2 handshake, while Apple's version of secureTransport in 10.9 that's used by all cocoa apps can only do TLS 1.1 handshake.

Tiny correction—Mac OS X 10.9 Mavericks actually supports TLS 1.2 (it was the first version of Mac OS to do so). What's missing aiui are modern cipher suites.

----------

Squid really shouldn't be using a noticeable amount of CPU, by the way. On my machine, Squid uses 0.01% CPU (with occasional spikes up to 0.10%) when I use curl to download a large file. I have a pretty fast CPU (Intel i7-4790K @ 4ghz), but I don't think this should be a big deal on pretty much any Intel Mac?

(I would like to try those Go proxies at some point, but "at some point" may be a very long time from now.)

 

[Xde]

Btw you don't need to run the squid server locally on your machine, you can always run it on an external raspberry pi or something if you're concerned about CPU usage.

I was thinking about an Raspberry as an option, but is there any easy to follow information available in the net?

Aside:
In mid dec my email provider announced, that they will support only TLS 1.2/1.3 in new year.
And so it was. Cannot use Mail with 10.9.5 anymore.

I have no knowledge about SSL/TSL and certificates.
I read some about OpenSSL which is Vers 0.98 in Mavericks,
and downloaded Vers 3.07 from openssl.org onto a test system, config and install and was able to get SSL Vers 3.07.

Anyway, Mail does still not work, and I dont know if this update could have any advantage for us ?

 

[Wowfunhappy]

Anyway, Mail does still not work, and I dont know if this update could have any advantage for us ?

There is a decent possibility the proxy will allow you to connect to your email provider in Apple Mail.

However I can't be sure because:

• I've long been a bit confused about how some of the old email protocols (POP/IMAP) are related (or unrelated) to HTTP. But if it's using SSL, then it must be HTTP...?
• Clearly, your email provider isn't giving you the full story as to what changed. Mavericks natively supports TLS 1.2, so if your provider merely dropped support for TLS 1.1, Mavericks wouldn't care. If Mavericks broke because your provider also dropped support for old cipher suites, the proxy will fix that. If Mavericks broke because your provider enabled some other esoteric hardening setting deep in Microsoft Exchange, then it's a crapshoot.

 

[f54da]

You can have TLS without needing to use HTTP at the application layer.

 

[Xde]

There is a decent possibility the proxy will allow you to connect to your email provider in Apple Mail.

I just tried it, but did not work.
(Aside of Squid, the two extra checkboxes I left unchecked)

Anyway thanks for your comment

 

[f54da]

squid is an http proxy only I think (just by skimming docs, but may be wrong), I don't know if it will terminate X over TLS for any application transport layer X. It should surely be possible to setup something that will work for you, but you'd have to figure out what exactly your email provider supports and cobble together something on your own from primitives (I suggest golang only because it seems to have decent networking libraries built in that I can glue together blindly, but even python would probably work if you pull in the right stuff)

 

[maverick28]

The Dictionary app fails to connect to Wikipedia again.

Is this bug tracker helpful?
Wikimedia Phabricator

 

[Wowfunhappy]

The Dictionary app fails to connect to Wikipedia again.

It's working for me.

 

[maverick28]

Not for me.

Could you share the output of the Info window on the process in Activity Monitor on your Mac?
I get the following entries:

/
/Applications/Dictionary.app/Contents/MacOS/Dictionary
/Applications/Dictionary.app/Contents/Frameworks/ProxyFix.dylib
/usr/share/icu/icudt51l.dat
/System/Library/ColorSync/Profiles/Generic Gray Gamma 2.2 Profile.icc
/System/Library/ColorSync/Profiles/Generic Gray Profile.icc
/System/Library/ColorSync/Profiles/sRGB Profile.icc
/System/Library/ColorSync/Profiles/Generic RGB Profile.icc
/System/Library/Keyboard Layouts/AppleKeyboardLayouts.bundle/Contents/Resources/AppleKeyboardLayouts-L.dat
/private/var/folders/pr/34n7_cwx0_1bz_k7xcy1hjyc0000gn/C/com.apple.IntlDataCache.le.kbdx
/private/var/db/mds/messages/se_SecurityMessages
/Users/home/Library/Caches/com.apple.Dictionary/Cache.db-shm
/private/var/folders/pr/34n7_cwx0_1bz_k7xcy1hjyc0000gn/C/com.apple.LaunchServices-044501.csstore
/System/Library/PrivateFrameworks/CoreUI.framework/Versions/A/Resources/SArtFile.bin
/System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/HIToolbox.framework/Versions/A/Resources/Extras2.rsrc
/System/Library/Fonts/LucidaGrande.ttc
/System/Library/Fonts/Helvetica.dfont
/System/Library/PrivateFrameworks/CoreUI.framework/Versions/A/Resources/ArtFile.bin
/System/Library/Frameworks/OpenCL.framework/Versions/A/Libraries/ImageFormats/unorm8_bgra.dylib
/System/Library/Frameworks/CoreFoundation.framework/Versions/A/Resources/tokruleLE.data
/System/Library/Frameworks/AppKit.framework/Versions/C/Resources/Latn1.data
/System/Library/Frameworks/AppKit.framework/Versions/C/Resources/Latn2.data
/System/Library/Fonts/Keyboard.ttf
/System/Library/PrivateFrameworks/DataDetectorsCore.framework/Versions/A/Resources/com.apple.datadetectorscore.cache.urlifier.system
/System/Library/ColorSync/Profiles/Generic CMYK Profile.icc
/Library/Fonts/Baskerville.ttc
/usr/share/langid/langid.inv
/usr/lib/dyld
/private/var/db/dyld/dyld_shared_cache_x86_64
/dev/null
/dev/null
/dev/null
count=2, state=0x2
/System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/HIToolbox.framework/Versions/A/Resources/Extras2.rsrc
/Users/home/Library/Caches/com.apple.Dictionary/Cache.db
/Users/home/Library/Caches/com.apple.Dictionary/Cache.db
/Users/home/Library/Caches/com.apple.Dictionary/Cache.db-wal
/Users/home/Library/Caches/com.apple.Dictionary/Cache.db-shm
/Users/home/Library/Caches/com.apple.Dictionary/Cache.db-wal
192.168.1.65:60194->a184-85-156-64.deploy.static.akamaitechnologies.com:http
192.168.1.65:60199->a184-85-156-64.deploy.static.akamaitechnologies.com:http
192.168.1.65:60194->a184-85-156-64.deploy.static.akamaitechnologies.com:http
->0x503269c805942aef
->0x503269c7ffcb5cdf
192.168.1.65:60195->a184-85-156-64.deploy.static.akamaitechnologies.com:http
192.168.1.65:60200->a184-85-156-64.deploy.static.akamaitechnologies.com:http
192.168.1.65:60195->a184-85-156-64.deploy.static.akamaitechnologies.com:http
192.168.1.65:60196->a184-85-156-64.deploy.static.akamaitechnologies.com:http
192.168.1.65:60196->a184-85-156-64.deploy.static.akamaitechnologies.com:http
192.168.1.65:60199->a184-85-156-64.deploy.static.akamaitechnologies.com:http
192.168.1.65:60200->a184-85-156-64.deploy.static.akamaitechnologies.com:http
192.168.1.65:60201->a184-85-156-64.deploy.static.akamaitechnologies.com:http
192.168.1.65:60201->a184-85-156-64.deploy.static.akamaitechnologies.com:http

Notice the connections made to the Akamai server. Is there everything OK? How could I check if the dylib was loaded?

 

[maverick28]

My feeling is that it stopped using the proxy. I can only remember 2 possible events the impact of which on the Dictionary's ability to fetch Wikipedia articles I'm not certain at all: I used lsregister on one occasion and flushed the DNS cache after editing the hosts file.

 

[maverick28]

It's now back to normal. I noticed the upload.wikimedia.org host among connected endpoints which was gone previously. Interestingly, it shows up both in Squid's and Dictionary's connection lists. In the former case, it uses HTTPS on port 443, and in the latter case, it connects to HTTP on port 80. Go figure.

 

[PowerHarryG4]

Should this proxy be able to fix git clone with https? I'm trying to clone a repo and it says "
SSL certificate problem: self-signed certificate in certificate chain"
Is this something i've messed up?

 

[Wowfunhappy]

Should this proxy be able to fix git clone with https?

Getting command line tools to respect the proxy and/or certificate is a bit hit-or-miss. Most of them work as long as you check the option to `Set Environment Variables` when installing the proxy, but not always.

You might consider telling git to to stop verifying SSL by running git config --global http.sslVerify false. This of course has security implications, but it probably isn't a major concern as long as you're cloning repositories on a trusted internet connection. You could also clone via SSH instead.

 

[PowerHarryG4]

Getting command line tools to respect the proxy and/or certificate is a bit hit-or-miss. Most of them work as long as you check the option to `Set Environment Variables` when installing the proxy, but not always.

You might consider telling git to to stop verifying SSL by running git config --global http.sslVerify false. This of course has security implications, but it probably isn't a major concern as long as you're cloning repositories on a trusted internet connection. You could also clone via SSH instead.

Ah yeah it’s for my work so probably not good to turn off the SSL. But you were right about the ssh, I got that to work thanks. Took me a while to find the right encryption method for a certificate that mavericks can do and GitHub also supports.