Multi-Party Relays (MPRs) such as INVISV Relay (Android) and Apple’s iCloud Private Relay (iOS) provide practical privacy for connecting to the Internet. In this post we detail how we created INVISV Relay.
INVISV Relay: Implementation
To ensure that all traffic from the phone uses Relay, the INVISV app uses functionality that was originally designed in Android for VPNs – all this does is route all data packets into the app, which then becomes responsible for getting those packets out to the Internet in a privacy-preserving manner. When INVISV Relay is first turned on, it establishes a TLS-encrypted tunnel from your phone through an INVISV server (by default the one that is likely to be fastest) to the Fastly server nearest to it. From there, your network traffic (itself encrypted end-to-end by your browser or other app) is sent to Fastly via TLS-encrypted HTTPS tunnels using the IETF MASQUE specification. But doing this isn’t straightforward, because to make INVISV Relay general-purpose, we wanted to avoid requiring app developers to change their apps to use INVISV Relay and/or only support a select few apps.
Packet streams and Byte streams
So, how can we put TLS-encrypted traffic, which is in the form of TCP/IP packets that are about to exit the phone, into more TLS tunnels? There’s a fundamental mismatch here from a networking perspective: Android apps encrypt their data using TLS and hand that encrypted data to the Android Linux kernel via a TCP socket. The kernel then turns those bytes into distinct packets of encrypted data and routes those packets to the Relay app, but we then need to put those packets into HTTPS tunnels, which are byte-stream oriented not packet-oriented. (While there is an early draft for supporting something called CONNECT-IP in MASQUE, it is not yet fully baked or supported in partner networks or server implementations.)
The way to do this translation from packet stream to byte stream involves a classic technique: to run a TCP/IP stack in userspace (of which there are many, such as gVisor and picoTCP). Traffic comes into the Relay userspace TCP/IP stack from the Android Linux kernel as packets and then leaves as byte streams, but no decryption or data manipulation takes place (and it can’t, because the data is all in TLS streams).
This conversion from packet stream to byte stream is key to being able to support unmodified apps, because from an app’s standpoint, it’s sending a TLS connection straight to the destination server with end-to-end encryption. INVISV Relay ensures that application data is routed through the intermediate TLS-encrypted MASQUE layers to protect user data and metadata. INVISV servers only see packets from some IP address; we log nothing about the traffic we see, but unlike a VPN we have nothing meaningful to log in the first place as we have no visibility into the traffic itself – this is the strength of the MPR architecture. The only information we store is aggregate load, which is coarse-grained statistics about the rate of traffic (Mbps on the network interface) on our servers on several minute intervals – to make sure that we have capacity when a server is heavily loaded.
Benchmarking INVISV Relay
Just to provide anecdotal information about the performance of Relay, we thought it might be good to do some basic performance testing. This is simply to get a sense of what the performance is like, and is not a comprehensive benchmark. Tests in different places with different devices will of course yield different results, and we’d love to hear about how the service performs so we can continue to improve it.
Testing Setup
We used home (Cable Internet) broadband in Southern California as the setting for the test, subscribed to a plan with 500 Mbps stated downstream bandwidth. We used an ordinary 802.11ac WiFi access point. We used a Samsung S22 (Android 12). We tested in the same room as the WiFi router. We also used a laptop running Firefox to establish a baseline. For bandwidth testing we used Ookla Speedtest because it is regarded as a canonical speedtest among Internet measurement experts.
Baseline Speedtest
We established a baseline reading of 533 Mbps using the laptop on speedtest.net, confirming the available downstream bandwidth claimed by the broadband provider.
INVISV Relay Speedtest
Next we opened speedtest.net on Android. Without INVISV Relay enabled, Ookla Speedtest (speedtest.net) reported 450 Mbps downstream in Chrome. With INVISV Relay enabled, speedtest.net reported 471 Mbps, 451 Mbps, and 484 Mbps successively. Each time it reported 29 ms latency and geolocated to a server in San Jose, California which is the location of the US-West INVISV Relay service. We ran multiple tests over the course of several days and continued to yield similar performance numbers and behavior.
Browsing
Web browsing in Chrome, Brave, and DuckDuckGo was smooth and behaved as normal.
Apps
INVISV Relay supported and provided privacy for many other apps we tested, including communications apps such as Signal and Matrix, streaming apps such as YouTube, Netflix, BBC, Amazon Prime, and ESPN, and a number of other common and uncommon apps. The few apps that didn’t work, such as classic email apps that directly connect using SMTP (rather than HTTPS) and apps that make connections over the local network, were blocked (e.g. due to the current INVISV/Fastly anti-spam policy) rather than leaking out. (Users can use the Relay app exception feature to allow specific apps to bypass Relay if needed.)
Summary
INVISV Relay is a simple architecture that provides practical privacy. We use Android VPN hooks to transport all device traffic over Relay, and we use a longstanding network technique to deal with the mismatch between packet and byte streams. In our small scale experiments, we’ve seen virtually no degradation in performance when Relay is protecting traffic.