Tweaking Starlink - PART II

By Bl@ckbird on maandag 22 november 2021 10:01 - Comments are closed
Category: Networking, Views: 3.630

In my previous blogpost, I covered the pretty devastating effects of high latency on internet throughput. Please read this blogpost first before continuing…

When you want to increase your internet speed, you have basically 3 options:
  • Optimize your existing internet connection.
  • Bond multiple internet connections together.
  • Smart steering of your internet traffic along multiple internet connections.
You can read more of Option 1 in my previous blogpost. Options 2 & 3 are more advanced (and more expensive) compared to Option 1. Therefore, they are more suitable for businesses who want to increase their internet bandwidth in remote locations and have, in general, more money to spend compared to consumers.

Getting a fiber connection in a remote, rural area can easily cost tens of thousands of dollars. And getting a 100Mbps internet connection on a yacht at sea, can cost up to $ 900.000, - per year. Getting a Starlink internet connection (or any alternative) can be very beneficial.

Optimizing internet traffic for businesses:
Bond multiple internet connections together.


When you have multiple internet connections that are about the same type, you can bond them together. For example, two xDSL connections, two Starlink connections, two 4G or 5G connections, etc. You can have different internet providers, e.g., two different 4G providers, but the characteristics of these internet connections are the same.

If you are used working with Linux, you can install and configure MultiPath-TCP (MPTCP). It’s available on various platforms including Linux, OpenWRT, Android, Amazon EC2 instance, and Raspberry PI. MultiPath-TCP is commercially available through Tessares. But they only deliver solutions to large internet service providers.

Another solution is to get a Peplink router and connect it to the SpeedFusion Cloud. You connect the router to multiple internet connections and your internet traffic is send over this bonded connection.

SpeedFusion Cloud is a paid subscription that depends on how much data you use. When you use lots of data, it’s cheaper to run your own FusionHub Virtual Appliance. You can use FusionHub Virtual Appliance in a bigger distributed deployment. When you need only one hub to connect your router, you can use FusionHub Solo. It’s free to use, except for the costs of running the virtual appliance in the cloud.

Optimizing internet traffic for businesses:
Smart steering of your internet traffic along multiple internet connections.


Traditionally, high latency (inter)network connections could be optimized using WAN acceleration appliances. There are several vendors e.g. Silver Peak, Riverbed, Cisco WAAS or WANOS. (Were the last one is more affordable for small businesses.) These WAN acceleration appliances optimize network traffic using different techniques:
  • LZ Compression. (Just like WinZIP. :) )
  • Caching. (Store data on hard disks / SSD’s on both sides of the network connection.)
  • DRE Data deduplication. (Send only the changes made to the data across the network connection.)
  • Quality of Service / QoS. (Give business critical application a higher priority over other applications.)
  • TCP flow optimization. (Use TCP with a congestion control algorithm that is better suited for high latency network environments, e.g., by using TCP BBR.)


Unfortunately, almost all internet traffic is encrypted these days. Cloud services, but also Facebook or even YouTube use HTTPS, which is encrypted. Good for security, but encrypted traffic is random traffic and cannot be optimized. Only QoS and TCP flow optimization can be used. Most WAN acceleration vendors therefore now moved to SD-WAN, as it’s a better solution for encrypted traffic. QoS and TCP flow optimization are often combined with SD-WAN.

Software Defined - Wide Area Network (SD-WAN)

When you have different internet connections available, you can make use of the different characteristics of these internet connections.

With SD-WAN, two or more VPNs are configured over two or more internet connections. These internet connections can be anything: xDSL, 4G/5G, Starlink, broadband cable, Fiber-to-the-Home, wireless ISP, etc. Traffic characteristics of these VPNs are real-time measured: latency, jitter, and packet loss.



When internet provider A fails, internet traffic will be automatically sent over internet connection B. This is a hard A or B configuration. You can also configure it, so if the quality of internet provider A decreases (latency, jitter, or packet loss becomes too high), internet traffic will be sent over internet connection B.

Starlink can be used as a primary internet connection, but it’s also great as a secondary (backup) internet connection for businesses. It’s completely separated from other networking infrastructure: When you provide power to a Starlink dish (e.g., through a backup UPS or solar arrays), you have internet access. Complete separated from anything else; xDSL, 4G/5G, Fiber-to-the-Home, etc.

Most SD-WAN solutions are application aware. You can configure it, so the best path / internet connection is used for a particular application. For example: Gaming, voice and video conferencing use a VDSL2 internet connection, because of low latency, jitter, or packet loss. Other traffic uses a Starlink connection because more bandwidth is available.

The overall best solution depends on what type of internet connections there are locally available.

There are many SD-WAN vendors available. One of the easiest SD-WAN solutions to configure is Meraki SD-WAN. Unfortunately, it does not support natively TCP flow optimization. Cisco SD-WAN (formally known as Viptela) does support TCP flow optimization, but it’s a more complex solution.

Meraki now supports SD-Internet. This is basically the same traffic steering concept as SD-WAN, but without the VPN part. In the future, SD-Internet will also support L7 application recognition.

If you have any questions, leave a comment, or ping me a message. (Pun intended :) )

Tweaking Starlink - PART I

By Bl@ckbird on maandag 22 november 2021 10:00 - Comments (5)
Category: Networking, Views: 3.937

Optimizing Starlink for Consumers
When you want to increase your internet speed, you have basically 3 options:
  • Optimize your existing internet connection.
  • Bond multiple internet connections together.
  • Smart steering of your internet traffic along multiple internet connections.
Options 2 & 3 are more suitable for businesses and I’ll cover them in my next blogpost. I take Starlink as an example, but most techniques and ideas I mention here can also be used with other (high latency) internet and WAN connections.

What is a good internet connection?
The quality of an internet connection can be defined by:
  • Throughput (How many IP packets can you send/receive per seconde?)
  • Latency (How long take IP packets to complete their journey?)
  • Jitter (How much variation is there in latency?)
  • Packet-loss (How many packets do not reach their destination and are lost in transit?)
Most gaming, voice and video applications use UDP packets. UDP uses a simple, connectionless communication model. UDP packets are just send along the way. When packets are dropped, you may lose a video-frame, but that’s OK: We humans don’t notice that.

When gaming, a high latency internet connection can make the experience a bit sluggish. Unfortunately, UDP traffic can’t be optimized much, as it is an efficient protocol.

Latency of Starlink is about 40 to 50ms, with peaks of up to 100ms. Packet-loss is between 0 and 5%, with peaks of up to 10%

For comparison: When you use traditional satellite communication, latency is about 600 to 850ms.
(A satellite in a geo-stationary orbit is 36.000km from the Earth’s surface. A roundtrip will take two times 2x 36.000km.) Latency and throughput of Starlink is quite good, :) but not compared to an average broadband connection.



Starlink uses radio signals to communicate with satellites at 550km above the Earth’s surface. Water blocks radio signals very well. You want to avoid any trees (that contain water), poles, buildings or other obstructions that blocks a clear view on the sky. In this video you can see how.

Latency & Packet loss vs. Throughput
Most applications don’t use UDP, but TCP. TCP is connection-oriented: A connection between client and server is established before data can be sent. Every few TCP packets needs to be acknowledged to make sure data has been successfully transferred.

This is OK when latency is low, but when latency is high, this has a significant impact on the maximum throughput you can get. Having only a few percent of packet-loss makes the situation even worse.

With the Mathis Equation, you can calculate how much throughput you can get from a network (internet) connection. Google the term gives more information on the topic, for example here.

Optimizing TCP
Though you can’t bend the laws of physics, you can reduce the effects of high latency on TCP traffic. (e.g. Slow download speeds.) To do this you can replace standard TCP (TCP Reno or CUBIC) by TCP BBR.

You can do this by:Another way is using Shadowsocks-libev Proxy Server on your VPS.

High latency reduces throughput of TCP traffic. But this is per TCP session. You can off course “stack” multiple TCP sessions on top of each other by using a download manager. (Who remembers Download Accelerator Plus? :) )

You can also enable multi-threaded downloads in Google Chrome, by going to:
chrome://flags/#enable-parallel-downloading
and enable it. This will of course only optimize your downloads; all other traffic will not be optimized.

Test Results
To test TCP BBR, I’ve configured a VPS with Wireguard and enabled TCP BBR. I used WANem as a WAN Simulator. With WANem you can introduce different levels of latency and packet-loss to your traffic. I’ve tested internet throughput under different scenarios with iPerf3:

Different Scenarios:
  • With and without VPN with TCP BBR enabled.
  • Single stream downloads / multi-stream downloads.
  • With 0 and 5% packet loss.
Different Latencies:
  • 0ms latency. (Just the normal internet latency, no additional latency introduced.)
  • 20ms latency. (Minimum latency of Starlink.)
  • 50ms latency. (Average latency of Starlink.)
  • 100ms latency. (Peak value latency of Starlink.)
  • 600ms latency. (Minimum latency of traditional satellite internet (geo-stationary orbit.))


Notes:
  • All tests were tested with an iPerf3 server running at iperf.par2.as49434.net
  • My internet connection is limited to 100/100 Mbps. (Fiber to the Home)
Performance Analysis
When you look at the average latency of Starlink, throughput increase will be between 30 and 100%. (by using TCP BBR) When you use multiple TCP streams (e.g. multiple download sessions) throughput will be even a little bit higher.

When you experience only 5% packet loss, using a VPN with TCP BBR will increase your throughput by 500 to 1700% !!

Some Graphs
Downloading Ubuntu Server 20.04.3 LTS:
  • Latency is 50ms.
  • Left: Without VPN.
  • Right: With VPN and TCP BBR enabled.
  • Latency is 50ms & 5% packet loss.
  • Left: Without VPN.
  • Right: With VPN and TCP BBR enabled.


TL;DR
Although I’m not a fan of public VPN services from a security point of view,
in my search for information I stumbled upon ProtonVPN referral

They use TCP BBR to optimize your TCP traffic on high latency internet connections. (Including Starlink.)
It’s the easiest way to optimize your Starlink connection, without having to setup your own VPN server.

When you want to test ProtonVPN, make sure you get a subscription, as the free servers are often oversubscribed. They use Wireguard VPN and performance increase should be about the same as mentioned in my test results.

In my next blogpost, I’ll cover optimizing Starlink for businesses.
If you have any questions, leave a comment, or ping me a message. (Pun intended :) )