test from my network running customized openwrt w/ cake tar file here: rtt_fair4be-2022-04-17T124830.462924.dss-cake.f... root@DESKTOP-DLUP89P:~# flent -x --socket-stats --step-size=.05 -H de.starlink.taht.net -H london.starlink.taht.net -H s ingapore.starlink.taht.net -H fremont.starlink.taht.net -t 'dss-cake' rtt_fair4be Started Flent 1.3.2 using Python 3.8.10. Starting rtt_fair4be test. Expected run time: 70 seconds. Data file written to ./rtt_fair4be-2022-04-17T124830.462924.dss-cake.flent.gz. Summary of rtt_fair4be test run from 2022-04-17 16:48:30.462924 Title: 'dss-cake' avg median # data pts Ping (ms) ICMP1 de.starlink.taht.net : 119.54 114.00 ms 1398 Ping (ms) ICMP2 london.starlink.taht.net : 107.97 104.00 ms 1398 Ping (ms) ICMP3 singapore.starlink.taht.net : 272.62 268.00 ms 1398 Ping (ms) ICMP4 fremont.starlink.taht.net : 89.15 84.60 ms 1398 Ping (ms) avg : 147.32 N/A ms 1398 TCP download BE1 de.starlink.taht.net : 25.01 24.81 Mbits/s 1398 TCP download BE2 london.starlink.taht.net : 23.00 22.75 Mbits/s 1398 TCP download BE3 singapore.starlink.taht.net : 20.26 21.01 Mbits/s 1398 TCP download BE4 fremont.starlink.taht.net : 35.64 32.94 Mbits/s 1398 TCP download avg : 25.98 N/A Mbits/s 1398 TCP download fairness : 0.95 N/A Mbits/s 1398 TCP download sum : 103.91 N/A Mbits/s 1398 TCP upload BE1 de.starlink.taht.net : 7.76 8.00 Mbits/s 1398 TCP upload BE1 de.starlink.taht.net::tcp_cwnd : 88.38 91.00 1067 TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate : 7.52 7.57 1064 TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate : 9.82 9.73 1066 TCP upload BE1 de.starlink.taht.net::tcp_rtt : 130.68 125.68 1062 TCP upload BE1 de.starlink.taht.net::tcp_rtt_var : 5.38 4.25 1062 TCP upload BE2 london.starlink.taht.net : 7.53 7.91 Mbits/s 1398 TCP upload BE2 london.starlink.taht.net::tcp_cwnd : 78.47 81.00 1059 TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate : 7.26 7.34 1056 TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate : 9.48 9.45 1057 TCP upload BE2 london.starlink.taht.net::tcp_rtt : 120.05 114.79 1056 TCP upload BE2 london.starlink.taht.net::tcp_rtt_var : 5.25 4.15 1056 TCP upload BE3 singapore.starlink.taht.net : 3.98 4.23 Mbits/s 1398 TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd : 100.96 105.00 1066 TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate : 3.95 4.10 1056 TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate : 5.07 5.15 1061 TCP upload BE3 singapore.starlink.taht.net::tcp_rtt : 291.42 286.12 1060 TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var : 7.62 5.64 1060 TCP upload BE4 fremont.starlink.taht.net : 8.41 8.56 Mbits/s 1398 TCP upload BE4 fremont.starlink.taht.net::tcp_cwnd : 75.40 73.00 1060 TCP upload BE4 fremont.starlink.taht.net::tcp_delivery_rate : 8.04 8.00 1057 TCP upload BE4 fremont.starlink.taht.net::tcp_pacing_rate : 10.62 10.31 1058 TCP upload BE4 fremont.starlink.taht.net::tcp_rtt : 103.74 98.50 1057 TCP upload BE4 fremont.starlink.taht.net::tcp_rtt_var : 4.97 3.96 1057 TCP upload avg : 6.92 N/A Mbits/s 1398 TCP upload fairness : 0.94 N/A Mbits/s 1398 TCP upload sum : 27.68 N/A Mbits/s 1398 root@DESKTOP-DLUP89P:~# On Sun, Apr 17, 2022 at 11:36 AM jordan--- via Bloat < bloat@lists.bufferbloat.net> wrote: > You're welcome! Here's the files Flent generated: > https://1drv.ms/u/s!Ap4u4Rte63FqjqoSQXX444OWLvl8FQ?e=f22Nt7 > > And thanks for the link! It honestly doesn't look too bad to implement so > I might give it a shot today if time allows. > > I've also been contemplating replacing it with something like a Mikrotik > since cake is now integrated into them. I've been testing one at home and > made a post with a bunch of flent tests to the cake thread in their forum > if you're interested in some data from a Charter Spectrum cable connection: > https://forum.mikrotik.com/viewtopic.php?t=179307#p925485 > > The OneDrive link to my flent files should still be active in that post as > well. > > -Jordan > > -----Original Message----- > From: Dave Taht > Sent: Sunday, April 17, 2022 10:57 AM > To: Jordan Szuch > Cc: flent-users ; bloat < > bloat@lists.bufferbloat.net>; starlink@lists.bufferbloat.net > Subject: Re: [Starlink] an rtt_fair test? > > Thank you for testing!!! If you could stash the *.flent.gz files somewhere > I could get at, that would be great. You can also browse the plots using > the flent-gui to see more detail over time. > > Cake, long available as an out of tree build for the edgerouter, has both > dsl compensation and an ack-filter which would help A LOT, in your case. i > wish I'd pushed harder to get dsl compensation into the edgerouter smart > queues code... > > > https://community.ui.com/questions/Cake-compiled-for-the-EdgeRouter-devices/fc1ff27c-f321-4344-8737-fcc755cae8a2 > > Given your 500kbit upload speed the minimum latency inflation under the > structure of this test (4 upload flows) should have been around 150ms (4 * > 25ms + ack overhead), but without looking at the *.gz files I don't know > what your baseline latency was to any of these sites. (it would be good if > flent reported on the text output what the minimum actually was) > > On Sun, Apr 17, 2022 at 7:35 AM wrote: > > > > Hi Dave, > > > > This isn't specifically cake related but I figured it could be > interesting, especially as these are from a location where I've been > investigating setting up something with cake. These are statistics from a > relatives location where they only have a DSL line from CenturyLink > available to them. There's a little Debian based VM I setup for a couple > random things so I was able to run flent from that. They have a EdgeRouter > X serving as the router so they can at least take advantage of fq_codel > through the routers Smart Queue feature. I ran a quick speed test, then ran > flent without sqm, and then with sqm. Sorry if this gets a bit long and > hopefully it stays readable. > > > > ############# > > Speedtest by Ookla > > > > Server: 123.Net, Inc. - Southfield, MI (id = 15044) > > ISP: CenturyLink > > Latency: 45.45 ms (194.34 ms jitter) > > Download: 8.88 Mbps (data used: 12.6 MB ) > > Upload: 0.64 Mbps (data used: 962.7 kB ) > > Packet Loss: 5.1% > > Result URL: > > https://www.speedtest.net/result/c/799ba46b-d3d4-4d54-ac76-118fac847b7 > > 3 > > > > ########### > > No SQM or QOS > > > > flent -x --socket-stats --step-size=.05 -H de.starlink.taht.net -H > > london.starlink.taht.net -H singapore.starlink.taht.net -H > fremont.starlink.taht.net -t > nosqm_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl rtt_fair4be Started > Flent 2.0.0 using Python 3.9.2. > > Starting rtt_fair4be test. Expected run time: 70 seconds. > > Data file written to > > ./rtt_fair4be-2022-04-17T145553.865990.nosqm_erx_rttfair4be_8Mbpsdown_ > > 500Kbpsup_centurylinkdsl.flent.gz > > > > Summary of rtt_fair4be test run from 2022-04-17 13:55:53.865990 > > Title: 'nosqm_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl' > > > > > avg median # data pts > > Ping (ms) ICMP1 de.starlink.taht.net : > 1026.06 920.00 ms 1383 > > Ping (ms) ICMP2 london.starlink.taht.net : > 1009.28 887.00 ms 1383 > > Ping (ms) ICMP3 singapore.starlink.taht.net : > 1121.09 1012.00 ms 1383 > > Ping (ms) ICMP4 fremont.starlink.taht.net : > 943.77 853.00 ms 1383 > > Ping (ms) avg : > 1025.05 N/A ms 1383 > > TCP download BE1 de.starlink.taht.net : > 1.36 2.00 Mbits/s 1383 > > TCP download BE2 london.starlink.taht.net : > 0.72 0.65 Mbits/s 1383 > > TCP download BE3 singapore.starlink.taht.net : > 1.03 1.74 Mbits/s 1383 > > TCP download BE4 fremont.starlink.taht.net : > 2.11 2.88 Mbits/s 1383 > > TCP download avg : > 1.30 N/A Mbits/s 1383 > > TCP download fairness : > 0.86 N/A Mbits/s 1383 > > TCP download sum : > 5.22 N/A Mbits/s 1383 > > TCP upload BE1 de.starlink.taht.net : > 0.12 0.20 Mbits/s 1383 > > TCP upload BE1 de.starlink.taht.net::tcp_cwnd : > 16.85 14.00 1044 > > TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate : > 0.12 0.12 1036 > > TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate : > 0.35 0.26 1042 > > TCP upload BE1 de.starlink.taht.net::tcp_rtt : > 1128.57 954.47 1020 > > TCP upload BE1 de.starlink.taht.net::tcp_rtt_var : > 68.22 50.08 1020 > > TCP upload BE2 london.starlink.taht.net : > 0.12 0.15 Mbits/s 1383 > > TCP upload BE2 london.starlink.taht.net::tcp_cwnd : > 17.33 11.00 1046 > > TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate : > 0.12 0.11 1041 > > TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate : > 0.38 0.26 1044 > > TCP upload BE2 london.starlink.taht.net::tcp_rtt : > 1166.74 991.48 1043 > > TCP upload BE2 london.starlink.taht.net::tcp_rtt_var : > 79.99 68.27 1043 > > TCP upload BE3 singapore.starlink.taht.net : > 0.07 0.13 Mbits/s 1383 > > TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd : > 13.60 9.00 1042 > > TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate : > 0.07 0.08 1013 > > TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate : > 0.23 0.20 1031 > > TCP upload BE3 singapore.starlink.taht.net::tcp_rtt : > 1302.23 1129.07 967 > > TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var : > 113.67 98.84 967 > > TCP upload BE4 fremont.starlink.taht.net : > 0.11 0.20 Mbits/s 1383 > > TCP upload BE4 fremont.starlink.taht.net::tcp_cwnd : > 17.55 11.00 1038 > > TCP upload BE4 fremont.starlink.taht.net::tcp_delivery_rate : > 0.11 0.10 1035 > > TCP upload BE4 fremont.starlink.taht.net::tcp_pacing_rate : > 0.32 0.23 1036 > > TCP upload BE4 fremont.starlink.taht.net::tcp_rtt : > 1094.64 887.54 1033 > > TCP upload BE4 fremont.starlink.taht.net::tcp_rtt_var : > 67.20 59.16 1033 > > TCP upload avg : > 0.10 N/A Mbits/s 1383 > > TCP upload fairness : > 0.96 N/A Mbits/s 1383 > > TCP upload sum : > 0.42 N/A Mbits/s 1383 > > > > ########## > > with smart queue (fq_codel) on the EdgeRouter > > > > flent -x --socket-stats --step-size=.05 -H de.starlink.taht.net -H > > london.starlink.taht.net -H singapore.starlink.taht.net -H > fremont.starlink.taht.net -t > fqcodel_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl rtt_fair4be > Started Flent 2.0.0 using Python 3.9.2. > > Starting rtt_fair4be test. Expected run time: 70 seconds. > > Data file written to > > ./rtt_fair4be-2022-04-17T150339.549817.fqcodel_erx_rttfair4be_8Mbpsdow > > n_500Kbpsup_centurylinkdsl.flent.gz > > > > Summary of rtt_fair4be test run from 2022-04-17 14:03:39.549817 > > Title: 'fqcodel_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl' > > > > > avg median # data pts > > Ping (ms) ICMP1 de.starlink.taht.net : > 392.93 208.00 ms 1384 > > Ping (ms) ICMP2 london.starlink.taht.net : > 373.94 196.00 ms 1384 > > Ping (ms) ICMP3 singapore.starlink.taht.net : > 508.15 319.00 ms 1384 > > Ping (ms) ICMP4 fremont.starlink.taht.net : > 346.23 152.00 ms 1384 > > Ping (ms) avg : > 405.31 N/A ms 1384 > > TCP download BE1 de.starlink.taht.net : > 1.36 1.62 Mbits/s 1384 > > TCP download BE2 london.starlink.taht.net : > 1.47 1.63 Mbits/s 1384 > > TCP download BE3 singapore.starlink.taht.net : > 1.55 1.60 Mbits/s 1384 > > TCP download BE4 fremont.starlink.taht.net : > 1.50 1.80 Mbits/s 1384 > > TCP download avg : > 1.47 N/A Mbits/s 1384 > > TCP download fairness : > 1.00 N/A Mbits/s 1384 > > TCP download sum : > 5.88 N/A Mbits/s 1384 > > TCP upload BE1 de.starlink.taht.net : > 0.09 0.85 Mbits/s 1384 > > TCP upload BE1 de.starlink.taht.net::tcp_cwnd : > 4.50 4.00 1035 > > TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate : > 0.08 0.08 1030 > > TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate : > 0.17 0.15 1032 > > TCP upload BE1 de.starlink.taht.net::tcp_rtt : > 458.11 318.82 1027 > > TCP upload BE1 de.starlink.taht.net::tcp_rtt_var : > 95.76 61.88 1027 > > TCP upload BE2 london.starlink.taht.net : > 0.09 0.93 Mbits/s 1384 > > TCP upload BE2 london.starlink.taht.net::tcp_cwnd : > 4.17 4.00 1038 > > TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate : > 0.08 0.08 1033 > > TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate : > 0.17 0.14 1036 > > TCP upload BE2 london.starlink.taht.net::tcp_rtt : > 452.67 322.49 1033 > > TCP upload BE2 london.starlink.taht.net::tcp_rtt_var : > 96.87 64.28 1033 > > TCP upload BE3 singapore.starlink.taht.net : > 0.06 0.44 Mbits/s 1384 > > TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd : > 4.16 4.00 1040 > > TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate : > 0.06 0.06 1031 > > TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate : > 0.12 0.11 1036 > > TCP upload BE3 singapore.starlink.taht.net::tcp_rtt : > 577.75 438.61 1027 > > TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var : > 107.05 72.97 1027 > > TCP upload BE4 fremont.starlink.taht.net : > 0.10 1.48 Mbits/s 1384 > > TCP upload BE4 fremont.starlink.taht.net::tcp_cwnd : > 4.45 4.00 1020 > > TCP upload BE4 fremont.starlink.taht.net::tcp_delivery_rate : > 0.09 0.09 1017 > > TCP upload BE4 fremont.starlink.taht.net::tcp_pacing_rate : > 0.21 0.16 1019 > > TCP upload BE4 fremont.starlink.taht.net::tcp_rtt : > 439.05 285.86 1018 > > TCP upload BE4 fremont.starlink.taht.net::tcp_rtt_var : > 81.90 56.33 1018 > > TCP upload avg : > 0.08 N/A Mbits/s 1384 > > TCP upload fairness : > 0.97 N/A Mbits/s 1384 > > TCP upload sum : > 0.34 N/A Mbits/s 1384 > > > > > > EdgeRouter Settings > > > > smart-queue DSL { > > download { > > ecn enable > > flows 1024 > > fq-quantum 300 > > limit 10240 > > rate 8mbit > > } > > upload { > > ecn enable > > flows 1024 > > fq-quantum 300 > > limit 10240 > > rate 600kbit > > target 15ms > > } > > wan-interface pppoe0 > > > > Also, here's the VMs qdisc settings. Which I now realize probably > > aren't the best > > > > sysctl -a | grep qdisc > > net.core.default_qdisc = pfifo_fast > > > > tc -s qdisc > > qdisc noqueue 0: dev lo root refcnt 2 > > Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b > > 0p requeues 0 qdisc mq 0: dev eth0 root Sent 628201 bytes 8643 pkt > > (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 qdisc > > pfifo_fast 0: dev eth0 parent :1 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 > > 1 1 1 1 1 Sent 628201 bytes 8643 pkt (dropped 0, overlimits 0 > > requeues 0) backlog 0b 0p requeues 0 > > > > > > > > -Jordan > > > > -----Original Message----- > > From: Starlink On Behalf Of > > Dave Taht > > Sent: Saturday, April 16, 2022 9:40 PM > > To: flent-users ; bloat > > ; starlink@lists.bufferbloat.net > > Subject: [Starlink] an rtt_fair test? > > > > if anyone has a bit of spare time this holiday, I'd like to collect a > few results from various networks around the web this week, not just on the > turris. Example command line and outputs here: > > > > https://forum.turris.cz/t/sqm-on-turris-flent-benchmarks/17048/ > > > > -- > > I tried to build a better future, a few times: > > https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org > > > > Dave Täht CEO, TekLibre, LLC > > _______________________________________________ > > Starlink mailing list > > Starlink@lists.bufferbloat.net > > https://lists.bufferbloat.net/listinfo/starlink > > > > > -- > I tried to build a better future, a few times: > https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org > > Dave Täht CEO, TekLibre, LLC > > _______________________________________________ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat >