From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: mail.toke.dk; spf=pass (mailfrom) smtp.mailfrom=inacomptc.com (client-ip=198.109.107.12; helo=cpanel.tc-mi.net; envelope-from=jordan@inacomptc.com; receiver=) Authentication-Results: mail.toke.dk; dkim=pass (2048-bit key; secure) header.d=inacomptc.com header.i=@inacomptc.com header.a=rsa-sha256 header.s=default header.b=b059YbrC Received: from cpanel.tc-mi.net (cpanel.tc-mi.net [198.109.107.12]) by mail.toke.dk (Postfix) with ESMTPS id 88412959D58 for ; Sun, 17 Apr 2022 17:36:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=inacomptc.com; s=default; h=Content-Transfer-Encoding:Content-Type: MIME-Version:Message-ID:Date:Subject:In-Reply-To:References:Cc:To:From:Sender :Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=wxhT3y43HDxROYLSEh16uhBya6kTVFZIWOvIJx5rMpQ=; b=b059YbrCRJlItt2eZ6UNGmp/OV SK5dpDXt2jelQW7HloIHY5CeXf5NgpE3QbVbBssRsXZB/8O9FY6i46iHBomdqN1/nGxMwmiyp5qrV 4zqcFMYeFD2aOyGUU8b87tHvvUG64NoZzwf+SKQa0GHpjFTGRVGIJJxP3nCojiOyy2gIpcEFNm3ta 2GDegJeb1lfSvfZ/PYwjIhUJxpYPbnHWM43LSw3TlvPU30CLXVNiEzOpd/oqm0+o1JNN0vZrj5UgH MEwbec+9yCMFb+GrETuPwYQa2WgjA4ERtzoLAPXDghU5kZHMm2zbuYQV1NQuVnR95MVPU2PcSj/og H/rcuZUg==; Received: from 172-222-114-185.res.spectrum.com ([172.222.114.185]:25972 helo=DESKTOPRC3REJK) by cpanel.tc-mi.net with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1ng6wh-0001ls-L9; Sun, 17 Apr 2022 11:36:15 -0400 From: To: "'Dave Taht'" References: <14d1f01d85268$68d55f00$3a801d00$@inacomptc.com> In-Reply-To: Date: Sun, 17 Apr 2022 11:36:03 -0400 Message-ID: <14d2801d85270$d9820020$8c860060$@inacomptc.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 16.0 Thread-Index: AQI0nJeVSODbepJ6WRxI9YjCWTQW0AEoIjfVAjxivRGsIII1kA== Content-Language: en-us X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - cpanel.tc-mi.net X-AntiAbuse: Original Domain - flent.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - inacomptc.com X-Get-Message-Sender-Via: cpanel.tc-mi.net: authenticated_id: jordan@inacomptc.com X-Authenticated-Sender: cpanel.tc-mi.net: jordan@inacomptc.com X-Source: X-Source-Args: X-Source-Dir: X-MailFrom: jordan@inacomptc.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation Message-ID-Hash: I4GIUYQG2SF7SY3XIOH4CZG4GNPEGMMQ X-Message-ID-Hash: I4GIUYQG2SF7SY3XIOH4CZG4GNPEGMMQ X-Mailman-Approved-At: Sun, 17 Apr 2022 21:01:26 +0200 CC: 'flent-users' , 'bloat' , starlink@lists.bufferbloat.net X-Mailman-Version: 3.3.5 Precedence: list Subject: [Flent-users] Re: [Starlink] an rtt_fair test? List-Id: Flent discussion list Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: You're welcome! Here's the files Flent generated: = https://1drv.ms/u/s!Ap4u4Rte63FqjqoSQXX444OWLvl8FQ?e=3Df22Nt7 And thanks for the link! It honestly doesn't look too bad to implement = so I might give it a shot today if time allows. I've also been contemplating replacing it with something like a Mikrotik = since cake is now integrated into them. I've been testing one at home = and made a post with a bunch of flent tests to the cake thread in their = forum if you're interested in some data from a Charter Spectrum cable = connection: https://forum.mikrotik.com/viewtopic.php?t=3D179307#p925485 The OneDrive link to my flent files should still be active in that post = as well. -Jordan -----Original Message----- From: Dave Taht =20 Sent: Sunday, April 17, 2022 10:57 AM To: Jordan Szuch Cc: flent-users ; bloat = ; starlink@lists.bufferbloat.net Subject: Re: [Starlink] an rtt_fair test? Thank you for testing!!! If you could stash the *.flent.gz files = somewhere I could get at, that would be great. You can also browse the = plots using the flent-gui to see more detail over time. Cake, long available as an out of tree build for the edgerouter, has = both dsl compensation and an ack-filter which would help A LOT, in your = case. i wish I'd pushed harder to get dsl compensation into the = edgerouter smart queues code... https://community.ui.com/questions/Cake-compiled-for-the-EdgeRouter-devic= es/fc1ff27c-f321-4344-8737-fcc755cae8a2 Given your 500kbit upload speed the minimum latency inflation under the = structure of this test (4 upload flows) should have been around 150ms (4 = * 25ms + ack overhead), but without looking at the *.gz files I don't = know what your baseline latency was to any of these sites. (it would be = good if flent reported on the text output what the minimum actually was) On Sun, Apr 17, 2022 at 7:35 AM wrote: > > Hi Dave, > > This isn't specifically cake related but I figured it could be = interesting, especially as these are from a location where I've been = investigating setting up something with cake. These are statistics from = a relatives location where they only have a DSL line from CenturyLink = available to them. There's a little Debian based VM I setup for a couple = random things so I was able to run flent from that. They have a = EdgeRouter X serving as the router so they can at least take advantage = of fq_codel through the routers Smart Queue feature. I ran a quick speed = test, then ran flent without sqm, and then with sqm. Sorry if this gets = a bit long and hopefully it stays readable. > > ############# > Speedtest by Ookla > > Server: 123.Net, Inc. - Southfield, MI (id =3D 15044) > ISP: CenturyLink > Latency: 45.45 ms (194.34 ms jitter) > Download: 8.88 Mbps (data used: 12.6 MB ) > Upload: 0.64 Mbps (data used: 962.7 kB ) > Packet Loss: 5.1% > Result URL:=20 > https://www.speedtest.net/result/c/799ba46b-d3d4-4d54-ac76-118fac847b7 > 3 > > ########### > No SQM or QOS > > flent -x --socket-stats --step-size=3D.05 -H de.starlink.taht.net -H=20 > london.starlink.taht.net -H singapore.starlink.taht.net -H = fremont.starlink.taht.net -t = nosqm_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl rtt_fair4be = Started Flent 2.0.0 using Python 3.9.2. > Starting rtt_fair4be test. Expected run time: 70 seconds. > Data file written to=20 > ./rtt_fair4be-2022-04-17T145553.865990.nosqm_erx_rttfair4be_8Mbpsdown_ > 500Kbpsup_centurylinkdsl.flent.gz > > Summary of rtt_fair4be test run from 2022-04-17 13:55:53.865990 > Title: 'nosqm_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl' > > = avg median # data pts > Ping (ms) ICMP1 de.starlink.taht.net : = 1026.06 920.00 ms 1383 > Ping (ms) ICMP2 london.starlink.taht.net : = 1009.28 887.00 ms 1383 > Ping (ms) ICMP3 singapore.starlink.taht.net : = 1121.09 1012.00 ms 1383 > Ping (ms) ICMP4 fremont.starlink.taht.net : = 943.77 853.00 ms 1383 > Ping (ms) avg : = 1025.05 N/A ms 1383 > TCP download BE1 de.starlink.taht.net : = 1.36 2.00 Mbits/s 1383 > TCP download BE2 london.starlink.taht.net : = 0.72 0.65 Mbits/s 1383 > TCP download BE3 singapore.starlink.taht.net : = 1.03 1.74 Mbits/s 1383 > TCP download BE4 fremont.starlink.taht.net : = 2.11 2.88 Mbits/s 1383 > TCP download avg : = 1.30 N/A Mbits/s 1383 > TCP download fairness : = 0.86 N/A Mbits/s 1383 > TCP download sum : = 5.22 N/A Mbits/s 1383 > TCP upload BE1 de.starlink.taht.net : = 0.12 0.20 Mbits/s 1383 > TCP upload BE1 de.starlink.taht.net::tcp_cwnd : = 16.85 14.00 1044 > TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate : = 0.12 0.12 1036 > TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate : = 0.35 0.26 1042 > TCP upload BE1 de.starlink.taht.net::tcp_rtt : = 1128.57 954.47 1020 > TCP upload BE1 de.starlink.taht.net::tcp_rtt_var : = 68.22 50.08 1020 > TCP upload BE2 london.starlink.taht.net : = 0.12 0.15 Mbits/s 1383 > TCP upload BE2 london.starlink.taht.net::tcp_cwnd : = 17.33 11.00 1046 > TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate : = 0.12 0.11 1041 > TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate : = 0.38 0.26 1044 > TCP upload BE2 london.starlink.taht.net::tcp_rtt : = 1166.74 991.48 1043 > TCP upload BE2 london.starlink.taht.net::tcp_rtt_var : = 79.99 68.27 1043 > TCP upload BE3 singapore.starlink.taht.net : = 0.07 0.13 Mbits/s 1383 > TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd : = 13.60 9.00 1042 > TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate : = 0.07 0.08 1013 > TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate : = 0.23 0.20 1031 > TCP upload BE3 singapore.starlink.taht.net::tcp_rtt : = 1302.23 1129.07 967 > TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var : = 113.67 98.84 967 > TCP upload BE4 fremont.starlink.taht.net : = 0.11 0.20 Mbits/s 1383 > TCP upload BE4 fremont.starlink.taht.net::tcp_cwnd : = 17.55 11.00 1038 > TCP upload BE4 fremont.starlink.taht.net::tcp_delivery_rate : = 0.11 0.10 1035 > TCP upload BE4 fremont.starlink.taht.net::tcp_pacing_rate : = 0.32 0.23 1036 > TCP upload BE4 fremont.starlink.taht.net::tcp_rtt : = 1094.64 887.54 1033 > TCP upload BE4 fremont.starlink.taht.net::tcp_rtt_var : = 67.20 59.16 1033 > TCP upload avg : = 0.10 N/A Mbits/s 1383 > TCP upload fairness : = 0.96 N/A Mbits/s 1383 > TCP upload sum : = 0.42 N/A Mbits/s 1383 > > ########## > with smart queue (fq_codel) on the EdgeRouter > > flent -x --socket-stats --step-size=3D.05 -H de.starlink.taht.net -H=20 > london.starlink.taht.net -H singapore.starlink.taht.net -H = fremont.starlink.taht.net -t = fqcodel_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl rtt_fair4be = Started Flent 2.0.0 using Python 3.9.2. > Starting rtt_fair4be test. Expected run time: 70 seconds. > Data file written to=20 > ./rtt_fair4be-2022-04-17T150339.549817.fqcodel_erx_rttfair4be_8Mbpsdow > n_500Kbpsup_centurylinkdsl.flent.gz > > Summary of rtt_fair4be test run from 2022-04-17 14:03:39.549817 > Title: 'fqcodel_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl' > > = avg median # data pts > Ping (ms) ICMP1 de.starlink.taht.net : = 392.93 208.00 ms 1384 > Ping (ms) ICMP2 london.starlink.taht.net : = 373.94 196.00 ms 1384 > Ping (ms) ICMP3 singapore.starlink.taht.net : = 508.15 319.00 ms 1384 > Ping (ms) ICMP4 fremont.starlink.taht.net : = 346.23 152.00 ms 1384 > Ping (ms) avg : = 405.31 N/A ms 1384 > TCP download BE1 de.starlink.taht.net : = 1.36 1.62 Mbits/s 1384 > TCP download BE2 london.starlink.taht.net : = 1.47 1.63 Mbits/s 1384 > TCP download BE3 singapore.starlink.taht.net : = 1.55 1.60 Mbits/s 1384 > TCP download BE4 fremont.starlink.taht.net : = 1.50 1.80 Mbits/s 1384 > TCP download avg : = 1.47 N/A Mbits/s 1384 > TCP download fairness : = 1.00 N/A Mbits/s 1384 > TCP download sum : = 5.88 N/A Mbits/s 1384 > TCP upload BE1 de.starlink.taht.net : = 0.09 0.85 Mbits/s 1384 > TCP upload BE1 de.starlink.taht.net::tcp_cwnd : = 4.50 4.00 1035 > TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate : = 0.08 0.08 1030 > TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate : = 0.17 0.15 1032 > TCP upload BE1 de.starlink.taht.net::tcp_rtt : = 458.11 318.82 1027 > TCP upload BE1 de.starlink.taht.net::tcp_rtt_var : = 95.76 61.88 1027 > TCP upload BE2 london.starlink.taht.net : = 0.09 0.93 Mbits/s 1384 > TCP upload BE2 london.starlink.taht.net::tcp_cwnd : = 4.17 4.00 1038 > TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate : = 0.08 0.08 1033 > TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate : = 0.17 0.14 1036 > TCP upload BE2 london.starlink.taht.net::tcp_rtt : = 452.67 322.49 1033 > TCP upload BE2 london.starlink.taht.net::tcp_rtt_var : = 96.87 64.28 1033 > TCP upload BE3 singapore.starlink.taht.net : = 0.06 0.44 Mbits/s 1384 > TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd : = 4.16 4.00 1040 > TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate : = 0.06 0.06 1031 > TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate : = 0.12 0.11 1036 > TCP upload BE3 singapore.starlink.taht.net::tcp_rtt : = 577.75 438.61 1027 > TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var : = 107.05 72.97 1027 > TCP upload BE4 fremont.starlink.taht.net : = 0.10 1.48 Mbits/s 1384 > TCP upload BE4 fremont.starlink.taht.net::tcp_cwnd : = 4.45 4.00 1020 > TCP upload BE4 fremont.starlink.taht.net::tcp_delivery_rate : = 0.09 0.09 1017 > TCP upload BE4 fremont.starlink.taht.net::tcp_pacing_rate : = 0.21 0.16 1019 > TCP upload BE4 fremont.starlink.taht.net::tcp_rtt : = 439.05 285.86 1018 > TCP upload BE4 fremont.starlink.taht.net::tcp_rtt_var : = 81.90 56.33 1018 > TCP upload avg : = 0.08 N/A Mbits/s 1384 > TCP upload fairness : = 0.97 N/A Mbits/s 1384 > TCP upload sum : = 0.34 N/A Mbits/s 1384 > > > EdgeRouter Settings > > smart-queue DSL { > download { > ecn enable > flows 1024 > fq-quantum 300 > limit 10240 > rate 8mbit > } > upload { > ecn enable > flows 1024 > fq-quantum 300 > limit 10240 > rate 600kbit > target 15ms > } > wan-interface pppoe0 > > Also, here's the VMs qdisc settings. Which I now realize probably=20 > aren't the best > > sysctl -a | grep qdisc > net.core.default_qdisc =3D pfifo_fast > > tc -s qdisc > qdisc noqueue 0: dev lo root refcnt 2 > Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b=20 > 0p requeues 0 qdisc mq 0: dev eth0 root Sent 628201 bytes 8643 pkt=20 > (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 qdisc=20 > pfifo_fast 0: dev eth0 parent :1 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 = > 1 1 1 1 1 Sent 628201 bytes 8643 pkt (dropped 0, overlimits 0=20 > requeues 0) backlog 0b 0p requeues 0 > > > > -Jordan > > -----Original Message----- > From: Starlink On Behalf Of=20 > Dave Taht > Sent: Saturday, April 16, 2022 9:40 PM > To: flent-users ; bloat=20 > ; starlink@lists.bufferbloat.net > Subject: [Starlink] an rtt_fair test? > > if anyone has a bit of spare time this holiday, I'd like to collect a = few results from various networks around the web this week, not just on = the turris. Example command line and outputs here: > > https://forum.turris.cz/t/sqm-on-turris-flent-benchmarks/17048/ > > -- > I tried to build a better future, a few times: > https://wayforward.archive.org/?site=3Dhttps%3A%2F%2Fwww.icei.org > > Dave T=C3=A4ht CEO, TekLibre, LLC > _______________________________________________ > Starlink mailing list > Starlink@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/starlink > -- I tried to build a better future, a few times: https://wayforward.archive.org/?site=3Dhttps%3A%2F%2Fwww.icei.org Dave T=C3=A4ht CEO, TekLibre, LLC