From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com [IPv6:2a00:1450:4864:20::630]) by mail.toke.dk (Postfix) with ESMTPS id 10B2B959CF6 for ; Sun, 17 Apr 2022 16:57:10 +0200 (CEST) Authentication-Results: mail.toke.dk; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=o9OQe2Kz Received: by mail-ej1-x630.google.com with SMTP id i27so23079817ejd.9 for ; Sun, 17 Apr 2022 07:57:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=BPFPrOvqE94W/oVDsts+u9fdQouyfte17SiCc+th5Gg=; b=o9OQe2Kz/W+ETKdvbFg15dgrYKemTiPAbqa36NNlTz2xT1r9UVxr9A041hCoJH1vcJ 2cpiu6MAxRJJbHm8ZUTn+iMIEsYmsCM7JbIYogtTFrNnZpzZRLYnisC30a0dP+O/B3WG z8x1lF/jENo/72LAApsD+KV7DQvD8wXSiMrwEpqbKrjI+6ReHTz0xs6p2auQula3MGSR 0meykIBgYzA22KSzsWwwczdVrkzsWUriLXAdaLffSb9HUuvdbTt5PwUCXS+C1iY6L8/o +EtOlud7fOeB8PkdFmJJAs/EOJwf1LlhEG3RRgz4dhU8NZiIYwAgafqgEIRMPxwchn4I P7Ug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=BPFPrOvqE94W/oVDsts+u9fdQouyfte17SiCc+th5Gg=; b=fWvW9OvPfphVRJooRdSI07Tj4pH8Ul/NXvONhNICdoPlMekAv6Lg1Mv0Ow7lzH31lh 7Ty51Z9lMZo6ihK1L5kx/NQoBU616xM3xMDAsQYnlRQti+zfb7CKjmQJrHhMz3r6SQPX SrxJ4JfjuR/l0dsdRKXrq7o8YZAz24TgBP+8R7MC2dBGN9kcLoDyd5+5IOxQSoAvsGaV QRFycpSDxNZtMWUabg396Ma4sYOx+a+EMGpnkujot/wLzj505GQZf31QIbm9bfvMSl00 49txUo3DV3Fuxa4p2oFhFr1BwijQPjgVaBwi25hIut+y08Of7VQIAL9DCrmEPIpjGlJ2 NZrQ== X-Gm-Message-State: AOAM533HXqPeBaRhm2QAUZOcGyxb8+NjT1m/TY2GVCsBINV/c7SrhMLB K65N+DDffB15ovrmXRwcfA4x92dCQF1Gkd3ayRXy9MYFIVxHNg== X-Google-Smtp-Source: ABdhPJxCevlIBFNSR5mzGERtnf2hUwuU9n00H/1UGE2bEXJ0vYTwqAdfQBsR8raA+/dpURpBoJXZFgxLsuhUP+EtU3w= X-Received: by 2002:a17:907:7296:b0:6e8:97c1:a7ef with SMTP id dt22-20020a170907729600b006e897c1a7efmr5865341ejc.262.1650207429779; Sun, 17 Apr 2022 07:57:09 -0700 (PDT) MIME-Version: 1.0 References: <14d1f01d85268$68d55f00$3a801d00$@inacomptc.com> In-Reply-To: <14d1f01d85268$68d55f00$3a801d00$@inacomptc.com> From: Dave Taht Date: Sun, 17 Apr 2022 07:56:57 -0700 Message-ID: To: Jordan Szuch Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Message-ID-Hash: OKK4EAPM3WAY37AXIIPUH7PWIOBDO374 X-Message-ID-Hash: OKK4EAPM3WAY37AXIIPUH7PWIOBDO374 X-MailFrom: dave.taht@gmail.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: flent-users , bloat , starlink@lists.bufferbloat.net X-Mailman-Version: 3.3.5 Precedence: list Subject: [Flent-users] Re: [Starlink] an rtt_fair test? List-Id: Flent discussion list Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Thank you for testing!!! If you could stash the *.flent.gz files somewhere I could get at, that would be great. You can also browse the plots using the flent-gui to see more detail over time. Cake, long available as an out of tree build for the edgerouter, has both dsl compensation and an ack-filter which would help A LOT, in your case. i wish I'd pushed harder to get dsl compensation into the edgerouter smart queues code... https://community.ui.com/questions/Cake-compiled-for-the-EdgeRouter-devices= /fc1ff27c-f321-4344-8737-fcc755cae8a2 Given your 500kbit upload speed the minimum latency inflation under the structure of this test (4 upload flows) should have been around 150ms (4 * 25ms + ack overhead), but without looking at the *.gz files I don't know what your baseline latency was to any of these sites. (it would be good if flent reported on the text output what the minimum actually was) On Sun, Apr 17, 2022 at 7:35 AM wrote: > > Hi Dave, > > This isn't specifically cake related but I figured it could be interestin= g, especially as these are from a location where I've been investigating se= tting up something with cake. These are statistics from a relatives locatio= n where they only have a DSL line from CenturyLink available to them. There= 's a little Debian based VM I setup for a couple random things so I was abl= e to run flent from that. They have a EdgeRouter X serving as the router so= they can at least take advantage of fq_codel through the routers Smart Que= ue feature. I ran a quick speed test, then ran flent without sqm, and then = with sqm. Sorry if this gets a bit long and hopefully it stays readable. > > ############# > Speedtest by Ookla > > Server: 123.Net, Inc. - Southfield, MI (id =3D 15044) > ISP: CenturyLink > Latency: 45.45 ms (194.34 ms jitter) > Download: 8.88 Mbps (data used: 12.6 MB ) > Upload: 0.64 Mbps (data used: 962.7 kB ) > Packet Loss: 5.1% > Result URL: https://www.speedtest.net/result/c/799ba46b-d3d4-4d54-ac76-1= 18fac847b73 > > ########### > No SQM or QOS > > flent -x --socket-stats --step-size=3D.05 -H de.starlink.taht.net -H lond= on.starlink.taht.net -H singapore.starlink.taht.net -H fremont.starlink.tah= t.net -t nosqm_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl rtt_fair4b= e > Started Flent 2.0.0 using Python 3.9.2. > Starting rtt_fair4be test. Expected run time: 70 seconds. > Data file written to ./rtt_fair4be-2022-04-17T145553.865990.nosqm_erx_rtt= fair4be_8Mbpsdown_500Kbpsup_centurylinkdsl.flent.gz > > Summary of rtt_fair4be test run from 2022-04-17 13:55:53.865990 > Title: 'nosqm_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl' > > = avg median # data pts > Ping (ms) ICMP1 de.starlink.taht.net : 102= 6.06 920.00 ms 1383 > Ping (ms) ICMP2 london.starlink.taht.net : 100= 9.28 887.00 ms 1383 > Ping (ms) ICMP3 singapore.starlink.taht.net : 112= 1.09 1012.00 ms 1383 > Ping (ms) ICMP4 fremont.starlink.taht.net : 94= 3.77 853.00 ms 1383 > Ping (ms) avg : 102= 5.05 N/A ms 1383 > TCP download BE1 de.starlink.taht.net : = 1.36 2.00 Mbits/s 1383 > TCP download BE2 london.starlink.taht.net : = 0.72 0.65 Mbits/s 1383 > TCP download BE3 singapore.starlink.taht.net : = 1.03 1.74 Mbits/s 1383 > TCP download BE4 fremont.starlink.taht.net : = 2.11 2.88 Mbits/s 1383 > TCP download avg : = 1.30 N/A Mbits/s 1383 > TCP download fairness : = 0.86 N/A Mbits/s 1383 > TCP download sum : = 5.22 N/A Mbits/s 1383 > TCP upload BE1 de.starlink.taht.net : = 0.12 0.20 Mbits/s 1383 > TCP upload BE1 de.starlink.taht.net::tcp_cwnd : 1= 6.85 14.00 1044 > TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate : = 0.12 0.12 1036 > TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate : = 0.35 0.26 1042 > TCP upload BE1 de.starlink.taht.net::tcp_rtt : 112= 8.57 954.47 1020 > TCP upload BE1 de.starlink.taht.net::tcp_rtt_var : 6= 8.22 50.08 1020 > TCP upload BE2 london.starlink.taht.net : = 0.12 0.15 Mbits/s 1383 > TCP upload BE2 london.starlink.taht.net::tcp_cwnd : 1= 7.33 11.00 1046 > TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate : = 0.12 0.11 1041 > TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate : = 0.38 0.26 1044 > TCP upload BE2 london.starlink.taht.net::tcp_rtt : 116= 6.74 991.48 1043 > TCP upload BE2 london.starlink.taht.net::tcp_rtt_var : 7= 9.99 68.27 1043 > TCP upload BE3 singapore.starlink.taht.net : = 0.07 0.13 Mbits/s 1383 > TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd : 1= 3.60 9.00 1042 > TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate : = 0.07 0.08 1013 > TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate : = 0.23 0.20 1031 > TCP upload BE3 singapore.starlink.taht.net::tcp_rtt : 130= 2.23 1129.07 967 > TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var : 11= 3.67 98.84 967 > TCP upload BE4 fremont.starlink.taht.net : = 0.11 0.20 Mbits/s 1383 > TCP upload BE4 fremont.starlink.taht.net::tcp_cwnd : 1= 7.55 11.00 1038 > TCP upload BE4 fremont.starlink.taht.net::tcp_delivery_rate : = 0.11 0.10 1035 > TCP upload BE4 fremont.starlink.taht.net::tcp_pacing_rate : = 0.32 0.23 1036 > TCP upload BE4 fremont.starlink.taht.net::tcp_rtt : 109= 4.64 887.54 1033 > TCP upload BE4 fremont.starlink.taht.net::tcp_rtt_var : 6= 7.20 59.16 1033 > TCP upload avg : = 0.10 N/A Mbits/s 1383 > TCP upload fairness : = 0.96 N/A Mbits/s 1383 > TCP upload sum : = 0.42 N/A Mbits/s 1383 > > ########## > with smart queue (fq_codel) on the EdgeRouter > > flent -x --socket-stats --step-size=3D.05 -H de.starlink.taht.net -H lond= on.starlink.taht.net -H singapore.starlink.taht.net -H fremont.starlink.tah= t.net -t fqcodel_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl rtt_fair= 4be > Started Flent 2.0.0 using Python 3.9.2. > Starting rtt_fair4be test. Expected run time: 70 seconds. > Data file written to ./rtt_fair4be-2022-04-17T150339.549817.fqcodel_erx_r= ttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl.flent.gz > > Summary of rtt_fair4be test run from 2022-04-17 14:03:39.549817 > Title: 'fqcodel_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl' > > = avg median # data pts > Ping (ms) ICMP1 de.starlink.taht.net : 39= 2.93 208.00 ms 1384 > Ping (ms) ICMP2 london.starlink.taht.net : 37= 3.94 196.00 ms 1384 > Ping (ms) ICMP3 singapore.starlink.taht.net : 50= 8.15 319.00 ms 1384 > Ping (ms) ICMP4 fremont.starlink.taht.net : 34= 6.23 152.00 ms 1384 > Ping (ms) avg : 40= 5.31 N/A ms 1384 > TCP download BE1 de.starlink.taht.net : = 1.36 1.62 Mbits/s 1384 > TCP download BE2 london.starlink.taht.net : = 1.47 1.63 Mbits/s 1384 > TCP download BE3 singapore.starlink.taht.net : = 1.55 1.60 Mbits/s 1384 > TCP download BE4 fremont.starlink.taht.net : = 1.50 1.80 Mbits/s 1384 > TCP download avg : = 1.47 N/A Mbits/s 1384 > TCP download fairness : = 1.00 N/A Mbits/s 1384 > TCP download sum : = 5.88 N/A Mbits/s 1384 > TCP upload BE1 de.starlink.taht.net : = 0.09 0.85 Mbits/s 1384 > TCP upload BE1 de.starlink.taht.net::tcp_cwnd : = 4.50 4.00 1035 > TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate : = 0.08 0.08 1030 > TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate : = 0.17 0.15 1032 > TCP upload BE1 de.starlink.taht.net::tcp_rtt : 45= 8.11 318.82 1027 > TCP upload BE1 de.starlink.taht.net::tcp_rtt_var : 9= 5.76 61.88 1027 > TCP upload BE2 london.starlink.taht.net : = 0.09 0.93 Mbits/s 1384 > TCP upload BE2 london.starlink.taht.net::tcp_cwnd : = 4.17 4.00 1038 > TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate : = 0.08 0.08 1033 > TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate : = 0.17 0.14 1036 > TCP upload BE2 london.starlink.taht.net::tcp_rtt : 45= 2.67 322.49 1033 > TCP upload BE2 london.starlink.taht.net::tcp_rtt_var : 9= 6.87 64.28 1033 > TCP upload BE3 singapore.starlink.taht.net : = 0.06 0.44 Mbits/s 1384 > TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd : = 4.16 4.00 1040 > TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate : = 0.06 0.06 1031 > TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate : = 0.12 0.11 1036 > TCP upload BE3 singapore.starlink.taht.net::tcp_rtt : 57= 7.75 438.61 1027 > TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var : 10= 7.05 72.97 1027 > TCP upload BE4 fremont.starlink.taht.net : = 0.10 1.48 Mbits/s 1384 > TCP upload BE4 fremont.starlink.taht.net::tcp_cwnd : = 4.45 4.00 1020 > TCP upload BE4 fremont.starlink.taht.net::tcp_delivery_rate : = 0.09 0.09 1017 > TCP upload BE4 fremont.starlink.taht.net::tcp_pacing_rate : = 0.21 0.16 1019 > TCP upload BE4 fremont.starlink.taht.net::tcp_rtt : 43= 9.05 285.86 1018 > TCP upload BE4 fremont.starlink.taht.net::tcp_rtt_var : 8= 1.90 56.33 1018 > TCP upload avg : = 0.08 N/A Mbits/s 1384 > TCP upload fairness : = 0.97 N/A Mbits/s 1384 > TCP upload sum : = 0.34 N/A Mbits/s 1384 > > > EdgeRouter Settings > > smart-queue DSL { > download { > ecn enable > flows 1024 > fq-quantum 300 > limit 10240 > rate 8mbit > } > upload { > ecn enable > flows 1024 > fq-quantum 300 > limit 10240 > rate 600kbit > target 15ms > } > wan-interface pppoe0 > > Also, here's the VMs qdisc settings. Which I now realize probably aren't = the best > > sysctl -a | grep qdisc > net.core.default_qdisc =3D pfifo_fast > > tc -s qdisc > qdisc noqueue 0: dev lo root refcnt 2 > Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) > backlog 0b 0p requeues 0 > qdisc mq 0: dev eth0 root > Sent 628201 bytes 8643 pkt (dropped 0, overlimits 0 requeues 0) > backlog 0b 0p requeues 0 > qdisc pfifo_fast 0: dev eth0 parent :1 bands 3 priomap 1 2 2 2 1 2 0 0 1 = 1 1 1 1 1 1 1 > Sent 628201 bytes 8643 pkt (dropped 0, overlimits 0 requeues 0) > backlog 0b 0p requeues 0 > > > > -Jordan > > -----Original Message----- > From: Starlink On Behalf Of Dave= Taht > Sent: Saturday, April 16, 2022 9:40 PM > To: flent-users ; bloat ; starlink@lists.bufferbloat.net > Subject: [Starlink] an rtt_fair test? > > if anyone has a bit of spare time this holiday, I'd like to collect a few= results from various networks around the web this week, not just on the tu= rris. Example command line and outputs here: > > https://forum.turris.cz/t/sqm-on-turris-flent-benchmarks/17048/ > > -- > I tried to build a better future, a few times: > https://wayforward.archive.org/?site=3Dhttps%3A%2F%2Fwww.icei.org > > Dave T=C3=A4ht CEO, TekLibre, LLC > _______________________________________________ > Starlink mailing list > Starlink@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/starlink > --=20 I tried to build a better future, a few times: https://wayforward.archive.org/?site=3Dhttps%3A%2F%2Fwww.icei.org Dave T=C3=A4ht CEO, TekLibre, LLC