Flent-users discussion archives
 help / color / mirror / Atom feed
* [Flent-users] an rtt_fair test?
@ 2022-04-17  1:39 Dave Taht
  2022-04-17 14:35 ` [Flent-users] Re: [Starlink] " jordan
  0 siblings, 1 reply; 6+ messages in thread
From: Dave Taht @ 2022-04-17  1:39 UTC (permalink / raw)
  To: flent-users, bloat, starlink

if anyone has a bit of spare time this holiday, I'd like to collect a
few results from various networks around the web this week, not just
on the turris. Example command line and outputs here:

https://forum.turris.cz/t/sqm-on-turris-flent-benchmarks/17048/

-- 
I tried to build a better future, a few times:
https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org

Dave Täht CEO, TekLibre, LLC

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [Flent-users] Re: [Starlink] an rtt_fair test?
  2022-04-17  1:39 [Flent-users] an rtt_fair test? Dave Taht
@ 2022-04-17 14:35 ` jordan
  2022-04-17 14:56   ` Dave Taht
  0 siblings, 1 reply; 6+ messages in thread
From: jordan @ 2022-04-17 14:35 UTC (permalink / raw)
  To: 'Dave Taht', 'flent-users', 'bloat', starlink

Hi Dave,

This isn't specifically cake related but I figured it could be interesting, especially as these are from a location where I've been investigating setting up something with cake. These are statistics from a relatives location where they only have a DSL line from CenturyLink available to them. There's a little Debian based VM I setup for a couple random things so I was able to run flent from that. They have a EdgeRouter X serving as the router so they can at least take advantage of fq_codel through the routers Smart Queue feature. I ran a quick speed test, then ran flent without sqm, and then with sqm. Sorry if this gets a bit long and hopefully it stays readable.

#############
Speedtest by Ookla

     Server: 123.Net, Inc. - Southfield, MI (id = 15044)
        ISP: CenturyLink
    Latency:    45.45 ms   (194.34 ms jitter)
   Download:     8.88 Mbps (data used: 12.6 MB )
     Upload:     0.64 Mbps (data used: 962.7 kB )
Packet Loss:     5.1%
 Result URL: https://www.speedtest.net/result/c/799ba46b-d3d4-4d54-ac76-118fac847b73

###########
No SQM or QOS

flent -x --socket-stats --step-size=.05 -H de.starlink.taht.net -H london.starlink.taht.net -H singapore.starlink.taht.net -H fremont.starlink.taht.net -t nosqm_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl rtt_fair4be
Started Flent 2.0.0 using Python 3.9.2.
Starting rtt_fair4be test. Expected run time: 70 seconds.
Data file written to ./rtt_fair4be-2022-04-17T145553.865990.nosqm_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl.flent.gz

Summary of rtt_fair4be test run from 2022-04-17 13:55:53.865990
  Title: 'nosqm_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl'

                                                                          avg       median          # data pts
 Ping (ms) ICMP1 de.starlink.taht.net                          :      1026.06       920.00 ms             1383
 Ping (ms) ICMP2 london.starlink.taht.net                      :      1009.28       887.00 ms             1383
 Ping (ms) ICMP3 singapore.starlink.taht.net                   :      1121.09      1012.00 ms             1383
 Ping (ms) ICMP4 fremont.starlink.taht.net                     :       943.77       853.00 ms             1383
 Ping (ms) avg                                                 :      1025.05          N/A ms             1383
 TCP download BE1 de.starlink.taht.net                         :         1.36         2.00 Mbits/s        1383
 TCP download BE2 london.starlink.taht.net                     :         0.72         0.65 Mbits/s        1383
 TCP download BE3 singapore.starlink.taht.net                  :         1.03         1.74 Mbits/s        1383
 TCP download BE4 fremont.starlink.taht.net                    :         2.11         2.88 Mbits/s        1383
 TCP download avg                                              :         1.30          N/A Mbits/s        1383
 TCP download fairness                                         :         0.86          N/A Mbits/s        1383
 TCP download sum                                              :         5.22          N/A Mbits/s        1383
 TCP upload BE1 de.starlink.taht.net                           :         0.12         0.20 Mbits/s        1383
 TCP upload BE1 de.starlink.taht.net::tcp_cwnd                 :        16.85        14.00                1044
 TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate        :         0.12         0.12                1036
 TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate          :         0.35         0.26                1042
 TCP upload BE1 de.starlink.taht.net::tcp_rtt                  :      1128.57       954.47                1020
 TCP upload BE1 de.starlink.taht.net::tcp_rtt_var              :        68.22        50.08                1020
 TCP upload BE2 london.starlink.taht.net                       :         0.12         0.15 Mbits/s        1383
 TCP upload BE2 london.starlink.taht.net::tcp_cwnd             :        17.33        11.00                1046
 TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate    :         0.12         0.11                1041
 TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate      :         0.38         0.26                1044
 TCP upload BE2 london.starlink.taht.net::tcp_rtt              :      1166.74       991.48                1043
 TCP upload BE2 london.starlink.taht.net::tcp_rtt_var          :        79.99        68.27                1043
 TCP upload BE3 singapore.starlink.taht.net                    :         0.07         0.13 Mbits/s        1383
 TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd          :        13.60         9.00                1042
 TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate :         0.07         0.08                1013
 TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate   :         0.23         0.20                1031
 TCP upload BE3 singapore.starlink.taht.net::tcp_rtt           :      1302.23      1129.07                 967
 TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var       :       113.67        98.84                 967
 TCP upload BE4 fremont.starlink.taht.net                      :         0.11         0.20 Mbits/s        1383
 TCP upload BE4 fremont.starlink.taht.net::tcp_cwnd            :        17.55        11.00                1038
 TCP upload BE4 fremont.starlink.taht.net::tcp_delivery_rate   :         0.11         0.10                1035
 TCP upload BE4 fremont.starlink.taht.net::tcp_pacing_rate     :         0.32         0.23                1036
 TCP upload BE4 fremont.starlink.taht.net::tcp_rtt             :      1094.64       887.54                1033
 TCP upload BE4 fremont.starlink.taht.net::tcp_rtt_var         :        67.20        59.16                1033
 TCP upload avg                                                :         0.10          N/A Mbits/s        1383
 TCP upload fairness                                           :         0.96          N/A Mbits/s        1383
 TCP upload sum                                                :         0.42          N/A Mbits/s        1383

##########
with smart queue (fq_codel) on the EdgeRouter

flent -x --socket-stats --step-size=.05 -H de.starlink.taht.net -H london.starlink.taht.net -H singapore.starlink.taht.net -H fremont.starlink.taht.net -t fqcodel_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl rtt_fair4be
Started Flent 2.0.0 using Python 3.9.2.
Starting rtt_fair4be test. Expected run time: 70 seconds.
Data file written to ./rtt_fair4be-2022-04-17T150339.549817.fqcodel_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl.flent.gz

Summary of rtt_fair4be test run from 2022-04-17 14:03:39.549817
  Title: 'fqcodel_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl'

                                                                          avg       median          # data pts
 Ping (ms) ICMP1 de.starlink.taht.net                          :       392.93       208.00 ms             1384
 Ping (ms) ICMP2 london.starlink.taht.net                      :       373.94       196.00 ms             1384
 Ping (ms) ICMP3 singapore.starlink.taht.net                   :       508.15       319.00 ms             1384
 Ping (ms) ICMP4 fremont.starlink.taht.net                     :       346.23       152.00 ms             1384
 Ping (ms) avg                                                 :       405.31          N/A ms             1384
 TCP download BE1 de.starlink.taht.net                         :         1.36         1.62 Mbits/s        1384
 TCP download BE2 london.starlink.taht.net                     :         1.47         1.63 Mbits/s        1384
 TCP download BE3 singapore.starlink.taht.net                  :         1.55         1.60 Mbits/s        1384
 TCP download BE4 fremont.starlink.taht.net                    :         1.50         1.80 Mbits/s        1384
 TCP download avg                                              :         1.47          N/A Mbits/s        1384
 TCP download fairness                                         :         1.00          N/A Mbits/s        1384
 TCP download sum                                              :         5.88          N/A Mbits/s        1384
 TCP upload BE1 de.starlink.taht.net                           :         0.09         0.85 Mbits/s        1384
 TCP upload BE1 de.starlink.taht.net::tcp_cwnd                 :         4.50         4.00                1035
 TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate        :         0.08         0.08                1030
 TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate          :         0.17         0.15                1032
 TCP upload BE1 de.starlink.taht.net::tcp_rtt                  :       458.11       318.82                1027
 TCP upload BE1 de.starlink.taht.net::tcp_rtt_var              :        95.76        61.88                1027
 TCP upload BE2 london.starlink.taht.net                       :         0.09         0.93 Mbits/s        1384
 TCP upload BE2 london.starlink.taht.net::tcp_cwnd             :         4.17         4.00                1038
 TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate    :         0.08         0.08                1033
 TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate      :         0.17         0.14                1036
 TCP upload BE2 london.starlink.taht.net::tcp_rtt              :       452.67       322.49                1033
 TCP upload BE2 london.starlink.taht.net::tcp_rtt_var          :        96.87        64.28                1033
 TCP upload BE3 singapore.starlink.taht.net                    :         0.06         0.44 Mbits/s        1384
 TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd          :         4.16         4.00                1040
 TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate :         0.06         0.06                1031
 TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate   :         0.12         0.11                1036
 TCP upload BE3 singapore.starlink.taht.net::tcp_rtt           :       577.75       438.61                1027
 TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var       :       107.05        72.97                1027
 TCP upload BE4 fremont.starlink.taht.net                      :         0.10         1.48 Mbits/s        1384
 TCP upload BE4 fremont.starlink.taht.net::tcp_cwnd            :         4.45         4.00                1020
 TCP upload BE4 fremont.starlink.taht.net::tcp_delivery_rate   :         0.09         0.09                1017
 TCP upload BE4 fremont.starlink.taht.net::tcp_pacing_rate     :         0.21         0.16                1019
 TCP upload BE4 fremont.starlink.taht.net::tcp_rtt             :       439.05       285.86                1018
 TCP upload BE4 fremont.starlink.taht.net::tcp_rtt_var         :        81.90        56.33                1018
 TCP upload avg                                                :         0.08          N/A Mbits/s        1384
 TCP upload fairness                                           :         0.97          N/A Mbits/s        1384
 TCP upload sum                                                :         0.34          N/A Mbits/s        1384


EdgeRouter Settings

smart-queue DSL {
        download {
            ecn enable
            flows 1024
            fq-quantum 300
            limit 10240
            rate 8mbit
        }
        upload {
            ecn enable
            flows 1024
            fq-quantum 300
            limit 10240
            rate 600kbit
            target 15ms
        }
        wan-interface pppoe0

Also, here's the VMs qdisc settings. Which I now realize probably aren't the best

sysctl -a | grep qdisc
net.core.default_qdisc = pfifo_fast

tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc mq 0: dev eth0 root
 Sent 628201 bytes 8643 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc pfifo_fast 0: dev eth0 parent :1 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sent 628201 bytes 8643 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0



-Jordan

-----Original Message-----
From: Starlink <starlink-bounces@lists.bufferbloat.net> On Behalf Of Dave Taht
Sent: Saturday, April 16, 2022 9:40 PM
To: flent-users <flent-users@flent.org>; bloat <bloat@lists.bufferbloat.net>; starlink@lists.bufferbloat.net
Subject: [Starlink] an rtt_fair test?

if anyone has a bit of spare time this holiday, I'd like to collect a few results from various networks around the web this week, not just on the turris. Example command line and outputs here:

https://forum.turris.cz/t/sqm-on-turris-flent-benchmarks/17048/

--
I tried to build a better future, a few times:
https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org

Dave Täht CEO, TekLibre, LLC
_______________________________________________
Starlink mailing list
Starlink@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/starlink


^ permalink raw reply	[flat|nested] 6+ messages in thread

* [Flent-users] Re: [Starlink] an rtt_fair test?
  2022-04-17 14:35 ` [Flent-users] Re: [Starlink] " jordan
@ 2022-04-17 14:56   ` Dave Taht
  2022-04-17 15:36     ` jordan
  0 siblings, 1 reply; 6+ messages in thread
From: Dave Taht @ 2022-04-17 14:56 UTC (permalink / raw)
  To: Jordan Szuch; +Cc: flent-users, bloat, starlink

Thank you for testing!!! If you could stash the *.flent.gz files
somewhere I could get at, that would be great. You can also browse the
plots using the flent-gui to see more detail over time.

Cake, long available as an out of tree build for the edgerouter, has
both dsl compensation and an ack-filter which would help A LOT, in
your case. i wish I'd pushed harder to get dsl compensation into the
edgerouter smart queues code...

https://community.ui.com/questions/Cake-compiled-for-the-EdgeRouter-devices/fc1ff27c-f321-4344-8737-fcc755cae8a2

Given your 500kbit upload speed the minimum latency inflation under
the structure of this test (4 upload flows) should have been around
150ms (4 * 25ms + ack overhead), but without looking at the *.gz files
I don't know what your baseline latency was to any of these sites. (it
would be good if flent reported on the text output what the minimum
actually was)

On Sun, Apr 17, 2022 at 7:35 AM <jordan@inacomptc.com> wrote:
>
> Hi Dave,
>
> This isn't specifically cake related but I figured it could be interesting, especially as these are from a location where I've been investigating setting up something with cake. These are statistics from a relatives location where they only have a DSL line from CenturyLink available to them. There's a little Debian based VM I setup for a couple random things so I was able to run flent from that. They have a EdgeRouter X serving as the router so they can at least take advantage of fq_codel through the routers Smart Queue feature. I ran a quick speed test, then ran flent without sqm, and then with sqm. Sorry if this gets a bit long and hopefully it stays readable.
>
> #############
> Speedtest by Ookla
>
>      Server: 123.Net, Inc. - Southfield, MI (id = 15044)
>         ISP: CenturyLink
>     Latency:    45.45 ms   (194.34 ms jitter)
>    Download:     8.88 Mbps (data used: 12.6 MB )
>      Upload:     0.64 Mbps (data used: 962.7 kB )
> Packet Loss:     5.1%
>  Result URL: https://www.speedtest.net/result/c/799ba46b-d3d4-4d54-ac76-118fac847b73
>
> ###########
> No SQM or QOS
>
> flent -x --socket-stats --step-size=.05 -H de.starlink.taht.net -H london.starlink.taht.net -H singapore.starlink.taht.net -H fremont.starlink.taht.net -t nosqm_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl rtt_fair4be
> Started Flent 2.0.0 using Python 3.9.2.
> Starting rtt_fair4be test. Expected run time: 70 seconds.
> Data file written to ./rtt_fair4be-2022-04-17T145553.865990.nosqm_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl.flent.gz
>
> Summary of rtt_fair4be test run from 2022-04-17 13:55:53.865990
>   Title: 'nosqm_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl'
>
>                                                                           avg       median          # data pts
>  Ping (ms) ICMP1 de.starlink.taht.net                          :      1026.06       920.00 ms             1383
>  Ping (ms) ICMP2 london.starlink.taht.net                      :      1009.28       887.00 ms             1383
>  Ping (ms) ICMP3 singapore.starlink.taht.net                   :      1121.09      1012.00 ms             1383
>  Ping (ms) ICMP4 fremont.starlink.taht.net                     :       943.77       853.00 ms             1383
>  Ping (ms) avg                                                 :      1025.05          N/A ms             1383
>  TCP download BE1 de.starlink.taht.net                         :         1.36         2.00 Mbits/s        1383
>  TCP download BE2 london.starlink.taht.net                     :         0.72         0.65 Mbits/s        1383
>  TCP download BE3 singapore.starlink.taht.net                  :         1.03         1.74 Mbits/s        1383
>  TCP download BE4 fremont.starlink.taht.net                    :         2.11         2.88 Mbits/s        1383
>  TCP download avg                                              :         1.30          N/A Mbits/s        1383
>  TCP download fairness                                         :         0.86          N/A Mbits/s        1383
>  TCP download sum                                              :         5.22          N/A Mbits/s        1383
>  TCP upload BE1 de.starlink.taht.net                           :         0.12         0.20 Mbits/s        1383
>  TCP upload BE1 de.starlink.taht.net::tcp_cwnd                 :        16.85        14.00                1044
>  TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate        :         0.12         0.12                1036
>  TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate          :         0.35         0.26                1042
>  TCP upload BE1 de.starlink.taht.net::tcp_rtt                  :      1128.57       954.47                1020
>  TCP upload BE1 de.starlink.taht.net::tcp_rtt_var              :        68.22        50.08                1020
>  TCP upload BE2 london.starlink.taht.net                       :         0.12         0.15 Mbits/s        1383
>  TCP upload BE2 london.starlink.taht.net::tcp_cwnd             :        17.33        11.00                1046
>  TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate    :         0.12         0.11                1041
>  TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate      :         0.38         0.26                1044
>  TCP upload BE2 london.starlink.taht.net::tcp_rtt              :      1166.74       991.48                1043
>  TCP upload BE2 london.starlink.taht.net::tcp_rtt_var          :        79.99        68.27                1043
>  TCP upload BE3 singapore.starlink.taht.net                    :         0.07         0.13 Mbits/s        1383
>  TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd          :        13.60         9.00                1042
>  TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate :         0.07         0.08                1013
>  TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate   :         0.23         0.20                1031
>  TCP upload BE3 singapore.starlink.taht.net::tcp_rtt           :      1302.23      1129.07                 967
>  TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var       :       113.67        98.84                 967
>  TCP upload BE4 fremont.starlink.taht.net                      :         0.11         0.20 Mbits/s        1383
>  TCP upload BE4 fremont.starlink.taht.net::tcp_cwnd            :        17.55        11.00                1038
>  TCP upload BE4 fremont.starlink.taht.net::tcp_delivery_rate   :         0.11         0.10                1035
>  TCP upload BE4 fremont.starlink.taht.net::tcp_pacing_rate     :         0.32         0.23                1036
>  TCP upload BE4 fremont.starlink.taht.net::tcp_rtt             :      1094.64       887.54                1033
>  TCP upload BE4 fremont.starlink.taht.net::tcp_rtt_var         :        67.20        59.16                1033
>  TCP upload avg                                                :         0.10          N/A Mbits/s        1383
>  TCP upload fairness                                           :         0.96          N/A Mbits/s        1383
>  TCP upload sum                                                :         0.42          N/A Mbits/s        1383
>
> ##########
> with smart queue (fq_codel) on the EdgeRouter
>
> flent -x --socket-stats --step-size=.05 -H de.starlink.taht.net -H london.starlink.taht.net -H singapore.starlink.taht.net -H fremont.starlink.taht.net -t fqcodel_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl rtt_fair4be
> Started Flent 2.0.0 using Python 3.9.2.
> Starting rtt_fair4be test. Expected run time: 70 seconds.
> Data file written to ./rtt_fair4be-2022-04-17T150339.549817.fqcodel_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl.flent.gz
>
> Summary of rtt_fair4be test run from 2022-04-17 14:03:39.549817
>   Title: 'fqcodel_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl'
>
>                                                                           avg       median          # data pts
>  Ping (ms) ICMP1 de.starlink.taht.net                          :       392.93       208.00 ms             1384
>  Ping (ms) ICMP2 london.starlink.taht.net                      :       373.94       196.00 ms             1384
>  Ping (ms) ICMP3 singapore.starlink.taht.net                   :       508.15       319.00 ms             1384
>  Ping (ms) ICMP4 fremont.starlink.taht.net                     :       346.23       152.00 ms             1384
>  Ping (ms) avg                                                 :       405.31          N/A ms             1384
>  TCP download BE1 de.starlink.taht.net                         :         1.36         1.62 Mbits/s        1384
>  TCP download BE2 london.starlink.taht.net                     :         1.47         1.63 Mbits/s        1384
>  TCP download BE3 singapore.starlink.taht.net                  :         1.55         1.60 Mbits/s        1384
>  TCP download BE4 fremont.starlink.taht.net                    :         1.50         1.80 Mbits/s        1384
>  TCP download avg                                              :         1.47          N/A Mbits/s        1384
>  TCP download fairness                                         :         1.00          N/A Mbits/s        1384
>  TCP download sum                                              :         5.88          N/A Mbits/s        1384
>  TCP upload BE1 de.starlink.taht.net                           :         0.09         0.85 Mbits/s        1384
>  TCP upload BE1 de.starlink.taht.net::tcp_cwnd                 :         4.50         4.00                1035
>  TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate        :         0.08         0.08                1030
>  TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate          :         0.17         0.15                1032
>  TCP upload BE1 de.starlink.taht.net::tcp_rtt                  :       458.11       318.82                1027
>  TCP upload BE1 de.starlink.taht.net::tcp_rtt_var              :        95.76        61.88                1027
>  TCP upload BE2 london.starlink.taht.net                       :         0.09         0.93 Mbits/s        1384
>  TCP upload BE2 london.starlink.taht.net::tcp_cwnd             :         4.17         4.00                1038
>  TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate    :         0.08         0.08                1033
>  TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate      :         0.17         0.14                1036
>  TCP upload BE2 london.starlink.taht.net::tcp_rtt              :       452.67       322.49                1033
>  TCP upload BE2 london.starlink.taht.net::tcp_rtt_var          :        96.87        64.28                1033
>  TCP upload BE3 singapore.starlink.taht.net                    :         0.06         0.44 Mbits/s        1384
>  TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd          :         4.16         4.00                1040
>  TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate :         0.06         0.06                1031
>  TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate   :         0.12         0.11                1036
>  TCP upload BE3 singapore.starlink.taht.net::tcp_rtt           :       577.75       438.61                1027
>  TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var       :       107.05        72.97                1027
>  TCP upload BE4 fremont.starlink.taht.net                      :         0.10         1.48 Mbits/s        1384
>  TCP upload BE4 fremont.starlink.taht.net::tcp_cwnd            :         4.45         4.00                1020
>  TCP upload BE4 fremont.starlink.taht.net::tcp_delivery_rate   :         0.09         0.09                1017
>  TCP upload BE4 fremont.starlink.taht.net::tcp_pacing_rate     :         0.21         0.16                1019
>  TCP upload BE4 fremont.starlink.taht.net::tcp_rtt             :       439.05       285.86                1018
>  TCP upload BE4 fremont.starlink.taht.net::tcp_rtt_var         :        81.90        56.33                1018
>  TCP upload avg                                                :         0.08          N/A Mbits/s        1384
>  TCP upload fairness                                           :         0.97          N/A Mbits/s        1384
>  TCP upload sum                                                :         0.34          N/A Mbits/s        1384
>
>
> EdgeRouter Settings
>
> smart-queue DSL {
>         download {
>             ecn enable
>             flows 1024
>             fq-quantum 300
>             limit 10240
>             rate 8mbit
>         }
>         upload {
>             ecn enable
>             flows 1024
>             fq-quantum 300
>             limit 10240
>             rate 600kbit
>             target 15ms
>         }
>         wan-interface pppoe0
>
> Also, here's the VMs qdisc settings. Which I now realize probably aren't the best
>
> sysctl -a | grep qdisc
> net.core.default_qdisc = pfifo_fast
>
> tc -s qdisc
> qdisc noqueue 0: dev lo root refcnt 2
>  Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>  backlog 0b 0p requeues 0
> qdisc mq 0: dev eth0 root
>  Sent 628201 bytes 8643 pkt (dropped 0, overlimits 0 requeues 0)
>  backlog 0b 0p requeues 0
> qdisc pfifo_fast 0: dev eth0 parent :1 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
>  Sent 628201 bytes 8643 pkt (dropped 0, overlimits 0 requeues 0)
>  backlog 0b 0p requeues 0
>
>
>
> -Jordan
>
> -----Original Message-----
> From: Starlink <starlink-bounces@lists.bufferbloat.net> On Behalf Of Dave Taht
> Sent: Saturday, April 16, 2022 9:40 PM
> To: flent-users <flent-users@flent.org>; bloat <bloat@lists.bufferbloat.net>; starlink@lists.bufferbloat.net
> Subject: [Starlink] an rtt_fair test?
>
> if anyone has a bit of spare time this holiday, I'd like to collect a few results from various networks around the web this week, not just on the turris. Example command line and outputs here:
>
> https://forum.turris.cz/t/sqm-on-turris-flent-benchmarks/17048/
>
> --
> I tried to build a better future, a few times:
> https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org
>
> Dave Täht CEO, TekLibre, LLC
> _______________________________________________
> Starlink mailing list
> Starlink@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/starlink
>


-- 
I tried to build a better future, a few times:
https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org

Dave Täht CEO, TekLibre, LLC

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [Flent-users] Re: [Starlink] an rtt_fair test?
  2022-04-17 14:56   ` Dave Taht
@ 2022-04-17 15:36     ` jordan
  2022-04-17 16:54       ` [Flent-users] Re: [Bloat] " Daniel Sterling
  2022-04-18  0:22       ` [Flent-users] " Dave Taht
  0 siblings, 2 replies; 6+ messages in thread
From: jordan @ 2022-04-17 15:36 UTC (permalink / raw)
  To: 'Dave Taht'; +Cc: 'flent-users', 'bloat', starlink

You're welcome! Here's the files Flent generated: https://1drv.ms/u/s!Ap4u4Rte63FqjqoSQXX444OWLvl8FQ?e=f22Nt7

And thanks for the link! It honestly doesn't look too bad to implement so I might give it a shot today if time allows.

I've also been contemplating replacing it with something like a Mikrotik since cake is now integrated into them. I've been testing one at home and made a post with a bunch of flent tests to the cake thread in their forum if you're interested in some data from a Charter Spectrum cable connection: https://forum.mikrotik.com/viewtopic.php?t=179307#p925485

The OneDrive link to my flent files should still be active in that post as well.

-Jordan

-----Original Message-----
From: Dave Taht <dave.taht@gmail.com> 
Sent: Sunday, April 17, 2022 10:57 AM
To: Jordan Szuch <jordan@inacomptc.com>
Cc: flent-users <flent-users@flent.org>; bloat <bloat@lists.bufferbloat.net>; starlink@lists.bufferbloat.net
Subject: Re: [Starlink] an rtt_fair test?

Thank you for testing!!! If you could stash the *.flent.gz files somewhere I could get at, that would be great. You can also browse the plots using the flent-gui to see more detail over time.

Cake, long available as an out of tree build for the edgerouter, has both dsl compensation and an ack-filter which would help A LOT, in your case. i wish I'd pushed harder to get dsl compensation into the edgerouter smart queues code...

https://community.ui.com/questions/Cake-compiled-for-the-EdgeRouter-devices/fc1ff27c-f321-4344-8737-fcc755cae8a2

Given your 500kbit upload speed the minimum latency inflation under the structure of this test (4 upload flows) should have been around 150ms (4 * 25ms + ack overhead), but without looking at the *.gz files I don't know what your baseline latency was to any of these sites. (it would be good if flent reported on the text output what the minimum actually was)

On Sun, Apr 17, 2022 at 7:35 AM <jordan@inacomptc.com> wrote:
>
> Hi Dave,
>
> This isn't specifically cake related but I figured it could be interesting, especially as these are from a location where I've been investigating setting up something with cake. These are statistics from a relatives location where they only have a DSL line from CenturyLink available to them. There's a little Debian based VM I setup for a couple random things so I was able to run flent from that. They have a EdgeRouter X serving as the router so they can at least take advantage of fq_codel through the routers Smart Queue feature. I ran a quick speed test, then ran flent without sqm, and then with sqm. Sorry if this gets a bit long and hopefully it stays readable.
>
> #############
> Speedtest by Ookla
>
>      Server: 123.Net, Inc. - Southfield, MI (id = 15044)
>         ISP: CenturyLink
>     Latency:    45.45 ms   (194.34 ms jitter)
>    Download:     8.88 Mbps (data used: 12.6 MB )
>      Upload:     0.64 Mbps (data used: 962.7 kB )
> Packet Loss:     5.1%
>  Result URL: 
> https://www.speedtest.net/result/c/799ba46b-d3d4-4d54-ac76-118fac847b7
> 3
>
> ###########
> No SQM or QOS
>
> flent -x --socket-stats --step-size=.05 -H de.starlink.taht.net -H 
> london.starlink.taht.net -H singapore.starlink.taht.net -H fremont.starlink.taht.net -t nosqm_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl rtt_fair4be Started Flent 2.0.0 using Python 3.9.2.
> Starting rtt_fair4be test. Expected run time: 70 seconds.
> Data file written to 
> ./rtt_fair4be-2022-04-17T145553.865990.nosqm_erx_rttfair4be_8Mbpsdown_
> 500Kbpsup_centurylinkdsl.flent.gz
>
> Summary of rtt_fair4be test run from 2022-04-17 13:55:53.865990
>   Title: 'nosqm_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl'
>
>                                                                           avg       median          # data pts
>  Ping (ms) ICMP1 de.starlink.taht.net                          :      1026.06       920.00 ms             1383
>  Ping (ms) ICMP2 london.starlink.taht.net                      :      1009.28       887.00 ms             1383
>  Ping (ms) ICMP3 singapore.starlink.taht.net                   :      1121.09      1012.00 ms             1383
>  Ping (ms) ICMP4 fremont.starlink.taht.net                     :       943.77       853.00 ms             1383
>  Ping (ms) avg                                                 :      1025.05          N/A ms             1383
>  TCP download BE1 de.starlink.taht.net                         :         1.36         2.00 Mbits/s        1383
>  TCP download BE2 london.starlink.taht.net                     :         0.72         0.65 Mbits/s        1383
>  TCP download BE3 singapore.starlink.taht.net                  :         1.03         1.74 Mbits/s        1383
>  TCP download BE4 fremont.starlink.taht.net                    :         2.11         2.88 Mbits/s        1383
>  TCP download avg                                              :         1.30          N/A Mbits/s        1383
>  TCP download fairness                                         :         0.86          N/A Mbits/s        1383
>  TCP download sum                                              :         5.22          N/A Mbits/s        1383
>  TCP upload BE1 de.starlink.taht.net                           :         0.12         0.20 Mbits/s        1383
>  TCP upload BE1 de.starlink.taht.net::tcp_cwnd                 :        16.85        14.00                1044
>  TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate        :         0.12         0.12                1036
>  TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate          :         0.35         0.26                1042
>  TCP upload BE1 de.starlink.taht.net::tcp_rtt                  :      1128.57       954.47                1020
>  TCP upload BE1 de.starlink.taht.net::tcp_rtt_var              :        68.22        50.08                1020
>  TCP upload BE2 london.starlink.taht.net                       :         0.12         0.15 Mbits/s        1383
>  TCP upload BE2 london.starlink.taht.net::tcp_cwnd             :        17.33        11.00                1046
>  TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate    :         0.12         0.11                1041
>  TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate      :         0.38         0.26                1044
>  TCP upload BE2 london.starlink.taht.net::tcp_rtt              :      1166.74       991.48                1043
>  TCP upload BE2 london.starlink.taht.net::tcp_rtt_var          :        79.99        68.27                1043
>  TCP upload BE3 singapore.starlink.taht.net                    :         0.07         0.13 Mbits/s        1383
>  TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd          :        13.60         9.00                1042
>  TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate :         0.07         0.08                1013
>  TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate   :         0.23         0.20                1031
>  TCP upload BE3 singapore.starlink.taht.net::tcp_rtt           :      1302.23      1129.07                 967
>  TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var       :       113.67        98.84                 967
>  TCP upload BE4 fremont.starlink.taht.net                      :         0.11         0.20 Mbits/s        1383
>  TCP upload BE4 fremont.starlink.taht.net::tcp_cwnd            :        17.55        11.00                1038
>  TCP upload BE4 fremont.starlink.taht.net::tcp_delivery_rate   :         0.11         0.10                1035
>  TCP upload BE4 fremont.starlink.taht.net::tcp_pacing_rate     :         0.32         0.23                1036
>  TCP upload BE4 fremont.starlink.taht.net::tcp_rtt             :      1094.64       887.54                1033
>  TCP upload BE4 fremont.starlink.taht.net::tcp_rtt_var         :        67.20        59.16                1033
>  TCP upload avg                                                :         0.10          N/A Mbits/s        1383
>  TCP upload fairness                                           :         0.96          N/A Mbits/s        1383
>  TCP upload sum                                                :         0.42          N/A Mbits/s        1383
>
> ##########
> with smart queue (fq_codel) on the EdgeRouter
>
> flent -x --socket-stats --step-size=.05 -H de.starlink.taht.net -H 
> london.starlink.taht.net -H singapore.starlink.taht.net -H fremont.starlink.taht.net -t fqcodel_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl rtt_fair4be Started Flent 2.0.0 using Python 3.9.2.
> Starting rtt_fair4be test. Expected run time: 70 seconds.
> Data file written to 
> ./rtt_fair4be-2022-04-17T150339.549817.fqcodel_erx_rttfair4be_8Mbpsdow
> n_500Kbpsup_centurylinkdsl.flent.gz
>
> Summary of rtt_fair4be test run from 2022-04-17 14:03:39.549817
>   Title: 'fqcodel_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl'
>
>                                                                           avg       median          # data pts
>  Ping (ms) ICMP1 de.starlink.taht.net                          :       392.93       208.00 ms             1384
>  Ping (ms) ICMP2 london.starlink.taht.net                      :       373.94       196.00 ms             1384
>  Ping (ms) ICMP3 singapore.starlink.taht.net                   :       508.15       319.00 ms             1384
>  Ping (ms) ICMP4 fremont.starlink.taht.net                     :       346.23       152.00 ms             1384
>  Ping (ms) avg                                                 :       405.31          N/A ms             1384
>  TCP download BE1 de.starlink.taht.net                         :         1.36         1.62 Mbits/s        1384
>  TCP download BE2 london.starlink.taht.net                     :         1.47         1.63 Mbits/s        1384
>  TCP download BE3 singapore.starlink.taht.net                  :         1.55         1.60 Mbits/s        1384
>  TCP download BE4 fremont.starlink.taht.net                    :         1.50         1.80 Mbits/s        1384
>  TCP download avg                                              :         1.47          N/A Mbits/s        1384
>  TCP download fairness                                         :         1.00          N/A Mbits/s        1384
>  TCP download sum                                              :         5.88          N/A Mbits/s        1384
>  TCP upload BE1 de.starlink.taht.net                           :         0.09         0.85 Mbits/s        1384
>  TCP upload BE1 de.starlink.taht.net::tcp_cwnd                 :         4.50         4.00                1035
>  TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate        :         0.08         0.08                1030
>  TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate          :         0.17         0.15                1032
>  TCP upload BE1 de.starlink.taht.net::tcp_rtt                  :       458.11       318.82                1027
>  TCP upload BE1 de.starlink.taht.net::tcp_rtt_var              :        95.76        61.88                1027
>  TCP upload BE2 london.starlink.taht.net                       :         0.09         0.93 Mbits/s        1384
>  TCP upload BE2 london.starlink.taht.net::tcp_cwnd             :         4.17         4.00                1038
>  TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate    :         0.08         0.08                1033
>  TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate      :         0.17         0.14                1036
>  TCP upload BE2 london.starlink.taht.net::tcp_rtt              :       452.67       322.49                1033
>  TCP upload BE2 london.starlink.taht.net::tcp_rtt_var          :        96.87        64.28                1033
>  TCP upload BE3 singapore.starlink.taht.net                    :         0.06         0.44 Mbits/s        1384
>  TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd          :         4.16         4.00                1040
>  TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate :         0.06         0.06                1031
>  TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate   :         0.12         0.11                1036
>  TCP upload BE3 singapore.starlink.taht.net::tcp_rtt           :       577.75       438.61                1027
>  TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var       :       107.05        72.97                1027
>  TCP upload BE4 fremont.starlink.taht.net                      :         0.10         1.48 Mbits/s        1384
>  TCP upload BE4 fremont.starlink.taht.net::tcp_cwnd            :         4.45         4.00                1020
>  TCP upload BE4 fremont.starlink.taht.net::tcp_delivery_rate   :         0.09         0.09                1017
>  TCP upload BE4 fremont.starlink.taht.net::tcp_pacing_rate     :         0.21         0.16                1019
>  TCP upload BE4 fremont.starlink.taht.net::tcp_rtt             :       439.05       285.86                1018
>  TCP upload BE4 fremont.starlink.taht.net::tcp_rtt_var         :        81.90        56.33                1018
>  TCP upload avg                                                :         0.08          N/A Mbits/s        1384
>  TCP upload fairness                                           :         0.97          N/A Mbits/s        1384
>  TCP upload sum                                                :         0.34          N/A Mbits/s        1384
>
>
> EdgeRouter Settings
>
> smart-queue DSL {
>         download {
>             ecn enable
>             flows 1024
>             fq-quantum 300
>             limit 10240
>             rate 8mbit
>         }
>         upload {
>             ecn enable
>             flows 1024
>             fq-quantum 300
>             limit 10240
>             rate 600kbit
>             target 15ms
>         }
>         wan-interface pppoe0
>
> Also, here's the VMs qdisc settings. Which I now realize probably 
> aren't the best
>
> sysctl -a | grep qdisc
> net.core.default_qdisc = pfifo_fast
>
> tc -s qdisc
> qdisc noqueue 0: dev lo root refcnt 2
>  Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)  backlog 0b 
> 0p requeues 0 qdisc mq 0: dev eth0 root  Sent 628201 bytes 8643 pkt 
> (dropped 0, overlimits 0 requeues 0)  backlog 0b 0p requeues 0 qdisc 
> pfifo_fast 0: dev eth0 parent :1 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 
> 1 1 1 1 1  Sent 628201 bytes 8643 pkt (dropped 0, overlimits 0 
> requeues 0)  backlog 0b 0p requeues 0
>
>
>
> -Jordan
>
> -----Original Message-----
> From: Starlink <starlink-bounces@lists.bufferbloat.net> On Behalf Of 
> Dave Taht
> Sent: Saturday, April 16, 2022 9:40 PM
> To: flent-users <flent-users@flent.org>; bloat 
> <bloat@lists.bufferbloat.net>; starlink@lists.bufferbloat.net
> Subject: [Starlink] an rtt_fair test?
>
> if anyone has a bit of spare time this holiday, I'd like to collect a few results from various networks around the web this week, not just on the turris. Example command line and outputs here:
>
> https://forum.turris.cz/t/sqm-on-turris-flent-benchmarks/17048/
>
> --
> I tried to build a better future, a few times:
> https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org
>
> Dave Täht CEO, TekLibre, LLC
> _______________________________________________
> Starlink mailing list
> Starlink@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/starlink
>


--
I tried to build a better future, a few times:
https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org

Dave Täht CEO, TekLibre, LLC


^ permalink raw reply	[flat|nested] 6+ messages in thread

* [Flent-users] Re: [Bloat] [Starlink] an rtt_fair test?
  2022-04-17 15:36     ` jordan
@ 2022-04-17 16:54       ` Daniel Sterling
  2022-04-18  0:22       ` [Flent-users] " Dave Taht
  1 sibling, 0 replies; 6+ messages in thread
From: Daniel Sterling @ 2022-04-17 16:54 UTC (permalink / raw)
  To: jordan; +Cc: Dave Taht, starlink, flent-users, bloat

[-- Attachment #1: Type: text/plain, Size: 20949 bytes --]

test from my network running customized openwrt w/ cake

tar file here:

 rtt_fair4be-2022-04-17T124830.462924.dss-cake.f...
<https://drive.google.com/file/d/1MzuHQfEsl7xQFXe0F7n506KYhRAE4jXI/view?usp=drive_web>


root@DESKTOP-DLUP89P:~# flent -x --socket-stats --step-size=.05 -H
de.starlink.taht.net -H london.starlink.taht.net -H s
ingapore.starlink.taht.net -H fremont.starlink.taht.net -t 'dss-cake'
rtt_fair4be
Started Flent 1.3.2 using Python 3.8.10.
Starting rtt_fair4be test. Expected run time: 70 seconds.
Data file written to
./rtt_fair4be-2022-04-17T124830.462924.dss-cake.flent.gz.

Summary of rtt_fair4be test run from 2022-04-17 16:48:30.462924
  Title: 'dss-cake'


avg       median          # data pts
 Ping (ms) ICMP1 de.starlink.taht.net                          :
119.54       114.00 ms             1398
 Ping (ms) ICMP2 london.starlink.taht.net                      :
107.97       104.00 ms             1398
 Ping (ms) ICMP3 singapore.starlink.taht.net                   :
272.62       268.00 ms             1398
 Ping (ms) ICMP4 fremont.starlink.taht.net                     :
 89.15        84.60 ms             1398
 Ping (ms) avg                                                 :
147.32          N/A ms             1398
 TCP download BE1 de.starlink.taht.net                         :
 25.01        24.81 Mbits/s        1398
 TCP download BE2 london.starlink.taht.net                     :
 23.00        22.75 Mbits/s        1398
 TCP download BE3 singapore.starlink.taht.net                  :
 20.26        21.01 Mbits/s        1398
 TCP download BE4 fremont.starlink.taht.net                    :
 35.64        32.94 Mbits/s        1398
 TCP download avg                                              :
 25.98          N/A Mbits/s        1398
 TCP download fairness                                         :
0.95          N/A Mbits/s        1398
 TCP download sum                                              :
103.91          N/A Mbits/s        1398
 TCP upload BE1 de.starlink.taht.net                           :
7.76         8.00 Mbits/s        1398
 TCP upload BE1 de.starlink.taht.net::tcp_cwnd                 :
 88.38        91.00                1067
 TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate        :
7.52         7.57                1064
 TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate          :
9.82         9.73                1066
 TCP upload BE1 de.starlink.taht.net::tcp_rtt                  :
130.68       125.68                1062
 TCP upload BE1 de.starlink.taht.net::tcp_rtt_var              :
5.38         4.25                1062
 TCP upload BE2 london.starlink.taht.net                       :
7.53         7.91 Mbits/s        1398
 TCP upload BE2 london.starlink.taht.net::tcp_cwnd             :
 78.47        81.00                1059
 TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate    :
7.26         7.34                1056
 TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate      :
9.48         9.45                1057
 TCP upload BE2 london.starlink.taht.net::tcp_rtt              :
120.05       114.79                1056
 TCP upload BE2 london.starlink.taht.net::tcp_rtt_var          :
5.25         4.15                1056
 TCP upload BE3 singapore.starlink.taht.net                    :
3.98         4.23 Mbits/s        1398
 TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd          :
100.96       105.00                1066
 TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate :
3.95         4.10                1056
 TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate   :
5.07         5.15                1061
 TCP upload BE3 singapore.starlink.taht.net::tcp_rtt           :
291.42       286.12                1060
 TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var       :
7.62         5.64                1060
 TCP upload BE4 fremont.starlink.taht.net                      :
8.41         8.56 Mbits/s        1398
 TCP upload BE4 fremont.starlink.taht.net::tcp_cwnd            :
 75.40        73.00                1060
 TCP upload BE4 fremont.starlink.taht.net::tcp_delivery_rate   :
8.04         8.00                1057
 TCP upload BE4 fremont.starlink.taht.net::tcp_pacing_rate     :
 10.62        10.31                1058
 TCP upload BE4 fremont.starlink.taht.net::tcp_rtt             :
103.74        98.50                1057
 TCP upload BE4 fremont.starlink.taht.net::tcp_rtt_var         :
4.97         3.96                1057
 TCP upload avg                                                :
6.92          N/A Mbits/s        1398
 TCP upload fairness                                           :
0.94          N/A Mbits/s        1398
 TCP upload sum                                                :
 27.68          N/A Mbits/s        1398
root@DESKTOP-DLUP89P:~#


On Sun, Apr 17, 2022 at 11:36 AM jordan--- via Bloat <
bloat@lists.bufferbloat.net> wrote:

> You're welcome! Here's the files Flent generated:
> https://1drv.ms/u/s!Ap4u4Rte63FqjqoSQXX444OWLvl8FQ?e=f22Nt7
>
> And thanks for the link! It honestly doesn't look too bad to implement so
> I might give it a shot today if time allows.
>
> I've also been contemplating replacing it with something like a Mikrotik
> since cake is now integrated into them. I've been testing one at home and
> made a post with a bunch of flent tests to the cake thread in their forum
> if you're interested in some data from a Charter Spectrum cable connection:
> https://forum.mikrotik.com/viewtopic.php?t=179307#p925485
>
> The OneDrive link to my flent files should still be active in that post as
> well.
>
> -Jordan
>
> -----Original Message-----
> From: Dave Taht <dave.taht@gmail.com>
> Sent: Sunday, April 17, 2022 10:57 AM
> To: Jordan Szuch <jordan@inacomptc.com>
> Cc: flent-users <flent-users@flent.org>; bloat <
> bloat@lists.bufferbloat.net>; starlink@lists.bufferbloat.net
> Subject: Re: [Starlink] an rtt_fair test?
>
> Thank you for testing!!! If you could stash the *.flent.gz files somewhere
> I could get at, that would be great. You can also browse the plots using
> the flent-gui to see more detail over time.
>
> Cake, long available as an out of tree build for the edgerouter, has both
> dsl compensation and an ack-filter which would help A LOT, in your case. i
> wish I'd pushed harder to get dsl compensation into the edgerouter smart
> queues code...
>
>
> https://community.ui.com/questions/Cake-compiled-for-the-EdgeRouter-devices/fc1ff27c-f321-4344-8737-fcc755cae8a2
>
> Given your 500kbit upload speed the minimum latency inflation under the
> structure of this test (4 upload flows) should have been around 150ms (4 *
> 25ms + ack overhead), but without looking at the *.gz files I don't know
> what your baseline latency was to any of these sites. (it would be good if
> flent reported on the text output what the minimum actually was)
>
> On Sun, Apr 17, 2022 at 7:35 AM <jordan@inacomptc.com> wrote:
> >
> > Hi Dave,
> >
> > This isn't specifically cake related but I figured it could be
> interesting, especially as these are from a location where I've been
> investigating setting up something with cake. These are statistics from a
> relatives location where they only have a DSL line from CenturyLink
> available to them. There's a little Debian based VM I setup for a couple
> random things so I was able to run flent from that. They have a EdgeRouter
> X serving as the router so they can at least take advantage of fq_codel
> through the routers Smart Queue feature. I ran a quick speed test, then ran
> flent without sqm, and then with sqm. Sorry if this gets a bit long and
> hopefully it stays readable.
> >
> > #############
> > Speedtest by Ookla
> >
> >      Server: 123.Net, Inc. - Southfield, MI (id = 15044)
> >         ISP: CenturyLink
> >     Latency:    45.45 ms   (194.34 ms jitter)
> >    Download:     8.88 Mbps (data used: 12.6 MB )
> >      Upload:     0.64 Mbps (data used: 962.7 kB )
> > Packet Loss:     5.1%
> >  Result URL:
> > https://www.speedtest.net/result/c/799ba46b-d3d4-4d54-ac76-118fac847b7
> > 3
> >
> > ###########
> > No SQM or QOS
> >
> > flent -x --socket-stats --step-size=.05 -H de.starlink.taht.net -H
> > london.starlink.taht.net -H singapore.starlink.taht.net -H
> fremont.starlink.taht.net -t
> nosqm_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl rtt_fair4be Started
> Flent 2.0.0 using Python 3.9.2.
> > Starting rtt_fair4be test. Expected run time: 70 seconds.
> > Data file written to
> > ./rtt_fair4be-2022-04-17T145553.865990.nosqm_erx_rttfair4be_8Mbpsdown_
> > 500Kbpsup_centurylinkdsl.flent.gz
> >
> > Summary of rtt_fair4be test run from 2022-04-17 13:55:53.865990
> >   Title: 'nosqm_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl'
> >
> >
>  avg       median          # data pts
> >  Ping (ms) ICMP1 de.starlink.taht.net                          :
> 1026.06       920.00 ms             1383
> >  Ping (ms) ICMP2 london.starlink.taht.net                      :
> 1009.28       887.00 ms             1383
> >  Ping (ms) ICMP3 singapore.starlink.taht.net                   :
> 1121.09      1012.00 ms             1383
> >  Ping (ms) ICMP4 fremont.starlink.taht.net                     :
>  943.77       853.00 ms             1383
> >  Ping (ms) avg                                                 :
> 1025.05          N/A ms             1383
> >  TCP download BE1 de.starlink.taht.net                         :
>  1.36         2.00 Mbits/s        1383
> >  TCP download BE2 london.starlink.taht.net                     :
>  0.72         0.65 Mbits/s        1383
> >  TCP download BE3 singapore.starlink.taht.net                  :
>  1.03         1.74 Mbits/s        1383
> >  TCP download BE4 fremont.starlink.taht.net                    :
>  2.11         2.88 Mbits/s        1383
> >  TCP download avg                                              :
>  1.30          N/A Mbits/s        1383
> >  TCP download fairness                                         :
>  0.86          N/A Mbits/s        1383
> >  TCP download sum                                              :
>  5.22          N/A Mbits/s        1383
> >  TCP upload BE1 de.starlink.taht.net                           :
>  0.12         0.20 Mbits/s        1383
> >  TCP upload BE1 de.starlink.taht.net::tcp_cwnd                 :
> 16.85        14.00                1044
> >  TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate        :
>  0.12         0.12                1036
> >  TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate          :
>  0.35         0.26                1042
> >  TCP upload BE1 de.starlink.taht.net::tcp_rtt                  :
> 1128.57       954.47                1020
> >  TCP upload BE1 de.starlink.taht.net::tcp_rtt_var              :
> 68.22        50.08                1020
> >  TCP upload BE2 london.starlink.taht.net                       :
>  0.12         0.15 Mbits/s        1383
> >  TCP upload BE2 london.starlink.taht.net::tcp_cwnd             :
> 17.33        11.00                1046
> >  TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate    :
>  0.12         0.11                1041
> >  TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate      :
>  0.38         0.26                1044
> >  TCP upload BE2 london.starlink.taht.net::tcp_rtt              :
> 1166.74       991.48                1043
> >  TCP upload BE2 london.starlink.taht.net::tcp_rtt_var          :
> 79.99        68.27                1043
> >  TCP upload BE3 singapore.starlink.taht.net                    :
>  0.07         0.13 Mbits/s        1383
> >  TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd          :
> 13.60         9.00                1042
> >  TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate :
>  0.07         0.08                1013
> >  TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate   :
>  0.23         0.20                1031
> >  TCP upload BE3 singapore.starlink.taht.net::tcp_rtt           :
> 1302.23      1129.07                 967
> >  TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var       :
>  113.67        98.84                 967
> >  TCP upload BE4 fremont.starlink.taht.net                      :
>  0.11         0.20 Mbits/s        1383
> >  TCP upload BE4 fremont.starlink.taht.net::tcp_cwnd            :
> 17.55        11.00                1038
> >  TCP upload BE4 fremont.starlink.taht.net::tcp_delivery_rate   :
>  0.11         0.10                1035
> >  TCP upload BE4 fremont.starlink.taht.net::tcp_pacing_rate     :
>  0.32         0.23                1036
> >  TCP upload BE4 fremont.starlink.taht.net::tcp_rtt             :
> 1094.64       887.54                1033
> >  TCP upload BE4 fremont.starlink.taht.net::tcp_rtt_var         :
> 67.20        59.16                1033
> >  TCP upload avg                                                :
>  0.10          N/A Mbits/s        1383
> >  TCP upload fairness                                           :
>  0.96          N/A Mbits/s        1383
> >  TCP upload sum                                                :
>  0.42          N/A Mbits/s        1383
> >
> > ##########
> > with smart queue (fq_codel) on the EdgeRouter
> >
> > flent -x --socket-stats --step-size=.05 -H de.starlink.taht.net -H
> > london.starlink.taht.net -H singapore.starlink.taht.net -H
> fremont.starlink.taht.net -t
> fqcodel_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl rtt_fair4be
> Started Flent 2.0.0 using Python 3.9.2.
> > Starting rtt_fair4be test. Expected run time: 70 seconds.
> > Data file written to
> > ./rtt_fair4be-2022-04-17T150339.549817.fqcodel_erx_rttfair4be_8Mbpsdow
> > n_500Kbpsup_centurylinkdsl.flent.gz
> >
> > Summary of rtt_fair4be test run from 2022-04-17 14:03:39.549817
> >   Title: 'fqcodel_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl'
> >
> >
>  avg       median          # data pts
> >  Ping (ms) ICMP1 de.starlink.taht.net                          :
>  392.93       208.00 ms             1384
> >  Ping (ms) ICMP2 london.starlink.taht.net                      :
>  373.94       196.00 ms             1384
> >  Ping (ms) ICMP3 singapore.starlink.taht.net                   :
>  508.15       319.00 ms             1384
> >  Ping (ms) ICMP4 fremont.starlink.taht.net                     :
>  346.23       152.00 ms             1384
> >  Ping (ms) avg                                                 :
>  405.31          N/A ms             1384
> >  TCP download BE1 de.starlink.taht.net                         :
>  1.36         1.62 Mbits/s        1384
> >  TCP download BE2 london.starlink.taht.net                     :
>  1.47         1.63 Mbits/s        1384
> >  TCP download BE3 singapore.starlink.taht.net                  :
>  1.55         1.60 Mbits/s        1384
> >  TCP download BE4 fremont.starlink.taht.net                    :
>  1.50         1.80 Mbits/s        1384
> >  TCP download avg                                              :
>  1.47          N/A Mbits/s        1384
> >  TCP download fairness                                         :
>  1.00          N/A Mbits/s        1384
> >  TCP download sum                                              :
>  5.88          N/A Mbits/s        1384
> >  TCP upload BE1 de.starlink.taht.net                           :
>  0.09         0.85 Mbits/s        1384
> >  TCP upload BE1 de.starlink.taht.net::tcp_cwnd                 :
>  4.50         4.00                1035
> >  TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate        :
>  0.08         0.08                1030
> >  TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate          :
>  0.17         0.15                1032
> >  TCP upload BE1 de.starlink.taht.net::tcp_rtt                  :
>  458.11       318.82                1027
> >  TCP upload BE1 de.starlink.taht.net::tcp_rtt_var              :
> 95.76        61.88                1027
> >  TCP upload BE2 london.starlink.taht.net                       :
>  0.09         0.93 Mbits/s        1384
> >  TCP upload BE2 london.starlink.taht.net::tcp_cwnd             :
>  4.17         4.00                1038
> >  TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate    :
>  0.08         0.08                1033
> >  TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate      :
>  0.17         0.14                1036
> >  TCP upload BE2 london.starlink.taht.net::tcp_rtt              :
>  452.67       322.49                1033
> >  TCP upload BE2 london.starlink.taht.net::tcp_rtt_var          :
> 96.87        64.28                1033
> >  TCP upload BE3 singapore.starlink.taht.net                    :
>  0.06         0.44 Mbits/s        1384
> >  TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd          :
>  4.16         4.00                1040
> >  TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate :
>  0.06         0.06                1031
> >  TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate   :
>  0.12         0.11                1036
> >  TCP upload BE3 singapore.starlink.taht.net::tcp_rtt           :
>  577.75       438.61                1027
> >  TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var       :
>  107.05        72.97                1027
> >  TCP upload BE4 fremont.starlink.taht.net                      :
>  0.10         1.48 Mbits/s        1384
> >  TCP upload BE4 fremont.starlink.taht.net::tcp_cwnd            :
>  4.45         4.00                1020
> >  TCP upload BE4 fremont.starlink.taht.net::tcp_delivery_rate   :
>  0.09         0.09                1017
> >  TCP upload BE4 fremont.starlink.taht.net::tcp_pacing_rate     :
>  0.21         0.16                1019
> >  TCP upload BE4 fremont.starlink.taht.net::tcp_rtt             :
>  439.05       285.86                1018
> >  TCP upload BE4 fremont.starlink.taht.net::tcp_rtt_var         :
> 81.90        56.33                1018
> >  TCP upload avg                                                :
>  0.08          N/A Mbits/s        1384
> >  TCP upload fairness                                           :
>  0.97          N/A Mbits/s        1384
> >  TCP upload sum                                                :
>  0.34          N/A Mbits/s        1384
> >
> >
> > EdgeRouter Settings
> >
> > smart-queue DSL {
> >         download {
> >             ecn enable
> >             flows 1024
> >             fq-quantum 300
> >             limit 10240
> >             rate 8mbit
> >         }
> >         upload {
> >             ecn enable
> >             flows 1024
> >             fq-quantum 300
> >             limit 10240
> >             rate 600kbit
> >             target 15ms
> >         }
> >         wan-interface pppoe0
> >
> > Also, here's the VMs qdisc settings. Which I now realize probably
> > aren't the best
> >
> > sysctl -a | grep qdisc
> > net.core.default_qdisc = pfifo_fast
> >
> > tc -s qdisc
> > qdisc noqueue 0: dev lo root refcnt 2
> >  Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)  backlog 0b
> > 0p requeues 0 qdisc mq 0: dev eth0 root  Sent 628201 bytes 8643 pkt
> > (dropped 0, overlimits 0 requeues 0)  backlog 0b 0p requeues 0 qdisc
> > pfifo_fast 0: dev eth0 parent :1 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1
> > 1 1 1 1 1  Sent 628201 bytes 8643 pkt (dropped 0, overlimits 0
> > requeues 0)  backlog 0b 0p requeues 0
> >
> >
> >
> > -Jordan
> >
> > -----Original Message-----
> > From: Starlink <starlink-bounces@lists.bufferbloat.net> On Behalf Of
> > Dave Taht
> > Sent: Saturday, April 16, 2022 9:40 PM
> > To: flent-users <flent-users@flent.org>; bloat
> > <bloat@lists.bufferbloat.net>; starlink@lists.bufferbloat.net
> > Subject: [Starlink] an rtt_fair test?
> >
> > if anyone has a bit of spare time this holiday, I'd like to collect a
> few results from various networks around the web this week, not just on the
> turris. Example command line and outputs here:
> >
> > https://forum.turris.cz/t/sqm-on-turris-flent-benchmarks/17048/
> >
> > --
> > I tried to build a better future, a few times:
> > https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org
> >
> > Dave Täht CEO, TekLibre, LLC
> > _______________________________________________
> > Starlink mailing list
> > Starlink@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/starlink
> >
>
>
> --
> I tried to build a better future, a few times:
> https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org
>
> Dave Täht CEO, TekLibre, LLC
>
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
>

[-- Attachment #2: Type: text/html, Size: 33517 bytes --]

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [Flent-users] Re: [Starlink] an rtt_fair test?
  2022-04-17 15:36     ` jordan
  2022-04-17 16:54       ` [Flent-users] Re: [Bloat] " Daniel Sterling
@ 2022-04-18  0:22       ` Dave Taht
  1 sibling, 0 replies; 6+ messages in thread
From: Dave Taht @ 2022-04-18  0:22 UTC (permalink / raw)
  To: Jordan Szuch; +Cc: flent-users, bloat, starlink

On Sun, Apr 17, 2022 at 8:36 AM <jordan@inacomptc.com> wrote:
>
> You're welcome! Here's the files Flent generated: https://1drv.ms/u/s!Ap4u4Rte63FqjqoSQXX444OWLvl8FQ?e=f22Nt7

Thank you for the data!

Part of with me unleashing this test on y'all is to validate my cloud,
which was looking weird yesterday. I still haven't quite got to the
problem fremont or london actually had, but it's much better now.

> And thanks for the link! It honestly doesn't look too bad to implement so I might give it a shot today if time allows.

Honestly, I hope y'all are out there spending time with your families
today. The Net can wait.

>
> I've also been contemplating replacing it with something like a Mikrotik since cake is now integrated into them. I've been testing one at home and made a post with a bunch of flent tests to the cake thread in their forum if you're interested in some data from a Charter Spectrum cable connection: https://forum.mikrotik.com/viewtopic.php?t=179307#p925485

That is a really wonderful addition to that thread, thank you.  For
those here that haven't gone through it, the pictures help a lot, but
requires you register for a free account. Perhaps at some point we can
eliminate the confusion and noise from it, but I often find it useful
to deeply understand what mistakes others will make. ( I have no idea
why anyone is using the bucket thing on that bug thread... no buckets
needed. I don't get it)

Seeing that horrible 1sec of induced latency on spectrum and knowing
that they have 28m subscribers whose networks still behave like that,
pains me. I would so hope that their gbit service, at least, was only
in the 120ms range.

Pro tip: tc -s qdisc show
shows you the drop/market/reschedules

on the client, you might see a few reschedules under the structure of
this test but no drops as "TCP small queues" keeps that together. On
router, the drop ratios are geometrically relative to the bandwidth.

If I have any one gripe left about the mikrotik release it is that
they make backlog, packet and ,drop statistics easily available across
their OSes. I'm pretty sure that at higher bandwidths their default
settings for SFQ and pfifo are too low...


>
> The OneDrive link to my flent files should still be active in that post as well.
>
> -Jordan
>
> -----Original Message-----
> From: Dave Taht <dave.taht@gmail.com>
> Sent: Sunday, April 17, 2022 10:57 AM
> To: Jordan Szuch <jordan@inacomptc.com>
> Cc: flent-users <flent-users@flent.org>; bloat <bloat@lists.bufferbloat.net>; starlink@lists.bufferbloat.net
> Subject: Re: [Starlink] an rtt_fair test?
>
> Thank you for testing!!! If you could stash the *.flent.gz files somewhere I could get at, that would be great. You can also browse the plots using the flent-gui to see more detail over time.
>
> Cake, long available as an out of tree build for the edgerouter, has both dsl compensation and an ack-filter which would help A LOT, in your case. i wish I'd pushed harder to get dsl compensation into the edgerouter smart queues code...
>
> https://community.ui.com/questions/Cake-compiled-for-the-EdgeRouter-devices/fc1ff27c-f321-4344-8737-fcc755cae8a2
>
> Given your 500kbit upload speed the minimum latency inflation under the structure of this test (4 upload flows) should have been around 150ms (4 * 25ms + ack overhead), but without looking at the *.gz files I don't know what your baseline latency was to any of these sites. (it would be good if flent reported on the text output what the minimum actually was)
>
> On Sun, Apr 17, 2022 at 7:35 AM <jordan@inacomptc.com> wrote:
> >
> > Hi Dave,
> >
> > This isn't specifically cake related but I figured it could be interesting, especially as these are from a location where I've been investigating setting up something with cake. These are statistics from a relatives location where they only have a DSL line from CenturyLink available to them. There's a little Debian based VM I setup for a couple random things so I was able to run flent from that. They have a EdgeRouter X serving as the router so they can at least take advantage of fq_codel through the routers Smart Queue feature. I ran a quick speed test, then ran flent without sqm, and then with sqm. Sorry if this gets a bit long and hopefully it stays readable.
> >
> > #############
> > Speedtest by Ookla
> >
> >      Server: 123.Net, Inc. - Southfield, MI (id = 15044)
> >         ISP: CenturyLink
> >     Latency:    45.45 ms   (194.34 ms jitter)
> >    Download:     8.88 Mbps (data used: 12.6 MB )
> >      Upload:     0.64 Mbps (data used: 962.7 kB )
> > Packet Loss:     5.1%
> >  Result URL:
> > https://www.speedtest.net/result/c/799ba46b-d3d4-4d54-ac76-118fac847b7
> > 3
> >
> > ###########
> > No SQM or QOS
> >
> > flent -x --socket-stats --step-size=.05 -H de.starlink.taht.net -H
> > london.starlink.taht.net -H singapore.starlink.taht.net -H fremont.starlink.taht.net -t nosqm_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl rtt_fair4be Started Flent 2.0.0 using Python 3.9.2.
> > Starting rtt_fair4be test. Expected run time: 70 seconds.
> > Data file written to
> > ./rtt_fair4be-2022-04-17T145553.865990.nosqm_erx_rttfair4be_8Mbpsdown_
> > 500Kbpsup_centurylinkdsl.flent.gz
> >
> > Summary of rtt_fair4be test run from 2022-04-17 13:55:53.865990
> >   Title: 'nosqm_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl'
> >
> >                                                                           avg       median          # data pts
> >  Ping (ms) ICMP1 de.starlink.taht.net                          :      1026.06       920.00 ms             1383
> >  Ping (ms) ICMP2 london.starlink.taht.net                      :      1009.28       887.00 ms             1383
> >  Ping (ms) ICMP3 singapore.starlink.taht.net                   :      1121.09      1012.00 ms             1383
> >  Ping (ms) ICMP4 fremont.starlink.taht.net                     :       943.77       853.00 ms             1383
> >  Ping (ms) avg                                                 :      1025.05          N/A ms             1383
> >  TCP download BE1 de.starlink.taht.net                         :         1.36         2.00 Mbits/s        1383
> >  TCP download BE2 london.starlink.taht.net                     :         0.72         0.65 Mbits/s        1383
> >  TCP download BE3 singapore.starlink.taht.net                  :         1.03         1.74 Mbits/s        1383
> >  TCP download BE4 fremont.starlink.taht.net                    :         2.11         2.88 Mbits/s        1383
> >  TCP download avg                                              :         1.30          N/A Mbits/s        1383
> >  TCP download fairness                                         :         0.86          N/A Mbits/s        1383
> >  TCP download sum                                              :         5.22          N/A Mbits/s        1383
> >  TCP upload BE1 de.starlink.taht.net                           :         0.12         0.20 Mbits/s        1383
> >  TCP upload BE1 de.starlink.taht.net::tcp_cwnd                 :        16.85        14.00                1044
> >  TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate        :         0.12         0.12                1036
> >  TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate          :         0.35         0.26                1042
> >  TCP upload BE1 de.starlink.taht.net::tcp_rtt                  :      1128.57       954.47                1020
> >  TCP upload BE1 de.starlink.taht.net::tcp_rtt_var              :        68.22        50.08                1020
> >  TCP upload BE2 london.starlink.taht.net                       :         0.12         0.15 Mbits/s        1383
> >  TCP upload BE2 london.starlink.taht.net::tcp_cwnd             :        17.33        11.00                1046
> >  TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate    :         0.12         0.11                1041
> >  TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate      :         0.38         0.26                1044
> >  TCP upload BE2 london.starlink.taht.net::tcp_rtt              :      1166.74       991.48                1043
> >  TCP upload BE2 london.starlink.taht.net::tcp_rtt_var          :        79.99        68.27                1043
> >  TCP upload BE3 singapore.starlink.taht.net                    :         0.07         0.13 Mbits/s        1383
> >  TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd          :        13.60         9.00                1042
> >  TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate :         0.07         0.08                1013
> >  TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate   :         0.23         0.20                1031
> >  TCP upload BE3 singapore.starlink.taht.net::tcp_rtt           :      1302.23      1129.07                 967
> >  TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var       :       113.67        98.84                 967
> >  TCP upload BE4 fremont.starlink.taht.net                      :         0.11         0.20 Mbits/s        1383
> >  TCP upload BE4 fremont.starlink.taht.net::tcp_cwnd            :        17.55        11.00                1038
> >  TCP upload BE4 fremont.starlink.taht.net::tcp_delivery_rate   :         0.11         0.10                1035
> >  TCP upload BE4 fremont.starlink.taht.net::tcp_pacing_rate     :         0.32         0.23                1036
> >  TCP upload BE4 fremont.starlink.taht.net::tcp_rtt             :      1094.64       887.54                1033
> >  TCP upload BE4 fremont.starlink.taht.net::tcp_rtt_var         :        67.20        59.16                1033
> >  TCP upload avg                                                :         0.10          N/A Mbits/s        1383
> >  TCP upload fairness                                           :         0.96          N/A Mbits/s        1383
> >  TCP upload sum                                                :         0.42          N/A Mbits/s        1383
> >
> > ##########
> > with smart queue (fq_codel) on the EdgeRouter
> >
> > flent -x --socket-stats --step-size=.05 -H de.starlink.taht.net -H
> > london.starlink.taht.net -H singapore.starlink.taht.net -H fremont.starlink.taht.net -t fqcodel_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl rtt_fair4be Started Flent 2.0.0 using Python 3.9.2.
> > Starting rtt_fair4be test. Expected run time: 70 seconds.
> > Data file written to
> > ./rtt_fair4be-2022-04-17T150339.549817.fqcodel_erx_rttfair4be_8Mbpsdow
> > n_500Kbpsup_centurylinkdsl.flent.gz
> >
> > Summary of rtt_fair4be test run from 2022-04-17 14:03:39.549817
> >   Title: 'fqcodel_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl'
> >
> >                                                                           avg       median          # data pts
> >  Ping (ms) ICMP1 de.starlink.taht.net                          :       392.93       208.00 ms             1384
> >  Ping (ms) ICMP2 london.starlink.taht.net                      :       373.94       196.00 ms             1384
> >  Ping (ms) ICMP3 singapore.starlink.taht.net                   :       508.15       319.00 ms             1384
> >  Ping (ms) ICMP4 fremont.starlink.taht.net                     :       346.23       152.00 ms             1384
> >  Ping (ms) avg                                                 :       405.31          N/A ms             1384
> >  TCP download BE1 de.starlink.taht.net                         :         1.36         1.62 Mbits/s        1384
> >  TCP download BE2 london.starlink.taht.net                     :         1.47         1.63 Mbits/s        1384
> >  TCP download BE3 singapore.starlink.taht.net                  :         1.55         1.60 Mbits/s        1384
> >  TCP download BE4 fremont.starlink.taht.net                    :         1.50         1.80 Mbits/s        1384
> >  TCP download avg                                              :         1.47          N/A Mbits/s        1384
> >  TCP download fairness                                         :         1.00          N/A Mbits/s        1384
> >  TCP download sum                                              :         5.88          N/A Mbits/s        1384
> >  TCP upload BE1 de.starlink.taht.net                           :         0.09         0.85 Mbits/s        1384
> >  TCP upload BE1 de.starlink.taht.net::tcp_cwnd                 :         4.50         4.00                1035
> >  TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate        :         0.08         0.08                1030
> >  TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate          :         0.17         0.15                1032
> >  TCP upload BE1 de.starlink.taht.net::tcp_rtt                  :       458.11       318.82                1027
> >  TCP upload BE1 de.starlink.taht.net::tcp_rtt_var              :        95.76        61.88                1027
> >  TCP upload BE2 london.starlink.taht.net                       :         0.09         0.93 Mbits/s        1384
> >  TCP upload BE2 london.starlink.taht.net::tcp_cwnd             :         4.17         4.00                1038
> >  TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate    :         0.08         0.08                1033
> >  TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate      :         0.17         0.14                1036
> >  TCP upload BE2 london.starlink.taht.net::tcp_rtt              :       452.67       322.49                1033
> >  TCP upload BE2 london.starlink.taht.net::tcp_rtt_var          :        96.87        64.28                1033
> >  TCP upload BE3 singapore.starlink.taht.net                    :         0.06         0.44 Mbits/s        1384
> >  TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd          :         4.16         4.00                1040
> >  TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate :         0.06         0.06                1031
> >  TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate   :         0.12         0.11                1036
> >  TCP upload BE3 singapore.starlink.taht.net::tcp_rtt           :       577.75       438.61                1027
> >  TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var       :       107.05        72.97                1027
> >  TCP upload BE4 fremont.starlink.taht.net                      :         0.10         1.48 Mbits/s        1384
> >  TCP upload BE4 fremont.starlink.taht.net::tcp_cwnd            :         4.45         4.00                1020
> >  TCP upload BE4 fremont.starlink.taht.net::tcp_delivery_rate   :         0.09         0.09                1017
> >  TCP upload BE4 fremont.starlink.taht.net::tcp_pacing_rate     :         0.21         0.16                1019
> >  TCP upload BE4 fremont.starlink.taht.net::tcp_rtt             :       439.05       285.86                1018
> >  TCP upload BE4 fremont.starlink.taht.net::tcp_rtt_var         :        81.90        56.33                1018
> >  TCP upload avg                                                :         0.08          N/A Mbits/s        1384
> >  TCP upload fairness                                           :         0.97          N/A Mbits/s        1384
> >  TCP upload sum                                                :         0.34          N/A Mbits/s        1384
> >
> >
> > EdgeRouter Settings
> >
> > smart-queue DSL {
> >         download {
> >             ecn enable
> >             flows 1024
> >             fq-quantum 300
> >             limit 10240
> >             rate 8mbit
> >         }
> >         upload {
> >             ecn enable
> >             flows 1024
> >             fq-quantum 300
> >             limit 10240
> >             rate 600kbit
> >             target 15ms
> >         }
> >         wan-interface pppoe0
> >
> > Also, here's the VMs qdisc settings. Which I now realize probably
> > aren't the best
> >
> > sysctl -a | grep qdisc
> > net.core.default_qdisc = pfifo_fast
> >
> > tc -s qdisc
> > qdisc noqueue 0: dev lo root refcnt 2
> >  Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)  backlog 0b
> > 0p requeues 0 qdisc mq 0: dev eth0 root  Sent 628201 bytes 8643 pkt
> > (dropped 0, overlimits 0 requeues 0)  backlog 0b 0p requeues 0 qdisc
> > pfifo_fast 0: dev eth0 parent :1 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1
> > 1 1 1 1 1  Sent 628201 bytes 8643 pkt (dropped 0, overlimits 0
> > requeues 0)  backlog 0b 0p requeues 0
> >
> >
> >
> > -Jordan
> >
> > -----Original Message-----
> > From: Starlink <starlink-bounces@lists.bufferbloat.net> On Behalf Of
> > Dave Taht
> > Sent: Saturday, April 16, 2022 9:40 PM
> > To: flent-users <flent-users@flent.org>; bloat
> > <bloat@lists.bufferbloat.net>; starlink@lists.bufferbloat.net
> > Subject: [Starlink] an rtt_fair test?
> >
> > if anyone has a bit of spare time this holiday, I'd like to collect a few results from various networks around the web this week, not just on the turris. Example command line and outputs here:
> >
> > https://forum.turris.cz/t/sqm-on-turris-flent-benchmarks/17048/
> >
> > --
> > I tried to build a better future, a few times:
> > https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org
> >
> > Dave Täht CEO, TekLibre, LLC
> > _______________________________________________
> > Starlink mailing list
> > Starlink@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/starlink
> >
>
>
> --
> I tried to build a better future, a few times:
> https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org
>
> Dave Täht CEO, TekLibre, LLC
>


-- 
I tried to build a better future, a few times:
https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org

Dave Täht CEO, TekLibre, LLC

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2022-04-18  0:22 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-04-17  1:39 [Flent-users] an rtt_fair test? Dave Taht
2022-04-17 14:35 ` [Flent-users] Re: [Starlink] " jordan
2022-04-17 14:56   ` Dave Taht
2022-04-17 15:36     ` jordan
2022-04-17 16:54       ` [Flent-users] Re: [Bloat] " Daniel Sterling
2022-04-18  0:22       ` [Flent-users] " Dave Taht

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox