Flent-users discussion archives
 help / color / mirror / Atom feed
* [Flent-users] [tohojo/flent] Incorrect "TCP upload sum" for tcp_nup, rrul, rrul_be tests (#182)
@ 2019-09-12 12:03 Rich Brown
  2019-09-12 12:09 ` Rich Brown
                   ` (4 more replies)
  0 siblings, 5 replies; 6+ messages in thread
From: Rich Brown @ 2019-09-12 12:03 UTC (permalink / raw)
  To: tohojo/flent; +Cc: Subscribed


[-- Attachment #1.1: Type: text/plain, Size: 6099 bytes --]

Using Flent 1.3.0 on macOS 10.12.6:

I notice that the "TCP upload sum" value shown in the upload portion of tests (tcp_nup, rrul, rrul_be) doesn't match the sum of the individual flows. In the first example below, the displayed sum (0.28 Mbps) is considerably lower than the sum of the individual flows (0.77 Mbps - which matches the expected speed of my 7Mbps/768kbps DSL link).

I attach the flent.gz and the log files from the first run shown below. (Earlier rrul and rrul_be test runs follow.) What other information could I gather? Thanks.

[tcp_nup-2019-09-12T074946.261029.flent-fremont_bufferbloat_net.log](https://github.com/tohojo/flent/files/3605312/tcp_nup-2019-09-12T074946.261029.flent-fremont_bufferbloat_net.log)
[tcp_nup-2019-09-12T074946.261029.flent-fremont_bufferbloat_net.flent.gz](https://github.com/tohojo/flent/files/3605313/tcp_nup-2019-09-12T074946.261029.flent-fremont_bufferbloat_net.flent.gz)

```
bash-3.2$ flent -x -L -H flent-fremont.bufferbloat.net  -t "flent-fremont.bufferbloat.net" --te=upload_streams=4 --te=ping_hosts=1.1.1.1 tcp_nup
Started Flent 1.3.0 using Python 2.7.15.
Starting tcp_nup test. Expected run time: 70 seconds.
Data file written to ./tcp_nup-2019-09-12T074946.261029.flent-fremont_bufferbloat_net.flent.gz.

Summary of tcp_nup test run from 2019-09-12 11:49:46.261029
  Title: 'flent-fremont.bufferbloat.net'

                                           avg       median          # data pts
 Ping (ms) ICMP                 :       100.34       100.50 ms              348
 Ping (ms) ICMP 1.1.1.1 (extra) :        23.40        23.40 ms              349
 TCP upload avg                 :         0.18         0.19 Mbits/s         108
 TCP upload sum                 :         0.28         0.28 Mbits/s         108
 TCP upload::1                  :         0.22         0.20 Mbits/s          11
 TCP upload::2                  :         0.16         0.14 Mbits/s           7
 TCP upload::3                  :         0.17         0.15 Mbits/s           8
 TCP upload::4                  :         0.22         0.20 Mbits/s          11
```
```
Started Flent 1.3.0 using Python 2.7.15.
Starting rrul test. Expected run time: 70 seconds.
Data file written to ./rrul-2019-09-11T125110.964940.Running_flent_from_Richs-MBP-10337_lan.flent.gz.

Summary of rrul test run from 2019-09-11 16:51:10.964940
  Title: 'Running_flent_from_Richs-MBP-10337.lan'

                                           avg       median          # data pts
 Ping (ms) ICMP                 :       104.75       104.00 ms              348
 Ping (ms) ICMP 1.1.1.1 (extra) :        28.96        28.80 ms              346
 Ping (ms) UDP BE               :       104.78       104.38 ms              261
 Ping (ms) UDP BK               :       105.18       104.71 ms              273
 Ping (ms) UDP EF               :       104.18       103.31 ms              247
 Ping (ms) avg                  :        88.23        89.39 ms              353
 TCP download BE                :         1.53         1.59 Mbits/s         282
 TCP download BK                :         1.56         1.60 Mbits/s         284
 TCP download CS5               :         1.46         1.51 Mbits/s         286
 TCP download EF                :         1.51         1.58 Mbits/s         282
 TCP download avg               :         1.51         1.55 Mbits/s         301
 TCP download sum               :         6.04         6.20 Mbits/s         301
 TCP totals                     :         6.06         6.21 Mbits/s         301
 TCP upload BE                  :         0.09         0.08 Mbits/s           4
 TCP upload BK                  :         0.09         0.07 Mbits/s           3
 TCP upload CS5                 :         0.11         0.09 Mbits/s           5
 TCP upload EF                  :         0.11         0.09 Mbits/s           5
 TCP upload avg                 :         0.09         0.08 Mbits/s          48
 TCP upload sum                 :         0.14         0.12 Mbits/s          48
```

```
Started Flent 1.3.0 using Python 2.7.15.
Starting rrul_be test. Expected run time: 70 seconds.
Data file written to ./rrul_be-2019-09-11T125231.578848.Running_flent_from_Richs-MBP-10337_lan.flent.gz.

Summary of rrul_be test run from 2019-09-11 16:52:31.578848
  Title: 'Running_flent_from_Richs-MBP-10337.lan'

                                           avg       median          # data pts
 Ping (ms) ICMP                 :       104.88       104.00 ms              350
 Ping (ms) ICMP 1.1.1.1 (extra) :        30.32        29.90 ms              349
 Ping (ms) UDP BE1              :       105.10       104.38 ms              248
 Ping (ms) UDP BE2              :       106.64       106.27 ms              277
 Ping (ms) UDP BE3              :       105.71       104.71 ms              279
 Ping (ms) avg                  :        89.55        90.38 ms              353
 TCP download BE                :         1.45         1.49 Mbits/s         278
 TCP download BE2               :         1.52         1.57 Mbits/s         280
 TCP download BE3               :         1.66         1.71 Mbits/s         281
 TCP download BE4               :         1.36         1.42 Mbits/s         278
 TCP download avg               :         1.49         1.53 Mbits/s         301
 TCP download sum               :         5.97         6.13 Mbits/s         301
 TCP totals                     :         6.00         6.15 Mbits/s         301
 TCP upload BE                  :         0.08         0.06 Mbits/s           3
 TCP upload BE2                 :         0.11         0.09 Mbits/s           5
 TCP upload BE3                 :         0.11         0.09 Mbits/s           5
 TCP upload BE4                 :         0.11         0.09 Mbits/s           5
 TCP upload avg                 :         0.09         0.09 Mbits/s          34
 TCP upload sum                 :         0.19         0.23 Mbits/s          34
bash-3.2$ 

```

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/tohojo/flent/issues/182

[-- Attachment #1.2: Type: text/html, Size: 7466 bytes --]

[-- Attachment #2: Type: text/plain, Size: 151 bytes --]

_______________________________________________
Flent-users mailing list
Flent-users@flent.org
http://flent.org/mailman/listinfo/flent-users_flent.org

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [Flent-users] [tohojo/flent] Incorrect "TCP upload sum" for tcp_nup, rrul, rrul_be tests (#182)
  2019-09-12 12:03 [Flent-users] [tohojo/flent] Incorrect "TCP upload sum" for tcp_nup, rrul, rrul_be tests (#182) Rich Brown
@ 2019-09-12 12:09 ` Rich Brown
  2019-09-13 16:13 ` Toke Høiland-Jørgensen
                   ` (3 subsequent siblings)
  4 siblings, 0 replies; 6+ messages in thread
From: Rich Brown @ 2019-09-12 12:09 UTC (permalink / raw)
  To: tohojo/flent; +Cc: Subscribed


[-- Attachment #1.1: Type: text/plain, Size: 392 bytes --]

Update: dtaht (in https://github.com/tohojo/flent/issues/178#issuecomment-528885331) said of the ping times, "I have seen this statistic be wildly wrong in the summary but correct in the graph."

Related?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/tohojo/flent/issues/182#issuecomment-530795911

[-- Attachment #1.2: Type: text/html, Size: 2125 bytes --]

[-- Attachment #2: Type: text/plain, Size: 151 bytes --]

_______________________________________________
Flent-users mailing list
Flent-users@flent.org
http://flent.org/mailman/listinfo/flent-users_flent.org

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [Flent-users] [tohojo/flent] Incorrect "TCP upload sum" for tcp_nup, rrul, rrul_be tests (#182)
  2019-09-12 12:03 [Flent-users] [tohojo/flent] Incorrect "TCP upload sum" for tcp_nup, rrul, rrul_be tests (#182) Rich Brown
  2019-09-12 12:09 ` Rich Brown
@ 2019-09-13 16:13 ` Toke Høiland-Jørgensen
  2019-09-13 16:13 ` Toke Høiland-Jørgensen
                   ` (2 subsequent siblings)
  4 siblings, 0 replies; 6+ messages in thread
From: Toke Høiland-Jørgensen @ 2019-09-13 16:13 UTC (permalink / raw)
  To: tohojo/flent; +Cc: Subscribed


[-- Attachment #1.1: Type: text/plain, Size: 828 bytes --]

Rich Brown <notifications@github.com> writes:

> Update: dtaht (in
> https://github.com/tohojo/flent/issues/178#issuecomment-528885331)
> said of the ping times, "I have seen this statistic be wildly wrong in
> the summary but correct in the graph."

Yes, this can be the case because of the way the summary values are
computed. However, it *should* be the case that what the summary outputs
is computed from the aggregate statistics that netperf outputs at the
end of the test where possible.

However, it seems there was a bug causing all this per-series metadata
to be cleared out before the file was saved. Not good! Will post a fix...


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/tohojo/flent/issues/182#issuecomment-531299423

[-- Attachment #1.2: Type: text/html, Size: 2269 bytes --]

[-- Attachment #2: Type: text/plain, Size: 151 bytes --]

_______________________________________________
Flent-users mailing list
Flent-users@flent.org
http://flent.org/mailman/listinfo/flent-users_flent.org

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [Flent-users] [tohojo/flent] Incorrect "TCP upload sum" for tcp_nup, rrul, rrul_be tests (#182)
  2019-09-12 12:03 [Flent-users] [tohojo/flent] Incorrect "TCP upload sum" for tcp_nup, rrul, rrul_be tests (#182) Rich Brown
  2019-09-12 12:09 ` Rich Brown
  2019-09-13 16:13 ` Toke Høiland-Jørgensen
@ 2019-09-13 16:13 ` Toke Høiland-Jørgensen
  2019-09-14  1:15 ` Rich Brown
  2019-09-14 15:39 ` Toke Høiland-Jørgensen
  4 siblings, 0 replies; 6+ messages in thread
From: Toke Høiland-Jørgensen @ 2019-09-13 16:13 UTC (permalink / raw)
  To: tohojo/flent; +Cc: Subscribed


[-- Attachment #1.1: Type: text/plain, Size: 239 bytes --]

Closed #182 via ff465203dd7de78d87e0d88717068621b8159c28.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/tohojo/flent/issues/182#event-2632860804

[-- Attachment #1.2: Type: text/html, Size: 2224 bytes --]

[-- Attachment #2: Type: text/plain, Size: 151 bytes --]

_______________________________________________
Flent-users mailing list
Flent-users@flent.org
http://flent.org/mailman/listinfo/flent-users_flent.org

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [Flent-users] [tohojo/flent] Incorrect "TCP upload sum" for tcp_nup, rrul, rrul_be tests (#182)
  2019-09-12 12:03 [Flent-users] [tohojo/flent] Incorrect "TCP upload sum" for tcp_nup, rrul, rrul_be tests (#182) Rich Brown
                   ` (2 preceding siblings ...)
  2019-09-13 16:13 ` Toke Høiland-Jørgensen
@ 2019-09-14  1:15 ` Rich Brown
  2019-09-14 15:39 ` Toke Høiland-Jørgensen
  4 siblings, 0 replies; 6+ messages in thread
From: Rich Brown @ 2019-09-14  1:15 UTC (permalink / raw)
  To: tohojo/flent; +Cc: Subscribed


[-- Attachment #1.1: Type: text/plain, Size: 2803 bytes --]

Sorry to re-open, but the new version (Flent 1.9.9-git-ff46520) seems to fix the upload values for the Average - the individual streams sum to the displayed sum.

But the Median values still don't look right. Here are two runs of the same command:

```
bash-3.2$ flent -x -L -H flent-fremont.bufferbloat.net  -t "flent-fremont.bufferbloat.net" --te=upload_streams=4 --te=ping_hosts=1.1.1.1 tcp_nup
Started Flent 1.9.9-git-ff46520 using Python 3.7.4.
Starting tcp_nup test. Expected run time: 70 seconds.
Data file written to ./tcp_nup-2019-09-13T210706.195971.flent-fremont_bufferbloat_net.flent.gz.

Summary of tcp_nup test run from 2019-09-14 01:07:06.195971
  Title: 'flent-fremont.bufferbloat.net'

                                           avg       median          # data pts
 Ping (ms) ICMP                 :       101.23       101.00 ms              347
 Ping (ms) ICMP 1.1.1.1 (extra) :        23.39        23.90 ms              347
 TCP upload avg                 :         0.17         0.18 Mbits/s         354
 TCP upload sum                 :         0.69         0.30 Mbits/s         354
 TCP upload::1                  :         0.16         0.17 Mbits/s         354
 TCP upload::2                  :         0.15         0.14 Mbits/s         354
 TCP upload::3                  :         0.19         0.20 Mbits/s         354
 TCP upload::4                  :         0.19         0.20 Mbits/s         354
```

```
bash-3.2$ flent -x -L -H flent-fremont.bufferbloat.net  -t "flent-fremont.bufferbloat.net" --te=upload_streams=4 --te=ping_hosts=1.1.1.1 tcp_nup
Started Flent 1.9.9-git-ff46520 using Python 3.7.4.
Starting tcp_nup test. Expected run time: 70 seconds.
Data file written to ./tcp_nup-2019-09-13T211010.793062.flent-fremont_bufferbloat_net.flent.gz.

Summary of tcp_nup test run from 2019-09-14 01:10:10.793062
  Title: 'flent-fremont.bufferbloat.net'

                                           avg       median          # data pts
 Ping (ms) ICMP                 :       102.19       102.00 ms              347
 Ping (ms) ICMP 1.1.1.1 (extra) :        24.91        24.80 ms              341
 TCP upload avg                 :         0.15         0.16 Mbits/s         354
 TCP upload sum                 :         0.62         0.16 Mbits/s         354
 TCP upload::1                  :         0.12         0.13 Mbits/s         354
 TCP upload::2                  :         0.18         0.19 Mbits/s         354
 TCP upload::3                  :         0.18         0.19 Mbits/s         354
 TCP upload::4                  :         0.14         0.15 Mbits/s         354
bash-3.2$
```

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/tohojo/flent/issues/182#issuecomment-531433159

[-- Attachment #1.2: Type: text/html, Size: 4205 bytes --]

[-- Attachment #2: Type: text/plain, Size: 151 bytes --]

_______________________________________________
Flent-users mailing list
Flent-users@flent.org
http://flent.org/mailman/listinfo/flent-users_flent.org

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [Flent-users] [tohojo/flent] Incorrect "TCP upload sum" for tcp_nup, rrul, rrul_be tests (#182)
  2019-09-12 12:03 [Flent-users] [tohojo/flent] Incorrect "TCP upload sum" for tcp_nup, rrul, rrul_be tests (#182) Rich Brown
                   ` (3 preceding siblings ...)
  2019-09-14  1:15 ` Rich Brown
@ 2019-09-14 15:39 ` Toke Høiland-Jørgensen
  4 siblings, 0 replies; 6+ messages in thread
From: Toke Høiland-Jørgensen @ 2019-09-14 15:39 UTC (permalink / raw)
  To: tohojo/flent; +Cc: Subscribed


[-- Attachment #1.1: Type: text/plain, Size: 912 bytes --]

Rich Brown <notifications@github.com> writes:

> Sorry to re-open, but the new version (Flent 1.9.9-git-ff46520) seems
> to fix the upload values for the Average - the individual streams sum
> to the displayed sum.
>
> But the Median values still don't look right. Here are two runs of the
> same command:

Ah, no, the median has the same issue, but there is no per-test output
we can use, so we need to calculate it. And since the summary output
calculates a per-series median, the sum doesn't match up (which is not
too surprising; the median of the total throughput is not necessarily
the sum of the medians of each individual flow).

We could remove the median from the calculated data series if that is
less confusing?


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/tohojo/flent/issues/182#issuecomment-531489398

[-- Attachment #1.2: Type: text/html, Size: 2369 bytes --]

[-- Attachment #2: Type: text/plain, Size: 151 bytes --]

_______________________________________________
Flent-users mailing list
Flent-users@flent.org
http://flent.org/mailman/listinfo/flent-users_flent.org

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2019-09-14 15:39 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-09-12 12:03 [Flent-users] [tohojo/flent] Incorrect "TCP upload sum" for tcp_nup, rrul, rrul_be tests (#182) Rich Brown
2019-09-12 12:09 ` Rich Brown
2019-09-13 16:13 ` Toke Høiland-Jørgensen
2019-09-13 16:13 ` Toke Høiland-Jørgensen
2019-09-14  1:15 ` Rich Brown
2019-09-14 15:39 ` Toke Høiland-Jørgensen

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox