From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mail.toke.dk X-Spam-Level: X-Spam-ASN: AS60781 77.235.32.0/19 X-Spam-Status: No, score=-101.9 required=5.0 tests=BAYES_00,SHORTCIRCUIT shortcircuit=ham autolearn=disabled version=3.4.1 Received: from mail.toke.dk by mail.toke.dk with LMTP id zmCGHdw9HlttWQAAOr1fkg (envelope-from ) for ; Mon, 11 Jun 2018 11:16:12 +0200 Authentication-Results: mail.toke.dk; none (SPF check N/A for local connections - client-ip=77.235.54.103; helo=web6.sd.eurovps.com; envelope-from=flent-users-bounces@flent.org; receiver=) Authentication-Results: mail.toke.dk; dkim=fail reason="key not found in DNS" (0-bit key) header.d=flent.org header.i=@flent.org header.b=GCWqdBvY; dkim=fail reason="signature verification failed" (1024-bit key) header.d=github.com header.i=@github.com header.b=JYZcB12N Received: from web6.sd.eurovps.com (web6.sd.eurovps.com [77.235.54.103]) by mail.toke.dk (Postfix) with ESMTPS id 892123EC96D for ; Mon, 11 Jun 2018 11:16:03 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=flent.org; s=default; h=Sender:Content-Type:Cc:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Subject:Mime-Version: References:In-Reply-To:Message-ID:To:From:Date:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=5ug/bWk9M0LGWdaa5uKnJSGSacy36u0u0uPMVTBp4q4=; b=GCWqdBvYuBzxMGd5/OC/hRSTmM MxZJzbQvUHxQeX6bJGoQ4l1cMEdXJRCZ2NQb+5lpqasvrkUWTTaW6tOHeH+VFZO3h/qbKj15dpSII 1HzbS6QXrPzFd0+SCW7kDrSKDNk0YrB5++i9xlrhBXU+gYuDOjFwwt2ZQulmiF8LKUL0TMCb7lepA xEbM6wQXMZdA8Q6vhlGGagOB3kj1WWMQT9sXNHZNpWoiruvzZ76PZdOSupZS+YjpbvBM8kr9R4RxE casVcF+5MiXgd27CsSBPlIuZ1CS5oGybPEgOC2IlGi6E+hn2PAe/W6eqeHhIAuPPNjFeRiAAq1OCz SE0itREA==; Received: from [::1] (port=49196 helo=web6.sd.eurovps.com) by web6.sd.eurovps.com with esmtp (Exim 4.91) (envelope-from ) id 1fSIvi-00EUdf-Fi; Mon, 11 Jun 2018 12:16:02 +0300 Received: from out-5.smtp.github.com ([192.30.252.196]:44816) by web6.sd.eurovps.com with esmtps (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.91) (envelope-from ) id 1fSIve-00EURM-0d for flent-users@flent.org; Mon, 11 Jun 2018 12:16:00 +0300 Date: Mon, 11 Jun 2018 02:15:17 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=github.com; s=pf2014; t=1528708517; bh=4innpq05gGsh33+R/qIZ+mhdKhJlt17gVhX8JIErPuY=; h=Date:From:Reply-To:To:Cc:In-Reply-To:References:Subject:List-ID: List-Archive:List-Post:List-Unsubscribe:From; b=JYZcB12Nit/tkqU14mOk++Hw/jlInoTtlN8/T6Xsod8fM6QiroaLL6i5gZvpNKStU c4CDEl2P/ZxHnTaDJOsqfUsvmHGxkdEEGujBzG2UO4F679U2bNrbdKQrD575y42c65 q6HTj3TJFWusEzj5NtalWyqtq8AcJwKnRFBQ1/jk= From: =?UTF-8?B?VG9rZSBIw7hpbGFuZC1Kw7hyZ2Vuc2Vu?= To: tohojo/flent Message-ID: In-Reply-To: References: Mime-Version: 1.0 Precedence: list X-GitHub-Sender: tohojo X-GitHub-Recipient: flent-users X-GitHub-Reason: subscribed X-Auto-Response-Suppress: All X-GitHub-Recipient-Address: flent-users@flent.org Subject: Re: [Flent-users] [tohojo/flent] completion delay for an iperf script (#138) X-BeenThere: flent-users@flent.org X-Mailman-Version: 2.1.26 List-Id: Flent discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: tohojo/flent Cc: Subscribed Content-Type: multipart/mixed; boundary="===============8166335911456891462==" Errors-To: flent-users-bounces@flent.org Sender: "Flent-users" X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - web6.sd.eurovps.com X-AntiAbuse: Original Domain - toke.dk X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - flent.org X-Get-Message-Sender-Via: web6.sd.eurovps.com: acl_c_authenticated_local_user: mailman/mailman X-Authenticated-Sender: web6.sd.eurovps.com: mailman@flent.org --===============8166335911456891462== Content-Type: multipart/alternative; boundary="--==_mimepart_5b1e3da555a5a_8e82ae02451af5c739bc"; charset=UTF-8 Content-Transfer-Encoding: 7bit ----==_mimepart_5b1e3da555a5a_8e82ae02451af5c739bc Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Matthieu Coudron writes: > I used to have > ``` > Runner aggregation finished > ERROR: No data to aggregate. Run with -L and check log file to investigate. > ``` > with in log > ``` > ELAPSED_TIME=0.00 Yeah, this is an issue with netperf; it'll only output time in 10-ms intervals. So you'll need to have enough data for the transfer to take longer than this. The 'no data to aggregate' is because there are no intermediate data points output by netperf. For very short transfers, you could fiddle the step size parameter to get output for smaller intervals. If you need to run tests that large, I guess the 'no data to aggregate' error could be made non-fatal in the case where there are valid metadata results. But I assume your 'real' tests are going to be on lower bandwidth links so they'll take longer (and thus this will be less of an issue)? > ... > -- OUTPUT START -->{'TCP netperf::1': [], > 'TCP netperf::2': [], > 'TCP netperf::3': [], > 'TCP netperf::4': [], > 'TCP netperf::5': []}<-- OUTPUT END -- > ``` > as if ELAPSED_TIME being 0.00 wouldn't count as a result. > > I increased the byte count and it started producing result > ``` > 2018-06-11 11:38:01,160 [flent.aggregators] DEBUG: Runner aggregation finished > -- OUTPUT START -->{'TCP netperf::1': [[1528684679.972, 41611.46]], > 'TCP netperf::2': [[1528684679.99, 45512.46]], > 'TCP netperf::3': [[1528684680.044, 45546.14]], > 'TCP netperf::4': [[1528684680.008, 46227.74]], > 'TCP netperf::5': [[1528684680.026, 46214.39]]}<-- OUTPUT END -- > > > ``` > but these don't seem to be the ELAPSED_TIME. Yeah, so this is because we are still using the timeseries run mode, so the actual data points being produced are the bandwidth measurements that netperf produce. So the elapsed time is stored as metadata, and we basically ignore the other results. > I've tried using this > ``` > DATA_SETS = o([ > ('TCP netperf', > { > 'test': 'TCP_STREAM', > 'host': HOST, > 'length': 10, > 'bytes': 100000000, > 'duplicates': 5, > # 'units': 'Mbits/s', > 'runner': 'netperf_demo',}) > ]) > > PLOTS['iperf_delay'] = {'description': 'Netperf completion time', > 'type': 'cdf_combine', > 'group_by': 'groups_concat', > 'series': [ > # when duplicates is used the name is changed to e.g.: > # Started NetperfDemoRunner idx 4 ('TCP netperf::5') > # thus we need to use the glob > > {'data': glob('TCP netperf*'), > 'label': 'completion delay (ms)', > 'combine_mode': 'meta:ELAPSED_TIME' > }] > } > ``` > and it displays an empty plot. when opening the tcp_iperf_delay-2018-06-07T143147.599875.flent it seems like all elapsed time were 0.02. I wanted to modify these values in place and regenerate a plot from the modified tcp_iperf_delay-2018-06-07T143147.599875.flent but couldn't find a way out. Well, this should more or less work, I think. However, you only get one data point to plot here; the 5 repetitions are not going to be five data points to plot a CDF over; they are going to be 5 different series with one datapoint each. To get multiple datapoints, you'd need to run multiple tests (`for s in $(seq 10); do flent ; done`), then plot all data files at once. > How can I get > ``` > 'TCP netperf::2': , > 'TCP netperf::3': , > 'TCP netperf::4': <-- OUTPUT END -- > ``` > and then plot the cdf of it ? maybe that's not how it works, and I shouldn't care about the `'TCP netperf::2': []` data since ELAPSED_TIME is metadata ? it really looks like magic as it's kind of DSL mixed with python. See above; you are right that you should just ignore those data points ;) And yeah, this is really a DSL; it's just Python because that was easier than coming up with a full DSL myself. I've been thinking about changing it, but, well, that is a different subject... -Toke -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/tohojo/flent/issues/138#issuecomment-396176959 ----==_mimepart_5b1e3da555a5a_8e82ae02451af5c739bc Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Matthieu Coudron <notifications@github.com> writes:

> I used to have
> ```
> Runner aggregation finished
> ERROR: No data to aggregate. Run with -L and check log file to inves= tigate.
> ```
> with in log
> ```
> ELAPSED_TIME=3D0.00

Yeah, this is an issue with netperf; it'll only output time in 10-ms<= br> intervals. So you'll need to have enough data for the transfer to tak= e
longer than this.

The 'no data to aggregate' is because there are no intermediate d= ata
points output by netperf. For very short transfers, you could fiddle the<= br> step size parameter to get output for smaller intervals.

If you need to run tests that large, I guess the 'no data to aggregat= e'
error could be made non-fatal in the case where there are valid metadata<= br> results. But I assume your 'real' tests are going to be on lower<= br> bandwidth links so they'll take longer (and thus this will be less of= an
issue)?

> ...
> -- OUTPUT START -->{'TCP netperf::1': [],
> 'TCP netperf::2': [],
> 'TCP netperf::3': [],
> 'TCP netperf::4': [],
> 'TCP netperf::5': []}<-- OUTPUT END --
> ```
> as if ELAPSED_TIME being 0.00 wouldn't count as a result.
>
> I increased the byte count and it started producing result
> ```
> 2018-06-11 11:38:01,160 [flent.aggregators] DEBUG: Runner aggregatio= n finished
> -- OUTPUT START -->{'TCP netperf::1': [[1528684679.972, 4= 1611.46]],
> 'TCP netperf::2': [[1528684679.99, 45512.46]],
> 'TCP netperf::3': [[1528684680.044, 45546.14]],
> 'TCP netperf::4': [[1528684680.008, 46227.74]],
> 'TCP netperf::5': [[1528684680.026, 46214.39]]}<-- OUTPU= T END --
>
>
> ```
> but these don't seem to be the ELAPSED_TIME.

Yeah, so this is because we are still using the timeseries run mode, so the actual data points being produced are the bandwidth measurements
that netperf produce. So the elapsed time is stored as metadata, and we basically ignore the other results.


> I've tried using this
> ```
> DATA_SETS =3D o([
> ('TCP netperf',
> {
> 'test': 'TCP_STREAM',
> 'host': HOST,
> 'length': 10,
> 'bytes': 100000000,
> 'duplicates': 5,
> # 'units': 'Mbits/s',
> 'runner': 'netperf_demo',})
> ])
>
> PLOTS['iperf_delay'] =3D {'description': 'Net= perf completion time',
> 'type': 'cdf_combine',<= br> > 'group_by': 'groups_concat',
> 'series': [
> # when duplicates is used the name is changed to e.g.:
> # Started NetperfDemoRunner idx 4 ('TCP netperf::5')=
> # thus we need to use the glob
>
> {'data': glob('TCP netp= erf*'),
> 'label': 'completion delay (ms)',
> 'combine_mode': 'meta:ELAPSED_TIM= E'
> }]
> }
> ```
> and it displays an empty plot. when opening the tcp_iperf_delay-201= 8-06-07T143147.599875.flent it seems like all elapsed time were 0.02. I w= anted to modify these values in place and regenerate a plot from the modi= fied tcp_iperf_delay-2018-06-07T143147.599875.flent but couldn't find= a way out.

Well, this should more or less work, I think. However, you only get one data point to plot here; the 5 repetitions are not going to be five data<= br> points to plot a CDF over; they are going to be 5 different series with one datapoint each.

To get multiple datapoints, you'd need to run multiple tests (`for s = in
$(seq 10); do flent <args>; done`), then plot all data files at onc= e.



> How can I get
> ```
> 'TCP netperf::2': <ElapsedTime2>,
> 'TCP netperf::3': <ElapsedTime3>,
> 'TCP netperf::4': <ElapsedTime4><-- OUTPUT END --<= br> > ```
> and then plot the cdf of it ? maybe that's not how it works, and= I shouldn't care about the `'TCP netperf::2': []` data sinc= e ELAPSED_TIME is metadata ? it really looks like magic as it's kind = of DSL mixed with python.

See above; you are right that you should just ignore those data points ;)=

And yeah, this is really a DSL; it's just Python because that was eas= ier
than coming up with a full DSL myself. I've been thinking about chang= ing
it, but, well, that is a different subject...

-Toke

&m= dash;
You are receiving this because you are subscribed to this thre= ad.
Reply to this email directly, view it on GitHub, or mute the thread.3D""

= ----==_mimepart_5b1e3da555a5a_8e82ae02451af5c739bc-- --===============8166335911456891462== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Flent-users mailing list Flent-users@flent.org http://flent.org/mailman/listinfo/flent-users_flent.org --===============8166335911456891462==--