From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mail.toke.dk X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-101.9 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_NONE, SHORTCIRCUIT shortcircuit=ham autolearn=disabled version=3.4.1 Received: from mail.toke.dk by mail.toke.dk with LMTP id QESFDyueylnhcgAAOr1fkg for ; Tue, 26 Sep 2017 20:36:27 +0200 Received: from web6.sd.eurovps.com (web6.sd.eurovps.com [77.235.54.103]) by mail.toke.dk (Postfix) with ESMTPS id C37DD1E1EF7 for ; Tue, 26 Sep 2017 20:36:25 +0200 (CEST) Authentication-Results: mail.toke.dk; dkim=fail reason="key not found in DNS" (0-bit key) header.d=flent.org header.i=@flent.org header.b=Q0bWFebM; dkim=fail reason="signature verification failed" (1024-bit key) header.d=github.com header.i=@github.com header.b=B1jyNE5/ DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=flent.org; s=default; h=Sender:Content-Type:Cc:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Subject:Mime-Version: References:In-Reply-To:Message-ID:To:From:Date:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=1REyRVR30pTO7bhLk/a0U6AwlT/vKepXpxIJCh8XynI=; b=Q0bWFebM1XFc1E0ovNDgOtX8T2 u2ZJ9XkvwaY9vBA9kyvpLG6BjT9lO7Q5DA45hpWM/q4c1N3SIPyLI6L7A1I4BaNisYKmJaJOadiEJ 8XgGkQXxDul3IfDfzQIYfO/SeamqbWk8GzM6nhxoBd+/TKmZQA2dlXNNghPE2AXpAm05NtrE5nfB7 6qzo3w+jXPNjA7FwdFYaMa0Ba1FMEpj70o2bLV9Gyliwvldi6U7Qt5W3rBfkAstzpUZZoGm9JvpML EW8fAQAWb1Cwed9HkR1Iz6zYQkegqkFRpOpYCW90oTsY4GGMt3jLNl5XBkeFVwlVG8ihFXwdPEf61 51YLmNbA==; Received: from [::1] (port=53738 helo=web6.sd.eurovps.com) by web6.sd.eurovps.com with esmtp (Exim 4.89) (envelope-from ) id 1dwuiY-002nj2-IV; Tue, 26 Sep 2017 21:36:26 +0300 Received: from github-smtp2-ext2.iad.github.net ([192.30.252.193]:40339 helo=github-smtp2b-ext-cp1-prd.iad.github.net) by web6.sd.eurovps.com with esmtps (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89) (envelope-from ) id 1dwuiU-002ncr-8R for flent-users@flent.org; Tue, 26 Sep 2017 21:36:25 +0300 Date: Tue, 26 Sep 2017 11:35:38 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=github.com; s=pf2014; t=1506450938; bh=vpl56OBxPKOXd41qnmwO6/AqEezsxciSpd62SrBlZTQ=; h=From:Reply-To:To:Cc:In-Reply-To:References:Subject:List-ID: List-Archive:List-Post:List-Unsubscribe:From; b=B1jyNE5/KduCDVTJCnWJ2xRsmu1SYy5nRV5q8V1FeBDtGCIMf8jv8JWrXEITB+ktz VGsh/X/7RoLk0k05hDU0yiOjfUfXD7Ql+MfCeKQdNHqQ82LwTvlpqaanIiHez4JHbs DSzTIfqjMPzgoBd1aLdgrfxFr4uofH3ASAeSA7Ko= From: =?UTF-8?B?VG9rZSBIw7hpbGFuZC1Kw7hyZ2Vuc2Vu?= To: tohojo/flent Message-ID: In-Reply-To: References: Mime-Version: 1.0 Precedence: list X-GitHub-Sender: tohojo X-GitHub-Recipient: flent-users X-GitHub-Reason: subscribed X-Auto-Response-Suppress: All X-GitHub-Recipient-Address: flent-users@flent.org Subject: Re: [Flent-users] [tohojo/flent] packet loss stats (#106) X-BeenThere: flent-users@flent.org X-Mailman-Version: 2.1.23 List-Id: Flent discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: tohojo/flent Cc: Subscribed Content-Type: multipart/mixed; boundary="===============8462000004236082895==" Errors-To: flent-users-bounces@flent.org Sender: "Flent-users" X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - web6.sd.eurovps.com X-AntiAbuse: Original Domain - toke.dk X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - flent.org X-Get-Message-Sender-Via: web6.sd.eurovps.com: acl_c_authenticated_local_user: mailman/mailman X-Authenticated-Sender: web6.sd.eurovps.com: mailman@flent.org --===============8462000004236082895== Content-Type: multipart/alternative; boundary="--==_mimepart_59ca9dfa3ac46_274f33f9accb72f7c799a3"; charset=UTF-8 Content-Transfer-Encoding: 7bit ----==_mimepart_59ca9dfa3ac46_274f33f9accb72f7c799a3 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Pete Heist writes: >> - The data points are missing sequence numbers; makes it hard to infer= >> loss, and to relate IPDV to RTT values. > > Yes, because the seqno is just the array index. I=E2=80=99ll add the se= qno > explicitly to make it easier to consume. So what happens if a packet is lost? There'll be an empty element in the array? That's... unusual... ;) >> - Some of the 'params' are not terribly useful: >> - What are the local and remote addresses of? Is this where the server= >> listens? I'm guessing the client doesn't connect to 0.0.0.0 at >> least... Would be better to know the values that were actually used. > > Yes, I can get the actual local address and fix that. The remote > address would be there but I removed it manually as I usually don=E2=80= =99t > post internal IPs if I remember to remove them. BTW, I may have an > option to keep out any =E2=80=9Cinternal=E2=80=9D info, like hostnames = and IPs. > > Come to think of it I=E2=80=99d rather have params be params (what was > supplied to the API), then I can have something separate after Dial > has occurred with resolved addresses, actual IP version, etc. Likewise > it can be the case that the server doesn=E2=80=99t support the timestam= p mode > you requested, or the length of the packet, or it doesn=E2=80=99t suppo= rt DSCP > or DF (in the case of Windows), etc. It would be good to know both > what you asked for and what you actually got. Presumably the caller knows what is being asked for, so the important thing is what will actually be used. Depends a little bit on what you envisage the output being used for; if you plan on storing it by itself, then the supplied parameters might be useful to include. But for Flent's usage, the command line options are stored in the data file... >> - Similarly, it would be more useful to know whether packets were actu= ally sent >> as IPv4 or IPv6, rather than what was selected. >> - Which fields are guaranteed to be present and which can be blank? > > I=E2=80=99m omitting some that can be blank, but I see that it may be e= asier > for consumption if I include everything instead of documenting what > may or may not be there. Well, whether they are omitted or can be blank, it'll be important to know what to expect :) >> - What is the send and receive rates? Are they always the same? And in= >> which direction? Do they include packet loss? > > They=E2=80=99re based on: > > - for send, the total UDP payload data written to the socket between > right before the first send and right after the last send > > - for receive, the total UDP payload data received from the socket > between right after the first receive and right after the last receive > (dups not included) > > They may differ due to packet loss. Right, that's what I assumed; and that seems reasonable. > BTW, later I want to have the server return the total data received > for a flow to distinguish between upstream and downstream packet loss > (and maybe some bits with a window into which packets were lost). Yup, that could be useful :) > I do not include duplicates in received data, which raises a point. I > consider duplicates something you shouldn=E2=80=99t see unless there is= a > problem or misconfiguration somewhere, so other than having a > duplicate counter and warning about them I don=E2=80=99t include them i= n other > stats (bytes received, bitrate, RTT/ OWD, etc). If that=E2=80=99s misgu= ided, > if seeing duplicates is an ordinary thing on the open Internet, let me > know and I may reconsider what I do with the stats. Meh, don't think duplication is a huge concern over the public internet. Probably save to ignore for now... > I was eyeballing it before, plus that was before I added the json > package, which may or may not have dependencies in common, now it's: > > Unstripped increase: 153600 bytes > Stripped increase: 108544 bytes > > It=E2=80=99s not important enough to me now for the size. > > Adding the JSON encoder was a relative =E2=80=9Cwhopper=E2=80=9D at aro= und 250K > unstripped (also eyeballed). If I=E2=80=99m looking for somewhere to cu= t, I > could later find another encoder or just write JSON by hand. I did > some gyrations to avoid pulling in regexp in a couple of cases. :) Huh, that's quite impressive (and not in a good way). But then it probably can't run on the tiniest of devices anyway since there's no MIPS support in Go (last I looked anyway)... -Toke -- = You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/tohojo/flent/issues/106#issuecomment-332293836= ----==_mimepart_59ca9dfa3ac46_274f33f9accb72f7c799a3 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Pete Heist <notifications@github.com> writes:

>> - The data points are missing sequence numbers; makes it hard to= infer
>> loss, and to relate IPDV to RTT values.
>
> Yes, because the seqno is just the array index. I=E2=80=99ll add the= seqno
> explicitly to make it easier to consume.

So what happens if a packet is lost? There'll be an empty element in = the
array? That's... unusual... ;)

>> - Some of the 'params' are not terribly useful:
>> - What are the local and remote addresses of? Is this where the = server
>> listens? I'm guessing the client doesn't connect to 0.0.= 0.0 at
>> least... Would be better to know the values that were actually u= sed.
>
> Yes, I can get the actual local address and fix that. The remote
= > address would be there but I removed it manually as I usually don=E2= =80=99t
> post internal IPs if I remember to remove them. BTW, I may have an > option to keep out any =E2=80=9Cinternal=E2=80=9D info, like hostnam= es and IPs.
>
> Come to think of it I=E2=80=99d rather have params be params (what w= as
> supplied to the API), then I can have something separate after Dial<= br> > has occurred with resolved addresses, actual IP version, etc. Likewi= se
> it can be the case that the server doesn=E2=80=99t support the times= tamp mode
> you requested, or the length of the packet, or it doesn=E2=80=99t su= pport DSCP
> or DF (in the case of Windows), etc. It would be good to know both > what you asked for and what you actually got.

Presumably the caller knows what is being asked for, so the important
= thing is what will actually be used. Depends a little bit on what you
= envisage the output being used for; if you plan on storing it by itself,<= br> then the supplied parameters might be useful to include. But for Flent= 9;s
usage, the command line options are stored in the data file...

>> - Similarly, it would be more useful to know whether packets wer= e actually sent
>> as IPv4 or IPv6, rather than what was selected.
>> - Which fields are guaranteed to be present and which can be bla= nk?
>
> I=E2=80=99m omitting some that can be blank, but I see that it may b= e easier
> for consumption if I include everything instead of documenting what<= br> > may or may not be there.

Well, whether they are omitted or can be blank, it'll be important to=
know what to expect :)

>> - What is the send and receive rates? Are they always the same? = And in
>> which direction? Do they include packet loss?
>
> They=E2=80=99re based on:
>
> - for send, the total UDP payload data written to the socket between=
> right before the first send and right after the last send
>
> - for receive, the total UDP payload data received from the socket > between right after the first receive and right after the last recei= ve
> (dups not included)
>
> They may differ due to packet loss.

Right, that's what I assumed; and that seems reasonable.

> BTW, later I want to have the server return the total data received<= br> > for a flow to distinguish between upstream and downstream packet los= s
> (and maybe some bits with a window into which packets were lost).
Yup, that could be useful :)

> I do not include duplicates in received data, which raises a point. = I
> consider duplicates something you shouldn=E2=80=99t see unless there= is a
> problem or misconfiguration somewhere, so other than having a
> duplicate counter and warning about them I don=E2=80=99t include the= m in other
> stats (bytes received, bitrate, RTT/ OWD, etc). If that=E2=80=99s mi= sguided,
> if seeing duplicates is an ordinary thing on the open Internet, let = me
> know and I may reconsider what I do with the stats.

Meh, don't think duplication is a huge concern over the public intern= et.
Probably save to ignore for now...

> I was eyeballing it before, plus that was before I added the json > package, which may or may not have dependencies in common, now it= 9;s:
>
> Unstripped increase: 153600 bytes
> Stripped increase: 108544 bytes
>
> It=E2=80=99s not important enough to me now for the size.
>
> Adding the JSON encoder was a relative =E2=80=9Cwhopper=E2=80=9D at = around 250K
> unstripped (also eyeballed). If I=E2=80=99m looking for somewhere to= cut, I
> could later find another encoder or just write JSON by hand. I did > some gyrations to avoid pulling in regexp in a couple of cases. :)
Huh, that's quite impressive (and not in a good way). But then it
= probably can't run on the tiniest of devices anyway since there's= no
MIPS support in Go (last I looked anyway)...

-Toke

&m= dash;
You are receiving this because you are subscribed to this thre= ad.
Reply to this email directly, view it on GitHub, or mute the thread.3D""

= ----==_mimepart_59ca9dfa3ac46_274f33f9accb72f7c799a3-- --===============8462000004236082895== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Flent-users mailing list Flent-users@flent.org http://flent.org/mailman/listinfo/flent-users_flent.org --===============8462000004236082895==--