[Flent-users] [tohojo/flent] ss_iterate.sh unnecessarily runs for a long time (#236)

[Flent-users] [tohojo/flent] ss_iterate.sh unnecessarily runs for a long time (#236)

[Shashank D]

[-- Attachment #1: Type: text/plain, Size: 1735 bytes --]

I tried running a `tcp_nup` test in flent with `length` set to 200 seconds & `step-size` set to 0.05 seconds. But the test took more than 10 minutes to complete. On checking the log, I found out that the `ss_iterate.sh` was running  even after other runners had finished execution.  

https://github.com/tohojo/flent/blob/9378031415f9d87b2ef500d082f4720abcec52da/flent/scripts/ss_iterate.sh#L43-L45

My guess is that execution time of ss command was about 0.1 ish  second on average (could see from the output timestamps). So each iteration took about 0.15 seconds to execute and hence total execution of the 4200 iterations took about 10 mins.

I resolved this issue by checking current time against endtime.
```bash
command_string=$(cat <<EOF
duration="$(echo "$count*$interval" | bc) sec";
endtime=\$(date -d "\$duration" +%s%N);
while (( \$(date +%s%N) <= \$endtime )); do
    ss -t -i -p -n state connected "dst $target $filter"
```

Infact the issue exists with `tc_iterate.sh` as well. But the execution time only exceeded by about 20 seconds.

Now this might be a bad idea in general since the use of `bc` and the additional `date` commands may add some overhead (although I didn't see any notable difference b/w timestamps).  

Does this fix seem like a good PR (to `tc_iterate.sh` and `ss_iterate.sh`)? (The endtime calculation can be done from python code and passed to this script instead of passing ```count (test_length / interval)```)
 > (I am using flent with network namespaces in Ubuntu 20.04 machine. I am unsure if this works with other environments)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/tohojo/flent/issues/236

[-- Attachment #2: Type: text/html, Size: 3636 bytes --]

[Flent-users] Re: [tohojo/flent] ss_iterate.sh unnecessarily runs for a long time (#236)

[Toke Høiland-Jørgensen]

[-- Attachment #1: Type: text/plain, Size: 473 bytes --]

Hmm, good question. So you're just running ss in a busy loop, or did you
keep the 'sleep'? If so, that seems a bit excessive to do by default.

I'd be a bit surprised if you could get this short a polling time out of
the shell script, actually; what interval were you able to achieve?


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/tohojo/flent/issues/236#issuecomment-918112784

[-- Attachment #2: Type: text/html, Size: 1759 bytes --]

[Flent-users] Re: [tohojo/flent] ss_iterate.sh unnecessarily runs for a long time (#236)

[Shashank D]

[-- Attachment #1: Type: text/plain, Size: 1416 bytes --]

> Hmm, good question. So you're just running ss in a busy loop, or did you keep the 'sleep'? If so, that seems a bit excessive to do by default. 
Yes I did keep the sleep in the loop. Here's the full loop:
```bash
command_string=$(cat <<EOF
duration="$(echo "$count*$interval" | bc) sec";
endtime=\$(date -d "\$duration" +%s%N);
while (( \$(date +%s%N) <= \$endtime )); do
    ss -t -i -p -n state connected "dst $target $filter"
    echo ''
    date '+Time: %s.%N';
    echo "---";
    sleep $interval || exit 1;
done
EOF
)
```
> I'd be a bit surprised if you could get this short a polling time out of the shell script, actually; what interval were you able to achieve?
Each iteration of this loop takes almost the same time as with the current implementation in flent. So this doesn't really help to set smaller `step-size`. But the total time of all iterations will be closer to the expected test duration set by the user. 

https://github.com/tohojo/flent/blob/9378031415f9d87b2ef500d082f4720abcec52da/flent/runners.py#L1978-L1979

Here, I feel that the calculated count is a bit excessive and keeps the loop running for much longer than the set test duration. Using end time calculations in the loop will solve this issue


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/tohojo/flent/issues/236#issuecomment-918892985

[-- Attachment #2: Type: text/html, Size: 3671 bytes --]

[Flent-users] Re: [tohojo/flent] ss_iterate.sh unnecessarily runs for a long time (#236)

[Toke Høiland-Jørgensen]

[-- Attachment #1: Type: text/plain, Size: 1022 bytes --]

Shashank D ***@***.***> writes:

>> Hmm, good question. So you're just running ss in a busy loop, or did you keep the 'sleep'? If so, that seems a bit excessive to do by default. 
> Yes I did keep the sleep in the loop. Here's the full loop:
> ```bash
> command_string=$(cat <<EOF
> duration="$(echo "$count*$interval" | bc) sec";
> endtime=\$(date -d "\$duration" +%s%N);
> while (( \$(date +%s%N) <= \$endtime )); do
>     ss -t -i -p -n state connected "dst $target $filter"
>     echo ''
>     date '+Time: %s.%N';
>     echo "---";
>     sleep $interval || exit 1;
> done
> EOF
> )
> ```

Right, makes sense. Feel free to open a PR with this, but please lose
the dependency on 'bc'. Either do the math in native shell, or just pass
the duration as a parameter from Flent.

Did you test whether this works on OpenWrt and on Dash?


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/tohojo/flent/issues/236#issuecomment-919035245

[-- Attachment #2: Type: text/html, Size: 2521 bytes --]

[Flent-users] Re: [tohojo/flent] ss_iterate.sh unnecessarily runs for a long time (#236)

[Shashank D]

[-- Attachment #1: Type: text/plain, Size: 294 bytes --]

> Did you test whether this works on OpenWrt and on Dash?
No. I could only test this on a Ubuntu machine.


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/tohojo/flent/issues/236#issuecomment-919302179

[-- Attachment #2: Type: text/html, Size: 1579 bytes --]

[Flent-users] Re: [tohojo/flent] ss_iterate.sh unnecessarily runs for a long time (#236)

[Toke Høiland-Jørgensen]

[-- Attachment #1: Type: text/plain, Size: 485 bytes --]

Shashank D ***@***.***> writes:

>> Did you test whether this works on OpenWrt and on Dash?
> No. I could only test this on a Ubuntu machine.

Right, okay. Well, the 'dash' shell should be available to install on
ubuntu as well. Feel free to open a PR with the change, I can test it on
OpenWrt...


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/tohojo/flent/issues/236#issuecomment-919500031

[-- Attachment #2: Type: text/html, Size: 1787 bytes --]