virtual_sdcard: fadvice reads

[nefelim4ag]

We know that os.read() will block, we assume that blocking is short enough so we can ignore that.
Generally it is true. I think there can be a situations where this assumption doesn't hold.
I think this is what happened in the log below:

Stats 215403.7: gcodein=0 mcu: mcu_awake=0.042 mcu_task_avg=0.000045 mcu_task_stddev=0.000044 bytes_write=108252997 bytes_read=33659186 bytes_retransmit=0 bytes_invalid=0 send_seq=2187724 receive_seq=2187724 retransmit_seq=0 srtt=0.001 rttvar=0.000 rto=0.025 ready_bytes=0 upcoming_bytes=0 freq=64001294 sd_pos=35013429 MCU: temp=36.5 CB1: temp=35.8 heater_bed: target=70 temp=70.1 pwm=0.197 print_time=108432.976 buffer_time=1.006 print_stall=6 extruder: target=210 temp=210.0 pwm=0.566 sysload=0.31 cputime=4980.999 memavail=487680
Stats 215405.4: gcodein=0 mcu: mcu_awake=0.042 mcu_task_avg=0.000045 mcu_task_stddev=0.000044 bytes_write=108256705 bytes_read=33661930 bytes_retransmit=0 bytes_invalid=0 send_seq=2187799 receive_seq=2187799 retransmit_seq=0 srtt=0.002 rttvar=0.002 rto=0.025 ready_bytes=25 upcoming_bytes=0 freq=64001295 sd_pos=35015094 MCU: temp=36.5 CB1: temp=35.9 heater_bed: target=70 temp=70.2 pwm=0.090 print_time=108433.623 buffer_time=0.018 print_stall=6 extruder: target=210 temp=210.2 pwm=0.410 sysload=0.31 cputime=4981.139 memavail=495468
<shutdown TTC>

It looks to me that introduce of the thread and all managment will be too combersome, to do a general solution for low probability case.
So, I purporse the fadvice WILLNEED hack, that will ask OS to readahead for us beforehand.
Alas, it will still do synchronous readahead sometimes.
So, there is initial query to preread the whole file (size 0).

I did test the sequential and random flag.
The thing is, readahead are synchronous. So, read if it triggers the readahead, will block longer.
I can reproduce 7ms stalls where normally read takes ~20us.
Random on the other hand simply disables the readahead logic and makes all reads consistently slow (0.4ms in my case).

How I profile that sudo strace -tT -e trace=read -p 6306 |& grep '= 8192'

14:49:26 read(26, " ; some bytes overhead\nG4 P20 ; "..., 8192) = 8192 <0.000017>
14:49:32 read(26, "overhead\nG4 P20 ; some bytes ove"..., 8192) = 8192 <0.000024>
14:49:38 read(26, "0 ; some bytes overhead\nG4 P20 ;"..., 8192) = 8192 <0.000027>
14:49:43 read(26, " overhead\nG4 P20 ; some bytes ov"..., 8192) = 8192 <0.000018>

And this is my test gcode:

for i in {1..50000}; do echo 'G4 P20 ; some bytes overhead' >> iotest.gcode; done

(iostat used to see IO size and sysctl vm.drop_caches=3 to reproduce something).

klippy_buffer_zero.log

Testing:

git fetch origin pull/7168/head
git checkout FETCH_HEAD
sudo systemctl restart klipper

[github-actions]

Thank you for your contribution to Klipper. Unfortunately, a reviewer has not assigned themselves to this GitHub Pull Request. All Pull Requests are reviewed before merging, and a reviewer will need to volunteer. Further information is available at: https://www.klipper3d.org/CONTRIBUTING.html

There are some steps that you can take now:

1. Perform a self-review of your Pull Request by following the steps at: https://www.klipper3d.org/CONTRIBUTING.html#what-to-expect-in-a-review
   If you have completed a self-review, be sure to state the results of that self-review explicitly in the Pull Request comments. A reviewer is more likely to participate if the bulk of a review has already been completed.
2. Consider opening a topic on the Klipper Discourse server to discuss this work. The Discourse server is a good place to discuss development ideas and to engage users interested in testing. Reviewers are more likely to prioritize Pull Requests with an active community of users.
3. Consider helping out reviewers by reviewing other Klipper Pull Requests. Taking the time to perform a careful and detailed review of others work is appreciated. Regular contributors are more likely to prioritize the contributions of other regular contributors.

Unfortunately, if a reviewer does not assign themselves to this GitHub Pull Request then it will be automatically closed. If this happens, then it is a good idea to move further discussion to the Klipper Discourse server. Reviewers can reach out on that forum to let you know if they are interested and when they are available.

Best regards,
~ Your friendly GitIssueBot

PS: I'm just an automated script, not a human being.

[nefelim4ag]

I think threads are a more appropriate way to handle that: #7230