After fio using the norandommap parameter, the verify: bad header error occurs. #1756
Open
1 task done
Comments
|
Please try fio 3.37 and simplify your job to include the minimal options that produce the issue. |
I ‘ve already tried the latest version,and it has the same problem. When the norandommap parameter is used, some blocks on the SSD may be read and written multiple times. It seems that the problem lies in this problem. If the norandommap parameter is not used, the fio obtains only the new offset, and the blocks will not be written/read for many times. |
|
You can't use norandommap with verify. I think fio allows it for fixed block sizes, but it's not sane if you're tracking generations of writes. |
|
@axboe ok，thank you.Do you consider supporting norandommap with verify in the future? |
vincentkfu
pushed a commit
to vincentkfu/fio
that referenced
this issue
Jan 28, 2025
When norandommap is enabled, fio logs the I/O entries in a RB tree. This is to account for offset overlaps and overwrites. Then during verify phase, the I/O entries are picked from the top and in this case the smallest offset is verified first and so on. This creates a mismatch during the header verification as the random seed generated at the time of read differs from what was logged during write. Overwrite the I/O entry random seed to prevent this. The existing non readwrite check to overwrite the I/O entry random seed was preventing this issue for write or read cases. But for mixed readwrite workloads this has been observed. This also alters the readwrite check to just write. Moreover, this also fixes any verify issues with async I/O engines with queue depth > 1. Update the documentation accordingly. fixes axboe#1756 Signed-off-by: Ankit Kumar <[email protected]>
ankit-sam
added a commit
to ankit-sam/fio
that referenced
this issue
Jan 29, 2025
When norandommap is enabled, fio logs the I/O entries in a RB tree. This is to account for offset overlaps and overwrites. Then during verify phase, the I/O entries are picked from the top and in this case the smallest offset is verified first and so on. This creates a mismatch during the header verification as the random seed generated at the time of read differs from what was logged during write. Overwrite the I/O entry random seed to prevent this. The existing non readwrite check to overwrite the I/O entry random seed was preventing this issue for write or read cases. But for mixed readwrite workloads this has been observed. This also alters the readwrite check to just write. fixes axboe#1756 Signed-off-by: Ankit Kumar <[email protected]>
ankit-sam
added a commit
to ankit-sam/fio
that referenced
this issue
Jan 30, 2025
When norandommap is enabled, fio logs the I/O entries in a RB tree. This is to account for offset overlaps and overwrites. Then during verify phase, the I/O entries are picked from the top and in this case the smallest offset is verified first and so on. This creates a mismatch during the header verification as the random seed generated at the time of read differs from what was logged during write. Overwrite the I/O entry random seed to prevent this. The existing non readwrite check to overwrite the I/O entry random seed was preventing this issue for write or read cases. But for mixed readwrite workloads this has been observed. This also alters the readwrite check to just write. fixes axboe#1756 Signed-off-by: Ankit Kumar <[email protected]>
ankit-sam
added a commit
to ankit-sam/fio
that referenced
this issue
Feb 4, 2025
When norandommap is enabled, fio logs the I/O entries in a RB tree. This is to account for offset overlaps and overwrites. Then during verify phase, the I/O entries are picked from the top and in this case the smallest offset is verified first and so on. This creates a mismatch during the header verification as the seed generated at the time of read differs from what was logged during write. Skip seed verification in this scenario. fixes axboe#1756 Signed-off-by: Ankit Kumar <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Please acknowledge the following before creating a ticket
Description of the bug:
It seems that the norandommap parameter of fio affects the consistency check result.
Environment:
[root@localhost ~]# uname -a
Linux localhost.localdomain 5.10.0-60.18.0.50.r1064_49.hce2.aarch64 #1 SMP Tue Nov 28 08:26:19 UTC 2023 aarch64 aarch64 aarch64 GNU/Linux
fio version:
fio-3.29,
Reproduction steps
1、without norandommap ，fio task success
fio --name=ssd --numjobs=1 --filename=/home/test.io --bs=1m --iodepth=256 --rw=randrw -rwmixread=50 --ioengine=libaio --direct=1 --sync=0 --group_reporting --runtime=100 --time_based --do_verify=1 --verify=md5 --size=10G
2、with norandommap， fio fail
The text was updated successfully, but these errors were encountered: