I am using FIO tool on linux to run some IO's. I am interested to look at data contents that are generated as part of the FIO command.
My command:
sudo fio --name=randwrite --ioengine=libaio --iodepth=1 --rw=write --bs=4k --direct=0 --size=512M --numjobs=1 --runtime=240 --group_reporting --filename=venkata --buffer_compress_percentage=50 --output=fioad
I am interested to see how the data is generated with 50% compress buffer option. Is there any way to look / output the FIO IO input data?
Is there any way to look / output the FIO IO input data?
Just examine the file the I/O is being done on (venkata) with a tool like hexdump? The one thing I'd caution is because your I/O file is 512 megabytes big you will almost certainly want to use the -n flag of hexdump or pipe the output to less to prevent your terminal overflowing... Here's a safe reduced example job to make analysis quicker/easier:
$ fio --name=bufcontents --filename=/tmp/fio.tmp --size=4k --bs=4k --buffer_compress_percentage=50
bufcontents: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
[...]
Run status group 0 (all jobs):
READ: bw=4000KiB/s (4096kB/s), 4000KiB/s-4000KiB/s (4096kB/s-4096kB/s), io=4096B (4096B), run=1-1msec
$ hexdump -C /tmp/fio.tmp
00000000 35 e0 28 cc 38 a0 99 16 06 9c 6a a9 f2 cd e9 0a |5.(.8.....j.....|
00000010 80 53 2a 07 09 e5 0d 15 70 4a 25 f7 0b 39 9d 18 |.S*.....pJ%..9..|
00000020 4e a9 ac d9 8e ab 9d 13 29 95 8e 86 9b 48 4e 12 |N.......)....HN.|
00000030 a5 52 3d 26 cc 05 db 1b 54 2a 75 db 9a 4d d8 1d |.R=&....T*u..M..|
00000040 4a a5 44 c6 f8 9b 39 00 a9 94 23 c6 5c d0 90 0c |J.D...9...#.\...|
00000050 95 f2 6f ce f9 b6 c2 13 52 7e 83 40 a7 6f ce 07 |[email protected]..|
00000060 ca 6f e7 28 b3 2d e4 10 f9 ed 37 ad 42 f1 48 0f |.o.(.-....7.B.H.|
00000070 bf 7d aa 5e 8c c7 d6 00 b7 cf f5 4c 9c a9 cd 08 |.}.^.......L....|
00000080 f6 39 c3 a1 b8 8e 8c 18 3e 67 3d 77 f5 40 ef 0b |.9......>g=w.@..|
00000090 e7 ac 48 fb 7f 2c 35 1c 9c 95 f5 a8 eb a7 d7 19 |..H..,5.........|
000000a0 b3 b2 50 aa 82 20 89 0f 56 96 f0 fb e7 ce d4 03 |..P.. ..V.......|
000000b0 ca 12 53 b4 e4 9b e0 17 59 62 25 0d 53 b9 0f 0e |..S.....Yb%.S...|
000000c0 4b 2c 78 b0 97 70 47 13 89 85 e9 df d6 15 6a 09 |K,x..pG.......j.|
000000d0 b1 b0 38 19 c6 d2 c0 0e 16 96 ce 6a bc 0d 0c 15 |..8........j....|
000000e0 c2 d2 4e 42 50 4c dd 08 58 da e8 9e 62 88 c1 15 |..NBPL..X...b...|
000000f0 4b 1b b1 e6 97 e1 ee 00 69 a3 30 6f da e8 9e 17 |K.......i.0o....|
00000100 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00000200 a8 71 09 48 d3 ad 5f c5 35 2e 2d b0 b5 51 5a 13 |.q.H.._.5.-..QZ.|
[...]
(100 in hex is 256 in decimal, 200 in hex is 512 in decimal etc)
Alternatively, since you asked this on Stack Overflow (which is for programming questions), fio is open source and the fio source is available on GitHub, we can read the source there (note that you didn't say WHICH version of fio you are using so I shall assume the very latest at the tile of writing which is fio-3.21):
fill_io_buffer() in io_u.c): when compress_percentage is non-zero, fio will try to repeatedly call fill_random_buf_percentage() until it has created a block sized buffer worth of data. In the example job you set a block size of 4k (so the min and max block sizes will both be 4k) and compress_chunk will be 512 (its default). Therefore fill_random_buf_percentage() will be called 8 times and each time with its segment and len parameters being 512.__fill_random_buf_percentage() in rand.c): This works out the percentage of the segment it is filling that needs to be random data and the rest is set to either zeros (the default) or the buffer_pattern (if set). With your example the first 256 bytes will be random data and the next 256 will be zeros.