使用FIO测试云主机IOPS及写入读取速度
先安装fio工具：
yum install fio -yfio参数说明：
filename=/dev/emcpowerb 支持文件系统或者裸设备，-filename=/dev/sda2或-filename=/dev/sdb
direct=1 测试过程绕过机器自带的buffer，使测试结果更真实
rw=randwread 测试随机读的I/O
rw=randwrite 测试随机写的I/O
rw=randrw 测试随机混合写和读的I/O
rw=read 测试顺序读的I/O
rw=write 测试顺序写的I/O
rw=rw 测试顺序混合写和读的I/O
bs=4k 单次io的块文件大小为4k
bsrange=512-2048 同上，提定数据块的大小范围
size=5g 本次的测试文件大小为5g，以每次4k的io进行测试
numjobs=30 本次的测试线程为30
runtime=1000 测试时间为1000秒，如果不写则一直将5g文件分4k每次写完为止
ioengine=psync io引擎使用pync方式，如果要使用libaio引擎，需要yum install libaio-devel包
rwmixwrite=30 在混合读写的模式下，写占30%
group_reporting 关于显示结果的，汇总每个进程的信息
此外
lockmem=1g 只使用1g内存进行测试
zero_buffers 用0初始化系统buffer
nrfiles=8 每个进程生成文件的数量测试命令(创建100G容量大小的文件)
fio -direct=1 -iodepth=128 -rw=write -ioengine=libaio -bs=4k -size=100G -numjobs=1 -runtime=1000 -group_reporting -name=test -filename=/data/test111运行结果：
test: (g=0): rw=write, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=128
fio-2.0.13
Starting 1 process
test: Laying out IO file(s) (1 file(s) / 102400MB)
Jobs: 1 (f=1): [W] [100.0% done] [0K/129.3M/0K /s] [0 /33.1K/0 iops] [eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=594: Mon May 14 10:27:54 2018
write: io=102400MB, bw=129763KB/s, iops=32440 , runt=808070msec
slat (usec): min=0 , max=80322 , avg=12.06, stdev=24.73
clat (usec): min=222 , max=254410 , avg=3932.47, stdev=5007.90
lat (usec): min=622 , max=254416 , avg=3944.82, stdev=5007.26
clat percentiles (usec):
| 1.00th=[ 1288], 5.00th=[ 1560], 10.00th=[ 1736], 20.00th=[ 1992],
| 30.00th=[ 2224], 40.00th=[ 2480], 50.00th=[ 2736], 60.00th=[ 3088],
| 70.00th=[ 3536], 80.00th=[ 4320], 90.00th=[ 6624], 95.00th=[ 9664],
| 99.00th=[21120], 99.50th=[37632], 99.90th=[54528], 99.95th=[78336],
| 99.99th=[177152]
bw (KB/s) : min=20720, max=215424, per=100.00%, avg=129801.50, stdev=19878.75
lat (usec) : 250=0.01%, 750=0.01%, 1000=0.09%
lat (msec) : 2=20.47%, 4=56.01%, 10=18.78%, 20=3.20%, 50=1.33%
lat (msec) : 100=0.09%, 250=0.02%, 500=0.01%
cpu : usr=4.88%, sys=36.64%, ctx=9409837, majf=0, minf=23
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
issued : total=r=0/w=26214400/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
WRITE: io=102400MB, aggrb=129763KB/s, minb=129763KB/s, maxb=129763KB/s, mint=808070msec, maxt=808070msec
Disk stats (read/write):
vdb: ios=0/26203032, merge=0/3962, ticks=0/90681446, in_queue=90672667, util=100.00%100%随机，100%读，4K
[root@test-db data]# fio -filename=/dev/vdb1 -direct=1 -iodepth 1 -thread -rw=randread -ioengine=psync -bs=4k -size=100G -numjobs=50 -runtime=180 -group_reporting -name=rand_100read_4k运行结果：
rand_100read_4k: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=psync, iodepth=1
...
rand_100read_4k: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=psync, iodepth=1
fio-2.0.13
Starting 50 threads
Jobs: 50 (f=50): [rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr] [100.0% done] [47544K/0K/0K /s] [11.9K/0 /0 iops] [eta 00m:00s]
rand_100read_4k: (groupid=0, jobs=50): err= 0: pid=25884: Mon May 14 14:59:44 2018
read : io=8454.3MB, bw=48091KB/s, iops=12022 , runt=180018msec
clat (usec): min=318 , max=167471 , avg=4156.60, stdev=8363.94
lat (usec): min=318 , max=167472 , avg=4156.89, stdev=8363.95
clat percentiles (usec):
| 1.00th=[ 1032], 5.00th=[ 1176], 10.00th=[ 1240], 20.00th=[ 1352],
| 30.00th=[ 1464], 40.00th=[ 1688], 50.00th=[ 1832], 60.00th=[ 1944],
| 70.00th=[ 2064], 80.00th=[ 2288], 90.00th=[15808], 95.00th=[18560],
| 99.00th=[20608], 99.50th=[39168], 99.90th=[134144], 99.95th=[144384],
| 99.99th=[156672]
bw (KB/s) : min= 231, max= 7248, per=2.00%, avg=961.78, stdev=384.46
lat (usec) : 500=0.05%, 750=0.20%, 1000=0.51%
lat (msec) : 2=64.37%, 4=19.62%, 10=2.88%, 20=10.62%, 50=1.52%
lat (msec) : 100=0.02%, 250=0.23%
cpu : usr=0.00%, sys=0.02%, ctx=1813752, majf=18446744073709550866, minf=18446744073698419008
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=2164296/w=0/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=8454.3MB, aggrb=48090KB/s, minb=48090KB/s, maxb=48090KB/s, mint=180018msec, maxt=180018msec
Disk stats (read/write):
vdb: ios=2163559/42, merge=23/10, ticks=8925036/223, in_queue=8922540, util=99.94%100%随机，100%写， 4K
[root@test-db data]# fio -filename=/dev/vdb1 -direct=1 -iodepth 1 -thread -rw=randwrite -ioengine=psync -bs=4k -size=100G -numjobs=50 -runtime=180 -group_reporting -name=rand_100write_4k运行结果：
rand_100write_4k: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=psync, iodepth=1
...
rand_100write_4k: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=psync, iodepth=1
fio-2.0.13
Starting 50 threads
Jobs: 50 (f=50): [wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww] [100.0% done] [0K/47996K/0K /s] [0 /11.1K/0 iops] [eta 00m:00s]
rand_100write_4k: (groupid=0, jobs=50): err= 0: pid=25963: Mon May 14 15:08:43 2018
write: io=8445.5MB, bw=48039KB/s, iops=12009 , runt=180024msec
clat (usec): min=558 , max=248717 , avg=4160.45, stdev=8315.33
lat (usec): min=559 , max=248717 , avg=4161.04, stdev=8315.34
clat percentiles (usec):
| 1.00th=[ 1256], 5.00th=[ 1528], 10.00th=[ 1656], 20.00th=[ 1816],
| 30.00th=[ 1928], 40.00th=[ 2024], 50.00th=[ 2128], 60.00th=[ 2224],
| 70.00th=[ 2384], 80.00th=[ 2704], 90.00th=[ 8768], 95.00th=[17792],
| 99.00th=[24448], 99.50th=[38656], 99.90th=[130560], 99.95th=[191488],
| 99.99th=[226304]
bw (KB/s) : min= 220, max= 4272, per=2.00%, avg=960.02, stdev=343.97
lat (usec) : 750=0.04%, 1000=0.28%
lat (msec) : 2=36.58%, 4=48.13%, 10=5.45%, 20=6.74%, 50=2.59%
lat (msec) : 100=0.05%, 250=0.14%
cpu : usr=0.00%, sys=0.08%, ctx=1907994, majf=0, minf=18446744073699280913
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=0/w=2162044/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
WRITE: io=8445.5MB, aggrb=48039KB/s, minb=48039KB/s, maxb=48039KB/s, mint=180024msec, maxt=180024msec
Disk stats (read/write):
vdb: ios=148/2158380, merge=22/10, ticks=256/8928184, in_queue=8926408, util=99.96%100%顺序，100%读 ，4K
fio -filename=/dev/vdb1 -direct=1 -iodepth 1 -thread -rw=read -ioengine=psync -bs=4k -size=100G -numjobs=50 -runtime=180 -group_reporting -name=sqe_100read_4k运行结果：
sqe_100read_4k: (g=0): rw=read, bs=4K-4K/4K-4K/4K-4K, ioengine=psync, iodepth=1
fio-2.0.13
Starting 50 threads
Jobs: 50 (f=50): [RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR] [100.0% done] [59996K/0K/0K /s] [14.1K/0 /0 iops] [eta 00m:00s]
sqe_100read_4k: (groupid=0, jobs=50): err= 0: pid=26047: Mon May 14 15:13:02 2018
read : io=10599MB, bw=60295KB/s, iops=15073 , runt=180006msec
clat (usec): min=253 , max=550591 , avg=3315.70, stdev=26406.30
lat (usec): min=254 , max=550591 , avg=3315.95, stdev=26406.30
clat percentiles (usec):
| 1.00th=[ 1176], 5.00th=[ 1240], 10.00th=[ 1256], 20.00th=[ 1288],
| 30.00th=[ 1336], 40.00th=[ 1368], 50.00th=[ 1416], 60.00th=[ 1480],
| 70.00th=[ 1528], 80.00th=[ 1592], 90.00th=[ 1736], 95.00th=[ 2800],
| 99.00th=[17792], 99.50th=[18816], 99.90th=[522240], 99.95th=[528384],
| 99.99th=[536576]
bw (KB/s) : min= 15, max= 2475, per=2.00%, avg=1206.47, stdev=634.32
lat (usec) : 500=0.01%, 750=0.01%, 1000=0.04%
lat (msec) : 2=92.64%, 4=3.87%, 10=1.05%, 20=2.03%, 50=0.01%
lat (msec) : 250=0.01%, 500=0.17%, 750=0.15%
cpu : usr=0.05%, sys=0.23%, ctx=2697397, majf=0, minf=18446744073708900853
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=2713362/w=0/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=10599MB, aggrb=60294KB/s, minb=60294KB/s, maxb=60294KB/s, mint=180006msec, maxt=180006msec
Disk stats (read/write):
vdb: ios=2711304/55, merge=1805/10, ticks=8899022/202, in_queue=8898860, util=100.00%100%顺序，100%写 ，4K
fio -filename=/dev/vdb1 -direct=1 -iodepth 1 -thread -rw=write -ioengine=psync -bs=4k -size=100G -numjobs=50 -runtime=180 -group_reporting -name=sqe_100write_4k
运行结果：
sqe_100write_4k: (g=0): rw=write, bs=4K-4K/4K-4K/4K-4K, ioengine=psync, iodepth=1
...
sqe_100write_4k: (g=0): rw=write, bs=4K-4K/4K-4K/4K-4K, ioengine=psync, iodepth=1
fio-2.0.13
Starting 50 threads
Jobs: 50 (f=50): [WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW] [100.0% done] [0K/55788K/0K /s] [0 /13.1K/0 iops] [eta 00m:00s]
sqe_100write_4k: (groupid=0, jobs=50): err= 0: pid=26100: Mon May 14 15:17:24 2018
write: io=10002MB, bw=56896KB/s, iops=14224 , runt=180019msec
clat (usec): min=583 , max=457330 , avg=3513.01, stdev=9958.13
lat (usec): min=584 , max=457331 , avg=3513.60, stdev=9958.13
clat percentiles (usec):
| 1.00th=[ 1224], 5.00th=[ 1384], 10.00th=[ 1480], 20.00th=[ 1592],
| 30.00th=[ 1672], 40.00th=[ 1752], 50.00th=[ 1832], 60.00th=[ 1928],
| 70.00th=[ 2096], 80.00th=[ 2480], 90.00th=[ 6368], 95.00th=[12480],
| 99.00th=[20608], 99.50th=[37120], 99.90th=[166912], 99.95th=[268288],
| 99.99th=[346112]
bw (KB/s) : min= 152, max= 2432, per=2.01%, avg=1141.03, stdev=401.10
lat (usec) : 750=0.01%, 1000=0.11%
lat (msec) : 2=65.06%, 4=21.62%, 10=7.36%, 20=4.40%, 50=1.24%
lat (msec) : 100=0.08%, 250=0.06%, 500=0.07%
cpu : usr=0.05%, sys=0.38%, ctx=2547790, majf=0, minf=18446744073708899536
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=0/w=2560604/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
WRITE: io=10002MB, aggrb=56896KB/s, minb=56896KB/s, maxb=56896KB/s, mint=180019msec, maxt=180019msec
Disk stats (read/write):
vdb: ios=63/2558949, merge=0/275, ticks=12/8897256, in_queue=8895411, util=100.00%100%随机，70%读，30%写 4K
fio -filename=/dev/vdb1 -direct=1 -iodepth 1 -thread -rw=randrw -rwmixread=70 -ioengine=psync -bs=4k -size=100G -numjobs=50 -runtime=180 -group_reporting -name=randrw_70read_4k
运行结果：
randrw_70read_4k: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=psync, iodepth=1
...
randrw_70read_4k: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=psync, iodepth=1
fio-2.0.13
Starting 50 threads
Jobs: 50 (f=50): [mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm] [100.0% done] [48012K/20800K/0K /s] [12.3K/5200 /0 iops] [eta 00m:00s]
randrw_70read_4k: (groupid=0, jobs=50): err= 0: pid=26162: Mon May 14 15:29:18 2018
read : io=8430.4MB, bw=47954KB/s, iops=11988 , runt=180020msec
clat (usec): min=304 , max=177969 , avg=3045.60, stdev=5103.08
lat (usec): min=304 , max=177970 , avg=3045.87, stdev=5103.08
clat percentiles (usec):
| 1.00th=[ 860], 5.00th=[ 1048], 10.00th=[ 1160], 20.00th=[ 1320],
| 30.00th=[ 1448], 40.00th=[ 1560], 50.00th=[ 1656], 60.00th=[ 1768],
| 70.00th=[ 1896], 80.00th=[ 2128], 90.00th=[ 4384], 95.00th=[17792],
| 99.00th=[20352], 99.50th=[28544], 99.90th=[41216], 99.95th=[57088],
| 99.99th=[127488]
bw (KB/s) : min= 206, max= 4424, per=2.00%, avg=959.77, stdev=408.18
write: io=3613.8MB, bw=20556KB/s, iops=5138 , runt=180020msec
clat (usec): min=569 , max=157336 , avg=2617.23, stdev=2821.20
lat (usec): min=570 , max=157337 , avg=2617.81, stdev=2821.20
clat percentiles (usec):
| 1.00th=[ 1096], 5.00th=[ 1432], 10.00th=[ 1592], 20.00th=[ 1768],
| 30.00th=[ 1912], 40.00th=[ 2040], 50.00th=[ 2160], 60.00th=[ 2288],
| 70.00th=[ 2416], 80.00th=[ 2576], 90.00th=[ 2864], 95.00th=[ 3472],
| 99.00th=[19328], 99.50th=[20352], 99.90th=[22400], 99.95th=[37632],
| 99.99th=[52992]
bw (KB/s) : min= 62, max= 1888, per=2.00%, avg=411.23, stdev=178.61
lat (usec) : 500=0.06%, 750=0.22%, 1000=2.28%
lat (msec) : 2=61.13%, 4=27.68%, 10=3.02%, 20=4.48%, 50=1.07%
lat (msec) : 100=0.04%, 250=0.01%
cpu : usr=0.01%, sys=0.08%, ctx=2743663, majf=0, minf=18446744073698904078
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=2158165/w=925122/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=8430.4MB, aggrb=47953KB/s, minb=47953KB/s, maxb=47953KB/s, mint=180020msec, maxt=180020msec
WRITE: io=3613.8MB, aggrb=20555KB/s, minb=20555KB/s, maxb=20555KB/s, mint=180020msec, maxt=180020msec
Disk stats (read/write):
vdb: ios=2154562/923535, merge=0/0, ticks=6503566/2394699, in_queue=8896553, util=99.93%执行结果说明：
io=执行了多少M的IO
bw=平均IO带宽
iops=IOPS
runt=线程运行时间
slat=提交延迟
clat=完成延迟
lat=响应时间
bw=带宽
cpu=利用率
IO depths=io队列
IO submit=单个IO提交要提交的IO数
IO complete=Like the above submit number, but for completions instead.
IO issued=The number of read/write requests issued, and how many of them were short.
IO latencies=IO完延迟的分布
io=总共执行了多少size的IO
aggrb=group总带宽
minb=最小.平均带宽.
maxb=最大平均带宽.
mint=group中线程的最短运行时间.
maxt=group中线程的最长运行时间.
ios=所有group总共执行的IO数.
merge=总共发生的IO合并数.
ticks=Number of ticks we kept the disk busy.
io_queue=花费在队列上的总共时间.
util=磁盘利用率
Tag标签:「云主机 速度 iops fio」更新时间:「2021-11-03 14:21:24」阅读次数:「1055」