美文网首页
测试工具:FIO --- IO引擎学习

测试工具:FIO --- IO引擎学习

作者: 小鱼哥儿 | 来源:发表于2020-05-27 19:40 被阅读0次

FIO是怎么将I/O发送到文件中的呢?是通过下面的引擎来进行操作的:

**sync**

    使用read()、write()、lseek()函数来进行IO的读写和定位

**psync**

    使用pread()、pwrite()来进行IO的读写,在所有系统都支持(除了windows)

    由于lseek和read 调用之间,内核可能会临时挂起进程,所以对同步造成了问题,调用pread相当于顺序调用了lseek和read,这两个操作相当于一个捆绑的原子操作。

    由于lseek和write 调用之间,内核可能会临时挂起进程,所以对同步问题造成了问题,调用pwrite相当于顺序调用了lseek和write,这两个操作相当于一个捆绑的原子操作。

    这两个函数无法中断其定位和读写操作,另外不更新文件指针

**vsync**

    模仿队列的IO合并功能,尽量减少提交数

    readv/writev:在一次函数调用中读、写多个非连续缓冲区,但是这些缓冲区已经用iovec表示好了,减少了系统调用的次数。

**pvsync**

    pwritev()系统调用结合了writev()和的功能pwrite ()。它执行与writev()相同的任务,但是添加了第四个参数偏移量,指定输出所在的文件偏移量要进行操作。这些系统调用不会更改文件偏移量。即,读写与定位的原子操作 + 队列合并。

**pvsync2**

    同pvsync类似,多了第五个参数,用于设置本次操作的属性,如:

    RWF_DSYNC (since Linux 4.7) Provide a per-write equivalent of theO_DSYNC open(2) flag. This flag is meaningful only forpwritev2(), and its effect applies only to the data range written by the system call.(O_DSYNC:每次write都等待物理I/O完成,但是如果写操作不影响读取刚写入的数据,则不等待文件属性更新)

    RWF_HIPRI (since Linux 4.6) High priority read/write. Allows block-based filesystems to use polling of the device, which provides lower latency, but may use additional resources. (Currently, this feature is usable only on a file descriptor opened using theO_DIRECTflag.)

    RWF_SYNC (since Linux 4.7) Provide a per-write equivalent of theO_SYNC open(2) flag. This flag is meaningful only forpwritev2(), and its effect applies only to the data range written by the system call.(O_SYNC:每次write都等到物理I/O完成,包括write引起的文件属性的更新)

    RWF_NOWAIT (since Linux 4.14) Do not wait for data which is not immediately available. If this flag is specified, thepreadv2() system call will return instantly if it would have to read data from the backing stor‐ age or wait for a lock. If some data was successfully read, it will return the number of bytes read. If no bytes were read, it will return -1 and seterrnotoEAGAIN. Currently, this flag is meaningful only forpreadv2().

    RWF_APPEND (since Linux 4.16) Provide a per-write equivalent of theO_APPEND open(2) flag. This flag is meaningful only forpwritev2(), and its effect applies only to the data range written by the system call. Theoffsetargument does not affect the write operation; the data is always appended to the end of the file. However, if theoffsetargument is -1, the current file offset is updated

**io_uring**

linux原生AIO的升级版,易用且高效,linux5.1内核版本开始支持.

**libaio**

    linux2.6内核之后就有的本地异步非阻塞IO调用,使用此引擎会指定部分选项。

**posixaio**

    POSIX1003.1b 实时扩展协议规定的标准异步 I/O 接口,即 aio_read 函数、 aio_write 函数、aio_fsync 函数、aio_cancel 函数、aio_error 函数、aio_return 函数、aio_suspend函数和 lio_listio 函数。这组 API 用来操作异步 I/O。

**solarisaio**

    使用Solaris系统本地的异步IO接口

**windowsaio**

    windows本地的IO接口

**mmap**

    文件通过内存映射到用户空间,使用memcpy写入和读出数据

**splice**

    使用splice和vmsplice在用户空间和内核之间传输数据

**sg**

    SCSI generic sg v3 io.可以是使用SG_IO ioctl来同步,或是目标是一个sg字符设备,我们使用read和write执行异步IO

**null**

    不传输任何数据,只是伪装成这样。主要用于训练使用fio,或是基本debug/test的目的

**net**

    根据给定的host:port通过网络传输数据。根据具体的协议,hostname,port,listen,filename这些选项将被用来说明建立哪种连接,协议选项将决定哪种协议被使用。

**netsplice**

    像net,但是使用splic/vmsplice来映射数据和发送/接收数据。

**cpuio**

    不传输任何的数据,但是要根据cpuload=和cpucycle=选项占用CPU周期.e.g. cpuload=85将使用job不做任何的实际IO,但要占用85%的CPU周期。在SMP机器上,使用numjobs=<no_of_cpu>来获取需要的CPU,因为cpuload仅会载入单个CPU,然后占用需要的比例。

**guasi**

    GUASI IO引擎是一般的用于异步IO的用户空间异步系统调用接口

**rdma**

    RDMA I/O引擎支持RDMA内存语义(RDMA_WRITE/RDMA_READ)和通道主义(Send/Recv)用于InfiniBand,RoCE和iWARP协议

    external指明要调用一个外部的IO引擎(二进制文件)。e.g. ioengine=external:/tmp/foo.o将载入/tmp下的foo.o这个IO引擎

**falloc**

I/O engine that does regular fallocate to simulate data transfer as

fio ioengine.

DDIR_READ

does fallocate(,mode = FALLOC_FL_KEEP_SIZE,).

DDIR_WRITE

does fallocate(,mode = 0).

DDIR_TRIM

does fallocate(,mode = FALLOC_FL_KEEP_SIZE|FALLOC_FL_PUNCH_HOLE).

**ftruncate**

I/O engine that sends :manpage:`ftruncate(2)` operations in response

to write (DDIR_WRITE) events. Each ftruncate issued sets the file's

size to the current block offset. :option:`blocksize` is ignored.

**e4defrag**

I/O engine that does regular EXT4_IOC_MOVE_EXT ioctls to simulate

defragment activity in request to DDIR_WRITE event.

**rados**

I/O engine supporting direct access to Ceph Reliable Autonomic

Distributed Object Store (RADOS) via librados. This ioengine

defines engine specific options.

**rbd**

I/O engine supporting direct access to Ceph Rados Block Devices

(RBD) via librbd without the need to use the kernel rbd driver. This

ioengine defines engine specific options.

**http**

I/O engine supporting GET/PUT requests over HTTP(S) with libcurl to

a WebDAV or S3 endpoint.  This ioengine defines engine specific options.

This engine only supports direct IO of iodepth=1; you need to scale this

via numjobs. blocksize defines the size of the objects to be created.

TRIM is translated to object deletion.

**gfapi**

Using GlusterFS libgfapi sync interface to direct access to

GlusterFS volumes without having to go through FUSE.  This ioengine

defines engine specific options.

**gfapi_async**

Using GlusterFS libgfapi async interface to direct access to

GlusterFS volumes without having to go through FUSE. This ioengine

defines engine specific options.

**libhdfs**

Read and write through Hadoop (HDFS).  The :option:`filename` option

is used to specify host,port of the hdfs name-node to connect.  This

engine interprets offsets a little differently.  In HDFS, files once

created cannot be modified so random writes are not possible. To

imitate this the libhdfs engine expects a bunch of small files to be

created over HDFS and will randomly pick a file from them

based on the offset generated by fio backend (see the example

job file to create such files, use ``rw=write`` option). Please

note, it may be necessary to set environment variables to work

with HDFS/libhdfs properly.  Each job uses its own connection to

HDFS.

**mtd**

Read, write and erase an MTD character device (e.g.,

:file:`/dev/mtd0`). Discards are treated as erases. Depending on the

underlying device type, the I/O may have to go in a certain pattern,

e.g., on NAND, writing sequentially to erase blocks and discarding

before overwriting. The `trimwrite` mode works well for this

constraint.

**pmemblk**

Read and write using filesystem DAX to a file on a filesystem

mounted with DAX on a persistent memory device through the PMDK

libpmemblk library.

**dev-dax**

Read and write using device DAX to a persistent memory device (e.g.,

/dev/dax0.0) through the PMDK libpmem library.

**external**

Prefix to specify loading an external I/O engine object file. Append

the engine filename, e.g. ``ioengine=external:/tmp/foo.o`` to load

ioengine :file:`foo.o` in :file:`/tmp`. The path can be either

absolute or relative. See :file:`engines/skeleton_external.c` for

details of writing an external I/O engine.

**filecreate**

Simply create the files and do no I/O to them.  You still need to

set  `filesize` so that all the accounting still occurs, but no

actual I/O will be done other than creating the file.

**filestat**

Simply do stat() and do no I/O to the file. You need to set 'filesize'

and 'nrfiles', so that files will be created.

This engine is to measure file lookup and meta data access.

**libpmem**

Read and write using mmap I/O to a file on a filesystem

mounted with DAX on a persistent memory device through the PMDK

libpmem library.

**ime_psync**

Synchronous read and write using DDN's Infinite Memory Engine (IME).

This engine is very basic and issues calls to IME whenever an IO is

queued.

**ime_psyncv**

Synchronous read and write using DDN's Infinite Memory Engine (IME).

This engine uses iovecs and will try to stack as much IOs as possible

(if the IOs are "contiguous" and the IO depth is not exceeded)

before issuing a call to IME.

**ime_aio**

Asynchronous read and write using DDN's Infinite Memory Engine (IME).

This engine will try to stack as much IOs as possible by creating

requests for IME. FIO will then decide when to commit these requests.

**libiscsi**

Read and write iscsi lun with libiscsi.

**nbd**

Read and write a Network Block Device (NBD).

参考:

【1】https://github.com/bvanassche/fio/blob/master/HOWTO

相关文章

  • 测试工具:FIO --- IO引擎学习

    FIO是怎么将I/O发送到文件中的呢?是通过下面的引擎来进行操作的: **sync** 使用read()、writ...

  • 磁盘性能测试

    FIO测试 FIO是测试IOPS的非常好的工具,用来对硬件进行压力测试和验证,支持13种不同的I/O引擎,包括:s...

  • fio命令

    FIO介绍FIO是测试IOPS的非常好的工具,用来对磁盘进行压力测试和验证。磁盘IO是检查磁盘性能的重要指标,可以...

  • IO测试工具

    今天看一个视频时,学习到了IO性能的测试工具,记录下备忘: Linux: fio, dd Windows: IOM...

  • FIO 之入门指南

    一、概述 FIO 是一个开源的主流的 Linux 磁盘 IO 测试工具,可模拟各种IO workload,广泛用于...

  • 使用FIO测试磁盘的iops

    FIO是测试IOPS的非常好的工具,用来对硬件进行压力测试和验证,支持13种不同的I/O引擎,包括:sync,mm...

  • 使用fio测试磁盘I/O性能

    前言 fio是测试IOPS的非常好的工具,用来对硬件进行压力测试和验证,支持13种不同的I/O引擎,包括:sync...

  • CentOS中使用FIO测试磁盘IO性能

    FIO是测试IOPS的非常好的工具,用来对硬件进行压力测试和验证,支持19种不同的I/O引擎,包括:sync, m...

  • FIO

    FIO是测试IOPS的非常好的工具,用来对磁盘进行压力测试和验证。磁盘IO是检查磁盘性能的重要指标,可以按照负载情...

  • 【fio】Centos7.x 磁盘IOPS压力测试

    fio是一种 I/O工具,用于基准测试和压力/硬件验证。它支持19种不同类型的I/O引擎(sync,mmap,li...

网友评论

      本文标题:测试工具:FIO --- IO引擎学习

      本文链接:https://www.haomeiwen.com/subject/lkhicxtx.html