APPEND ONLY MODE部分:
1.设置开启aof选项:
# By default Redis asynchronously dumps the dataset on disk. This mode is
# good enough in many applications, but an issue with the Redis process or
# a power outage may result into a few minutes of writes lost (depending on
# the configured save points).
#
# The Append Only File is an alternative persistence mode that provides
# much better durability. For instance using the default data fsync policy
# (see later in the config file) Redis can lose just one second of writes in a
# dramatic event like a server power outage, or a single write if something
# wrong with the Redis process itself happens, but the operating system is
# still running correctly.
#
# AOF and RDB persistence can be enabled at the same time without problems.
# If the AOF is enabled on startup Redis will load the AOF, that is the file
# with the better durability guarantees.
#
# Please check http://redis.io/topics/persistence for more information.
appendonly no
默认情况下,Redis在磁盘上异步转储数据集。这种模式在许多应用程序中已经足够好了,但是Redis进程的问题或断电可能会导致几分钟的写操作丢失(取决于配置的保存点)
Append Only File 是一种替代的持久性模式,它提供了更好的持久性。例如,使用默认的数据fsync策略(请参阅配置文件的后面部分)Redis在服务器断电之类的重大事件中可能会丢失一秒钟的写入,或者在Redis进程本身出现问题时丢失一次写入,但操作系统仍在正常运行
可以同时启用AOF和RDB持久性,而不会出现问题。如果在启动时启用了AOF,Redis将加载AOF,即具有更好的持久性保证的文件
当同时开启rdb和aof的时候,redis在启动时首先加载aof文件
2.文件名称:
# The name of the append only file (default: "appendonly.aof")
appendfilename "appendonly.aof"
3.aof持久化策略:
# The fsync() call tells the Operating System to actually write data on disk
# instead of waiting for more data in the output buffer. Some OS will really flush
# data on disk, some other OS will just try to do it ASAP.
#
# Redis supports three different modes:
#
# no: don't fsync, just let the OS flush the data when it wants. Faster.
# always: fsync after every write to the append only log. Slow, Safest.
# everysec: fsync only one time every second. Compromise.
#
# The default is "everysec", as that's usually the right compromise between
# speed and data safety. It's up to you to understand if you can relax this to
# "no" that will let the operating system flush the output buffer when
# it wants, for better performances (but if you can live with the idea of
# some data loss consider the default persistence mode that's snapshotting),
# or on the contrary, use "always" that's very slow but a bit safer than
# everysec.
#
# More details please check the following article:
# http://antirez.com/post/redis-persistence-demystified.html
#
# If unsure, use "everysec".
# appendfsync always
appendfsync everysec
# appendfsync no
fsync()调用告诉操作系统实际在磁盘上写入数据,而不是在输出缓冲区中等待更多数据。一些操作系统将真正刷新磁盘上的数据,一些其他操作系统将尝试尽快这样做
1.不要fsync,只要让操作系统在需要的时候刷新数据就行了 快速
2.fsync,在每次写入仅追加日志之后,安全
3.fsync,每秒钟只同步一次,折中
默认是每秒同步一次
4.重写时不开启同步aof:
# When the AOF fsync policy is set to always or everysec, and a background
# saving process (a background save or AOF log background rewriting) is
# performing a lot of I/O against the disk, in some Linux configurations
# Redis may block too long on the fsync() call. Note that there is no fix for
# this currently, as even performing fsync in a different thread will block
# our synchronous write(2) call.
#
# In order to mitigate this problem it's possible to use the following option
# that will prevent fsync() from being called in the main process while a
# BGSAVE or BGREWRITEAOF is in progress.
#
# This means that while another child is saving, the durability of Redis is
# the same as "appendfsync none". In practical terms, this means that it is
# possible to lose up to 30 seconds of log in the worst scenario (with the
# default Linux settings).
#
# If you have latency problems turn this to "yes". Otherwise leave it as
# "no" that is the safest pick from the point of view of durability.
no-appendfsync-on-rewrite no
当aof fsync策略设置为always或everysec,并且后台保存进程(后台保存或AOF日志后台重写)正在对磁盘执行大量I/O时,在某些Linux配置中,Redis可能会在fsync()调用上阻塞太长时间。请注意,目前还没有对此进行修复,因为即使在不同的线程中执行fsync,也会阻止同步写入(2)调用。
为了缓解这个问题,可以使用以下选项,防止在BGSAVE或bgrowriteaof正在进行时在主进程中调用fsync()
这意味着,当另一个子节点正在保存时,Redis的持久性与“appendfsync none”相同。实际上,这意味着在最坏的情况下(使用默认的Linux设置),可能会丢失长达30秒的日志
如果您有延迟问题,请将其改为“是”。否则,从耐久性的角度来看,这是最安全的选择
同时在执行bgrewriteaof操作和主进程写aof文件的操作,两者都会操作磁盘,而bgrewriteaof往往会涉及大量磁盘操作,这样就会造成主进程在写aof文件的时候出现阻塞的情形,现在no-appendfsync-on-rewrite参数出场了。如果该参数设置为no,是最安全的方式,不会丢失数据,但是要忍受阻塞的问题。如果设置为yes呢?这就相当于将appendfsync设置为no,这说明并没有执行磁盘操作,只是写入了缓冲区,因此这样并不会造成阻塞(因为没有竞争磁盘),但是如果这个时候redis挂掉,就会丢失数据。丢失多少数据呢?在linux的操作系统的默认设置下,最多会丢失30s的数据,因此,如果应用系统无法忍受延迟,而可以容忍少量的数据丢失,则设置为yes。如果应用系统无法忍受数据丢失,则设置为no
5.重写触发条件:
# Automatic rewrite of the append only file.
# Redis is able to automatically rewrite the log file implicitly calling
# BGREWRITEAOF when the AOF log size grows by the specified percentage.
#
# This is how it works: Redis remembers the size of the AOF file after the
# latest rewrite (if no rewrite has happened since the restart, the size of
# the AOF at startup is used).
#
# This base size is compared to the current size. If the current size is
# bigger than the specified percentage, the rewrite is triggered. Also
# you need to specify a minimal size for the AOF file to be rewritten, this
# is useful to avoid rewriting the AOF file even if the percentage increase
# is reached but it is still pretty small.
#
# Specify a percentage of zero in order to disable the automatic AOF
# rewrite feature.
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
自动重写 append only file.。Redis能够在AOF日志大小增长指定百分比时隐式地调用BGREWRITEAOF来自动重写日志文件
这就是它的工作原理:Redis会记住最近一次重写之后AOF文件的大小(如果自重新启动之后没有发生重写,则使用启动时AOF的大小)
将此基础大小与当前大小进行比较。如果当前大小大于指定的百分比,则会触发重写。此外,还需要为要重写的AOF文件指定最小大小,这对于避免重写AOF文件很有用,即使达到了百分比增长,但仍然很小
指定0的百分比以禁用自动AOF重写功能
当aof文件超过上次重写的百分比之后并且,aof文件大小超过设置大小之后才会重写
6.是否忽略最后有问题的命令:
# An AOF file may be found to be truncated at the end during the Redis
# startup process, when the AOF data gets loaded back into memory.
# This may happen when the system where Redis is running
# crashes, especially when an ext4 filesystem is mounted without the
# data=ordered option (however this can't happen when Redis itself
# crashes or aborts but the operating system still works correctly).
#
# Redis can either exit with an error when this happens, or load as much
# data as possible (the default now) and start if the AOF file is found
# to be truncated at the end. The following option controls this behavior.
#
# If aof-load-truncated is set to yes, a truncated AOF file is loaded and
# the Redis server starts emitting a log to inform the user of the event.
# Otherwise if the option is set to no, the server aborts with an error
# and refuses to start. When the option is set to no, the user requires
# to fix the AOF file using the "redis-check-aof" utility before to restart
# the server.
#
# Note that if the AOF file will be found to be corrupted in the middle
# the server will still exit with an error. This option only applies when
# Redis will try to read more data from the AOF file but not enough bytes
# will be found.
aof-load-truncated yes
在Redis启动过程中,当AOF数据被加载回内存时,可能会发现AOF文件在末尾被截断。当Redis运行的系统崩溃时,可能会发生这种情况,特别是当ext4文件系统在没有data=ordered选项的情况下挂载时
Redis可以在出现错误时退出,或者加载尽可能多的数据(现在是默认值),如果发现AOF文件在结尾处被截断,则可以启动。以下选项控制此行为
如果aof-load-truncated设置为yes,则加载截断的aof文件,Redis服务器开始发出日志来通知用户事件。否则,如果将该选项设置为“否”,服务器将因错误而中止并拒绝启动。当选项设置为no时,用户需要在重新启动服务器之前使用“redis check AOF”实用程序修复AOF文件
请注意,如果在中间发现AOF文件已损坏,服务器仍将退出并出现错误。此选项仅适用于Redis试图从AOF文件读取更多数据但找不到足够字节的情况
指redis在恢复时,会忽略最后一条可能存在问题的指令。默认值yes。即在aof写入时,可能存在指令写错的问题(突然断电,写了一半),这种情况下,yes会log并继续,而no会直接恢复失败
7.开启混合持久化:
# When rewriting the AOF file, Redis is able to use an RDB preamble in the
# AOF file for faster rewrites and recoveries. When this option is turned
# on the rewritten AOF file is composed of two different stanzas:
#
# [RDB file][AOF tail]
#
# When loading Redis recognizes that the AOF file starts with the "REDIS"
# string and loads the prefixed RDB file, and continues loading the AOF
# tail.
aof-use-rdb-preamble yes
当重写AOF文件时,Redis能够在AOF文件中使用RDB前导码,以便更快地重写和恢复。启用此选项时,重写的AOF文件由两个不同的节组成
[RDB文件][AOF tail]
加载Redis时,会识别出AOF文件以“Redis”字符串开头并加载前缀RDB文件,然后继续加载AOF尾部
混合持久化同样也是通过bgrewriteaof完成的,不同的是当开启混合持久化时,fork出的子进程先将共享的内存副本全量的以RDB方式写入aof文件,然后在将重写缓冲区的增量命令以AOF方式写入到文件,写入完成后通知主进程更新统计信息,并将新的含有RDB格式和AOF格式的AOF文件替换旧的的AOF文件。简单的说:新的AOF文件前半段是RDB格式的全量数据后半段是AOF格式的增量数据
网友评论