Innodb undo之 undo物理结构的初始化

作者: 重庆八怪 | 来源:发表于2019-01-23 12:32 被阅读0次

水平有限，如果有误请指出。

一直以来未对Innodb 的undo进行好好的学习，最近刚好有点时间准备学习一下，通过阿里内核月报和自己看代码的综合总结一下。本文环境：

代码版本 percona 5.7.22
参数 innodb_undo_tablespaces = 4 及使用了4个undo tablespace
参数 innodb_rollback_segments = 128

本文描述使用如上参数的设置。

一、undo 表空间物理文件的建立

本过程调用函数srv_undo_tablespaces_init进行，栈帧如下：

#0  srv_undo_tablespaces_init (create_new_db=true, n_conf_tablespaces=4, n_opened=0x2ef55b0)
    at /root/mysqlc/percona-server-locks-detail-5.7.22/storage/innobase/srv/srv0start.cc:824
#1  0x0000000001bbd7e0 in innobase_start_or_create_for_mysql () at /root/mysqlc/percona-server-locks-detail-5.7.22/storage/innobase/srv/srv0start.cc:2188
#2  0x00000000019ca74e in innobase_init (p=0x2f2a420) at /root/mysqlc/percona-server-locks-detail-5.7.22/storage/innobase/handler/ha_innodb.cc:4409
#3  0x0000000000f7ec2a in ha_initialize_handlerton (plugin=0x2fca110) at /root/mysqlc/percona-server-locks-detail-5.7.22/sql/handler.cc:871
#4  0x00000000015f9edf in plugin_initialize (plugin=0x2fca110) at /root/mysqlc/percona-server-locks-detail-5.7.22/sql/sql_plugin.cc:1252

本过程主要有如下几个步骤：

根据参数innodb_undo_tablespaces 的配置通过调用srv_undo_tablespace_create分别进行文件建立，默认建立的大小为10M：

for (i = 0; create_new_db && i < n_conf_tablespaces; ++i) //n_conf_tablespaces 为innodb_undo_tablespaces的配置的个数

/** Default undo tablespace size in UNIV_PAGEs count (10MB). */
const ulint SRV_UNDO_TABLESPACE_SIZE_IN_PAGES =
    ((1024 * 1024) * 10) / UNIV_PAGE_SIZE_DEF;
...
        err = srv_undo_tablespace_create(
            name, SRV_UNDO_TABLESPACE_SIZE_IN_PAGES); //建立undo文件
...

本步骤会有一个注释如下：

/* Create the undo spaces only if we are creating a new
    instance. We don't allow creating of new undo tablespaces
    in an existing instance (yet).  This restriction exists because
    we check in several places for SYSTEM tablespaces to be less than
    the min of user defined tablespace ids. Once we implement saving
    the location of the undo tablespaces and their space ids this
    restriction will/should be lifted. */

简单的讲就是建立undo tablespace只能在初始化实例的时候，因为space id已经固定了。

分别对4个undo tablespace调用srv_undo_tablespace_open 其主要调用fil_space_create 和 fil_node_create将新建立的undo tablespace加入Innodb的文件体系。

for (i = 0; i < n_undo_tablespaces; ++i) {
....
err = srv_undo_tablespace_open(name, undo_tablespace_ids[i]); //打开UNDO文件 建立 file node
...
}

分别对4个undo tablespace 进行fsp header初始化

for (i = 0; i < n_undo_tablespaces; ++i) {

            fsp_header_init( //初始化fsp header 明显 space id 已经写入
                undo_tablespace_ids[i],
                SRV_UNDO_TABLESPACE_SIZE_IN_PAGES, &mtr); //SRV_UNDO_TABLESPACE_SIZE_IN_PAGES 默认的undo大小 10MB
        }

其中fsp_header_init部分代码如下：

    mlog_write_ulint(header + FSP_SPACE_ID, space_id, MLOG_4BYTES, mtr);
    mlog_write_ulint(header + FSP_NOT_USED, 0, MLOG_4BYTES, mtr);

    mlog_write_ulint(header + FSP_SIZE, size, MLOG_4BYTES, mtr);
    mlog_write_ulint(header + FSP_FREE_LIMIT, 0, MLOG_4BYTES, mtr);
    mlog_write_ulint(header + FSP_SPACE_FLAGS, space->flags,
             MLOG_4BYTES, mtr);
    mlog_write_ulint(header + FSP_FRAG_N_USED, 0, MLOG_4BYTES, mtr);

    flst_init(header + FSP_FREE, mtr);
    flst_init(header + FSP_FREE_FRAG, mtr);
    flst_init(header + FSP_FULL_FRAG, mtr);
    flst_init(header + FSP_SEG_INODES_FULL, mtr);
    flst_init(header + FSP_SEG_INODES_FREE, mtr);

这些都是fsp的内容。

做完这个步骤只是生成了4个大小为10MB的 undo tablespace文件，并且已经加入到Innodb文件体系，但是里面没有任何类容。

二、ibdata中system segment header的初始化

本步骤调用 trx_sys_create_sys_pages->trx_sysf_create进行，本步骤除了初始化transaction system segment以外还会初始化其header( ibdata page no 5))信息如下：

/* Create the trx sys file block in a new allocated file segment */
    block = fseg_create(TRX_SYS_SPACE, 0, TRX_SYS + TRX_SYS_FSEG_HEADER,
                mtr); //建立segment
    buf_block_dbg_add_level(block, SYNC_TRX_SYS_HEADER);

    ut_a(block->page.id.page_no() == TRX_SYS_PAGE_NO);

    page = buf_block_get_frame(block); //获取内存位置

    mlog_write_ulint(page + FIL_PAGE_TYPE, FIL_PAGE_TYPE_TRX_SYS, //写入block 的类型
             MLOG_2BYTES, mtr);

    ...
    /* Start counting transaction ids from number 1 up */
    mach_write_to_8(sys_header + TRX_SYS_TRX_ID_STORE, 1); // 初始化TRX_SYS_TRX_ID_STORE

    /* Reset the rollback segment slots.  Old versions of InnoDB
    define TRX_SYS_N_RSEGS as 256 (TRX_SYS_OLD_N_RSEGS) and expect
    that the whole array is initialized. */
    ptr = TRX_SYS_RSEGS + sys_header;
    len = ut_max(TRX_SYS_OLD_N_RSEGS, TRX_SYS_N_RSEGS)
        * TRX_SYS_RSEG_SLOT_SIZE;//TRX_SYS_OLD_N_RSEGS 为256个
    memset(ptr, 0xff, len); //将slot的信息的全部初始化为ff
    ptr += len;
    ut_a(ptr <= page + (UNIV_PAGE_SIZE - FIL_PAGE_DATA_END));

    /* Initialize all of the page.  This part used to be uninitialized. */
    memset(ptr, 0, UNIV_PAGE_SIZE - FIL_PAGE_DATA_END + page - ptr); //将剩下的空间设置为0x00

    mlog_log_string(sys_header, UNIV_PAGE_SIZE - FIL_PAGE_DATA_END
            + page - sys_header, mtr);

    /* Create the first rollback segment in the SYSTEM tablespace */
    slot_no = trx_sysf_rseg_find_free(mtr, false, 0);
    page_no = trx_rseg_header_create(TRX_SYS_SPACE, univ_page_size,
                     ULINT_MAX, slot_no, mtr); //将第一个slot固定在ibdata中

完成了这一步过后ibdata的 block 5 就初始化完了，而且我们看到所有的rollback segment slots 都初始化完成(源码所示有256个，实际上最多只会有128个，其中0号solt固定在ibdata中)，注意这里的槽大小是TRX_SYS_RSEG_SLOT_SIZE设置的大小为8字节，4字节space id ,4字节 page no，它们会指向 rollback segment header所在的位置。

下面是system segment header的定位：

/** Transaction system header */
/*------------------------------------------------------------- @{ */
#define TRX_SYS_TRX_ID_STORE    0   /*!< the maximum trx id or trx
                    number modulo
                    TRX_SYS_TRX_ID_UPDATE_MARGIN
                    written to a file page by any
                    transaction; the assignment of
                    transaction ids continues from
                    this number rounded up by
                    TRX_SYS_TRX_ID_UPDATE_MARGIN
                    plus
                    TRX_SYS_TRX_ID_UPDATE_MARGIN
                    when the database is
                    started */  //最大的事物ID，下次实例启动会加上TRX_SYS_TRX_ID_UPDATE_MARGIN启动
#define TRX_SYS_FSEG_HEADER 8   /*!< segment header for the
                    tablespace segment the trx
                    system is created into */
#define TRX_SYS_RSEGS       (8 + FSEG_HEADER_SIZE)
                    /*!< the start of the array of
                    rollback segment specification
                    slots *///指向rollback segment header的槽
/*------------------------------------------------------------- @} */

三、进行rollback segment header的初始化

调用 trx_sys_create_rsegs进行：

说明一下关于innodb_undo_logs参数和innodb_rollback_segments参数，他们作用就是设置rollback segment 的个数，本文以128为例。

根据注释和代码innodb_undo_logs已经是个淘汰的参数，应该用innodb_rollback_segments代替。
这两个参数默认是就是TRX_SYS_N_RSEGS及 128 其实不用设置的。本文也用128进行讨论。

参数 innodb_rollback_segments

static MYSQL_SYSVAR_ULONG(rollback_segments, srv_rollback_segments,
  PLUGIN_VAR_OPCMDARG,
  "Number of rollback segments to use for storing undo logs.",
  NULL, NULL,
  TRX_SYS_N_RSEGS,  /* Default setting */
  1,            /* Minimum value */
  TRX_SYS_N_RSEGS, 0);  /* Maximum value */

参数 innodb_undo_logs

static MYSQL_SYSVAR_ULONG(undo_logs, srv_undo_logs,
  PLUGIN_VAR_OPCMDARG,
  "Number of rollback segments to use for storing undo logs. (deprecated)",
  NULL, innodb_undo_logs_update,
  TRX_SYS_N_RSEGS,  /* Default setting */
  1,            /* Minimum value */
  TRX_SYS_N_RSEGS, 0);  /* Maximum value */

TRX_SYS_N_RSEGS 就是128

下面是注释和代码

    /* Deprecate innodb_undo_logs.  But still use it if it is set to
    non-default and innodb_rollback_segments is default. */

    if (srv_undo_logs < TRX_SYS_N_RSEGS) {
        ib::warn() << deprecated_undo_logs;
        if (srv_rollback_segments == TRX_SYS_N_RSEGS) {
            srv_rollback_segments = srv_undo_logs;
        }
    }

初始化rollback segments 段

n_noredo_created = trx_sys_create_noredo_rsegs(n_tmp_rsegs); //创建 32个 临时rollback segments

我们这里不准备考虑临时rollback segments

建立 95个(33-128) 普通rollback segments

ulint   new_rsegs = n_rsegs - n_used; //eg:128 -33 = 95

        for (i = 0; i < new_rsegs; ++i) { //对每个rollback segment进行初始化
            ulint   space_id;
            space_id = (n_spaces == 0) ? 0
                : (srv_undo_space_id_start + i % n_spaces); //获取 undo space_id 采用 取模的方式循环初始化 1 2 3 4

            ut_ad(n_spaces == 0
                  || srv_is_undo_tablespace(space_id));

            if (trx_rseg_create(space_id, 0) != NULL)

我们能够注意到这里是i % n_spaces的取模方式n_spaces为我们innodb_undo_tablespaces参数设置的值，因此每个rollback segment 是轮序的方式分布到4个不同的undo tablespace中的。

具体的rollback segment header初始化过程

如上是trx_rseg_create调用trx_rseg_header_create完成的。步骤大概如下：

1、建立rollback segment

block = fseg_create(space, 0, TRX_RSEG + TRX_RSEG_FSEG_HEADER, mtr); //建立一个回滚段，返回段头所在的块

2、初始化TRX_RSEG_MAX_SIZE和TRX_RSEG_HISTORY_SIZE信息

    /* Initialize max size field */
    mlog_write_ulint(rsegf + TRX_RSEG_MAX_SIZE, max_size,
             MLOG_4BYTES, mtr);

    /* Initialize the history list */

    mlog_write_ulint(rsegf + TRX_RSEG_HISTORY_SIZE, 0, MLOG_4BYTES, mtr);
    flst_init(rsegf + TRX_RSEG_HISTORY, mtr);

3、初始化每个undo segment header所在的page no

    for (i = 0; i < TRX_RSEG_N_SLOTS; i++) { //TRX_RSEG_N_SLOTS 为1024 初始化每个槽 值为 4字节指向 undo segment header的page no

        trx_rsegf_set_nth_undo(rsegf, i, FIL_NULL, mtr);
    }

初始化的情况下我们看到指向的page no都是 FIL_NULL，说明没有分配任何实际的undo segment。

4、整个rollback segment 初始化完成后将space id和page no 写回到 transaction system segment header中。

sys_header = trx_sysf_get(mtr); //获取 5号 block指针 跳过  FIL_PAGE_DATA 38U 

trx_sysf_rseg_set_space(sys_header, rseg_slot_no, space, mtr); //设置space

trx_sysf_rseg_set_page_no(sys_header, rseg_slot_no, page_no, mtr); //设置 no

下面是 rollback segment header的结构

/* Transaction rollback segment header */
/*-------------------------------------------------------------*/
#define TRX_RSEG_MAX_SIZE   0   /* Maximum allowed size for rollback
                    segment in pages */
#define TRX_RSEG_HISTORY_SIZE   4   /* Number of file pages occupied
                    by the logs in the history list */ //history 链表大小
#define TRX_RSEG_HISTORY    8   /* The update undo logs for committed
                    transactions */ //链表头base node  他们通常调用include/fut0lst.ic中的函数进行更改
#define TRX_RSEG_FSEG_HEADER    (8 + FLST_BASE_NODE_SIZE)
                    /* Header for the file segment where
                    this page is placed */
#define TRX_RSEG_UNDO_SLOTS (8 + FLST_BASE_NODE_SIZE + FSEG_HEADER_SIZE)
                    /* Undo log segment slots */ //
/*-------------------------------------------------------------*/

作为 base node的 TRX_RSEG_HISTORY我们可以看到定义如下

/* We define the field offsets of a base node for the list */
#define FLST_LEN    0   /* 32-bit list length field */
#define FLST_FIRST  4   /* 6-byte address of the first element
                of the list; undefined if empty list */
#define FLST_LAST   (4 + FIL_ADDR_SIZE) /* 6-byte address of the
                last element of the list; undefined
                if empty list */

#define FIL_ADDR_PAGE   0   /* first in address is the page offset */
#define FIL_ADDR_BYTE   4   /* then comes 2-byte byte offset within page*/
#endif /* !UNIV_INNOCHECKSUM */
#define FIL_ADDR_SIZE   6   /* address size is 6 bytes */

多了一个长度

到这里128 rollback segment已经初始化完成，并且每个都包含1024个 undo segment slots。

四、整个过程初始化完成后的分布图

为了让图更加美观和好理解，我这里使用的是innodb_undo_tablespaces=2的情况下作图，也就是只有2个 undo tablespace的情况。其实4个也是同样的道理，因为rollback segment slot是轮询在表空间分配的。

undo phy5.jpg

最终我们看到初始化完成后undo segment slot指向的都是FIL_NULL，及没有指向，当实际分配的时候这些slot就会指向我们的undo segment header。

同时我们可以看看undotablespace到底包含哪些类型块，使用自制的小工具读取如下：

./myblock undo001 -d|more

current read blocks is : 0 --This Block is file space header blocks!
current read blocks is : 1 --This Block is insert buffer bitmap  blocks!
current read blocks is : 2 --This Block is inode blocks!
current read blocks is : 3 --This Block is system   blocks!
current read blocks is : 4 --This Block is system   blocks!
current read blocks is : 5 --This Block is system   blocks!
current read blocks is : 6 --This Block is system   blocks!
current read blocks is : 7 --This Block is system   blocks!
current read blocks is : 8 --This Block is system   blocks!
current read blocks is : 9 --This Block is system   blocks!
current read blocks is : 10 --This Block is system   blocks!
current read blocks is : 11 --This Block is system   blocks!
current read blocks is : 12 --This Block is system   blocks!
current read blocks is : 13 --This Block is system   blocks!
current read blocks is : 14 --This Block is system   blocks!
current read blocks is : 15 --This Block is system   blocks!
current read blocks is : 16 --This Block is system   blocks!
current read blocks is : 17 --This Block is system   blocks!
current read blocks is : 18 --This Block is system   blocks!
current read blocks is : 19 --This Block is system   blocks!
current read blocks is : 20 --This Block is system   blocks!
current read blocks is : 21 --This Block is system   blocks!
current read blocks is : 22 --This Block is system   blocks!
current read blocks is : 23 --This Block is system   blocks!
current read blocks is : 24 --This Block is system   blocks!
current read blocks is : 25 --This Block is system   blocks!
current read blocks is : 26 --This Block is system   blocks!
current read blocks is : 27 --This Block is undo blocks!
current read blocks is : 28 --This Block is undo blocks!
current read blocks is : 29 --This Block is undo blocks!
current read blocks is : 30 --This Block is undo blocks!
current read blocks is : 31 --This Block is undo blocks!
current read blocks is : 32 --This Block is undo blocks!
current read blocks is : 33 --This Block is undo blocks!
current read blocks is : 34 --This Block is undo blocks!
current read blocks is : 35 --This Block is undo blocks!
current read blocks is : 36 --This Block is undo blocks!
current read blocks is : 37 --This Block is undo blocks!
current read blocks is : 38 --This Block is new allocate blocks!
current read blocks is : 39 --This Block is new allocate blocks!
current read blocks is : 40 --This Block is new allocate blocks!
current read blocks is : 41 --This Block is new allocate blocks!
current read blocks is : 42 --This Block is new allocate blocks!

这里 block3-block26 就是我们的rollback segment header block。我这里当然是 4个undo tablespace的情况，看的是undo tablespace 1。看来没有问题。分析正确。

五、总结

普通的undo segment的关联方式是：ibdata的block 5 system segment header通过33-128这些 rollback segment slot 轮询指向不同的undo tablespace 的rollback segment header，然后每个rollback segment header中有1024个slot来指向实际的undo segment header，来实现的。实际的undo block会挂载到undo segment header下的链表中。
undo tablespaces数量的变化只能通过重新初始化实例来改变，space id是固定了，所以要考虑清楚
innodb_undo_tablespaces是undo tablespace的数量而innodb_rollback_segments是 rollback segment的数量，参数innodb_undo_logs已经过时了，它和innodb_rollback_segments是同样的功能，默认他们都是128
rollback segment slot 0 固定在 ibdata中，而 rollback segment slot 1-32 为临时rollback segment，33-128才是普通事物的rollback segment。

参考文献：
http://mysql.taobao.org/monthly/2015/04/01/ 阿里内核月报

作者微信:gp_22389860

Innodb undo之 undo物理结构的初始化

水平有限，如果有误请指出。

一、undo 表空间物理文件的建立

二、ibdata中system segment header的初始化

三、进行rollback segment header的初始化

四、整个过程初始化完成后的分布图

五、总结

相关文章

网友评论

延伸阅读

深度阅读

栏目导航

热点阅读

MySQL