buf_flush_page_cleaner_coordinator协调线程的主循环主线程以最多1s的间隔或者收到buf_flush_event事件就会触发进行一轮的刷脏。
批量刷脏主要有3个场景。
- 同步刷脏
如果 buf_flush_sync_lsn > 0, 则因为redo log free space 不够了, 那么我们需要进入同步刷脏阶段了。同步刷脏场景下，所有需要写脏数据库的用户线程都会堵塞，这是很严重的情况。 - 正常刷脏
最常见逻辑 srv_check_activity(last_activity), 也就是系统有正常活动，有DML/DDL, 这个时候会通过 page_cleaner_flush_pages_recommendation() 函数去合理的判断应该刷多少个page, 既不抖动, 也能够满足刷脏需求 - 空闲刷脏
如果系统没有DML\DDL活动，且ret_sleep == OS_SYNC_TIME_EXCEEDED，说明比较空闲。空闲的情况下因为服务器IO比较空闲，所以Innodb使用buf_flush_page_cleaner_coordinator线程本身进行刷新，刷新的块数计算比较简单就是innodb_io_capacity设置的值。
/******************************************************************//**
page_cleaner thread tasked with flushing dirty pages from the buffer
pools. As of now we'll have only one coordinator.
@return a dummy parameter */
extern "C"
os_thread_ret_t
DECLARE_THREAD(buf_flush_page_cleaner_coordinator)(
/*===============================================*/
void* arg MY_ATTRIBUTE((unused)))
/*!< in: a dummy parameter required by
os_thread_create */
{
/* 忽略一些逻辑 */
while (srv_shutdown_state == SRV_SHUTDOWN_NONE) {
if (ret_sleep != OS_SYNC_TIME_EXCEEDED
&& srv_flush_sync
&& buf_flush_sync_lsn > 0) {
/* 场景1 */
} else if (srv_check_activity(last_activity)) {
ulint n_to_flush;
lsn_t lsn_limit = 0;
/* Estimate pages from flush_list to be flushed */
if (ret_sleep == OS_SYNC_TIME_EXCEEDED) {
last_activity = srv_get_activity_count();
n_to_flush =
page_cleaner_flush_pages_recommendation(
&lsn_limit, last_pages);
} else {
n_to_flush = 0;
}
/* 场景2 */
} else if (ret_sleep == OS_SYNC_TIME_EXCEEDED) {
/* 场景3 */
/* no activity, slept enough */
}
} else {
/* no activity, but woken up by event */
n_flushed = 0;
}
}
」
三种场景下的具体工作
同步刷脏
pc_request(ULINT_MAX, lsn_limit),会把lsn小于lsn_limit的都flush到硬盘，同时coordinator线程本身也会参与刷脏。
/* woke up for flush_sync */
mutex_enter(&page_cleaner->mutex);
lsn_t lsn_limit = buf_flush_sync_lsn;
buf_flush_sync_lsn = 0;
mutex_exit(&page_cleaner->mutex);
/* Request flushing for threads */
pc_request(ULINT_MAX, lsn_limit);
ib_time_monotonic_ms_t tm = ut_time_monotonic_ms();
/* Coordinator also treats requests */
while (pc_flush_slot() > 0) {}
/* only coordinator is using these counters,
so no need to protect by lock. */
page_cleaner->flush_time += ut_time_monotonic_ms() - tm;
page_cleaner->flush_pass++;
/* Wait for all slots to be finished */
ulint n_flushed_lru = 0;
ulint n_flushed_list = 0;
pc_wait_finished(&n_flushed_lru, &n_flushed_list);
if (n_flushed_list > 0 || n_flushed_lru > 0) {
buf_flush_stats(n_flushed_list, n_flushed_lru);
MONITOR_INC_VALUE_CUMULATIVE(
MONITOR_FLUSH_SYNC_TOTAL_PAGE,
MONITOR_FLUSH_SYNC_COUNT,
MONITOR_FLUSH_SYNC_PAGES,
n_flushed_lru + n_flushed_list);
}
n_flushed = n_flushed_lru + n_flushed_list;
正常刷脏
通过page_cleaner_flush_pages_recommendation计算需要刷新的页。
ulint n_to_flush;
lsn_t lsn_limit = 0;
/* Estimate pages from flush_list to be flushed */
if (ret_sleep == OS_SYNC_TIME_EXCEEDED) {
last_activity = srv_get_activity_count();
n_to_flush =
page_cleaner_flush_pages_recommendation(
&lsn_limit, last_pages);
} else {
n_to_flush = 0;
}
/* Request flushing for threads */
pc_request(n_to_flush, lsn_limit);
ib_time_monotonic_ms_t tm = ut_time_monotonic_ms();
/* Coordinator also treats requests */
while (pc_flush_slot() > 0) {
/* No op */
}
/* only coordinator is using these counters,
so no need to protect by lock. */
page_cleaner->flush_time += ut_time_monotonic_ms() - tm;
page_cleaner->flush_pass++ ;
/* Wait for all slots to be finished */
ulint n_flushed_lru = 0;
ulint n_flushed_list = 0;
pc_wait_finished(&n_flushed_lru, &n_flushed_list);
if (n_flushed_list > 0 || n_flushed_lru > 0) {
buf_flush_stats(n_flushed_list, n_flushed_lru);
}
if (ret_sleep == OS_SYNC_TIME_EXCEEDED) {
last_pages = n_flushed_list;
}
n_evicted += n_flushed_lru;
n_flushed_last += n_flushed_list;
n_flushed = n_flushed_lru + n_flushed_list;
if (n_flushed_lru) {
MONITOR_INC_VALUE_CUMULATIVE(
MONITOR_LRU_BATCH_FLUSH_TOTAL_PAGE,
MONITOR_LRU_BATCH_FLUSH_COUNT,
MONITOR_LRU_BATCH_FLUSH_PAGES,
n_flushed_lru);
}
if (n_flushed_list) {
MONITOR_INC_VALUE_CUMULATIVE(
MONITOR_FLUSH_ADAPTIVE_TOTAL_PAGE,
MONITOR_FLUSH_ADAPTIVE_COUNT,
MONITOR_FLUSH_ADAPTIVE_PAGES,
n_flushed_list);
}
空闲刷脏
空闲刷脏是coordinator自己进行，直接按照PCT_IO(100)来生成刷新数量。
#define PCT_IO(p) ((ulong) (srv_io_capacity * ((double) (p) / 100.0)))
buf_flush_lists(PCT_IO(100), LSN_MAX, &n_flushed);
n_flushed_last += n_flushed;
if (n_flushed) {
MONITOR_INC_VALUE_CUMULATIVE(
MONITOR_FLUSH_BACKGROUND_TOTAL_PAGE,
MONITOR_FLUSH_BACKGROUND_COUNT,
MONITOR_FLUSH_BACKGROUND_PAGES,
n_flushed);
}
http://mysql.taobao.org/monthly/2018/09/02/
https://www.jianshu.com/p/6991304a8e26
https://mp.weixin.qq.com/s/o2OlvRiybIsqi7WU_Kvhiw
网友评论