indexdb写入流程

逻辑大致流程

image.png

源码大致流程

image.png

结构体介绍

indexItems

lib/storage/index_db.go
type indexItems struct {
    B     []byte
    Items [][]byte

    start int
}

B: 索引序列化的数据
Items: 索引数组
主要是创建各种索引: Tag->MetricID; MetricID->TSID等. 属性B则是存储所有的索引数据,Items则是按照索引类型存储数据,最后把Items对象写入到下面的inmemoryBlock

inmemoryBlock

lib/mergeset/encoding.go
type inmemoryBlock struct {
    // commonPrefix contains common prefix for all the items stored in the inmemoryBlock
    commonPrefix []byte

    // data contains source data for items
    data []byte

    // items contains items stored in inmemoryBlock.
    // Every item contains the prefix specified at commonPrefix.
    items []Item
}

lib/mergeset/encoding.go
type Item struct {
    // Start is start offset for the item in data.
    Start uint32

    // End is end offset for the item in data.
    End uint32
}

索引先写入inmemoryBlock 原始索引数据——没压缩

data: 所有的索引数据
items: 不同类型的索引数据边界

rawItemsShards

lib/mergeset/table.go
type rawItemsShards struct {
    flushDeadlineMs atomic.Int64

    shardIdx atomic.Uint32

    // shards reduce lock contention when adding rows on multi-CPU systems.
    shards []rawItemsShard

    ibsToFlushLock sync.Mutex
    ibsToFlush     []*inmemoryBlock
}

flushDeadlineMs
- 延迟初始化: lib/mergeset/table.go:func (riss *rawItemsShards) updateFlushDeadline()
shardIdx
- 逻辑: lib/mergeset/table.go:func (riss *rawItemsShards) addItems(tb *Table, items [][]byte)
- 原子操作决定索引数据写到哪个内存shard——rawItemsShard
shards
- 初始化: lib/mergeset/table.go:func (riss *rawItemsShards) init() vCPU的倍数
- 内存shard数量——数据都存储在rawItemsShard对象, 每一个rawItemsShard有N个inmemoryBlock——即存储原始索引数据的对象
ibsToFlushLock
- 互斥锁
ibsToFlush
- 写入到inmemoryPart的inmemoryBlock数据