2020-12-15 rocksdb研究

参考文章：gorocksdb 的安装与使用搭建环境，文章里描述的几个错误我都没有遇到，很顺利的完成了，因此列下来
1、开发环境
ubuntu18
2、下载 rocksdb 最新的发行版

wget https://github.com/facebook/rocksdb/archive/v6.14.6.tar.gz
tar xvzf v6.14.6.tar.gz
cd rocksdb-6.14.6/
make shared_lib -j9
sudo make install

这里就一次性成功了，也没有修改代码
3、安装其他依赖
文档写要安装

zlib - a library for data compression.
bzip2 - a library for data compression.
lz4 - a library for extremely fast data compression.
snappy - a library for fast data compression.
zstandard - Fast real-time compression algorithm.

最后发现只有zlib ，bzip2 安装成功了，其他3个根本找不到安装包，也就算了
4、安装 gorocksdb
也没有那么复杂直接执行

 go get github.com/tecbot/gorocksdb

5、测试代码
原文的代码有个小错误，这里修正了

package main

import (
    "github.com/tecbot/gorocksdb"
    "log"
)

func main() {
    opts := gorocksdb.NewDefaultOptions()
    opts.SetCreateIfMissing(true)
    opts.SetCompression(gorocksdb.NoCompression)
    opts.SetWriteBufferSize(671088640)
    db, err := gorocksdb.OpenDb(opts, "test")
    wopt := gorocksdb.NewDefaultWriteOptions()
    if err != nil {
        log.Printf("%v\n", err)
    }
    defer db.Close()
    db.Put(wopt, []byte("data"), []byte("value"))
}

然后编译执行，也很顺利

go build example.go
./example

不过这个例子看不到执行结果
然后又抄了个rocksdb使用说明

package main

import (
"errors"
"github.com/tecbot/gorocksdb"
"log"
"strconv"
)

const (
DB_PATH = "gorocksdb"
)

func main() {
db, err := OpenDB()
if err != nil {
log.Println("fail to open db,", nil, db)
}

readOptions := gorocksdb.NewDefaultReadOptions()
readOptions.SetFillCache(true)

writeOptions := gorocksdb.NewDefaultWriteOptions()
writeOptions.SetSync(true)

for i := 0; i < 10; i++ {
keyStr := "test" + strconv.Itoa(i)
var key []byte = []byte(keyStr)
db.Put(writeOptions, key, key)
log.Println(i, keyStr)
slice, err2 := db.Get(readOptions, key)
if err2 != nil {
log.Println("get data exception：", key, err2)
continue
}
log.Println("get data：", slice.Size(), string(slice.Data()))
}

}

// opendb
func OpenDB() (*gorocksdb.DB, error) {
options := gorocksdb.NewDefaultOptions()
options.SetCreateIfMissing(true)

bloomFilter := gorocksdb.NewBloomFilter(10)

readOptions := gorocksdb.NewDefaultReadOptions()
readOptions.SetFillCache(false)

rateLimiter := gorocksdb.NewRateLimiter(10000000, 10000, 10)
options.SetRateLimiter(rateLimiter)
options.SetCreateIfMissing(true)
options.EnableStatistics()
options.SetWriteBufferSize(8 * 1024)
options.SetMaxWriteBufferNumber(3)
options.SetMaxBackgroundCompactions(10)
// options.SetCompression(gorocksdb.SnappyCompression)
// options.SetCompactionStyle(gorocksdb.UniversalCompactionStyle)

options.SetHashSkipListRep(2000000, 4, 4)

blockBasedTableOptions := gorocksdb.NewDefaultBlockBasedTableOptions()
blockBasedTableOptions.SetBlockCache(gorocksdb.NewLRUCache(64 * 1024))
blockBasedTableOptions.SetFilterPolicy(bloomFilter)
blockBasedTableOptions.SetBlockSizeDeviation(5)
blockBasedTableOptions.SetBlockRestartInterval(10)
blockBasedTableOptions.SetBlockCacheCompressed(gorocksdb.NewLRUCache(64 * 1024))
blockBasedTableOptions.SetCacheIndexAndFilterBlocks(true)
blockBasedTableOptions.SetIndexType(gorocksdb.KHashSearchIndexType)

options.SetBlockBasedTableFactory(blockBasedTableOptions)
//log.Println(bloomFilter, readOptions)
options.SetPrefixExtractor(gorocksdb.NewFixedPrefixTransform(3))

options.SetAllowConcurrentMemtableWrites(false)

db, err := gorocksdb.OpenDb(options, DB_PATH)

if err != nil {
log.Fatalln("OPEN DB error", db, err)
db.Close()
return nil, errors.New("fail to open db")
} else {
log.Println("OPEN DB success", db)
}
return db, nil
}

算是比较顺利，只是有个小错误

go build example2.go
./example2

这个就有执行结果了：

2020/12/15 16:58:25 OPEN DB success &{0x2073260 gorocksdb 0xc00005c040}
2020/12/15 16:58:25 0 test0
2020/12/15 16:58:25 get data： 5 test0
2020/12/15 16:58:25 1 test1
2020/12/15 16:58:25 get data： 5 test1
2020/12/15 16:58:25 2 test2
2020/12/15 16:58:25 get data： 5 test2
2020/12/15 16:58:25 3 test3
2020/12/15 16:58:25 get data： 5 test3
2020/12/15 16:58:25 4 test4
2020/12/15 16:58:25 get data： 5 test4
2020/12/15 16:58:25 5 test5
2020/12/15 16:58:25 get data： 5 test5
2020/12/15 16:58:25 6 test6
2020/12/15 16:58:25 get data： 5 test6
2020/12/15 16:58:25 7 test7
2020/12/15 16:58:25 get data： 5 test7
2020/12/15 16:58:25 8 test8
2020/12/15 16:58:25 get data： 5 test8
2020/12/15 16:58:25 9 test9
2020/12/15 16:58:25 get data： 5 test9