Redis删除大Key

作者: 我是杨正 | 来源:发表于2018-10-14 11:10 被阅读1次

    原文链接:https://www.dubby.cn/detail.html?id=9112

    这里说的大key是指包含很多元素的set,sorted set,list和hash。

    删除操作,我们一般想到有2种,delexpire

    DEL

    Time complexity: O(N) where N is the number of keys that will be removed. When a key to remove holds a value other than a string, the individual complexity for this key is O(M) where M is the number of elements in the list, set, sorted set or hash. Removing a single key that holds a string value is O(1).

    如果要删除的key是一个集合,包含了很多元素,那么DEL时的耗时和元素个数成正比,所以如果直接DEL,会很慢。

    EXPIRE

    Note that calling EXPIRE/PEXPIRE with a non-positive timeout or EXPIREAT/PEXPIREAT with a time in the past will result in the key being deleted rather than expired (accordingly, the emitted key event will be del, not expired).

    想着expire会不会可以不是直接删除,可惜官网的描述让我心灰意冷,如果expire后指定的timeout不是正数,也就是<=0,那其实就是DEL

    一点一点删

    我们知道Redis的工作线程是单线程的,如果一个command堵塞了,那所有请求都会超时,这时候,一些骚操作也许可以帮助你。

    其实如果想删除key,可以分解成2个目的,1:不想让其他人访问到这个key,2:释放空间。

    那其实我们可以分解成两步,先用RENAME把原先的key rename成另一个key,比如:

    RENAME userInfo:123 "deleteKey:userInfo:123"
    

    然后可以慢慢去删"deleteKey:userInfo:123",如果是set,那么可以用SREM慢慢删,最后再用DEL彻底删掉。

    这里可以搞个task去SCAN deleteKey:*,然后慢慢删除。

    UNLINK

    Redis 4.0.0提供了一个更加方便的命令

    Available since 4.0.0.

    Time complexity: O(1) for each key removed regardless of its size. Then the command does O(N) work in a different thread in order to reclaim memory, where N is the number of allocations the deleted objects where composed of.

    UNLINK其实是直接返回,然后在后台线程慢慢删除。

    如果你的Redis版本>=4.0.0,那么强烈建议使用UNLINK来删除。

    删除耗时测试结果

    单位:微秒

    Set个数 DEL EXPIRE UNLINK
    1 90 97 75
    10 79 67 100
    100 51 49 47
    1000 303 296 49
    10000 2773 2592 52
    100000 31210 33157 51
    1000000 549388 501536 62
    package main
    
    import (
        "github.com/go-redis/redis"
        "fmt"
        "time"
    )
    
    func main() {
        client := redis.NewClient(&redis.Options{
            Addr:         "localhost:6379",
            Password:     "",
            DB:           0,
            ReadTimeout:  1000 * 1000 * 1000 * 60 * 60 * 24,
            WriteTimeout: 1000 * 1000 * 1000 * 60 * 60 * 24,
        })
    
        maxLength := int64(10000 * 100)
    
        for n := int64(1); n <= maxLength; n *= 10 {
            fmt.Println("Set个数", n)
            TestDelBigSet(client, n)
            TestExpireBigSet(client, n)
            TestUnlinkBigSet(client, n)
            fmt.Println()
        }
    }
    
    func TestDelBigSet(client *redis.Client, count int64) {
        redisKey := fmt.Sprintf("%s%d", "del:", time.Now().Nanosecond())
    
        for n := int64(0); n < count; n++ {
            err := client.SAdd(redisKey, fmt.Sprintf("%d", n)).Err()
            if err != nil {
                panic(err)
            }
        }
    
        startTime := CurrentTimestampInMicroSecond()
        client.Del(redisKey)
        endTime := CurrentTimestampInMicroSecond()
    
        fmt.Println("Del", endTime-startTime)
    }
    
    func TestUnlinkBigSet(client *redis.Client, count int64) {
        redisKey := fmt.Sprintf("%s%d", "unlink:", time.Now().Nanosecond())
    
        for n := int64(0); n < count; n++ {
            err := client.SAdd(redisKey, fmt.Sprintf("%d", n)).Err()
            if err != nil {
                panic(err)
            }
        }
        startTime := CurrentTimestampInMicroSecond()
        client.Unlink(redisKey)
        endTime := CurrentTimestampInMicroSecond()
    
        fmt.Println("Unlink", endTime-startTime)
    }
    
    func TestExpireBigSet(client *redis.Client, count int64) {
        redisKey := fmt.Sprintf("%s%d", "expire:", time.Now().Nanosecond())
    
        for n := int64(0); n < count; n++ {
            err := client.SAdd(redisKey, fmt.Sprintf("%d", n)).Err()
            if err != nil {
                panic(err)
            }
        }
        startTime := CurrentTimestampInMicroSecond()
        client.Expire(redisKey, 0)
        endTime := CurrentTimestampInMicroSecond()
    
        fmt.Println("Expire", endTime-startTime)
    }
    
    func CurrentTimestampInMicroSecond() int64 {
        return time.Now().UnixNano() / 1000
    }
    
    

    相关文章

      网友评论

        本文标题:Redis删除大Key

        本文链接:https://www.haomeiwen.com/subject/klszaftx.html