问题现象
客户需要批量删除符合一定匹配规则的 key 以清理不需要的数据,释放内存和磁盘空间,本文展示如何使用 Python 脚本批量删除。
排查步骤
- 使用 keys 命令 查找匹配的键
keys 是高危命令,默认火山 Redis 是禁用,不建议生产使用,如果需要打开使用,需将对应的命令从禁用命令参数(即 disabled-commands
参数)的默认值中移除,具体参考官网文档 https://www.volcengine.com/docs/6293/97276
lxbtestfor.redis.ivolces.com:6379> keys *demo*
1) "cdemo:yellow"
2) "ademo:apple"
3) "bdemo:orange"
- 建议使用 scan 命令迭代查询
lxbtestfor.redis.ivolces.com:6379> scan 0 MATCH *demo*
1) "0"
2) 1) "ademo:apple"
2) "cdemo:blue"
3) "bdemo:yellow"
如果生产的 key 数量比较多的,不一定一次迭代就可以获取到匹配的数据。
解决方案
- 使用代码实现删除符合一定匹配规则的key
import redis
import json
def bulkdel_redis_conn(pattern):
try:
connect= redis.Redis(
host="***.redis.ivolces.com",
port=6379,
password="$password",
username='default',
db=0,
)
cursor=0
while True:
cursor, keys = connect.scan(cursor, match="*"+pattern + '*')
if cursor == 0:
break
if len(keys)==0:
print("no key found in this iteration,the cousor is",cursor)
continue
for key in keys:
connect.delete(key)
print("delete key",key)
except Exception as e:
print(f"delete key failed: {e}")
if __name__ == '__main__':
bulkdel_redis_conn('demo')
~
- 运行脚本并重定向删除结果到文件中,示例如下:
分析结果文件可以看到 ademo:apple
, cdemo:blue
bdemo:yellow
在迭代查询中被删除掉。
root@lxb-jms Redis]# python3 bulk_delete_redis.py > delete_result.txt
[root@lxb-jms Redis]# grep -C 2 "delete" delete_result.txt
no key found in this iteration,the cousor is 65512
no key found in this iteration,the cousor is 262168
delete key b'cdemo:blue'
no key found in this iteration,the cousor is 442392
no key found in this iteration,the cousor is 204824
--
no key found in this iteration,the cousor is 430966
no key found in this iteration,the cousor is 54134
delete key b'ademo:apple'
no key found in this iteration,the cousor is 357238
no key found in this iteration,the cousor is 396150
--
no key found in this iteration,the cousor is 358389
no key found in this iteration,the cousor is 456693
delete key b'bdemo:yellow'
no key found in this iteration,the cousor is 20469
no key found in this iteration,the cousor is 274421
- 查看确认 key 已经删除
因为是测试环境,这里直接用 keys 命令去查看
lxbtestfor.redis.ivolces.com:6379> keys *demo*
(empty array)
本文代码示例仅供思路参考,不可以直接将代码用于生产环境,生产业务需要根据实际情况调整匹配规则,生产中删除前一定要先查询匹配的数据是否符合预期后,在执行删除操作。