docker.mem Metrics from DataDog

Name

• docker.mem.cache

• bytes
• The amount of memory that is being used to cache data from disk (e.g. memory contents that can be associated precisely with a block on a block device)

• docker.mem.rss

• bytes
• The amount of non-cache memory that belongs to the container's processes. Used for stacks, heaps, etc.

• docker.mem.swap

• bytes
• The amount of swap currently used by the container, if set

• docker.mem.limit

• bytes
• The memory limit for the container, if set

• docker.mem.sw_limit

• bytes
• The swap + memory limit for the container, if set

• docker.mem.in_use

• fraction
• The fraction of used memory to available memory, IF THE LIMIT IS SET

• docker.mem.sw_in_use

• fraction
• The fraction of used swap + memory to available swap + memory, if the limit is set

• docker.mem.soft_limit

• bytes
• The memory reservation limit for the container, if set

计算公式

docker.mem.cache = cache
docker.mem.rss = rss
docker.mem.swap = swap
docker.mem.limit = hierarchical_memory_limit
docker.mem.sw_limit = hierarchical_memsw_limit
docker.mem.in_use = rss / hierarchical_memory_limit
docker.mem.sw_in_use = (swap + rss) / hierarchical_memsw_limit
docker.mem.soft_limit = `cat memory.soft_limit_in_bytes`

文件位置

容器对应的memory.stat or memory.soft_limit_in_bytes文件可能在如下三个位置：

• /sys/fs/cgroup/memory/docker/<container_id>/
• /sys/fs/cgroup/memory/kubepods/burstable/<pod_id>/<container_id>/
• /sys/fs/cgroup/memory/kubepods/besteffort/<pod_id>/<container_id>/
• /sys/fs/cgroup/memory/kubepods/<pod_id>/<container_id>/

容器对应的memory.stat文件示例内容如下：

cat memory.stat

cache 171200512
rss 17965056
rss_huge 0
mapped_file 9031680
swap 0
pgpgin 72026
pgpgout 25843
pgfault 24624
pgmajfault 36
inactive_anon 4096
active_anon 17965056
inactive_file 72552448
active_file 98643968
unevictable 0
hierarchical_memory_limit 9223372036854771712
hierarchical_memsw_limit 9223372036854771712
total_cache 171200512
total_rss 17965056
total_rss_huge 0
total_mapped_file 9031680
total_swap 0
total_pgpgin 72026
total_pgpgout 25843
total_pgfault 24624
total_pgmajfault 36
total_inactive_anon 4096
total_active_anon 17965056
total_inactive_file 72552448
total_active_file 98643968
total_unevictable 0


# cat memory.soft_limit_in_bytes

9223372036854771712

获取容器相关的MetaData

容器的PID

1. 通过docker ps获取节点所有容器基本信息
2. 扫描/proc文件夹下的所有进程文件夹中的cgroup文件，获取PIDs与container_id的对应关系

# cat /proc/<PID>/cgroup

11:devices:/docker/626a2040be97646c3ef12099d8bb3f614135b01ddb2a4873b9ff202d5b9ae11d
10:blkio:/docker/626a2040be97646c3ef12099d8bb3f614135b01ddb2a4873b9ff202d5b9ae11d
9:memory:/docker/626a2040be97646c3ef12099d8bb3f614135b01ddb2a4873b9ff202d5b9ae11d
8:perf_event:/docker/626a2040be97646c3ef12099d8bb3f614135b01ddb2a4873b9ff202d5b9ae11d
7:hugetlb:/docker/626a2040be97646c3ef12099d8bb3f614135b01ddb2a4873b9ff202d5b9ae11d
6:freezer:/docker/626a2040be97646c3ef12099d8bb3f614135b01ddb2a4873b9ff202d5b9ae11d
5:pids:/docker/626a2040be97646c3ef12099d8bb3f614135b01ddb2a4873b9ff202d5b9ae11d
4:cpuacct,cpu:/docker/626a2040be97646c3ef12099d8bb3f614135b01ddb2a4873b9ff202d5b9ae11d
3:net_prio,net_cls:/docker/626a2040be97646c3ef12099d8bb3f614135b01ddb2a4873b9ff202d5b9ae11d
2:cpuset:/docker/626a2040be97646c3ef12099d8bb3f614135b01ddb2a4873b9ff202d5b9ae11d
1:name=systemd:/docker/626a2040be97646c3ef12099d8bb3f614135b01ddb2a4873b9ff202d5b9ae11d