美文网首页
[mydocker]---构造容器02-实现资源限制01

[mydocker]---构造容器02-实现资源限制01

作者: nicktming | 来源:发表于2019-05-11 09:55 被阅读0次

实现容器memory限制

在构造容器01-实现run命令的基础上需要做以下改动.

1. 修改run命令

加入参数-m表示接受memory限制(command/command.go)

var RunCommand = cli.Command{
    Name: "run",
    Flags: []cli.Flag {
        cli.BoolFlag{
            Name: "it",
            Usage: "enable tty",
        },
        cli.StringFlag{
            Name: "m",
            Usage: "limit memory usage",
        },
    },
    Action: func(c *cli.Context) error {
        tty     := c.Bool("it")
        memory  := c.String("m")
        command := c.Args().Get(0)
        Run(command, tty, memory)
        return nil
    },
}

2. 实现一些utils函数

当前结构如下所示

root@nicktming:~/go/src/github.com/nicktming# tree mydocker
mydocker
|-- cgroups
|   |-- cgroup-manager.go
|   |-- subsystems
|   |   |-- memory.go
|   |   `-- utils.go
|   `-- utils_test.go
|-- command
|   |-- command.go
|   |-- init.go
|   `-- run.go
|-- main.go
|-- README.md
|-- test
|   `-- syscall
|       `-- TestExec.go
`-- urfave-cli-examples
    |-- test01.go
    |-- test02.go
    `-- test03.go
2.1 实现找到对应subsystem的目录位置

根据subsystem的类型找到对应的hierarchy, 从而可以在该hierarchy创建子cgroup, 进而把进程添加到此cgroup的限制中, 从而达到在此subsystem上限制进程的作用. 在cgroups/utils-test中.

func Test000(t *testing.T)  {
    mountPath := FindCgroupMountPoint("memory")
    log.Printf("mountPath:%s\n", mountPath)
}

cgroups/subsystems/utils.go如下:

func FindCgroupMountPoint(subsystem string) string {
    f, err := os.Open("/proc/self/mountinfo")
    if err != nil {
        log.Printf("Error open file error : %v\n", err)
        return ""
    }
    defer f.Close()

    bfRd := bufio.NewReader(f)
    for {
        line, err := bfRd.ReadBytes('\n')
        if err != nil {
            if err == io.EOF {
                return ""
            }
        }
        parts := strings.Split(string(line), " ")
        if strings.Contains(parts[len(parts) - 1], subsystem) {
            return parts[4]
        }
    }
}

运行结果如下:

root@nicktming:~/go/src/github.com/nicktming/mydocker/cgroups# go test -v utils_test.go -test.run Test000
=== RUN   Test000
2019/03/31 19:21:46 mountPath:/sys/fs/cgroup/memory
--- PASS: Test000 (0.00s)
PASS
ok      command-line-arguments  0.002s
2.2 找到当前容器所在subsystem的hierarchy的绝对路径

根据subsystem需要找到当前容器的cgroup位置,这样才可以往里面加入相关的限制. 在cgroups/subsystems/utils-test中.
这是当前路径memory的文件.

root@nicktming:/sys/fs/cgroup/memory# ls
cgroup.clone_children  memory.kmem.failcnt             memory.kmem.tcp.max_usage_in_bytes  memory.numa_stat            memory.usage_in_bytes
cgroup.event_control   memory.kmem.limit_in_bytes      memory.kmem.tcp.usage_in_bytes      memory.oom_control          memory.use_hierarchy
cgroup.procs           memory.kmem.max_usage_in_bytes  memory.kmem.usage_in_bytes          memory.pressure_level       notify_on_release
cgroup.sane_behavior   memory.kmem.slabinfo            memory.limit_in_bytes               memory.soft_limit_in_bytes  release_agent
memory.failcnt         memory.kmem.tcp.failcnt         memory.max_usage_in_bytes           memory.stat                 tasks
memory.force_empty     memory.kmem.tcp.limit_in_bytes  memory.move_charge_at_immigrate     memory.swappiness           test-limit-memory

root@nicktming:~/go/src/github.com/nicktming# cat mydocker/cgroups/cgroup-manager.go 
package cgroups
const (
    ResourceName = "mydocker"
)

代码(cgroups/subsystems/utils-test.go)如下:

func FindAbsolutePath(subsystem string) string {
    path := FindCgroupMountPoint(subsystem)
    if path != "" {
        absolutePath := path + "/" + cgroups.ResourceName
        exist, err := PathExists(absolutePath)
        if err != nil {
            log.Printf("PathExists error : %v\n", err)
            return ""
        }
        if !exist {
            err := os.Mkdir(absolutePath, os.ModePerm)
            if err != nil {
                log.Printf("Mkdir absolutePath:%s error : %v\n", err)
                return ""
            }
        }
        return absolutePath
    }
    return ""
}

func PathExists(path string) (bool, error) {
    _, err := os.Stat(path)
    if err == nil {
        return true, nil
    }
    if os.IsNotExist(err) {
        return false, nil
    }
    return false, err
}

func FindCgroupMountPoint(subsystem string) string {
    f, err := os.Open("/proc/self/mountinfo")
    if err != nil {
        log.Printf("Error open file error : %v\n", err)
        return ""
    }
    defer f.Close()

    bfRd := bufio.NewReader(f)
    for {
        line, err := bfRd.ReadBytes('\n')
        if err != nil {
            if err == io.EOF {
                return ""
            }
        }
        parts := strings.Split(string(line), " ")
        if strings.Contains(parts[len(parts) - 1], subsystem) {
            return parts[4]
        }
    }
}

cgroups/utils_test.go如下:

func Test001(t *testing.T)  {
    absolutePath := subsystems.FindAbsolutePath("memory")
    log.Printf("absolutePath:%s\n", absolutePath)
}

运行结果如下:

root@nicktming:~/go/src/github.com/nicktming/mydocker/cgroups# go test -v utils_test.go -test.run Test001
=== RUN   Test001
2019/03/31 19:34:13 absolutePath:/sys/fs/cgroup/memory/mydocker
--- PASS: Test001 (0.00s)
PASS
ok      command-line-arguments  0.002s
// 可以看下是否已经生成
root@nicktming:/sys/fs/cgroup/memory# ls
cgroup.clone_children  memory.kmem.failcnt             memory.kmem.tcp.max_usage_in_bytes  memory.numa_stat            memory.usage_in_bytes  test-limit-memory
cgroup.event_control   memory.kmem.limit_in_bytes      memory.kmem.tcp.usage_in_bytes      memory.oom_control          memory.use_hierarchy
cgroup.procs           memory.kmem.max_usage_in_bytes  memory.kmem.usage_in_bytes          memory.pressure_level       mydocker
cgroup.sane_behavior   memory.kmem.slabinfo            memory.limit_in_bytes               memory.soft_limit_in_bytes  notify_on_release
memory.failcnt         memory.kmem.tcp.failcnt         memory.max_usage_in_bytes           memory.stat                 release_agent
memory.force_empty     memory.kmem.tcp.limit_in_bytes  memory.move_charge_at_immigrate     memory.swappiness           tasks

可以看到已经生成了用于限制memory的当前容器的cgroup mydocker

root@nicktming:/sys/fs/cgroup/memory# ls mydocker/
cgroup.clone_children  memory.kmem.failcnt             memory.kmem.tcp.limit_in_bytes      memory.max_usage_in_bytes        memory.soft_limit_in_bytes  notify_on_release
cgroup.event_control   memory.kmem.limit_in_bytes      memory.kmem.tcp.max_usage_in_bytes  memory.move_charge_at_immigrate  memory.stat                 tasks
cgroup.procs           memory.kmem.max_usage_in_bytes  memory.kmem.tcp.usage_in_bytes      memory.numa_stat                 memory.swappiness
memory.failcnt         memory.kmem.slabinfo            memory.kmem.usage_in_bytes          memory.oom_control               memory.usage_in_bytes
memory.force_empty     memory.kmem.tcp.failcnt         memory.limit_in_bytes               memory.pressure_level            memory.use_hierarchy

3. 实现资源限制

从2中已经可以看到生成当前容器关于某个subsystem的cgroup, 所以这部分将会把相关限制比如内存加入进来.

root@nicktming:/sys/fs/cgroup/memory# pwd
/sys/fs/cgroup/memory
root@nicktming:/sys/fs/cgroup/memory# cat mydocker/tasks
root@nicktming:/sys/fs/cgroup/memory# cat mydocker/memory.limit_in_bytes 
18446744073709551615

代码如下cgroups/subsystems/memory.go

func Set(content string) error {
    absolutePath := ""
    if absolutePath = FindAbsolutePath("memory"); absolutePath == "" {
        log.Printf("ERROR: absoutePath is empty!\n")
        return fmt.Errorf("ERROR: absoutePath is empty!\n")
    }
    if err := ioutil.WriteFile(path.Join(absolutePath, "memory.limit_in_bytes"), []byte(content),0644); err != nil {
        log.Printf("ERROR write content:%s.\n", content)
        return fmt.Errorf("ERROR write content:%s.\n", content)
    }
    return nil
}

func Apply(pid string) error {
    absolutePath := ""
    if absolutePath = FindAbsolutePath("memory"); absolutePath == "" {
        log.Printf("ERROR: absoutePath is empty!\n")
        return fmt.Errorf("ERROR: absoutePath is empty!\n")
    }
    log.Printf("Apply absolutePath:%s, taskPath:%s\n", absolutePath, path.Join(absolutePath, "tasks"))
    if err := ioutil.WriteFile(path.Join(absolutePath, "tasks"), []byte(pid),0644); err != nil {
        log.Printf("ERROR write pid:%s.\n", pid)
        return fmt.Errorf("ERROR write pid:%s.\n", pid)
    } else {
        log.Printf("err : %v\n", err)
    }
    return nil
}

测试如下:

func Test002(t *testing.T)  {
    subsystems.Set("10M")
    pid := os.Getpid()
    log.Printf("current pid : %s\n", strconv.Itoa(pid))
    subsystems.Apply(strconv.Itoa(pid))
    for i := 0; i < 100; i++ {
        time.Sleep(1 * time.Second)
    }
}

执行结果如下:

root@nicktming:~/go/src/github.com/nicktming/mydocker/cgroups# go test -v utils_test.go -test.run Test002
=== RUN   Test002
2019/03/31 19:44:44 current pid : 18781
2019/03/31 19:44:44 Apply absolutePath:/sys/fs/cgroup/memory/mydocker, taskPath:/sys/fs/cgroup/memory/mydocker/tasks
2019/03/31 19:44:44 err : <nil>

打开另外一个terminal

root@nicktming:/sys/fs/cgroup/memory# cat mydocker/tasks 
18781
18785
root@nicktming:/sys/fs/cgroup/memory# cat mydocker/memory.limit_in_bytes 
10485760
// 运行结束后tasks里面已经没有进程了
root@nicktming:/sys/fs/cgroup/memory# cat mydocker/tasks 
root@nicktming:/sys/fs/cgroup/memory# cat mydocker/memory.limit_in_bytes 
10485760

4.实现容器资源隔离

在command/run.go中加入对memory的限制, 调用Set和Apply方法.

package command

import (
    "github.com/nicktming/mydocker/cgroups/subsystems"
    "log"
    "os"
    "os/exec"
    "strconv"
    "syscall"
)

func Run(command string, tty bool, memory string)  {
    //cmd := exec.Command(command)

    cmd := exec.Command("/proc/self/exe", "init", command)

    cmd.SysProcAttr = &syscall.SysProcAttr{
        Cloneflags: syscall.CLONE_NEWUTS | syscall.CLONE_NEWPID | syscall.CLONE_NEWNS | syscall.CLONE_NEWNET | syscall.CLONE_NEWIPC,
    }

    if tty {
        cmd.Stderr = os.Stderr
        cmd.Stdout = os.Stdout
        cmd.Stdin = os.Stdin
    }
    /**
     *   Start() will not block, so it needs to use Wait()
     *   Run() will block
     */
    if err := cmd.Start(); err != nil {
        log.Printf("Run Start err: %v.\n", err)
        log.Fatal(err)
    }
    log.Printf("222 before process pid:%d, memory:%s\n", cmd.Process.Pid, memory)

    subsystems.Set(memory)
    subsystems.Apply(strconv.Itoa(cmd.Process.Pid))

    cmd.Wait()
}

在test中加入memory.c做为测试.

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>

#define MB (1024 * 1024)

int main(int argc, char *argv[])
{
    char *p;
    int i = 0;
    while(1) {
        p = (char *)malloc(MB);
        memset(p, 0, MB);
        printf("%dM memory allocated\n", ++i);
        sleep(1);
    }

    return 0;
}

测试使用.

-------------------------------shell 01-----------------------------------
root@nicktming:~/go/src/github.com/nicktming/mydocker# go build .
root@nicktming:~/go/src/github.com/nicktming/mydocker# ./mydocker run -it -m 5M /bin/sh
// 此时可以打开另外一个terminal
-------------------------------shell 02-----------------------------------
root@nicktming:/sys/fs/cgroup/memory# pwd
/sys/fs/cgroup/memory
root@nicktming:/sys/fs/cgroup/memory# cat mydocker/tasks
23829
root@nicktming:/sys/fs/cgroup/memory# cat mydocker/memory.limit_in_bytes 
5242880
-------------------------------shell 01-----------------------------------
2019/03/31 20:32:56 222 before process pid:23829, memory:5M
2019/03/31 20:32:56 Apply absolutePath:/sys/fs/cgroup/memory/mydocker, taskPath:/sys/fs/cgroup/memory/mydocker/tasks
2019/03/31 20:32:56 err : <nil>
# cp /root/memory .
# ls
cgroups  command  main.go  memory  mydocker  README.md  test  urfave-cli-examples
# ./memory
1M memory allocated
2M memory allocated
3M memory allocated
4M memory allocated
Killed

可以看到确实是可以限制住内存的值. 已经基本成功了, 接下来可以加入删除功能.

5. 实现资源删除

资源删除其实在进程结束的时候把限制解除, 其实就把对应的文件夹给删除.
在cgroups/subsystems/memory.go中加入Remove方法.

func Remove() error {
    absolutePath := ""
    if absolutePath = FindAbsolutePath("memory"); absolutePath == "" {
        log.Printf("ERROR: absoutePath is empty!\n")
        return fmt.Errorf("ERROR: absoutePath is empty!\n")
    }
    if err := os.RemoveAll(absolutePath); err != nil {
        log.Printf("ERROR: remove absolutePath error:%v\n", err)
        return fmt.Errorf("ERROR: remove absolutePath error:%v\n", err)
    }
    return nil
}

在command/run.go中加入Remove方法.

func Run(command string, tty bool, memory string)  {
    //cmd := exec.Command(command)

    cmd := exec.Command("/proc/self/exe", "init", command)

    cmd.SysProcAttr = &syscall.SysProcAttr{
        Cloneflags: syscall.CLONE_NEWUTS | syscall.CLONE_NEWPID | syscall.CLONE_NEWNS | syscall.CLONE_NEWNET | syscall.CLONE_NEWIPC,
    }

    if tty {
        cmd.Stderr = os.Stderr
        cmd.Stdout = os.Stdout
        cmd.Stdin = os.Stdin
    }
    /**
     *   Start() will not block, so it needs to use Wait()
     *   Run() will block
     */
    if err := cmd.Start(); err != nil {
        log.Printf("Run Start err: %v.\n", err)
        log.Fatal(err)
    }
    log.Printf("222 before process pid:%d, memory:%s\n", cmd.Process.Pid, memory)

    subsystems.Set(memory)
    subsystems.Apply(strconv.Itoa(cmd.Process.Pid))
// 修改处
    defer subsystems.Remove()

    cmd.Wait()
}

至此一个简单的容器资源限制就结束了.

整体如下

root@nicktming:~/go/src/github.com/nicktming# git clone https://github.com/nicktming/mydocker.git
root@nicktming:~/go/src/github.com/nicktming# cd mydocker
root@nicktming:~/go/src/github.com/nicktming/mydocker# git checkout code-3.2.1
root@nicktming:~/go/src/github.com/nicktming/mydocker# ls
cgroups  command  main.go  memory  pictures  README.md  test  urfave-cli-examples
root@nicktming:~/go/src/github.com/nicktming/mydocker# go build .
root@nicktming:~/go/src/github.com/nicktming/mydocker# ./mydocker run -it -m 12M /bin/sh
2019/04/01 00:52:55 222 before process pid:22014, memory:12M
2019/04/01 00:52:55 Apply absolutePath:/sys/fs/cgroup/memory/mydocker, taskPath:/sys/fs/cgroup/memory/mydocker/tasks
2019/04/01 00:52:55 err : <nil>
# ./memory
1M memory allocated
2M memory allocated
3M memory allocated
4M memory allocated
5M memory allocated
6M memory allocated
7M memory allocated
8M memory allocated
9M memory allocated
10M memory allocated
11M memory allocated
Killed

时序图如下

时序.png

参考

1. https://www.jianshu.com/p/7790ca1bc8f6
2. 自己动手写docker.(基本参考此书,加入一些自己的理解,加深对docker的理解)

全部内容

mydocker.png

1. [mydocker]---环境说明
2. [mydocker]---urfave cli 理解
3. [mydocker]---Linux Namespace
4. [mydocker]---Linux Cgroup
5. [mydocker]---构造容器01-实现run命令
6. [mydocker]---构造容器02-实现资源限制01
7. [mydocker]---构造容器02-实现资源限制02
8. [mydocker]---构造容器03-实现增加管道
9. [mydocker]---通过例子理解存储驱动AUFS
10. [mydocker]---通过例子理解chroot 和 pivot_root
11. [mydocker]---一步步实现使用busybox创建容器
12. [mydocker]---一步步实现使用AUFS包装busybox
13. [mydocker]---一步步实现volume操作
14. [mydocker]---实现保存镜像
15. [mydocker]---实现容器的后台运行
16. [mydocker]---实现查看运行中容器
17. [mydocker]---实现查看容器日志
18. [mydocker]---实现进入容器Namespace
19. [mydocker]---实现停止容器
20. [mydocker]---实现删除容器
21. [mydocker]---实现容器层隔离
22. [mydocker]---实现通过容器制作镜像
23. [mydocker]---实现cp操作
24. [mydocker]---实现容器指定环境变量
25. [mydocker]---网际协议IP
26. [mydocker]---网络虚拟设备veth bridge iptables
27. [mydocker]---docker的四种网络模型与原理实现(1)
28. [mydocker]---docker的四种网络模型与原理实现(2)
29. [mydocker]---容器地址分配
30. [mydocker]---网络net/netlink api 使用解析
31. [mydocker]---网络实现
32. [mydocker]---网络实现测试

相关文章

网友评论

      本文标题：[mydocker]---构造容器02-实现资源限制01

      本文链接：https://www.haomeiwen.com/subject/ygvjbqtx.html

      延伸阅读

      深度阅读

      • 您也可以注册成为美文阅读网的作者，发表您的原创作品、分享您的心情！

      栏目导航

      热点阅读

      关于我们|服务条款|联系我们|[mydocker]---构造容器02-实现资源限制01|投稿指南|网站地图|RSS订阅|排版工具|手机版

      提供经典美文摘抄,优美散文欣赏,现代诗歌精选,短篇小说,心情随笔,表白情书范文,故事会在线阅读欣赏

      Copyright © 2014-2023 Haomeiwen.com All Rights Reserved. 好美文阅读网 版权所有

      备案信息：桂公网安备 45052102000051号 · 桂ICP备13007215号-3

      本站所收录作品、热点评论等信息部分来源互联网，目的只是为了系统归纳学习和传递资讯

      所有作品版权归原创作者所有，与本站立场无关，如不慎侵犯了你的权益，请联系我们告知，我们将做删除处理！