利用LXCFS提升容器资源可见性

由于默认情况下容器挂载的是宿主机的硬件配置信息,导致有些应用根据这些信息来决定启动内存等的大小,导致应用内存溢出等问题。

LXCFS简介

社区中常见的做法是利用 lxcfs来提供容器中的资源可见性。lxcfs 是一个开源的FUSE(用户态文件系统)实现来支持LXC容器,它也可以支持Docker容器。

LXCFS通过用户态文件系统,在容器中提供下列 procfs 的文件。

1
2
3
4
5
6
/proc/cpuinfo
/proc/diskstats
/proc/meminfo
/proc/stat
/proc/swaps
/proc/uptime

LXCFS的示意图如下:

LXCFS的示意图

比如,把宿主机的 /var/lib/lxcfs/proc/memoinfo 文件挂载到Docker容器的/proc/meminfo位置后。容器中进程读取相应文件内容时,LXCFS的FUSE实现会从容器对应的Cgroup中读取正确的内存限制。从而使得应用获得正确的资源约束设定。

安装lxcfs ,先安装需要使用的依赖包:

yum install http://mirror.centos.org/centos/7/os/x86_64/Packages/fuse-libs-2.9.2-10.el7.x86_64.rpm

用deamonset方式在每个节点启动一个lxcfs,lxcfs-daemonset.yaml配置如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
apiVersion: apps/v1beta2
kind: DaemonSet
metadata:
name: lxcfs
labels:
app: lxcfs
spec:
selector:
matchLabels:
app: lxcfs
template:
metadata:
labels:
app: lxcfs
spec:
hostPID: true
tolerations:
- key: node-role.kubernetes.io/master
effect: NoSchedule
containers:
- name: lxcfs
image: reg.test.sui.internal/library/lxcfs:2.0.8-1
imagePullPolicy: Always
securityContext:
privileged: true
volumeMounts:
- name: cgroup
mountPath: /sys/fs/cgroup
- name: lxcfs
mountPath: /var/lib/lxcfs
mountPropagation: Bidirectional
- name: usr-local
mountPath: /usr/local
volumes:
- name: cgroup
hostPath:
path: /sys/fs/cgroup
- name: usr-local
hostPath:
path: /usr/local
- name: lxcfs
hostPath:
path: /var/lib/lxcfs
type: DirectoryOrCreate

然后在发版平台deploy的模板配置资源限制的信息,主要信息如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
volumeMounts:
- mountPath: /proc/cpuinfo
name: lxcfs-proc-cpuinfo
- mountPath: /proc/meminfo
name: lxcfs-proc-meminfo
- mountPath: /proc/diskstats
name: lxcfs-proc-diskstats
- mountPath: /proc/stat
name: lxcfs-proc-stat
- mountPath: /proc/swaps
name: lxcfs-proc-swaps
- mountPath: /proc/uptime
name: lxcfs-proc-uptime
restartPolicy: Always
imagePullSecrets:
- name: pull-registry-secret
volumes:
- hostPath:
path: /var/lib/lxcfs/proc/cpuinfo
type: ""
name: lxcfs-proc-cpuinfo
- hostPath:
path: /var/lib/lxcfs/proc/diskstats
type: ""
name: lxcfs-proc-diskstats
- hostPath:
path: /var/lib/lxcfs/proc/meminfo
type: ""
name: lxcfs-proc-meminfo
- hostPath:
path: /var/lib/lxcfs/proc/stat
type: ""
name: lxcfs-proc-stat
- hostPath:
path: /var/lib/lxcfs/proc/swaps
type: ""
name: lxcfs-proc-swaps
- hostPath:
path: /var/lib/lxcfs/proc/uptime
type: ""
name: lxcfs-proc-uptime

启动应用之后即可看到内存大小就是cgroup分配的内存大小,注意不要使用alpine镜像,这个镜像挂载仍然有问题。

# 推荐文章
  1.深刻理解Docker镜像大小
  2.k8s专题[1.k8s基础概念]
  3.k8s专题[10.使用Spinnaker持续发布应用]
  4.k8s专题[2.k8s设计原则]
  5.k8s专题[3.k8s基础组件]

评论


:D 一言句子获取中...