Kubernetes 動態卷—NFS

Like

Like Love Haha Wow Sad Angry

一、基礎概念

  動態卷供應（Dynamic Volume Provisioning）允許按需建立儲存卷。如果沒有動態供應，群集管理員必須手動呼叫其雲或儲存提供商來建立新的儲存卷，然後建立 PV 物件以在 Kubernetes 中表示它們。動態供應功能消除了叢集管理員預先供應儲存的需要，相反，它會在使用者請求時自動配置儲存。

  動態卷的實現是基於 StorageClass，每個 StorageClass 物件都指定一個卷外掛（也稱為 provisioner），該外掛提供一個卷和一組引數，以便在設定時傳遞給該 provisioner。

 

二、NFS 動態卷配置

  這裡我們使用 Kubernetes NFS Subdir External Provisioner，它是一個自動供應器，它使用您現有的和已經配置的 NFS 伺服器來支援通過 PVC 動態供應 PV。

  參考文件： https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner

  配置流程:

- 配置 NFS 伺服器
- 獲取 NFS Subdir External Provisioner 檔案，地址：https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner/tree/master/deploy
- 如果叢集啟用了 RBAC，設定授權。
- 配置 NFS subdir external provisioner
- 建立 Storage Class
- 建立 PVC 和 Pod 進行測試

 

1、配置 NFS

[[email protected] ~]# yum install -y nfs-utils
[[email protected] ~]# mkdir /nfs
[[email protected] ~]# echo "/nfs *(rw,no_root_squash)" > /etc/exports
[[email protected] ~]# systemctl restart nfs rpcbind
[[email protected] ~]# showmount -e localhost
Export list for localhost:
/nfs *

 

2、獲取檔案

[[email protected] k8s]# git clone https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner.git
[[email protected] k8s]# cd nfs-subdir-external-provisioner/
[[email protected] nfs-subdir-external-provisioner]# rm -rf !(deploy)
[[email protected] nfs-subdir-external-provisioner]# mv deploy/*.yaml ./ && rm -rf deploy
[[email protected] nfs-subdir-external-provisioner]# ll
total 20
-rw-r--r-- 1 root root  255 Oct  4 04:31 class.yaml
-rw-r--r-- 1 root root 1064 Oct  4 04:31 deployment.yaml
-rw-r--r-- 1 root root 1900 Oct  4 04:31 rbac.yaml
-rw-r--r-- 1 root root  199 Oct  4 04:31 test-claim.yaml
-rw-r--r-- 1 root root  401 Oct  4 04:31 test-pod.yaml

 

3、配置 RBAC

直接使用獲取的原始檔案，只改變了名稱空間：

[[email protected] nfs-subdir-external-provisioner]# sed -i'' "s/namespace:.*/namespace: nfs/g" rbac.yaml deployment.yaml
[[email protected] nfs-subdir-external-provisioner]# cat rbac.yaml 
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: nfs
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nfs-client-provisioner-runner
rules:
  - apiGroups: [""]
    resources: ["nodes"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: run-nfs-client-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: nfs
roleRef:
  kind: ClusterRole
  name: nfs-client-provisioner-runner
  apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: nfs
rules:
  - apiGroups: [""]
    resources: ["endpoints"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: nfs
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: nfs
roleRef:
  kind: Role
  name: leader-locking-nfs-client-provisioner
  apiGroup: rbac.authorization.k8s.io

 

4、配置 NFS subdir external provisioner

填寫之前配置的 NFS Server 資訊：

[[email protected] nfs-subdir-external-provisioner]# cat deployment.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nfs-client-provisioner
  labels:
    app: nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: nfs
spec:
  replicas: 1
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: nfs-client-provisioner
  template:
    metadata:
      labels:
        app: nfs-client-provisioner
    spec:
      serviceAccountName: nfs-client-provisioner
      containers:
        - name: nfs-client-provisioner
          image: k8s.gcr.io/sig-storage/nfs-subdir-external-provisioner:v4.0.2
          volumeMounts:
            - name: nfs-client-root
              mountPath: /persistentvolumes
          env:
            - name: PROVISIONER_NAME
              value: k8s-sigs.io/nfs-subdir-external-provisioner
            - name: NFS_SERVER
              value: 10.60.77.22
            - name: NFS_PATH
              value: /nfs
      volumes:
        - name: nfs-client-root
          nfs:
            server: 10.60.77.22
            path: /nfs

執行：

[[email protected] nfs-subdir-external-provisioner]# kubectl apply -f rbac.yaml -f deployment.yaml 
[[email protected] nfs-subdir-external-provisioner]# kubectl get pod -n nfs
NAME                                     READY   STATUS    RESTARTS   AGE
nfs-client-provisioner-d89bb6fcd-tkwwb   1/1     Running   0          2s

 

5、建立 Storage class

[[email protected] nfs-subdir-external-provisioner]# cat class.yaml 
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: managed-nfs-storage
provisioner: k8s-sigs.io/nfs-subdir-external-provisioner # or choose another name, must match deployment's env PROVISIONER_NAME'
parameters:
  archiveOnDelete: "false"

[[email protected] nfs-subdir-external-provisioner]# kubectl apply -f class.yaml 
[[email protected] nfs-subdir-external-provisioner]# kubectl get sc
NAME                  PROVISIONER                                   RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
managed-nfs-storage   k8s-sigs.io/nfs-subdir-external-provisioner   Delete          Immediate           false                  9s

 

6、測試

建立 PVC 測試：

[[email protected] nfs-subdir-external-provisioner]# cat test-claim.yaml 
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: test-claim
spec:
  storageClassName: managed-nfs-storage
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Mi

[[email protected] nfs-subdir-external-provisioner]# kubectl apply -f test-claim.yaml 
[[email protected] nfs-subdir-external-provisioner]# kubectl get pvc
NAME         STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS          AGE
test-claim   Bound    pvc-feafa663-67e9-49fc-bdc3-0b199de77845   1Mi        RWX            managed-nfs-storage   5s

[[email protected] nfs-subdir-external-provisioner]# kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                STORAGECLASS          REASON   AGE
pvc-b5a0ea83-4f73-416f-862b-a0ffda2c3054   1Mi        RWX            Delete           Bound    default/test-claim   managed-nfs-storage            7s

建立 Pod 測試：

[[email protected] nfs-subdir-external-provisioner]# cat test-pod.yaml 
kind: Pod
apiVersion: v1
metadata:
  name: test-pod
spec:
  containers:
  - name: test-pod
    image: busybox:stable
    command:
      - "/bin/sh"
    args:
      - "-c"
      - "touch /mnt/SUCCESS && exit 0 || exit 1"
    volumeMounts:
      - name: nfs-pvc
        mountPath: "/mnt"
  restartPolicy: "Never"
  volumes:
    - name: nfs-pvc
      persistentVolumeClaim:
        claimName: test-claim

[[email protected] nfs-subdir-external-provisioner]# kubectl apply -f test-pod.yaml 
[[email protected] nfs-subdir-external-provisioner]# kubectl get pod
NAME                                     READY   STATUS      RESTARTS   AGE
test-pod                                 0/1     Completed   0          28s
[[email protected] nfs-subdir-external-provisioner]# tree /nfs
/nfs
└── default-test-claim-pvc-feafa663-67e9-49fc-bdc3-0b199de77845
    └── SUCCESS

預設目錄路徑為： ${namespace}-${pvcName}-${pvName}

 

7、Storage Class 相關引數

NFS subdir external provisioner 的 Storage Class 相關引數：

引數	描述	備註
onDelete	如果存在且值為delete，則刪除該目錄；如果存在值為retain，則儲存該目錄。	將在共享上以名稱歸檔：archived-
archiveOnDelete	預設為true，即刪除時會儲存目錄。如果存在且值為false，則刪除該目錄。如果onDelete存在，archiveOnDelete將被忽略。	將在共享上以名稱歸檔：archived-
pathPattern	指定用於通過 PVC 元資料（例如標籤、註釋、名稱或名稱空間）建立目錄路徑的模板。要指定元資料，請使用${.PVC. }。示例：如果資料夾應命名為 – ，則使用${.PVC.namespace}-${.PVC.name} 作為路徑模式。	n/a

示例：

[[email protected] nfs-subdir-external-provisioner]# cat class-test.yaml 
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: nfs-test
provisioner: k8s-sigs.io/nfs-subdir-external-provisioner
parameters:
  pathPattern: "${.PVC.namespace}/${.PVC.annotations.nfs.io/storage-path}"
  onDelete: delete

[[email protected] nfs-subdir-external-provisioner]# cat test-pvc.yaml 
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: test-claim
spec:
  storageClassName: nfs-test
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Mi

[[email protected] nfs-subdir-external-provisioner]# kubectl apply -f class-test.yaml -f test-pvc.yaml -f test-pod.yaml
[[email protected] nfs-subdir-external-provisioner]# tree /nfs
/nfs
└── default
    └── SUCCESS