2

我正在尝试在 k8s 集群内的命名空间之间共享一个 cephfs 卷。我将 ceph-csi 与 cephfs 一起使用。

按照https://github.com/ceph/ceph-csi/blob/devel/docs/static-pvc.md#cephfs-static-pvc在两个命名空间中创建静态 pv+pvc。如果我不在同一个节点上启动两个 pod，则可以使用。

如果两个 pod 在同一个节点上，第二个 pod 会出现错误：

MountVolume.SetUp failed for volume "team-test-vol-pv" : rpc error: code = Internal desc = failed to bind-mount /var/lib/kubelet/plugins/k
ubernetes.io/csi/pv/team-test-vol-pv/globalmount to /var/lib/kubelet/pods/007fc605-7fa4-4dc6-890f-fc0dabe5740b/volumes/kubernetes.io~csi/team-test-vol-pv/mount: an error (exit status 32) occurred while running mount arg
s: [-o bind,_netdev /var/lib/kubelet/plugins/kubernetes.io/csi/pv/team-test-vol-pv/globalmount /var/lib/kubelet/pods/007fc605-7fa4-4dc6-890f-fc0dabe5740b/volumes/kubernetes.io~csi/team-test-vol-pv/moun

任何想法如何解决这个问题或如何在不同的 NS 中使用单个 RWX 卷？

团队-x 的 PV+PVC：

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: test-vol
  namespace: team-x
spec:
  storageClassName: ""
  accessModes:
  - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
  volumeMode: Filesystem
  # volumeName should be same as PV name
  volumeName: team-x-test-vol-pv

---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: team-x-test-vol-pv
spec:
  claimRef:
    namespace: team-x
    name: test-vol
  storageClassName: ""
  accessModes:
  - ReadWriteMany
  capacity:
    storage: 1Gi
  csi:
    driver: cephfs.csi.ceph.com
    nodeStageSecretRef:
      name: csi-cephfs-secret-hd
      namespace: ceph-csi
    volumeAttributes:
      "clusterID": "cd79ae11-1804-4c06-a97e-aeeb961b84b0"
      "fsName": "cephfs"
      "staticVolume": "true"
      "rootPath": /volumes/team/share/8b73d3bb-282e-4c32-b13a-97459419bd5b
    # volumeHandle can be anything, need not to be same
    # as PV name or volume name. keeping same for brevity
    volumeHandle: team-share
  persistentVolumeReclaimPolicy: Retain
  volumeMode: Filesystem

团队用 PV+PVC

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: test-vol
  namespace: team-y
spec:
  storageClassName: ""
  accessModes:
  - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
  volumeMode: Filesystem
  # volumeName should be same as PV name
  volumeName: team-y-test-vol-pv

---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: team-y-test-vol-pv
spec:
  claimRef:
    namespace: team-y
    name: test-vol
  storageClassName: ""
  accessModes:
  - ReadWriteMany
  capacity:
    storage: 1Gi
  csi:
    driver: cephfs.csi.ceph.com
    nodeStageSecretRef:
      name: csi-cephfs-secret-hd
      namespace: ceph-csi
    volumeAttributes:
      "clusterID": "cd79ae11-1804-4c06-a97e-aeeb961b84b0"
      "fsName": "cephfs"
      "staticVolume": "true"
      "rootPath": /volumes/team-y/share/8b73d3bb-282e-4c32-b13a-97459419bd5b
    # volumeHandle can be anything, need not to be same
    # as PV name or volume name. keeping same for brevity
    volumeHandle: team-share
  persistentVolumeReclaimPolicy: Retain
  volumeMode: Filesystem
4

2 回答 2

1

volumeHandle: xyz对每个 pv 都具有唯一性就可以了。测试在 3 个不同的命名空间中部署 3xdaemonsets。

于 2021-12-21T05:51:31.633 回答
0

您可能需要提供ReadWriteMany选项参考链接：https ://kubernetes.io/docs/concepts/storage/persistent-volumes/

于 2021-12-18T10:09:49.227 回答