Freigeben über

Bereitstellen und Konfigurieren eines Valkey-Clusters auf Azure Kubernetes Service (AKS)

  • Artikel

In diesem Artikel zeigen wir, wie ein Valkey-Cluster auf Azure Kubernetes Service (AKS) konfiguriert und bereitgestellt werden kann.

Hinweis

Dieser Artikel enthält Verweise auf die Begriffe Master und Slave, die von Microsoft nicht mehr verwendet werden. Sobald der Begriff aus der Valkey-Software entfernt wurde, wird er auch aus diesem Artikel entfernt.

Erstellen eines Namespace

  1. Erstellen Sie für den Valkey-Cluster einen Namespace, indem Sie den Befehl kubectl create namespace verwenden.

    kubectl create namespace ${SERVICE_ACCOUNT_NAMESPACE} --dry-run=client --output yaml | kubectl apply -f -

    Beispielausgabe:

    namespace/valkey created

Erstellen von Geheimnissen

  1. Generieren Sie ein zufälliges Kennwort für den Valkey-Cluster mithilfe von OpenSSL, und speichern Sie es mithilfe des Befehls az keyvault secret set in Ihrem Azure Key Vault. Legen Sie die Richtlinie so fest, dass die benutzerseitig zugewiesene Identität das Geheimnis mithilfe des Befehls az keyvault set-policy abrufen kann.

    SECRET=$(openssl rand -base64 32)
echo requirepass $SECRET > /tmp/valkey-password-file.conf
echo primaryauth $SECRET >> /tmp/valkey-password-file.conf
az keyvault secret set --vault-name $MY_KEYVAULT_NAME --name valkey-password-file --file /tmp/valkey-password-file.conf --output table
rm /tmp/valkey-password-file.conf
az keyvault set-policy --name $MY_KEYVAULT_NAME --object-id $userAssignedObjectID --secret-permissions get --output table

  2. Erstellen Sie eine SecretProviderClass-Ressource, um auf das im Schlüsseltresor gespeicherte Valkey-Kennwort zuzugreifen, indem Sie den Befehl kubectl apply verwenden.

    kubectl apply -f - <<EOF
---
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: valkey-password
  namespace: valkey
spec:
  provider: azure
  parameters:
    usePodIdentity: "false"
    useVMManagedIdentity: "true"
    userAssignedIdentityID: "${userAssignedIdentityID}"
    keyvaultName: ${MY_KEYVAULT_NAME}              # the name of the AKV instance
    objects: |
      array:
        - |
          objectName: valkey-password-file
          objectAlias: valkey-password-file.conf
          objectType: secret
    tenantId: "${TENANT_ID}" # the tenant ID of the AKV instance
EOF

Bereitstellen des Valkey-Clusters

  1. Erstellen Sie eine ConfigMap, die als Volume im Valkey-StatefulSet bereitgestellt wird, um den Valkey-Cluster zu konfigurieren, indem Sie den Befehl kubectl apply verwenden.

    kubectl apply -f - <<EOF
apiVersion: v1
kind: ConfigMap
metadata:
  name: valkey-cluster
  namespace: valkey
data:
  valkey.conf:  |+
    cluster-enabled yes
    cluster-node-timeout 15000
    cluster-config-file /data/nodes.conf
    appendonly yes
    protected-mode yes
    dir /data
    port 6379
    include /etc/valkey-password/valkey-password-file.conf
EOF

    Beispielausgabe:

    configmap/valkey-cluster created

  2. Erstellen Sie eine StatefulSet-Ressource mit dem spec.affinity-Ziel, alle primären Replikate in Zone 1 und Zone 2 beizubehalten, vorzugsweise auf verschiedenen Knoten, indem Sie den Befehl kubectl apply verwenden.

    kubectl apply -f - <<EOF
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: valkey-masters
  namespace: valkey
spec:
  serviceName: "valkey-masters"
  replicas: 3
  selector:
    matchLabels:
      app: valkey
  template:
    metadata:
      labels:
        app: valkey
        appCluster: valkey-masters
    spec:
      terminationGracePeriodSeconds: 20
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: agentpool
                operator: In
                values:
                - valkey
              - key: topology.kubernetes.io/zone
                operator: In
                values:
                - ${MY_LOCATION}-1
            - matchExpressions:
              - key: agentpool
                operator: In
                values:
                - valkey
              - key: topology.kubernetes.io/zone
                operator: In
                values:
                - ${MY_LOCATION}-2
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 90
            podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: app
                  operator: In
                  values:
                  - valkey
              topologyKey: topology.kubernetes.io/zone
          - weight: 90
            podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: app
                  operator: In
                  values:
                  - valkey
              topologyKey: kubernetes.io/hostname
      containers:
      - name: valkey
        image: "${MY_ACR_REGISTRY}.azurecr.io/valkey:latest"
        env:
        - name: VALKEY_PASSWORD_FILE
          value: "/etc/valkey-password/valkey-password-file.conf"
        - name: MY_POD_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        command:
          - "valkey-server"
        args:
          - "/conf/valkey.conf"
          - "--cluster-announce-ip"
          - "\$(MY_POD_IP)"
        resources:
          requests:
            cpu: "100m"
            memory: "100Mi"
        ports:
            - name: valkey
              containerPort: 6379
              protocol: "TCP"
            - name: cluster
              containerPort: 16379
              protocol: "TCP"
        volumeMounts:
        - name: conf
          mountPath: /conf
          readOnly: false
        - name: data
          mountPath: /data
          readOnly: false
        - name: valkey-password
          mountPath: /etc/valkey-password
          readOnly: true
      volumes:
      - name: valkey-password
        csi:
          driver: secrets-store.csi.k8s.io
          readOnly: true
          volumeAttributes:
            secretProviderClass: valkey-password
      - name: conf
        configMap:
          name: valkey-cluster
          defaultMode: 0755
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: managed-csi-premium
      resources:
        requests:
          storage: 20Gi
EOF

    Beispielausgabe:

    statefulset.apps/valkey-masters created

  3. Erstellen Sie eine zweite StatefulSet-Ressource für die sekundären Valkey-Replikate mit dem spec.affinity-Ziel, alle Replikate in Zone 3 beizubehalten, vorzugsweise auf verschiedenen Knoten, indem Sie den Befehl kubectl apply verwenden.

    kubectl apply -f - <<EOF
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: valkey-replicas
  namespace: valkey
spec:
  serviceName: "valkey-replicas"
  replicas: 3
  selector:
    matchLabels:
      app: valkey
  template:
    metadata:
      labels:
        app: valkey
        appCluster: valkey-replicas
    spec:
      terminationGracePeriodSeconds: 20
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: agentpool
                operator: In
                values:
                - valkey
              - key: topology.kubernetes.io/zone
                operator: In
                values:
                - ${MY_LOCATION}-3
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 90
            podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: app
                  operator: In
                  values:
                  - valkey
              topologyKey: kubernetes.io/hostname
      containers:
      - name: valkey
        image: "${MY_ACR_REGISTRY}.azurecr.io/valkey:latest"
        env:
        - name: VALKEY_PASSWORD_FILE
          value: "/etc/valkey-password/valkey-password-file.conf"
        - name: MY_POD_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        command:
          - "valkey-server"
        args:
          - "/conf/valkey.conf"
          - "--cluster-announce-ip"
          - "\$(MY_POD_IP)"
        resources:
          requests:
            cpu: "100m"
            memory: "100Mi"
        ports:
            - name: valkey
              containerPort: 6379
              protocol: "TCP"
            - name: cluster
              containerPort: 16379
              protocol: "TCP"
        volumeMounts:
        - name: conf
          mountPath: /conf
          readOnly: false
        - name: data
          mountPath: /data
          readOnly: false
        - name: valkey-password
          mountPath: /etc/valkey-password
          readOnly: true
      volumes:
      - name: valkey-password
        csi:
          driver: secrets-store.csi.k8s.io
          readOnly: true
          volumeAttributes:
            secretProviderClass: valkey-password
      - name: conf
        configMap:
          name: valkey-cluster
          defaultMode: 0755
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: managed-csi-premium
      resources:
        requests:
          storage: 20Gi
EOF

    Beispielausgabe:

    statefulset.apps/valkey-replicas created

  4. Stellen Sie mithilfe der Befehle kubectl get nodes und kubectl get pods sicher, dass master-N und replica-N in verschiedenen Knoten und Zonen ausgeführt werden.

    kubectl get pods -n valkey -o wide
kubectl get node -o custom-columns=Name:.metadata.name,Zone:".metadata.labels.topology\.kubernetes\.io/zone"

    Beispielausgabe:

    NAME                READY   STATUS    RESTARTS   AGE     IP             NODE                             NOMINATED NODE   READINESS GATES
valkey-masters-0    1/1     Running   0          2m55s   10.224.0.4     aks-valkey-18693609-vmss000004   <none>           <none>
valkey-masters-1    1/1     Running   0          2m31s   10.224.0.137   aks-valkey-18693609-vmss000000   <none>           <none>
valkey-masters-2    1/1     Running   0          2m7s    10.224.0.222   aks-valkey-18693609-vmss000001   <none>           <none>
valkey-replicas-0   1/1     Running   0          88s     10.224.0.237   aks-valkey-18693609-vmss000005   <none>           <none>
valkey-replicas-1   1/1     Running   0          70s     10.224.0.18    aks-valkey-18693609-vmss000002   <none>           <none>
valkey-replicas-2   1/1     Running   0          48s     10.224.0.242   aks-valkey-18693609-vmss000005   <none>           <none>
Name                                Zone
aks-nodepool1-17621399-vmss000000   centralus-1
aks-nodepool1-17621399-vmss000001   centralus-2
aks-nodepool1-17621399-vmss000003   centralus-3
aks-valkey-18693609-vmss000000      centralus-1
aks-valkey-18693609-vmss000001      centralus-2
aks-valkey-18693609-vmss000002      centralus-3
aks-valkey-18693609-vmss000003      centralus-1
aks-valkey-18693609-vmss000004      centralus-2
aks-valkey-18693609-vmss000005      centralus-3

    Warten Sie, bis alle Pods ausgeführt werden, bevor Sie mit dem nächsten Schritt fortfahren.

  5. Erstellen Sie mithilfe des Befehls kubectl apply drei monitorlose Service-Ressourcen (die erste für den gesamten Cluster, die zweite für die primären Replikate und die dritte für die sekundären Replikate), um die IP-Adressen der Valkey-Pods abzurufen.

    kubectl apply -f - <<EOF
apiVersion: v1
kind: Service
metadata:
  name: valkey-cluster
  namespace: valkey
spec:
  clusterIP: None
  ports:
  - name: valkey-port
    port: 6379
    protocol: TCP
    targetPort: 6379
  selector:
    app: valkey
  sessionAffinity: None
  type: ClusterIP
EOF

kubectl apply -f - <<EOF
apiVersion: v1
kind: Service
metadata:
  name: valkey-masters
  namespace: valkey
spec:
  clusterIP: None
  ports:
  - name: valkey-port
    port: 6379
    protocol: TCP
    targetPort: 6379
  selector:
    app: valkey
    appCluster: valkey-masters
  sessionAffinity: None
  type: ClusterIP
EOF

kubectl apply -f - <<EOF
apiVersion: v1
kind: Service
metadata:
  name: valkey-replicas
  namespace: valkey
spec:
  clusterIP: None
  ports:
  - name: valkey-port
    port: 6379
    protocol: TCP
    targetPort: 6379
  selector:
    app: valkey
    appCluster: valkey-replicas
  sessionAffinity: None
  type: ClusterIP
EOF

    Beispielausgabe:

    service/valkey-cluster created
service/valkey-masters created
service/valkey-replicas created

Ausführen des Valkey-Clusters

  1. Fügen Sie die primären Valkey-Replikate dem Cluster in den Zonen 1 und 2 mithilfe des Befehls kubectl exec hinzu.

    kubectl exec -it -n valkey valkey-masters-0 -- valkey-cli --cluster create --cluster-yes --cluster-replicas 0 \
                    valkey-masters-0.valkey-masters.valkey.svc.cluster.local:6379 \
                    valkey-masters-1.valkey-masters.valkey.svc.cluster.local:6379 \
                    valkey-masters-2.valkey-masters.valkey.svc.cluster.local:6379 \
                    --pass ${SECRET}

    Beispielausgabe:

    >>> Performing hash slots allocation on 3 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
M: ee6ac1d00d3f016b6f46c7ce11199bc1a7809a35 valkey-masters-0.valkey-masters.valkey.svc.cluster.local:6379
   slots:[0-5460] (5461 slots) master
M: fd1fb98db83976478e05edd3d2a02f9a13badd80 valkey-masters-1.valkey-masters.valkey.svc.cluster.local:6379
   slots:[5461-10922] (5462 slots) master
M: ea47bf57ae7080ef03164a4d48b662c7b4c8770e valkey-masters-2.valkey-masters.valkey.svc.cluster.local:6379
   slots:[10923-16383] (5461 slots) master
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
...
>>> Performing Cluster Check (using node valkey-masters-0.valkey-masters.valkey.svc.cluster.local:6379)
M: ee6ac1d00d3f016b6f46c7ce11199bc1a7809a35 valkey-masters-0.valkey-masters.valkey.svc.cluster.local:6379
   slots:[0-5460] (5461 slots) master
M: ea47bf57ae7080ef03164a4d48b662c7b4c8770e 10.224.0.176:6379
   slots:[10923-16383] (5461 slots) master
M: fd1fb98db83976478e05edd3d2a02f9a13badd80 10.224.0.247:6379
   slots:[5461-10922] (5462 slots) master
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

  2. Fügen Sie die Valkey-Replikate dem Cluster in Zone 3 mithilfe des Befehls kubectl exec hinzu.

    kubectl exec -ti -n valkey valkey-masters-0 -- valkey-cli --cluster add-node \
                    valkey-replicas-0.valkey-replicas.valkey.svc.cluster.local:6379 \
                    valkey-masters-0.valkey-masters.valkey.svc.cluster.local:6379  --cluster-slave \
                    --pass ${SECRET}

kubectl exec -ti -n valkey valkey-masters-0 -- valkey-cli --cluster add-node \
                    valkey-replicas-1.valkey-replicas.valkey.svc.cluster.local:6379 \
                    valkey-masters-1.valkey-masters.valkey.svc.cluster.local:6379  --cluster-slave \
                    --pass ${SECRET}

kubectl exec -ti -n valkey valkey-masters-0 -- valkey-cli --cluster add-node \
                    valkey-replicas-2.valkey-replicas.valkey.svc.cluster.local:6379 \
                    valkey-masters-2.valkey-masters.valkey.svc.cluster.local:6379  --cluster-slave \
                    --pass ${SECRET}

    Beispielausgabe:

    >>> Adding node valkey-replicas-0.valkey-replicas.valkey.svc.cluster.local:6379 to cluster valkey-masters-0.valkey-masters.valkey.svc.cluster.local:6379
>>> Performing Cluster Check (using node valkey-masters-0.valkey-masters.valkey.svc.cluster.local:6379)
M: ee6ac1d00d3f016b6f46c7ce11199bc1a7809a35 valkey-masters-0.valkey-masters.valkey.svc.cluster.local:6379
   slots:[0-5460] (5461 slots) master
M: ea47bf57ae7080ef03164a4d48b662c7b4c8770e 10.224.0.176:6379
   slots:[10923-16383] (5461 slots) master
M: fd1fb98db83976478e05edd3d2a02f9a13badd80 10.224.0.247:6379
   slots:[5461-10922] (5462 slots) master
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
Automatically selected master valkey-masters-0.valkey-masters.valkey.svc.cluster.local:6379
>>> Send CLUSTER MEET to node valkey-replicas-0.valkey-replicas.valkey.svc.cluster.local:6379 to make it join the cluster.
Waiting for the cluster to join

>>> Configure node as replica of valkey-masters-0.valkey-masters.valkey.svc.cluster.local:6379.
[OK] New node added correctly.
>>> Adding node valkey-replicas-1.valkey-replicas.valkey.svc.cluster.local:6379 to cluster valkey-masters-1.valkey-masters.valkey.svc.cluster.local:6379
>>> Performing Cluster Check (using node valkey-masters-1.valkey-masters.valkey.svc.cluster.local:6379)
M: fd1fb98db83976478e05edd3d2a02f9a13badd80 valkey-masters-1.valkey-masters.valkey.svc.cluster.local:6379
   slots:[5461-10922] (5462 slots) master
S: 0ebceb60cbcc31da9040159440a1f4856b992907 10.224.0.224:6379
   slots: (0 slots) slave
   replicates ee6ac1d00d3f016b6f46c7ce11199bc1a7809a35
M: ea47bf57ae7080ef03164a4d48b662c7b4c8770e 10.224.0.176:6379
   slots:[10923-16383] (5461 slots) master
M: ee6ac1d00d3f016b6f46c7ce11199bc1a7809a35 10.224.0.14:6379
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
Automatically selected master valkey-masters-1.valkey-masters.valkey.svc.cluster.local:6379
>>> Send CLUSTER MEET to node valkey-replicas-1.valkey-replicas.valkey.svc.cluster.local:6379 to make it join the cluster.
Waiting for the cluster to join

>>> Configure node as replica of valkey-masters-1.valkey-masters.valkey.svc.cluster.local:6379.
[OK] New node added correctly.
>>> Adding node valkey-replicas-2.valkey-replicas.valkey.svc.cluster.local:6379 to cluster valkey-masters-2.valkey-masters.valkey.svc.cluster.local:6379
>>> Performing Cluster Check (using node valkey-masters-2.valkey-masters.valkey.svc.cluster.local:6379)
M: ea47bf57ae7080ef03164a4d48b662c7b4c8770e valkey-masters-2.valkey-masters.valkey.svc.cluster.local:6379
   slots:[10923-16383] (5461 slots) master
S: 0ebceb60cbcc31da9040159440a1f4856b992907 10.224.0.224:6379
   slots: (0 slots) slave
   replicates ee6ac1d00d3f016b6f46c7ce11199bc1a7809a35
S: fa44edff683e2e01ee5c87233f9f3bc35c205dce 10.224.0.103:6379
   slots: (0 slots) slave
   replicates fd1fb98db83976478e05edd3d2a02f9a13badd80
M: ee6ac1d00d3f016b6f46c7ce11199bc1a7809a35 10.224.0.14:6379
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
M: fd1fb98db83976478e05edd3d2a02f9a13badd80 10.224.0.247:6379
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
Automatically selected master valkey-masters-2.valkey-masters.valkey.svc.cluster.local:6379
>>> Send CLUSTER MEET to node valkey-replicas-2.valkey-replicas.valkey.svc.cluster.local:6379 to make it join the cluster.
Waiting for the cluster to join

>>> Configure node as replica of valkey-masters-2.valkey-masters.valkey.svc.cluster.local:6379.
[OK] New node added correctly.

  3. Überprüfen Sie die Rollen der Pods mithilfe der folgenden Befehle:

    for x in $(seq 0 2); do echo "valkey-masters-$x"; kubectl exec -n valkey valkey-masters-$x  -- valkey-cli --pass ${SECRET} role; echo; done
for x in $(seq 0 2); do echo "valkey-replicas-$x"; kubectl exec -n valkey valkey-replicas-$x -- valkey-cli --pass ${SECRET} role; echo; done

    Beispielausgabe:

    valkey-masters-0
master
84
10.224.0.224
6379
84

valkey-masters-1
master
84
10.224.0.103
6379
84

valkey-masters-2
master
70
10.224.0.200
6379
70

valkey-replicas-0
slave
10.224.0.14
6379
connected
98

valkey-replicas-1
slave
10.224.0.247
6379
connected
98

valkey-replicas-2
slave
10.224.0.176
6379
connected
84

Nächste Schritte

Weitere Informationen zum Bereitstellen von Open-Source-Software auf Azure Kubernetes Service (AKS) finden Sie in den folgenden Artikeln:

Beitragende

Microsoft pflegt diesen Artikel. Die folgenden Mitwirkenden haben es ursprünglich geschrieben:

  • Nelly Kiboi | Servicetechnikerin
  • Saverio Proto | Principal Customer Experience Engineer
  • Don High | Principal Customer Engineer
  • LaBrina Loving | Principal Service Engineer
  • Ken Kilty | Principal TPM
  • Russell de Pina | Principal TPM
  • Colin Mixon | Produktmanager
  • Ketan Chawda | Senior Customer Engineer
  • Naveed Kharadi | Customer Experience Engineer
  • Erin Schaffer | Content Developer 2