Postgres on Azure kubernetes volume permission error

Question

I'm trying to deploy Postgresql on Azure Kubernetes with data persistency. So I'm using PVC. I searched lots of posts on here, most of them offered yaml files like below, but it's giving the error below;

chmod: changing permissions of '/var/lib/postgresql/data/pgdata': Operation not permitted
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

initdb: error: could not change permissions of directory "/var/lib/postgresql/data/pgdata": Operation not permitted
fixing permissions on existing directory /var/lib/postgresql/data/pgdata ...

deployment yaml file is below;

apiVersion: apps/v1
kind: Deployment
metadata:
  name: postgresql
spec:
  replicas: 1
  selector:
    matchLabels:
      app: postgresql
  template:
    metadata:
      labels:
        app: postgresql
    spec:
      containers:
        - name: postgresql
          image: postgres:13.2
          securityContext:
            runAsUser: 999
          imagePullPolicy: "IfNotPresent"
          ports:
            - containerPort: 5432
          envFrom:
            - secretRef:
                name: postgresql-secret
          volumeMounts:
            - mountPath: /var/lib/postgresql/data
              name: postgredb-kap
      volumes:
        - name: postgredb-kap
          persistentVolumeClaim:
            claimName: postgresql-pvc

Secret yaml is below;

apiVersion: v1
kind: Secret
metadata:
  name: postgresql-secret
type: Opaque
data:
  POSTGRES_DB: a2V5sd4=
  POSTGRES_USER: cG9zdGdyZXNhZG1pbg==
  POSTGRES_PASSWORD: c234Rw==
  PGDATA: L3Za234dGF0YQ==

pvc and sc yaml files are below:

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: postgresql-pvc
  labels:
    app: postgresql
spec:
  storageClassName: postgresql-sc
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
---
allowVolumeExpansion: true
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: postgresql-sc
mountOptions:
- dir_mode=0777
- file_mode=0777
- uid=1000
- gid=1000
parameters:
  skuName: Standard_LRS
provisioner: kubernetes.io/azure-file
reclaimPolicy: Retain

So when I use the mountpath like "- mountPath: /var/lib/postgresql/", it's working. I can reach the DB and it's good. But when I delete the pod and recreating, there is no DB! So no data persistency.

Can you please help, what am I missing here?

Thanks!

Could you please share output for "kubectl get pv" and "kubectl get pvc" comands? — Andriy Bilous
– Andriy Bilous, Commented Mar 30, 2021 at 9:55

Matt · Accepted Answer · 2021-03-30 10:24:05Z

4

One thing you could try is to change uid=1000,gid=1000 in mount options to 999 since this is the uid of postgres user in postgres conatiner (I didn't test this).

Another solution that will for certain solve this issue involves init conatainers.

Postgres container requires to start as root to be able to chown pgdata dir since its mounted as root dir. After it does this, it drops root permisions and runs as postgres user.

But you can use init container (running as root) to chmod the volume dir so that you can run main container as non-root.

Here is an example:

  initContainers:
    - name: init
      image: alpine
      command: ["sh", "-c", "chown 999:999 /var/lib/postgresql/data"]
      volumeMounts:
        - mountPath: /var/lib/postgresql/data
          name: postgredb-kap

answered Mar 30, 2021 at 10:24

Matt

8,2121 gold badge15 silver badges25 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

zingi Over a year ago

Also note, that you can't change permissions on an azure file storage after you have mounted it. So having an init-container like this will do nothing. Probably OP solved it, by changing uid and gid in the StorageClass definition.

zingi Over a year ago

Also azure files don' support hard links by default, which is required by postgres. see you have to create a nfs azure file storage for it to work

Andy W. · Accepted Answer · 2022-05-23 15:44:50Z

1

Based on the helpful answer from Matt. For bitnami postgresql the initContainer also works but with a slightly different configuration:

      initContainers:
        - name: init
          image: alpine
          command: ["sh", "-c", "chown 1001:1001 /bitnami/postgresql"]
          volumeMounts:
            - mountPath: /bitnami/postgresql
              name: postgres-volume

answered May 23, 2022 at 15:44

Andy W.

112 bronze badges

Comments

pablo.bueti · Accepted Answer · 2024-03-13 19:35:33Z

As @zingi pointed out in a comment, you can create a storage class that uses nfs providing full POSIX file system support as stated in the Microsoft documentation. This would be then the storage class definition:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: azurefile-csi-nfs
provisioner: file.csi.azure.com
allowVolumeExpansion: true
parameters:
  protocol: nfs
mountOptions:
  - nconnect=4
  - noresvport
  - actimeo=30

Just point spec.storageClassName on your PVC to it. I also have set the environment variable PGDATA of the container to /var/lib/postgresql/data/pgdata to avoid the initialization routines on the postgres container trying to change the permissions on the mountpoint, this is explained in the image documentation.

Collectives™ on Stack Overflow

Postgres on Azure kubernetes volume permission error

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related