Data Platform/Systems/PostgreSQL/Clusters

Using the CloudnativePG operator, it becomes quite easy to deploy PostgreSQL clusters in Kubernetes.

We have created a custom cloudnative-pg-cluster chart, allowing users to create a cluster from a very simple YAML values file, such as this one.

fullnameOverride: postgres-airflow-analytics

By default, each cluster will upload base backups and WALs to a dedicated S3 bucket, and performs a daily backup at midnight UTC.

Observability

Dashboard

All cloudnative-pg clusters are instrumented using Prometheus and can be observed via this Grafana dashboard.

Monitoring

https://gerrit.wikimedia.org/r/plugins/gitiles/operations/alerts/+/refs/heads/master/team-data-platform/cloudnative-pg.yaml

Logs

https://logstash.wikimedia.org/app/dashboards#/view/1a9f3aa0-8095-11ef-a846-9f500bbb7f51?_g=(filters%3A!()%2CrefreshInterval%3A(pause%3A!t%2Cvalue%3A0)%2Ctime%3A(from%3Anow-15m%2Cto%3Anow))

Operations and Troubleshooting

Please see: Data Platform/Systems/PostgreSQL/Operations

Backup and Restore

Please see: Data Platform/Systems/PostgreSQL/Backup_and_Restore