-
Notifications
You must be signed in to change notification settings - Fork 9
chore : switch to official postgres #2037
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
45a80e1
3d85663
6edf8c4
3c1d161
327b79c
f67da9e
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
This file was deleted.
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,25 @@ | ||||||||||||||||||||||||||||||||||||||||||||||
| #!/bin/bash | ||||||||||||||||||||||||||||||||||||||||||||||
| set -euo pipefail | ||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||
| echo "Configuring PostgreSQL master for replication..." | ||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||
| # Update postgresql.conf for replication | ||||||||||||||||||||||||||||||||||||||||||||||
| cat >> "$PGDATA/postgresql.conf" <<EOF | ||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||
| # Replication settings | ||||||||||||||||||||||||||||||||||||||||||||||
| wal_level = replica | ||||||||||||||||||||||||||||||||||||||||||||||
| max_wal_senders = 3 | ||||||||||||||||||||||||||||||||||||||||||||||
| max_replication_slots = 3 | ||||||||||||||||||||||||||||||||||||||||||||||
| hot_standby = on | ||||||||||||||||||||||||||||||||||||||||||||||
| hot_standby_feedback = on | ||||||||||||||||||||||||||||||||||||||||||||||
| listen_addresses = '*' | ||||||||||||||||||||||||||||||||||||||||||||||
| EOF | ||||||||||||||||||||||||||||||||||||||||||||||
|
Comment on lines
+7
to
+16
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🛠️ Refactor suggestion Add essential PG settings for modern auth and dev replication. Ensure SCRAM and retain WAL for short outages. cat >> "$PGDATA/postgresql.conf" <<EOF
# Replication settings
wal_level = replica
max_wal_senders = 3
max_replication_slots = 3
hot_standby = on
hot_standby_feedback = on
listen_addresses = '*'
+password_encryption = scram-sha-256
+wal_keep_size = 64MB
EOF📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents |
||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||
| # Update pg_hba.conf to allow replication connections | ||||||||||||||||||||||||||||||||||||||||||||||
| cat >> "$PGDATA/pg_hba.conf" <<EOF | ||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||
| # Replication connections | ||||||||||||||||||||||||||||||||||||||||||||||
| host replication repl_user 0.0.0.0/0 md5 | ||||||||||||||||||||||||||||||||||||||||||||||
| EOF | ||||||||||||||||||||||||||||||||||||||||||||||
|
Comment on lines
+19
to
+23
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Use SCRAM and avoid world-open CIDR for replication access. md5 will fail if users are stored with SCRAM (default in newer Postgres). Also, 0.0.0.0/0 is unnecessarily broad; restrict to your Docker network or make it configurable. cat >> "$PGDATA/pg_hba.conf" <<EOF
# Replication connections
-host replication repl_user 0.0.0.0/0 md5
+# Prefer a narrowed CIDR (e.g., 172.18.0.0/16) or pass REPLICA_CIDR via env.
+host replication repl_user ${REPLICA_CIDR:-172.18.0.0/16} scram-sha-256
EOF📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents |
||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||
| echo "Master configuration complete" | ||||||||||||||||||||||||||||||||||||||||||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,62 @@ | ||
| name: read-replica-postgresql | ||
| services: | ||
| postgresql-master: | ||
| image: 'postgres:17' | ||
| ports: | ||
| - '5432:5432' | ||
| volumes: | ||
| - 'postgresql_master_data:/var/lib/postgresql/data' | ||
| - './init-master.sql:/docker-entrypoint-initdb.d/01-init-master.sql:ro' | ||
| - './configure-master.sh:/docker-entrypoint-initdb.d/02-configure-master.sh:ro' | ||
| environment: | ||
| - POSTGRES_USER=postgres_write | ||
| - POSTGRES_PASSWORD=postgres_write | ||
| - POSTGRES_DB=my_database | ||
| healthcheck: | ||
| test: ["CMD-SHELL", "pg_isready -U postgres_write -d my_database -h 127.0.0.1 -p 5432"] | ||
| interval: 5s | ||
| timeout: 3s | ||
| retries: 20 | ||
| start_period: 10s | ||
| postgresql-slave: | ||
| image: 'postgres:17' | ||
| ports: | ||
| - '15432:5432' | ||
| depends_on: | ||
| postgresql-master: | ||
| condition: service_healthy | ||
| volumes: | ||
| - 'postgresql_slave_data:/var/lib/postgresql/data' | ||
| - './setup-slave.sh:/setup-slave.sh:ro' | ||
| environment: | ||
| - POSTGRES_USER=postgres_write | ||
| - POSTGRES_PASSWORD=postgres_write | ||
| - PGUSER=postgres_write | ||
| - POSTGRES_MASTER_HOST=postgresql-master | ||
| - POSTGRES_MASTER_PORT=5432 | ||
| - REPLICATION_USER=repl_user | ||
| - REPLICATION_PASSWORD=repl_password | ||
| entrypoint: ["/setup-slave.sh"] | ||
| healthcheck: | ||
| test: ["CMD-SHELL", "pg_isready -U postgres_write -h 127.0.0.1 -p 5432"] | ||
| interval: 5s | ||
| timeout: 3s | ||
| retries: 20 | ||
| start_period: 15s | ||
|
|
||
| pgadmin4: | ||
| image: dpage/pgadmin4 | ||
| ports: | ||
| - "5050:80" | ||
| depends_on: | ||
| postgresql-master: | ||
| condition: service_healthy | ||
| environment: | ||
| - [email protected] | ||
| - PGADMIN_DEFAULT_PASSWORD=admin | ||
|
|
||
| volumes: | ||
| postgresql_master_data: | ||
| driver: local | ||
| postgresql_slave_data: | ||
| driver: local |
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,18 @@ | ||||||||||||||||||||||||||
| -- Create replication user | ||||||||||||||||||||||||||
| CREATE USER repl_user WITH REPLICATION ENCRYPTED PASSWORD 'repl_password'; | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
|
Comment on lines
+1
to
+3
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🛠️ Refactor suggestion Create replication user with SCRAM; avoid deprecated ENCRYPTED keyword. Rely on server password_encryption=scram-sha-256 and use PASSWORD. --- Create replication user
-CREATE USER repl_user WITH REPLICATION ENCRYPTED PASSWORD 'repl_password';
+CREATE USER repl_user WITH REPLICATION PASSWORD 'repl_password';📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents |
||||||||||||||||||||||||||
| -- Grant necessary permissions | ||||||||||||||||||||||||||
| GRANT CONNECT ON DATABASE my_database TO repl_user; | ||||||||||||||||||||||||||
| GRANT USAGE ON SCHEMA public TO repl_user; | ||||||||||||||||||||||||||
| GRANT SELECT ON ALL TABLES IN SCHEMA public TO repl_user; | ||||||||||||||||||||||||||
| GRANT SELECT ON ALL SEQUENCES IN SCHEMA public TO repl_user; | ||||||||||||||||||||||||||
|
Comment on lines
+4
to
+8
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Separate app read-only user from replication user. Don’t use the replication role for application queries. Create a distinct least-privileged user. -- Grant necessary permissions
-GRANT CONNECT ON DATABASE my_database TO repl_user;
-GRANT USAGE ON SCHEMA public TO repl_user;
-GRANT SELECT ON ALL TABLES IN SCHEMA public TO repl_user;
-GRANT SELECT ON ALL SEQUENCES IN SCHEMA public TO repl_user;
+CREATE USER app_readonly PASSWORD 'readonly_password';
+GRANT CONNECT ON DATABASE my_database TO app_readonly;
+GRANT USAGE ON SCHEMA public TO app_readonly;
+GRANT SELECT ON ALL TABLES IN SCHEMA public TO app_readonly;
+GRANT SELECT ON ALL SEQUENCES IN SCHEMA public TO app_readonly;
+ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO app_readonly;Follow-up: point spring.replica.datasource.username to app_readonly in application.yml. 📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents |
||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| -- Update pg_hba.conf to allow replication connections | ||||||||||||||||||||||||||
| -- This will be added to the end of pg_hba.conf by PostgreSQL init process | ||||||||||||||||||||||||||
| DO $$ | ||||||||||||||||||||||||||
| BEGIN | ||||||||||||||||||||||||||
| -- Add replication entry to pg_hba.conf | ||||||||||||||||||||||||||
| -- Note: The official postgres image handles pg_hba.conf configuration | ||||||||||||||||||||||||||
| -- The replication connection will be allowed via the default settings | ||||||||||||||||||||||||||
| RAISE NOTICE 'Replication user created successfully'; | ||||||||||||||||||||||||||
| END $$; | ||||||||||||||||||||||||||
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,49 @@ | ||||||||||||||||||||||||||||||||||||
| #!/bin/bash | ||||||||||||||||||||||||||||||||||||
| set -Eeuo pipefail | ||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||
| export PGDATA=/var/lib/postgresql/data | ||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||
| echo "Setting up PostgreSQL slave..." | ||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||
| # Wait for master to be ready | ||||||||||||||||||||||||||||||||||||
| echo "Waiting for master to be ready..." | ||||||||||||||||||||||||||||||||||||
| until PGPASSWORD="$POSTGRES_PASSWORD" psql -h "$POSTGRES_MASTER_HOST" -p "$POSTGRES_MASTER_PORT" -U "$POSTGRES_USER" -d "my_database" -c '\l' > /dev/null 2>&1 | ||||||||||||||||||||||||||||||||||||
| do | ||||||||||||||||||||||||||||||||||||
| echo "Waiting for master database..." | ||||||||||||||||||||||||||||||||||||
| sleep 3 | ||||||||||||||||||||||||||||||||||||
| done | ||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||
| echo "Master is ready. Checking if slave data exists..." | ||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||
| if [ "$(ls -A $PGDATA 2>/dev/null)" ]; then | ||||||||||||||||||||||||||||||||||||
| echo "Data directory exists, starting PostgreSQL..." | ||||||||||||||||||||||||||||||||||||
| # Change ownership and start as postgres user | ||||||||||||||||||||||||||||||||||||
| chown -R postgres:postgres "$PGDATA" | ||||||||||||||||||||||||||||||||||||
| exec gosu postgres postgres -c hot_standby=on -c hot_standby_feedback=on | ||||||||||||||||||||||||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🛠️ Refactor suggestion Pass data dir explicitly to postgres. Be explicit with - exec gosu postgres postgres -c hot_standby=on -c hot_standby_feedback=on
+ exec gosu postgres postgres -D "$PGDATA" -c hot_standby=on -c hot_standby_feedback=on📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents |
||||||||||||||||||||||||||||||||||||
| else | ||||||||||||||||||||||||||||||||||||
|
Comment on lines
+18
to
+23
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🛠️ Refactor suggestion Make the data-dir check robust and quote variables.
-if [ "$(ls -A $PGDATA 2>/dev/null)" ]; then
+if [ -s "$PGDATA/PG_VERSION" ]; then📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents |
||||||||||||||||||||||||||||||||||||
| echo "Data directory is empty, creating base backup..." | ||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||
| # Create base backup from master | ||||||||||||||||||||||||||||||||||||
| PGPASSWORD="$REPLICATION_PASSWORD" pg_basebackup \ | ||||||||||||||||||||||||||||||||||||
| -h "$POSTGRES_MASTER_HOST" \ | ||||||||||||||||||||||||||||||||||||
| -p "$POSTGRES_MASTER_PORT" \ | ||||||||||||||||||||||||||||||||||||
| -U "$REPLICATION_USER" \ | ||||||||||||||||||||||||||||||||||||
| -D "$PGDATA" \ | ||||||||||||||||||||||||||||||||||||
| -W \ | ||||||||||||||||||||||||||||||||||||
| -v \ | ||||||||||||||||||||||||||||||||||||
| -R \ | ||||||||||||||||||||||||||||||||||||
| -X stream | ||||||||||||||||||||||||||||||||||||
|
Comment on lines
+27
to
+35
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Remove -W; it forces an interactive password prompt and can hang CI.
- -W \
-v \
-R \
-X stream📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents |
||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||
| # Set proper permissions | ||||||||||||||||||||||||||||||||||||
| chmod 700 "$PGDATA" | ||||||||||||||||||||||||||||||||||||
| chown -R postgres:postgres "$PGDATA" | ||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||
| # Add standby configuration | ||||||||||||||||||||||||||||||||||||
| cat >> "$PGDATA/postgresql.conf" <<EOF | ||||||||||||||||||||||||||||||||||||
| hot_standby = on | ||||||||||||||||||||||||||||||||||||
| hot_standby_feedback = on | ||||||||||||||||||||||||||||||||||||
| EOF | ||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||
| echo "Base backup complete. Starting PostgreSQL in standby mode..." | ||||||||||||||||||||||||||||||||||||
| exec gosu postgres postgres | ||||||||||||||||||||||||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🛠️ Refactor suggestion Same: start with -D. - exec gosu postgres postgres
+ exec gosu postgres postgres -D "$PGDATA"📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents |
||||||||||||||||||||||||||||||||||||
| fi | ||||||||||||||||||||||||||||||||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,12 +1,15 @@ | ||
| package com.example.demo.readreplica; | ||
|
|
||
| import java.util.TimeZone; | ||
| import org.springframework.boot.SpringApplication; | ||
| import org.springframework.boot.autoconfigure.SpringBootApplication; | ||
|
|
||
| @SpringBootApplication | ||
| public class ReadReplicaApplication { | ||
|
|
||
| public static void main(String[] args) { | ||
| // Set default timezone to UTC to avoid timezone conflicts | ||
| TimeZone.setDefault(TimeZone.getTimeZone("UTC")); | ||
| SpringApplication.run(ReadReplicaApplication.class, args); | ||
| } | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🛠️ Refactor suggestion
Use healthchecks and
--waitto deflake CI; drop arbitrary sleep.Rely on Compose health status rather than
sleep 5.Follow-up: add healthchecks in the compose file (see next comment).
📝 Committable suggestion
🤖 Prompt for AI Agents