How to recover from PANIC: could not locate a valid checkpoint record.
Status: Downloaded newer image for postgres:13.5
PostgreSQL Database directory appears to contain a database; Skipping initialization
2024-04-16 18:34:27.383 UTC [1] LOG: starting PostgreSQL 13.5 (Debian 13.5-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2024-04-16 18:34:27.384 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
2024-04-16 18:34:27.384 UTC [1] LOG: listening on IPv6 address "::", port 5432
2024-04-16 18:34:27.389 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2024-04-16 18:34:27.410 UTC [26] LOG: database system was shut down at 2024-04-16 11:38:03 UTC
2024-04-16 18:34:27.532 UTC [26] LOG: invalid resource manager ID 72 at 2/72A46D68
2024-04-16 18:34:27.532 UTC [26] LOG: invalid primary checkpoint record
2024-04-16 18:34:27.532 UTC [26] PANIC: could not locate a valid checkpoint record
2024-04-16 18:34:27.641 UTC [1] LOG: startup process (PID 26) was terminated by signal 6: Aborted
2024-04-16 18:34:27.641 UTC [1] LOG: aborting startup due to startup process failure
2024-04-16 18:34:27.644 UTC [1] LOG: database system is shut downMake a copy of the corrupted data
Stop the running (& failing) container (or scale the service to 0).
Make a backup before trying to recover from the corruption (replace your paths; -rp will do a recursive copy preserving permissions, timestamps…):
sudo cp -rp <original_postgres_data> <backup_postgres_data>Reset the WAL
Run a bash command under the postgres data volume (replace by your volume name or bind path, also replace the postgres image version):
docker run -it --rm -u 999 -v <postgres_data_volume>:/var/lib/postgresql/data postgres:<version> bashThen go to the data path & reset the WAL:
$ cd /var/lib/postgresql/data
$ pg_resetwal .
Write-ahead log resetAll is done, now you can run your postgres container normally.
No way
What to do if this does not work… recover your backup, because you do regular backups, don’t you?
