db/pg_probackup
db/pg_probackup
1
0
Fork 0
mirror of https://github.com/postgrespro/pg_probackup.git synced 2025-01-05 13:20:31 +02:00
Code Issues Releases Activity

Use asciidoc and xmlto to automatic generation of man and html docs

Browse Source 
This has the merit to put all the documentation of the project into a
single banner, and to centralize all the project in a single place at
code level.

Compiling documentation can be made by setting the variables ASCIIDOC
and XMLTO. As PostgreSQL extension system is not that smart for doc
generation, some custom Makefile path is used to install man pages into
a folder that could directly by used in MANPATH.
This commit is contained in:
Michael Paquier 2014-01-12 03:03:30 +09:00
parent 1c3512fadf
commit 6639785305
5 changed files with 288 additions and 207 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

36
Makefile
View File

@ -18,11 +18,19 @@ SRCS = \
pgut/pgut.c \
pgut/pgut-port.c
OBJS = $(SRCS:.c=.o)
# pg_crc.c and are copied from PostgreSQL source tree.
DOCS = pg_rman.txt
DOCS = doc/pg_rman.txt
# asciidoc and xmlto are present, so install the html documentation and man
# pages as well. html is part of the vanilla documentation. Man pages need a
# special handling at installation.
ifneq ($(ASCIIDOC),)
ifneq ($(XMLTO),)
man_DOCS = doc/pg_rman.1
DOCS += doc/pg_rman.html doc/README.html
endif # XMLTO
endif # ASCIIDOC
# XXX for debug, add -g and disable optimization
PG_CPPFLAGS = -I$(libpq_srcdir)
PG_LIBS = $(libpq_pgport)
@ -44,3 +52,25 @@ LIBS := $(filter-out -lxml2, $(LIBS))
LIBS := $(filter-out -lxslt, $(LIBS))
$(OBJS): pg_rman.h
# Part related to documentation
# Compile documentation as well is ASCIIDOC and XMLTO are defined
ifneq ($(ASCIIDOC),)
ifneq ($(XMLTO),)
all: docs
docs:
$(MAKE) -C doc/
# Special handling for man pages, they need to be in a dedicated folder
install: install-man
install-man:
$(MKDIR_P) '$(DESTDIR)$(docdir)/$(docmoduledir)/man1/'
$(INSTALL_DATA) $(man_DOCS) '$(DESTDIR)$(docdir)/$(docmoduledir)/man1/'
endif # XMLTO
endif # ASCIIDOC
# Clean up documentation as well
clean: clean-docs
clean-docs:
$(MAKE) -C doc/ clean

22
README
View File

@ -23,13 +23,16 @@ Compiling pg_rman requires a PostgreSQL installation and can be done in
two ways:
1. Put pg_rman project directory inside PostgreSQL source tree as
contrib/pg_rman, and use "make" to compile
contrib/pg_rman, and use this command for compilation:
or
$ cd $POSTGRES_SOURCE/contrib/pg_rman
$ make
2. Keep the project directory as-is and use the PGXS development
infrastructure provided by a PostgreSQL installation to perform the
compilation: "make USE_PGXS=1"
compilation:
$ make USE_PGXS=1
In addition, you must have pg_config in $PATH.
@ -44,7 +47,18 @@ pg_rman has been tested on Linux and Unix-based platforms.
Documentation
-------------
All the documentation is written and maintained in pg_rman.txt.
All the documentation is maintained in doc/ as text file, that is then
fetched by asciidoc to generate automatically man pages and html
documentation.
In order to generate the documentation, the variables XMLTO and ASCIIDOC
need to be set to proper values indicating where are located the binaries
of respectively xmlto and asciidoc. An example of build is as follows:
$ make USE_PGXS=1 ASCIIDOC=asciidoc XMLTO=xmlto
They could as well be set as environment variables for development
purposes.
Regression tests
----------------

4
doc/.gitignore vendored Normal file
View File

@ -0,0 +1,4 @@
# Documentation entries
/*.html
/*.xml
/pg_rman.1

33
doc/Makefile Normal file
View File

@ -0,0 +1,33 @@
manpages = pg_rman.1
EXTRA_DIST = pg_rman.txt Makefile $(manpages)
htmls = pg_rman.html README.html
# We have asciidoc and xmlto, so build everything and define correct
# rules for build.
ifneq ($(ASCIIDOC),)
ifneq ($(XMLTO),)
dist_man_MANS = $(manpages)
doc_DATA = $(htmls)
pg_rman.1: pg_rman.xml $(doc_DATA)
$(XMLTO) man $<
%.xml: %.txt
$(ASCIIDOC) -b docbook -d manpage -o $@ $<
%.html: %.txt
$(ASCIIDOC) -a toc -o $@ $<
README.html: ../README
$(ASCIIDOC) -a toc -o $@ $<
endif # XMLTO
endif # ASCIIDOC
clean:
rm -rf $(manpages) *.html *.xml
.PHONY: clean

400
pg_rman.txt → doc/pg_rman.txt
View File

@ -1,20 +1,10 @@
pg_rman, Documentation
======================
= pg_rman(1) =
pg_rman is a backup and recovery manager for PostgreSQL.
== NAME ==
It proposes the features below:
- Backup while database runs including tablespaces with just one
command
- Recovery from backup with just one command, with customized targets
to facilitate the use of PITR.
- Support for full, incremental and archive backup
- Compression of backup files
- Management of backups with integrated catalog
pg_rman - Backup and recovery manager for PostgreSQL
========
Synopsis
========
== SYNOPSIS ==
pg_rman [ OPTIONS ]
{ init |
@ -28,41 +18,47 @@ DATE is the start time of the target backup in ISO-format:
(YYYY-MM-DD HH:MI:SS). Prefix match is used to compare DATE and backup
files.
========
Commands
========
pg_rman supports the following commands. See also Options for details of
OPTIONS.
init
Initialize a backup catalog.
backup
Take an online backup.
restore
Perform restore.
show
Show backup history.
The timeline option shows timeline of the backup and the parent's
timeline for each backup.
validate
Validate backup files.
delete
Delete backup files.
===========
Description
===========
== DESCRIPTION ==
pg_rman is a utility program to backup and restore PostgreSQL database.
It takes a physical online backup of whole database cluster, archive
WALs, and server logs.
- Initialize a backup catalog
It proposes the following features:
- Backup while database runs including tablespaces with just one
command
- Recovery from backup with just one command, with customized targets
to facilitate the use of PITR.
- Support for full, incremental and archive backup
- Compression of backup files
- Management of backups with integrated catalog
== COMMANDS ==
pg_rman supports the following commands. See also Options for details of
OPTIONS.
init::
Initialize a backup catalog.
backup::
Take an online backup.
restore::
Perform restore.
show::
Show backup history. The timeline option shows timeline of the backup
and the parent's timeline for each backup.
validate::
Validate backup files.
delete::
Delete backup files.
=== INITIALIZATION ===
First, you need to create "a backup catalog" to store backup files and
their metadata. It is recommended to setup archive_mode and archive_command
in postgresql.conf before initializing the backup catalog. If the variables
@ -70,24 +66,24 @@ are initialized, pg_rman can adjust the config file to the setting. In this
case, you have to specify the database cluster path for PostgreSQL. Please
specify it in PGDATA environmental variable or -D/--pgdata option.
$ pg_rman init -B <a backup catalog path>
$ pg_rman init -B <a backup catalog path>
- Backup
Backup target can be one of the following types. Also serverlogs can be added.
=== BACKUP ===
Full backup
Backup a whole database cluster.
Backup target can be one of the following types also serverlogs can be added):
Incremental backup
Backup only files or pages modified after the last verified backup.
- Full backup, backup a whole database cluster.
Archive WAL backup
Backup only archive WAL files.
- Incremental backup, backup only files or pages modified after the last
verified backup.
- Archive WAL backup, Backup only archive WAL files.
It is recommended to verify backup files as soon as possible after backup.
Unverified backup cannot be used in restore and in incremental backup.
- Restore
=== RESTORE ===
PostgreSQL server should be stopped before performing a restore. If database
cluster still exists, restore command will save unarchived transaction log
and delete all database files. You can retry recovery until a new backup is
@ -103,45 +99,48 @@ TimeLineID in control file ($PGDATA/global/pg_control) will be the restore
target. If pg_control is not present, TimeLineID in the full backup used by
the restore will be a restore target.
========
Examples
========
== Examples ==
To reduce the number of command line arguments, you can set BACKUP_PATH,
an environment variable, to the absolute path of the backup catalog and
write default configuration into ${BACKUP_PATH}/pg_rman.ini.
$ cat $BACKUP_PATH/pg_rman.ini
ARCLOG_PATH = /home/postgres/arclog
SRVLOG_PATH = /home/postgres/pgdata/pg_log
BACKUP_MODE = F
COMPRESS_DATA = YES
KEEP_ARCLOG_FILES = 10
KEEP_ARCLOG_DAYS = 10
KEEP_DATA_GENERATIONS = 3
KEEP_DATA_DAYS = 120
KEEP_SRVLOG_FILES = 10
KEEP_SRVLOG_DAYS = 10
$ cat $BACKUP_PATH/pg_rman.ini
ARCLOG_PATH = /home/postgres/arclog
SRVLOG_PATH = /home/postgres/pgdata/pg_log
BACKUP_MODE = F
COMPRESS_DATA = YES
KEEP_ARCLOG_FILES = 10
KEEP_ARCLOG_DAYS = 10
KEEP_DATA_GENERATIONS = 3
KEEP_DATA_DAYS = 120
KEEP_SRVLOG_FILES = 10
KEEP_SRVLOG_DAYS = 10
=== TAKE A BACKUP ===
- Take a backup.
This example takes a full backup of the whole database with server logs.
Then, it validates all unvalidated backups.
$ pg_rman backup --backup-mode=full --with-serverlog
$ pg_rman validate
$ pg_rman backup --backup-mode=full --with-serverlog
$ pg_rman validate
- Restore from a backup.
$ pg_ctl stop -m immediate
$ pg_rman restore
$ pg_ctl start
=== RESTORE FROM A BACKUP ===
- Show a backup
$ pg_rman show
==================================================================================================
Start Mode Current TLI Parent TLI Time Data WAL Log Backup Status
==================================================================================================
2013-12-25 03:02:31 INCR 1 0 0m 203kB 67MB ---- 67MB DONE
2013-12-25 03:02:31 INCR 1 0 0m 0B ---- ---- 0B ERROR
2013-12-25 03:02:25 FULL 1 0 0m 33MB 33MB ---- 64MB OK
Here are some commands to restore from a backup:
$ pg_ctl stop -m immediate
$ pg_rman restore
$ pg_ctl start
=== SHOW A BACKUP ===
$ pg_rman show
==================================================================================================
Start Mode Current TLI Parent TLI Time Data WAL Log Backup Status
==================================================================================================
2013-12-25 03:02:31 INCR 1 0 0m 203kB 67MB ---- 67MB DONE
2013-12-25 03:02:31 INCR 1 0 0m 0B ---- ---- 0B ERROR
2013-12-25 03:02:25 FULL 1 0 0m 33MB 33MB ---- 64MB OK
The fields are:
- Start : start time of backup
@ -166,27 +165,27 @@ The fields are:
-- ERROR : backup is unavailable because some errors occur during backup.
-- CORRUPT : backup is unavailable because it is broken.
When a date is specified, more details about a backup is retrived:
$ pg_rman show '2011-11-27 19:15:45'
# configuration
BACKUP_MODE=FULL
WITH_SERVERLOG=false
COMPRESS_DATA=false
# result
TIMELINEID=1
START_LSN=0/08000020
STOP_LSN=0/080000a0
START_TIME='2011-11-27 19:15:45'
END_TIME='2011-11-27 19:19:02'
RECOVERY_XID=1759
RECOVERY_TIME='2011-11-27 19:15:53'
TOTAL_DATA_BYTES=1242420184
READ_DATA_BYTES=25420184
READ_ARCLOG_BYTES=32218912
WRITE_BYTES=242919520
BLOCK_SIZE=8192
XLOG_BLOCK_SIZE=8192
STATUS=OK
When a date is specified, more details about a backup is retrieved:
$ pg_rman show '2011-11-27 19:15:45'
# configuration
BACKUP_MODE=FULL
WITH_SERVERLOG=false
COMPRESS_DATA=false
# result
TIMELINEID=1
START_LSN=0/08000020
STOP_LSN=0/080000a0
START_TIME='2011-11-27 19:15:45'
END_TIME='2011-11-27 19:19:02'
RECOVERY_XID=1759
RECOVERY_TIME='2011-11-27 19:15:53'
DATA_BYTES=25420184
ARCLOG_BYTES=32218912
BACKUP_BYTES=242919520
BLOCK_SIZE=8192
XLOG_BLOCK_SIZE=8192
STATUS=OK
You can check the "RECOVERY_XID" and "RECOVERY_TIME" which are used for
restore option "--recovery-target-xid", "--recovery-target-time".
@ -194,45 +193,42 @@ restore option "--recovery-target-xid", "--recovery-target-time".
The delete command deletes backup files not required by recovery after
the specified date.
=======
Options
=======
== OPTIONS ==
pg_rman accepts the following command line parameters. Some of them can
be also sepcified as environment variables. See also Parameters for the
details.
- Common options
=== COMMON OPTIONS ===
As a general rule, paths for data location need to be specified as
absolute paths; relative paths are not allowed.
-D PATH / --pgdata=PATH
-D PATH / --pgdata=PATH::
The absolute path of database cluster. Required on backup and
restore.
-A PATH / --arclog-path=PATH
-A PATH / --arclog-path=PATH::
The absolute path of archive WAL directory. Required on backup
and restore.
-S PATH / --srvlog-path=PATH
-S PATH / --srvlog-path=PATH::
The absolute path of server log directory. Required on backup with
server logs and restore.
-B PATH / --backup-path=PATH
-B PATH / --backup-path=PATH::
The absolute path of backup catalog. This option is mandatory.
-c / --check
-c / --check::
If specifed, pg_rman doesn't perform actual jobs but only checks
parameters and required resources. The option is typically used with
--verbose option to verify the operation.
-v / --verbose
-v / --verbose::
If specified, pg_rman works in verbose mode.
- Backup options
=== BACKUP OPTIONS ===
-b { full | incremental | archive } /
--backup-mode={ full | incremental | archive }
-b { full | incremental | archive } --backup-mode={ full | incremental | archive }::
Specify backup target files. Available options are: "full" backup,
"incremental" backup, and "archive" backup. Abbreviated forms
(prefix match) are also available. For example, -b f means "full"
@ -242,30 +238,30 @@ absolute paths; relative paths are not allowed.
incremental : Incremental backup and archive backup
archive : Only archive backup
-s / --with-serverlog
-s / --with-serverlog::
Backup server log files if specified.
-Z / --compress-data
-Z / --compress-data::
Compress backup files with zlib if specified.
-C / --smooth-checkpoint
-C / --smooth-checkpoint::
Checkpoint is performed on every backups. If the option is specified,
do smooth checkpoint then. See also the second argument for
pg_start_backup().
--keep-data-generations / --keep-data-days
--keep-data-generations / --keep-data-days::
Specify how long backuped data files will be kept.
--keep-data-generations means number of backup generations.
--keep-data-days means days to be kept.
Only files exceeded one of those settings are deleted.
--keep-arclog-files / --keep-arclog-days
--keep-arclog-files / --keep-arclog-days::
Specify how long backuped archive WAL files will be kept.
--keep-arclog-files means number of backup files.
--keep-arclog-days means days to be kept. When backup is run, only
files exceeded one of those settings are deleted from archive storage.
--keep-srvlog-files / --keep-srvlog-days
--keep-srvlog-files / --keep-srvlog-days::
Specify how long backuped serverlog files will be kept.
--keep-srvlog-files means number of backup files.
--keep-srvlog-days means days to be kept.
@ -274,61 +270,62 @@ absolute paths; relative paths are not allowed.
when you specify --with-serverlog and --srvlog-path options in backup
command.
- Restore options
=== RESTORE OPTIONS ===
The parameters whose name start are started with --recovery refer to
the same parameters as the ones in recovery.confin recovery.conf.
--recovery-target-timeline TIMELINE
--recovery-target-timeline TIMELINE::
Specifies recovering into a particular timeline.
If not specified, the current timeline is used.
--recovery-target-time TIMESTAMP
--recovery-target-time TIMESTAMP::
This parameter specifies the timestamp up to which recovery will
proceed.
--recovery-target-xid XID
--recovery-target-xid XID::
This parameter specifies the transaction ID up to which recovery
will proceed.
--recovery-target-inclusive
--recovery-target-inclusive::
Specifies whether server pauses when recovery target is reached.
--hard-copy
--hard-copy::
Archived WAL files are copied to archive WAL storage area. If you
do not specify --hard-copy, pg_rman makes symlink to archive WAL
where are in the backup catalog directory.
- Catalog options
=== CATALOG OPTIONS ===
-a / --show-all
-a / --show-all::
Show all existing backups, including the deleted ones.
- Connection options
=== CONNECTION OPTIONS ===
Parameters to connect PostgreSQL server.
-d DBNAME / --dbname=DBNAME
-d DBNAME / --dbname=DBNAME::
The database name to execute pg_start_backup() and pg_stop_backup().
-h HOSTNAME / --host=HOSTNAME
-h HOSTNAME / --host=HOSTNAME::
Specifies the host name of the machine on which the server is running.
If the value begins with a slash, it is used as the directory for the
Unix domain socket.
-p PORT / --port=PORT
-p PORT / --port=PORT::
Specifies the TCP port or local Unix domain socket file extension on
which the server is listening for connections.
-U USERNAME / --username=USERNAME
-U USERNAME / --username=USERNAME::
User name to connect as.
-w / --no-password
-w / --no-password::
Never issue a password prompt. If the server requires password
authentication and a password is not available by other means such as
a .pgpass file, the connection attempt will fail. This option can be
useful in batch jobs and scripts where no user is present to enter a
password.
-W / --password
-W / --password::
Force pg_rman to prompt for a password before connecting to a database.
This option is never essential, since pg_rman will automatically
prompt for a password if the server demands password authentication.
@ -336,51 +333,49 @@ Parameters to connect PostgreSQL server.
if the server wants a password. In some cases it is worth typing -W
to avoid the extra connection attempt.
- Generic options
=== GLOBAL OPTIONS ===
--help
--help::
Print help, then exit.
-V / --version
-V / --version::
Print version information, then exit.
-! / --debug
-! / --debug::
Show debug information.
==========
Parameters
==========
== PARAMETERS ==
Some of parameters can be specified as command line arguments, environment
variables or in configuration file as follows:
Short Long Environment Conf file
-h --host PGHOST No
-p --port PGPORT No
-d --dbname PGDATABASE No
-U --username PGUSER No
PGPASSWORD No
-w --password No
-W --no-password No
-D --pgdata PGDATA Yes
-B --backup-path BACKUP_PATH Yes
-A --arclog-path ARCLOG_PATH Yes
-S --srvlog-path SRVLOG_PATH Yes
-b --backup-mode BACKUP_MODE Yes
-s --with-serverlog WITH_SERVERLOG Yes
-Z --compress-data COMPRESS_DATA Yes
-C --smooth-checkpoint SMOOTH_CHECKPOINT Yes
--keep-data-generations KEEP_DATA_GENERATIONS Yes
--keep-data-days KEEP_DATA_DAYS Yes
--keep-srvlog-files KEEP_SRVLOG_FILES Yes
--keep-srvlog-days KEEP_SRVLOG_DAYS Yes
--keep-arclog-files KEEP_ARCLOG_FILES Yes
--keep-arclog-days KEEP_ARCLOG_DAYS Yes
--recovery-target-timeline RECOVERY_TARGET_TIMELINE Yes
--recovery-target-xid RECOVERY_TARGET_XID Yes
--recovery-target-time RECOVERY_TARGET_TIME Yes
--recovery-target-inclusive RECOVERY_TARGET_INCLUSIVE Yes
--hard-copy HARD_COPY Yes
Short Long Env File
-h --host PGHOST No
-p --port PGPORT No
-d --dbname PGDATABASE No
-U --username PGUSER No
PGPASSWORD No
-w --password No
-W --no-password No
-D --pgdata PGDATA Yes
-B --backup-path BACKUP_PATH Yes
-A --arclog-path ARCLOG_PATH Yes
-S --srvlog-path SRVLOG_PATH Yes
-b --backup-mode BACKUP_MODE Yes
-s --with-serverlog WITH_SERVERLOG Yes
-Z --compress-data COMPRESS_DATA Yes
-C --smooth-checkpoint SMOOTH_CHECKPOINT Yes
--keep-data-generations KEEP_DATA_GENERATIONS Yes
--keep-data-days KEEP_DATA_DAYS Yes
--keep-srvlog-files KEEP_SRVLOG_FILES Yes
--keep-srvlog-days KEEP_SRVLOG_DAYS Yes
--keep-arclog-files KEEP_ARCLOG_FILES Yes
--keep-arclog-days KEEP_ARCLOG_DAYS Yes
--recovery-target-timeline RECOVERY_TARGET_TIMELINE Yes
--recovery-target-xid RECOVERY_TARGET_XID Yes
--recovery-target-time RECOVERY_TARGET_TIME Yes
--recovery-target-inclusive RECOVERY_TARGET_INCLUSIVE Yes
--hard-copy HARD_COPY Yes
Variable names in configuration file are the same as long names or names
of environment variables. The password can not be specified in command
@ -389,9 +384,8 @@ line and configuration file for security reason.
This utility, like most other PostgreSQL utilities, also uses the
environment variables supported by libpq (see Environment Variables)
============
Restrictions
============
== RESTRICTIONS ==
pg_rman has the following restrictions.
- Requires to read database cluster directory and write backup catalog
@ -405,40 +399,46 @@ pg_rman has the following restrictions.
- Incremental backup is not able to take necessary files after a database
creation, so take a full backup once a new database is created.
=======
Details
=======
- Recovery to Point-in-Time
== DETAILS ==
=== RECOVERY TO POINT-IN-TIME ===
pg_rman can recover to point-in-time if timeline, transaction ID, or
timestamp is specified in recovery.conf. xlogdump is a contrib module of
PostgreSQL core that allows checking in the content of WAL files and
determine when to recover. This might help.
- Configuration file
=== CONFIGURATION FILE ===
Setting parameters in configuration file is done as "name=value". Quotes
are required if the value contains whitespaces. Comments should start with
"#" and are automatically ignored. Whitespaces and tabs are ignored
excluding values.
- Exit codes
=== EXIT CODE ===
pg_rman returns exit codes for each error status.
Code Name Description
0 SUCCESS Operation succeeded.
1 HELP Print help, then exit
2 ERROR Generic error
3 FATAL Exit because of repeated errors
4 PANIC Unknown critical condition
10 ERROR_SYSTEM I/O or system error
11 ERROR_NOMEM Out of memory
12 ERROR_ARGS Invalid input parameters
13 ERROR_INTERRUPTED Interrupted by user (Ctrl+C etc.)
14 ERROR_PG_COMMAND SQL error
15 ERROR_PG_CONNECT Cannot connect to PostgreSQL server
20 ERROR_ARCHIVE_FAILED Cannot archive WAL files
21 ERROR_NO_BACKUP Backup file not found
22 ERROR_CORRUPTED Backup file is broken
23 ERROR_ALREADY_RUNNING Cannot start because another pg_rman is running
24 ERROR_PG_INCOMPATIBLE Version conflicted with server
25 ERROR_PG_RUNNING Error due to server running
26 ERROR_PID_BROKEN postmaster.pid is broken
Code Name Description
0 SUCCESS Operation succeeded.
1 HELP Print help, then exit
2 ERROR Generic error
3 FATAL Exit because of repeated errors
4 PANIC Unknown critical condition
10 ERROR_SYSTEM I/O or system error
11 ERROR_NOMEM Out of memory
12 ERROR_ARGS Invalid input parameters
13 ERROR_INTERRUPTED Interrupted by user (Ctrl+C etc.)
14 ERROR_PG_COMMAND SQL error
15 ERROR_PG_CONNECT Cannot connect to PostgreSQL server
20 ERROR_ARCHIVE_FAILED Cannot archive WAL files
21 ERROR_NO_BACKUP Backup file not found
22 ERROR_CORRUPTED Backup file is broken
23 ERROR_ALREADY_RUNNING Cannot start because another pg_rman
is running
24 ERROR_PG_INCOMPATIBLE Version conflicted with server
25 ERROR_PG_RUNNING Error due to server running
26 ERROR_PID_BROKEN postmaster.pid is broken
== AUTHOR ==
pg_rman was originally written by NTT, and maintained in some way by Michael
Paquier.
Please report bug reports at <https://github.com/michaelpq/pg_rman>.