1
0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2024-12-14 10:13:05 +02:00
Commit Graph

102 Commits

Author SHA1 Message Date
David Steele
5d25e508ae Add io-timeout option.
Timeout used for connections and read/write operations.

Note that the entire read/write operation does not need to complete within this timeout but some progress must be made, even if it is only a single byte.
2020-04-17 09:18:52 -04:00
David Steele
789e364e6b Rename tcp-keep-alive option to sck-keep-alive.
This is really a socket option so the new name is clearer.

Since common/io/socket/tcp will contains a mix of options it makes sense to rename it to socket and cascade name changes as needed.
2020-04-01 15:44:51 -04:00
David Steele
5c6fb88bef TCP keep-alive options are configurable.
Prior to 2.25 the individual TCP keep-alive options were not being configured due to a missing header. In 2.25 they were being configured incorrectly due to a disconnect between the timeout specified in ms and what was expected by the TCP options, i.e. seconds.

Instead make the TCP keep-alive options directly configurable, with correct units and better testing. Keep-alive is enabled by default (though it can be defaulted to the system setting instead) and the rest of the options are not set by default. This is in line with what PostgreSQL does, though PostgreSQL does not allow keep-alive to be defaulted.

Also move configuration of TCP options before connect() as PostgreSQL does.
2020-03-31 18:13:11 -04:00
Adrian Vondendriesch
e1c72f6f97 Fix typos. 2020-03-28 17:48:57 -04:00
Cynthia Shang
2fa69af8da Add --dry-run option to the expire command.
Use dry-run to see which backups/archive would be removed by the expire command without actually removing anything.
2020-03-16 13:56:52 -04:00
David Steele
c279a00279 Add lz4 compression support.
LZ4 compresses data faster than gzip but at a lower ratio.  This can be a good tradeoff in certain scenarios.

Note that setting compress-type=lz4 will make new backups and archive incompatible (unrestorable) with prior versions of pgBackRest.
2020-03-10 14:45:27 -04:00
David Steele
d3c83453de Add repo-create, repo-get, repo-put, and repo-rm commands.
These commands are generally useful but more importantly they allow removing LibC by providing the Perl integration tests an alternate way to work with repository storage.

All the commands are currently internal only and should not be used on production repositories.
2020-03-09 17:15:03 -04:00
David Steele
5e1291a29f Rename ls command to repo-ls.
This command only makes sense for the repository storage since other storage (e.g. pg and spool) must be located on a local Posix filesystem and can be listed using standard unix commands.  Since the repo storage can be located lots of places having a common way to list it makes sense.

Prefix with repo- to make the scope of this command clear.

Update documentation to reflect this change.
2020-03-09 16:41:04 -04:00
David Steele
438b957f9c Add infrastructure for multiple compression type support.
Add compress-type option and deprecate compress option. Since the compress option is boolean it won't work with multiple compression types. Add logic to cfgLoadUpdateOption() to update compress-type if it is not set directly. The compress option should no longer be referenced outside the cfgLoadUpdateOption() function.

Add common/compress/helper module to contain interface functions that work with multiple compression types. Code outside this module should no longer call specific compression drivers, though it may be OK to reference a specific compression type using the new interface (e.g., saving backup history files in gz format).

Unit tests only test compression using the gz format because other formats may not be available in all builds. It is the job of integration tests to exercise all compression types.

Additional compression types will be added in future commits.
2020-03-06 14:41:03 -05:00
David Steele
fe263e87b1 Allow path-style URIs in S3 driver.
Although path-style URIs have been deprecated by AWS, they may still be used with products like Minio because no additional DNS configuration is required.

Path-style URIs must be explicitly enabled since it is not clear how they can be auto-detected reliably.  More importantly, faulty detection could cause regressions in current installations.
2020-01-12 11:31:06 -07:00
Mike Palmiotto
dc1e7ca22d Add pg-user option.
Specifies the database user name when connecting to PostgreSQL.

If not specified pgBackRest will connect with the local OS user or PGUSER, which was the previous behavior.
2019-12-19 11:26:38 -05:00
David Steele
5f03471445 Remove --force option from stanza-create documentation.
This should have been removed when the support for the option was removed in c7333190.

The option cannot be removed entirely because we don't want to error in the case where --force was specified but the stanza is valid.
2019-11-19 15:56:13 -05:00
Brad Nicholson
887ec97e07 Document maximum version for auto-stop option. 2019-10-03 06:58:52 -04:00
Cynthia Shang
f96c54c4ba Add info command set option for detailed text output.
The additional details include databases that can be used for selective restore and a list of tablespaces and symlinks with their default destinations.

This information is not included in the JSON output because it requires reading the manifest which is too IO intensive to do for all manifests.  We plan to include this information for JSON in a future release.
2019-09-30 12:39:38 -04:00
David Steele
c41fb575fb Add standby restore type.
This restore type automatically adds standby_mode=on to recovery.conf.

This could be accomplished previously by setting --recovery-option=standby_mode=on but PostgreSQL 12 requires standby mode to be enabled by a special file named standby.signal.

The new restore type allows us to maintain a common interface between PostgreSQL versions.
2019-09-26 17:39:45 -04:00
Cynthia Shang
49b421d372 Add documentation clarifications regarding standby repositories.
Clarify that placing the repository on a standby is not a robust configuration and recommend against it.
2019-09-16 13:56:25 -04:00
David Steele
f57e119b25 Document the relationship between db-timeout and protocol-timeout.
Contributed by Cynthia Shang.
Suggested by James Chanco Jr.
2019-09-13 12:28:39 -04:00
David Steele
15d04ca19c Add recursion and json output to the ls command.
These features finally make the ls command practical.

Currently the JSON contains only name, type, and size.  We may add more fields in the future, but these seem like the minimum needed to be useful.
2019-09-12 16:29:50 -04:00
David Steele
554d98746a Add repo-s3-port option for setting a non-standard S3 service port.
If this option is set then ports appended to repo-s3-endpoint or repo-s3-host will be ignored.

Setting this option explicitly may be the only way to use a bare ipv6 address with S3 (since multiple colons confuse the parser) but we plan to improve this in the future.
2019-07-25 17:36:51 -04:00
David Steele
404284b90f Add internal flag for commands.
Allow commands to be skipped by default in the command help but still work if help is requested for the command directly.  There may be other uses for the flag in the future.

Update help for ls now that it is exposed.
2019-05-28 12:18:05 -04:00
David Steele
20e5b92f36 Add ls command.
Allows listing repo paths/files from the command-line, to be used primarily for testing and debugging.

This command is internal-only so the interface may change at any time without notice.
2019-05-28 10:03:48 -04:00
David Steele
e3fe3434b4 Rename repo-s3-verify-ssl option to repo-s3-verify-tls.
The new name is preferred because pgBackRest does not support any SSL protocol versions (they are all considered to be insecure).

The old name will continue to be accepted.
2019-05-21 10:14:41 -04:00
David Steele
200fd3aa05 Clarify that encryption is always performed client-side.
Suggested by Bruce Burdick.
2019-01-04 12:25:58 +02:00
Cynthia Shang
72865ca33b Add admonitions to documentation renderers.
Admonitions call out places where the user should take special care.

Support added for HTML, PDF, Markdown and help text renderers.  XML files have been updated accordingly.

Contributed by Cynthia Shang.
2018-12-30 16:40:20 +02:00
David Steele
5b52f02021 Clarify when target-action is effective and PostgreSQL version support.
Suggested by Keith Fiske.
2018-12-05 15:58:45 -05:00
Cynthia Shang
880fbb5e57 Add checksum delta for incremental backups.
Use checksums rather than timestamps to determine if files have changed.  This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby.

If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full.  Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well.

Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time.  A future commit will address automatically enabling the feature in cases where it seems likely to be useful.

Contributed by Cynthia Shang.
2018-09-19 11:12:45 -04:00
Cynthia Shang
bf0691576a Allow delta option to be specified in the pgBackRest configuration file.
This option was previously allowed on the command-line only for no particular reason that we could determine.

Being able to specify it in the config file seems like a good idea and won't change current usage.

Contributed by Cynthia Shang.
2018-09-18 13:29:05 -04:00
Cynthia Shang
cabb3f8577 List allowable values for the buffer-size option in the configuration reference.
Contributed by Cynthia Shang.
Suggested by Stéphane Schildknecht.
2018-09-10 13:33:39 -04:00
David Steele
c638490451 Documentation updates for exclude feature based on review.
Reviewed by Cynthia Shang.
2018-08-28 16:49:29 -04:00
David Steele
bef58a7974 Allow arbitrary directories and/or files to be excluded from a backup.
Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 15:51:05 -04:00
David Steele
2276567027 Add log-subprocess option to allow file logging for local and remote subprocesses. 2018-08-22 20:05:49 -04:00
David Steele
d0b9f986a0 Allow any option to be set in an environment variable.
This includes options that previously could only be specified on the command line, e.g. stanza.
2018-08-15 10:52:53 -04:00
David Steele
40924e8f5c Fix invalid log level in log-path option reference.
Reported by Camilo Aguilar.
2018-08-13 07:47:26 -04:00
David Steele
3434240097 Remove incompletely implemented online option from the check command.
Offline operation runs counter to the purpose of this command, which is to check if archiving and backups are working correctly.

Reported by Jason O'Donnell.
2018-08-12 19:24:21 -04:00
David Steele
6084e9be04 Fix default location of pgbackrest.conf in option reference. 2018-06-16 13:07:01 -04:00
Yogesh Sharma
6a40c916d4 Add repo-s3-token option to allow temporary credentials tokens to be configured.
pgBackRest currently has no way to request new credentials so the entire command (e.g. backup, restore) must complete before the credentials expire.

Contributed by Yogesh Sharma.
2018-05-02 14:06:40 -04:00
David Steele
54dd6f3ed4 Add asynchronous, parallel archive-get.
This feature maintains a queue of WAL segments to help reduce latency when PostgreSQL requests a WAL segment with restore_command.
2018-04-30 17:27:39 -04:00
Cynthia Shang
721432f6ae Update the archive-push-queue-max, manifest-save-threshold, and buffer-size options to accept values in KB, MB, GB, TB, or PB where the multiplier is a power of 1024.
Contributed by Cynthia Shang.
2018-04-25 15:46:05 -04:00
Cynthia Shang
27aa5f756e Add support for additional pgBackRest configuration files.
Configuration files are loaded from the directory specified by the --config-include-path option.

Add --config-path option for overriding the default base path of the --config and --config-include-path option.

Contributed by Cynthia Shang.
2018-04-19 11:09:39 -04:00
David Steele
79b7552940 Rename archive-queue-max option to archive-push-queue-max.
This avoids confusion with the archive-get-queue-max option. The old option name will continue to be accepted.
2018-04-18 19:37:35 -04:00
David Steele
e9790fd3b1 Update out-of-date description for the spool-path option. 2018-04-17 17:38:25 -04:00
David Steele
90f980fe91 Remove documentation describing info --output=json as experimental. 2018-03-30 21:33:30 -04:00
David Steele
a50b1eb5be Remove references to the C library being optional. 2018-03-05 10:03:10 -05:00
David Steele
64a7196cb6 Rename retention-* options to repo-retention-*. 2018-02-19 15:07:24 -05:00
Cynthia Shang
f75ba7db94 Update backup host to repository host in the documentation.
Contributed by Cynthia Shang.
2018-02-19 13:17:58 -05:00
David Steele
72cbb9a9d4 Call Perl with built-in interpreter instead of execvp().
Exec'ing Perl worked fine but generated a very large command line in the process list and potentially exposed secrets.
2018-02-14 17:19:54 -05:00
David Steele
7d13087c4b Rename -no to -reset for non-boolean options. 2018-02-06 11:26:06 -05:00
David Steele
be90028100 Rename db-* options to pg-* and backup-* options to repo-* to improve consistency.
* repo-* options are now indexed although only one is allowed.
* List deprecated option names in documentation and command-line help.
2018-02-03 18:27:38 -05:00
David Steele
b4d434b41c Update stanza-create --force documentation to urge caution when using.
Suggested by Jason O'Donnell.
2018-01-31 17:34:27 -05:00
David Steele
b58a45e067 Implement help command in C. 2018-01-23 13:34:24 -05:00