1
0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2024-12-12 10:04:14 +02:00
Commit Graph

36 Commits

Author SHA1 Message Date
David Steele
1add35624d v2.50: Performance Improvements and Bug Fixes
Bug Fixes:

* Fix short read in block incremental restore. (Reviewed by Stephen Frost, Brent Graveland. Reported by Adol Rodriguez, Brent Graveland.)
* Fix overflow suppressing backup progress in info output. (Fixed by Robert Donovan. Reviewed by Joe Wildish.)

Improvements:

* Preserve partial files during block incremental delta restore. (Reviewed by Stephen Frost.)
* Add support for alternate compile-time page sizes. (Contributed by Viktor Kurilko. Reviewed by David Steele.)
* Skip files truncated during backup when bundling. (Contributed by Georgy Shelkovy. Reviewed by David Steele.)
* Improve SFTP storage error messages. (Contributed by Reid Thompson. Reviewed by David Steele.)
2024-01-22 09:54:59 -03:00
David Steele
85bc9f27d8 Begin v2.50 development. 2023-11-27 09:06:53 -03:00
David Steele
3cb891e3ca v2.49: Remove PostgreSQL 9.3 Support
Bug Fixes:

* Fix regression in retries. (Reviewed by Stephen Frost. Reported by Norman Adkins, Tanel Suurhans, Jordan English, Timothée Peignier.)
* Fix recursive path remove in SFTP storage driver. (Fixed by Reid Thompson. Reviewed by Stephen Frost. Reported by Luc.)

Improvements:

* Remove support for PostgreSQL 9.3. (Reviewed by Stephen Frost.)

Documentation Features:

* Document maintainer options. (Reviewed by Stefan Fercot.)
* Update point-in-time recovery documentation for PostgreSQL >= 13.

Test Suite Improvements:

* Allow config/load unit test to run without libssh2 installed. (Contributed by Reid Thompson. Reviewed by David Steele. Suggested by Wu Ning.)
2023-11-27 08:55:56 -03:00
David Steele
1eb0162208 Build command and configuration reference in C.
Migrate generation of these files from help.xml to the intermediate documentation format. This allows us to share a lot of code that is already in C and remove duplicated code in Perl. More duplicate code can be removed in Perl once man generation is migrated.

Also update the unit test harness to allow testing of modules in the doc directory.
2023-10-09 14:03:43 -04:00
David Steele
084c8e1691 Begin v2.49 development. 2023-09-25 09:40:45 -04:00
David Steele
a7ab686d0e v2.48: Repository Storage Tags
Bug Fixes:

* Fix issue restoring block incremental without a block list. (Reviewed by Stephen Frost, Burak Yurdakul. Reported by Burak Yurdakul.)

Features:

* Add --repo-storage-tag option to create object tags. (Reviewed by Stephen Frost, Stefan Fercot, Timothée Peignier.)
* Add known hosts checking for SFTP storage driver. (Contributed by Reid Thompson. Reviewed by Stephen Frost, David Steele.)
* Support for dual stack connections. (Reviewed by Stephen Frost.)
* Add backup size completed/total to info command JSON output. (Contributed by Stefan Fercot. Reviewed by David Steele.)

Improvements:

* Multi-stanza check command. (Reviewed by Stephen Frost.)
* Retry reads of pg_control until checksum is valid. (Reviewed by Stefan Fercot, Stephen Frost.)
* Optimize WAL segment check after successful backup. (Reviewed by Stephen Frost.)
* Improve GCS multi-part performance. (Reviewed by Reid Thompson.)
* Allow archive-get command to run when stanza is stopped. (Reviewed by Tom Swartz, David Christensen, Reid Thompson.)
* Accept leading tilde in paths for SFTP public/private keys. (Contributed by Reid Thompson. Reviewed by David Steele.)
* Reload GCS credentials before renewing authentication token. (Reviewed by Stephen Frost. Suggested by Daniel Farina.)

Documentation Bug Fixes:

* Fix configuration reference example for the tls-server-address option. (Fixed by Hartmut Goebel. Reviewed by David Steele.)
* Fix command reference example for the filter option.

Test Suite Improvements:

* Allow storage/sftp unit test to run without libssh2 installed. (Contributed by Reid Thompson. Reviewed by David Steele. Suggested by Wu Ning.)
2023-09-25 09:32:15 -04:00
David Steele
cbafcfabf2 Remove UNCONSTIFY() from gz compression module.
The ZLIB_CONST macro may be used since 1.2.5.2 (17 Dec 2011) to specify that the input buffers are const. This is sufficiently old to cover all non-EOL distributions, i.e. everything we test on.

Compiling on older distributions may generate warnings but will continue to work.
2023-08-07 12:38:33 +01:00
David Steele
c585ddaf48 Begin v2.48 development. 2023-07-24 09:15:38 +02:00
David Steele
b5b033cc1c v2.47: Performance Improvements and Bug Fixes
Bug Fixes:

* Preserve block incremental info in manifest during delta backup. (Reviewed by Stephen Frost. Reported by Francisco Miguel Biete Banon.)
* Fix block incremental file names in verify command. (Reviewed by Reid Thompson. Reported by Francisco Miguel Biete Banon.)
* Fix spurious automatic delta backup on backup from standby. (Reviewed by Stephen Frost. Reported by krmozejko, Don Seiler.)
* Skip recovery.signal for PostgreSQL >= 12 when recovery type=none. (Reviewed by Stefan Fercot. Reported by T.Anastacio.)
* Fix unique label generation for diff/incr backup. (Fixed by Andrey Sokolov. Reviewed by David Steele.)
* Fix time-based archive expiration when no backups are expired. (Reviewed by Stefan Fercot.)

Improvements:

* Improve performance of SFTP storage driver. (Contributed by Stephen Frost, Reid Thompson. Reviewed by David Steele.)
* Add timezone offset to info command date/time output. (Reviewed by Stefan Fercot, Philip Hurst. Suggested by Philip Hurst.)
* Centralize error handling for unsupported features. (Reviewed by Stefan Fercot.)

Documentation Improvements:

* Clarify preference to install from packages in the user guide. (Reviewed by Stefan Fercot. Suggested by dr-kd.)
2023-07-24 09:12:30 +02:00
David Steele
9dbf76d8e8 Begin v2.47 development. 2023-05-22 11:19:00 +03:00
David Steele
b461f7c6f8 v2.46: Block Incremental Backup and SFTP Storage
Features:

* Block incremental backup. (Reviewed by John Morris, Stephen Frost, Stefan Fercot.)
* SFTP support for repository storage. (Contributed by Reid Thompson. Reviewed by Stephen Frost, David Steele.)
* PostgreSQL 16 support. (Reviewed by Stefan Fercot.)

Improvements:

* Allow page header checks to be skipped. (Reviewed by David Christensen. Suggested by David Christensen.)
* Avoid chown() on recovery files during restore. (Reviewed by Stefan Fercot, Marcelo Henrique Neppel. Suggested by Marcelo Henrique Neppel.)
* Add error retry detail for HTTP retries.

Documentation Improvements:

* Add warning about using recovery type=none. (Reviewed by Stefan Fercot.)
* Add note about running stanza-create on already-created repositories.
2023-05-22 11:13:13 +03:00
Reid Thompson
87087fac23
SFTP support for repository storage. 2023-05-13 19:16:16 +03:00
David Steele
5b5786c082 Begin v2.46 development. 2023-03-20 10:25:44 +08:00
David Steele
6ad79d16ca v2.45: Block Incremental Backup (BETA)
Bug Fixes:

* Skip writing recovery.signal by default for restores of offline backups. (Reviewed by Stefan Fercot. Reported by Marcel Borger.)

Features:

* Block incremental backup (BETA). (Reviewed by John Morris, Stephen Frost, Stefan Fercot.)

Improvements:

* Keep only one all-default group index. (Reviewed by Stefan Fercot.)

Documentation Improvements:

* Add explicit instructions for upgrading between 2.x versions. (Contributed by Christophe Courtois. Reviewed by David Steele.)
* Remove references to SSH made obsolete when TLS was introduced.
2023-03-20 09:37:23 +08:00
David Steele
f91af30584 Add FN_PRINTF() macro.
Bring the format(printf) attribute in line with the FN_NO_RETURN and FN_INLINE_ALWAYS macros.

This is simpler to read and can be customized for different compilers.
2023-01-30 10:51:48 +07:00
David Steele
240312110c Begin v2.45 development. 2023-01-30 09:27:04 +07:00
David Steele
053468bfb1 v2.44: Remove PostgreSQL 9.0/9.1/9.2 Support
Improvements:

* Remove support for PostgreSQL 9.0/9.1/9.2. (Reviewed by Stefan Fercot.)
* Restore errors when no backup matches the current version of PostgreSQL. (Contributed by Stefan Fercot. Reviewed by David Steele. Suggested by Soulou.)
* Add compress-level range checking for each compress-type. (Reviewed by Stefan Fercot. Suggested by gkleen, ViperRu.)

Documentation Improvements:

* Add warning about enabling "hierarchical namespace" on Azure storage. (Reviewed by Stefan Fercot. Suggested by Vojtech Galda, Pluggi, asjonos.)
* Add replacement for linefeeds in monitoring example. (Reviewed by Stefan Fercot. Suggested by rudonx, gmustdie, Ivan Shelestov.)
* Clarify target-action behavior on various PostgreSQL versions. (Contributed by Chris Bandy. Reviewed by David Steele, Anton Kurochkin, Stefan Fercot. Suggested by Anton Kurochkin, Chris Bandy.)
* Updates and clarifications to index page. (Reviewed by Stefan Fercot.)
* Add dark mode to the website. (Suggested by Stephen Frost.)
2023-01-30 09:15:44 +07:00
David Steele
c826429123 Improve libbacktrace implementation.
The libbacktrace feature has not been working since the move to meson because libbacktrace detection was not added to the meson build. Add libbacktrace to meson and improve the feature so that it can be compiled into release builds.

The prior implementation fetched line numbers with each stack trace push. Not only was this slow but it missed any functions that were not being tracked on our stack.

Instead just examine the backtrace when an error happens and merge it with the info we have on our stack. If the backtrace is not available then the output remains as before.

Also remove --backtrace from test.pl since the library is now auto-detected.

Leave this library out of the production build for now to give it a little time to shake out in testing.
2023-01-11 11:19:26 +07:00
David Steele
4fb8a0ecdd Add meson unity build and tests.
This is immediately useful because it will detect any extern'd functions or variables that are not being used. It also detects functions or variables that are declared but not defined.

If a FV/VR_EXTERN macro is missing it will be detected either because of a mismatch in the declaration/definition or because a new defined symbol will appear in the nm test.

Eventually the unity build will be used to create a more optimized pgbackrest binary but that will need to wait.
2022-12-31 17:13:41 +07:00
David Steele
c972a9359b Begin v2.44 development. 2022-11-28 17:56:59 +08:00
David Steele
cc2ffd8264 v2.43: Bug Fix
Bug Fixes:

* Fix missing reference in diff/incr backup. (Reviewed by Stefan Fercot. Reported by Marcel Borger, ulfedf, jaymefSO.)

Improvements:

* Add hint when an option is specified without an index. (Reviewed by Stefan Fercot.)
2022-11-28 17:47:48 +08:00
David Steele
092e254794 Begin v2.43 development. 2022-11-22 10:27:20 +08:00
David Steele
70b75532bf v2.42: Bug Fixes
Bug Fixes:

* Fix memory leak in file bundle backup/restore. (Reviewed by John Morris, Oscar. Reported by Oscar.)
* Fix protocol error on short read of remote file. (Reviewed by Stephen Frost.)

Improvements:

* Do not store references for zero-length files when bundling. (Reviewed by Stefan Fercot.)
* Use more generic descriptions for pg_start_backup()/pg_stop_backup(). (Reviewed by Greg Sabino Mullane, David Christensen. Suggested by Greg Sabino Mullane.)

Test Suite Improvements:

* Update test.pl --psql-bin option to match command-line help. (Contributed by Koshi Shibagaki. Reviewed by David Steele.)
2022-11-22 10:20:59 +08:00
David Steele
ab4209ebcb Begin v2.42 development. 2022-09-19 10:17:25 -07:00
David Steele
6b355806cc v2.41: Backup Annotations
Bug Fixes:

* Fix incorrect time expiration being used for non-default repositories. (Reviewed by Stefan Fercot. Reported by Adam Brusselback.)
* Fix issue when listing directories recursively with a filter. (Reviewed by Stephen Frost. Reported by Efremov Egor.)

Features:

* Backup key/value annotations. (Contributed by Stefan Fercot. Reviewed by David Steele. Suggested by Adam Berlin.)

Improvements:

* Support --set in JSON output for info command. (Contributed by Stefan Fercot. Reviewed by David Steele. Suggested by Anton Kurochkin.)
* Update archive.info timestamps after a successful backup. (Reviewed by Stefan Fercot. Suggested by Alex Richman.)
* Move standby timeline check after checkpoint. (Reviewed by Stefan Fercot, Keith Fiske. Suggested by Keith Fiske.)
* Improve warning message on backup resume. (Suggested by Cynthia Shang.)

Documentation Improvements:

* Add absolute path for kill in pgbackrest.service. (Suggested by Don Seiler.)
2022-09-19 10:08:10 -07:00
David Steele
8fb61a809d
Add FN_INLINE_ALWAYS macro.
Eliminate a lot of useless repetition for a commonly used pattern.
2022-09-08 18:36:03 -06:00
David Steele
79d9884141 Move release build check to src/meson.build.
This allows the C harness to perform release builds which are required for some tests.
2022-07-25 09:39:54 -04:00
David Steele
20f9f72647 Begin v2.41 development. 2022-07-18 10:49:58 -04:00
David Steele
1ff531090b v2.40: OpenSSL 3 Support
NOTE TO PACKAGERS: An experimental meson build has been added but packagers should continue to use the autoconf/make build for the foreseeable future.

Improvements:

* OpenSSL 3 support. (Reviewed by Stephen Frost.)
* Create snapshot when listing contents of a path. (Reviewed by John Morris, Stephen Frost.)
* Force target-timeline=current when restore type=immediate. (Reviewed by Stephen Frost.)
* Truncate files during delta restore when they are larger than expected. (Reviewed by Stephen Frost.)
* Disable incremental manifest save when resume=n. (Contributed by Reid Thompson. Reviewed by David Steele.)
* Set backup percent complete to zero before copy start. (Contributed by Reid Thompson. Reviewed by David Steele.)
* Use S3 IsTruncated flag to determine list continuation. (Reviewed by John Morris, Soulou. Suggested by Christian Montagne.)

Documentation Bug Fixes:

* Skip internal options in the configuration reference. (Reported by Francisco Miguel Biete.)

Documentation Improvements:

* Add link to PostgreSQL configuration in repository host section. (Reviewed by Stefan Fercot. Suggested by Julien Cigar.)

Test Suite Improvements:

* Add experimental Meson build. (Reviewed by Eli Schwartz, Sam Bassaly.)
* Allow any path to be passed to the --test-path option. (Contributed by Andrey Sokolov. Reviewed by David Steele.)
* Fix compile error when DEBUG_EXEC_TIME is defined without DEBUG. (Contributed by Andrey Sokolov. Reviewed by David Steele.)
2022-07-18 09:32:30 -04:00
David Steele
0eccbc8bf4 Meson builds work on version 0.45.
v0.45 ships with Ubuntu 18.04, which is currently the oldest distro we support. We may never do a Meson release on Ubuntu 18.04 but this allows us to start running unit tests with Meson in the meantime.

Some more granular options are not available so we use buildtype in more places.

The check for a in-tree autoconf/make build had to be removed since the filesystem APIs are not available.

Finally, alias_target was removed. This means that full paths must be used for build targets, which does not seem too bad. For instance, test/src/test-pgbackrest must now be used as a build target instead of simple test-pgbackrest.
2022-07-06 18:17:52 -04:00
David Steele
59f148bf6e Generate -fmacro-prefix-map to strip relative path from __FILE__.
This provides reproducible builds and minimizes the file path in debug messages, just like an in-tree make build.

For test source, prefix with test/ in case there are any module name collisions.
2022-06-26 08:42:43 -04:00
David Steele
f863fc9888 Add experimental unit test harness written in C.
Having the test harness in C will allow us to remove duplicated Perl code and test on systems where Perl support is not present.

Custom harnesses and shims are currently not implemented, which means only the following tests in the common module will run: error, stack-trace, type-convert, assert-on, mem-context, time, encode, type-object, type-string, type-list, type-buffer, type-variant, reg-exp, log.

The experimental test harness is being committed with partial functionality so it can be used in Windows development. The remaining features will follow as needed.
2022-06-23 12:20:56 -04:00
David Steele
9fd85a104a Disable meson for all but debug builds.
The meson builds are still experimental so for now the configure/make build process is preferred for release builds. This message should help prevent any automated build systems from picking up meson instead.
2022-06-21 09:50:38 -04:00
David Steele
665da12ae7 Update meson.build include comment to be more general.
Eventually multiple subdirs will be included so update the comment to reflect that.
2022-06-17 16:41:48 -04:00
Sam Bassaly
0dabf88e9d
Add FN_NO_RETURN macro.
Change all instances of __attribute__((__noreturn__)) to a macro in meson.build / build.auto.h.in.

As compiler attributes written in the form of __attribute__ are not supported by MSVC, this is one of several commits to make the code-base more robust and allow using MSVC-style attributes later.
2022-06-08 17:43:23 -04:00
David Steele
b8fc20d5b8
Add experimental Meson build.
Meson is a new build system that offers simpler syntax and superior performance to autoconf/make. In addition, Windows is supported natively.

The Meson build appears complete, but currently is used only for auto-generation of code and the host build of pgbackrest. Some container upgrades will be required before Meson can be used for container builds.

Also patch the Debian package to force autoconf/make rather than Meson.
2022-06-03 14:13:56 -04:00