1
0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2024-12-14 10:13:05 +02:00
Commit Graph

190 Commits

Author SHA1 Message Date
David Steele
f1e92804ce Docker container improvements.
* Containers now use a squid proxy for apt/yum to speed builds.
* Obsolete containers are removed by the <br-option>--vm-force</br-option> option.
* Greatly reduced the quantity of Docker containers built by default. Containers are only built for PostgreSQL versions specified in db-minimal and those required to build documentation. Additional containers can be built with --db-version=all or by specifying a version, e.g. --db-version=9.4.
2016-06-12 09:00:16 -04:00
David Steele
eaa981c5aa Added execution cache for document generation.
Added an execution cache so that documentation can be generated without setting up the full container environment. This is useful for packaging, keeps the documentation consistent for a release, and speeds up generation when no changes are made in the execution list.
2016-06-02 09:32:56 -04:00
David Steele
174b16165b Replaced overzealous perl -cW check.
It failed on Perl 5.22 so replaced with with perl -cw.
2016-05-26 16:57:35 -04:00
David Steele
bfec2d2f89 Fixed release version test when the current version is development. 2016-05-26 11:03:10 -04:00
David Steele
c8d68bcf2d More detailed release notes.
Release notes are now broken into sections so that bugs, features, and refactors are clearly delineated.  An "Additional Notes" section has been added for changes to documentation and the test suite that do not affect the core code.
2016-05-26 10:34:10 -04:00
Cynthia Shang
5a85122841 Moved change log to website.
The change log was the last piece of documentation to be rendered in Markdown only.  Wrote a converter so the document can be output by the standard renderers.  The change log will now be located on the website and has been renamed to "Releases".
2016-05-26 09:20:55 -04:00
David Steele
e7d4a25d6a Added Ubuntu 16.04 and Debian 8 to regression tests.
Also added some new fields to make container generation simpler for distros that share a common package base.
2016-05-24 08:17:13 -04:00
David Steele
f2a14a06e5 Moved version checking code so it doesn't run in containers.
The required modules were not loaded so errors were occurring during regression tests.
2016-05-23 20:04:36 -04:00
David Steele
c604b6e657 Test release version using change-log.xml instead of CHANGELOG.md.
This is in preparation for moving the change log to the website and removing it from Github.
2016-05-23 19:33:02 -04:00
David Steele
512d006346 Refactor database version identification for archive and backup commands.
Added database version constants and changed version identification code to use hash tables instead of if-else.  Propagated the db version constants to the rest of the code and in passing fixed some path/filename constants.

Added new regression tests to check that specific files are never copied.
2016-05-14 10:33:12 -04:00
David Steele
e430f0a054 Added --db-version=minimal option as default.
This change assigns each version of PostgreSQL to a specific OS version for testing to minimize the number of tests being run.  In general, older versions of PostgreSQL are assigned to older OS versions.

The old behavior can be enabled with `--db-version=all`.
2016-05-11 08:59:34 -04:00
David Steele
a6a19e3735 Test directories are now located on the host VM rather than in the Docker container.
This change allows for easier testing since all files are local on the host VM and can be easily accessed without using `docker exec`.  In addition, this change is required to allow multiple Docker containers per test case which is coming soon.
2016-05-10 18:12:37 -04:00
David Steele
18fd25233b New simpler configuration and consistent project/exe/path naming.
* The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required.

* Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case.

* The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally.

* The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest.

* Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory.

* Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory.

* Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 09:30:54 -04:00
David Steele
64b2858bb5 Test VM build improvements.
* Specific VMs can now be built by using --vm along with --vm-build.
* Docker caching can be disabled with --vm-force.
* ControlMaster is now used for al VMs to improve test speed.
2016-04-13 18:47:29 -04:00
David Steele
048571e23f Closed #173: Add static source code analysis
Perl Critic added and passes on gentle.  A policy file has been created with some permanent exceptions and a list of policies to be fixed in approximately the order they should be fixed in.
2016-02-23 09:25:22 -05:00
David Steele
02405e851c Closed #163: Document Generation Issue
Fixed an issue where document generation failed because some OSs are not tolerant of having multiple installed versions of PostgreSQL. A separate VM is now created for each version. Also added a sleep after database starts during document generation to ensure the database is running before the next command runs.

Reported by John Harvey.
2016-01-22 18:01:21 -05:00
David Steele
fce7261536 Only allow --no-cleanup option when a single test is selected. 2016-01-15 23:22:30 -05:00
David Steele
67fb7a23bc Master test log is output to file. 2016-01-14 19:37:06 -05:00
David Steele
c2b1620c53 Logging enhancements:
1) Show log output realtime when --max-process=1 or only 1 test is being run.
2) Dry run now shows test output when --vm-out is specified.
2016-01-13 22:43:26 -05:00
David Steele
0f88943b7b Database versions are now run as separate tests. 2016-01-10 19:30:51 -05:00
David Steele
c8f863fbab Text execution improvements:
1) Tests for all operating systems can now be run with a single command.
2) Tests can be run in parallel with --process-max.
3) Container generation now integrated into test.pl
4) Some basic test documentation.
2016-01-09 08:21:53 -05:00
David Steele
7f630aec49 Switched to using docker for all unit tests as this allows for greater automation. 2015-12-29 13:57:10 -05:00
David Steele
ddd4c14773 Implemented issue #129: Stop/Start Commands.
Implemented issue #136: Add stderr back into protocol layer.

There were also miscellaneous bug fixes and improvements to buffering in IO.pm.
2015-10-08 11:43:56 -04:00
David Steele
e0dafbff7f Improvements to issue #132: Improved command-line help.
Regression tests are now more comprehensive by default.
Better handling for errors in safeExit().
Release notes.
2015-09-09 15:40:54 -04:00
David Steele
adc5568b15 Fixed issue #132: Improved command-line help.
Some platform specific fixes and added regression tests to detect issues.
2015-09-08 12:58:13 -04:00
David Steele
6a9377a0a9 Code cleanup and refactoring to standardize on patterns that have evolved over time. 2015-08-29 14:20:46 -04:00
David Steele
d3262822ef Fixed issue #128: Replace config name in test logs
It was actually the bin path that was causing the issue.
2015-08-27 15:55:29 -04:00
David Steele
2edf5d4bf7 Split most of README.md out into CHANGELOG.md and USERGUIDE.md.
Added release dates to change log.
2015-08-07 14:43:53 -04:00
David Steele
073e3b7c00 Minor for regression tests.
Improved CentOS 6 vagrant config.
2015-07-11 22:47:22 -04:00
David Steele
b777525f62 Added --exe param to allow testing of packaged versions.
Updates to u12 vagrant vm.
2015-07-11 17:16:35 -04:00
David Steele
8b7fd99637 Added vagrant configuration for Ubuntu 12.04.
Changes to regression tests to work better with VMs.
2015-07-06 21:00:48 -04:00
David Steele
c59adfc68d Shorted test command line params. 2015-06-27 11:12:44 -04:00
David Steele
760baab6b2 Implemented issue #111: Add the psql path for *BSD.
Author: Eric Radman
Modified by Committer
2015-06-22 14:35:25 -04:00
David Steele
f210fe99c3 Implemented issue #109: Move VERSION into source code.
Also stopped replacing FORMAT number which explains the large number of test log changes.  FORMAT should change very rarely and cause test log failures when it does.
2015-06-22 13:11:07 -04:00
David Steele
61dfeca3c1 Fixed issue #108: Incompatibility with Perl 5.10.1 2015-06-21 12:06:13 -04:00
David Steele
22e126eda7 Fix for logic that removes trailing spaces from code. 2015-06-17 15:40:12 -04:00
David Steele
b865070edd Experimental 9.5 support. Unit tests are not working yet. 2015-06-14 10:12:36 -04:00
David Steele
148836fe44 Implemented issue #26: Info command.
* Includes updating the manifest to format 4.  It turns out the manifest and .info files were not very good for providing information.  A format update was required anyway so worked through the backlog of changes that would require a format change.

* Multiple database versions are now supported in the archive.  Does't actually work yet but the structure should be good.

* Tests use more constants now that test logs can catch name regressions.
2015-06-13 18:25:49 -04:00
David Steele
a0d985b7ad Fixed an issue where an absolute path was not written into recovery.conf when the restore was run with a relative path. 2015-05-07 17:37:25 -05:00
David Steele
56588f6fdd Log testing can now be enabled for certain deterministic tests. This works by comparing the generated logs against a previous copy. Currently only enabled for the backup/synthetic tests. 2015-05-05 11:08:48 -06:00
David Steele
cb153f7242 Docs and comment fixes. 2015-04-19 17:27:40 -04:00
David Steele
43d86e64a4 First pass at tests comparing rsync to backrest. Decent results, but room for improvement.
All tests local over SSH with rsync default compression, 4 threads and default compression on backrest.  Backrest default is gzip = 6, assuming rsync is the same.

On a 1GB DB:

rsync time = 32.82
backrest time = 19.48

backrest is 171% faster.

On a 5GB DB:

rsync time = 171.16
backrest time = 86.97

backrest is 196% faster.
2015-04-07 18:36:59 -04:00
David Steele
3f651a8ce8 Unit tests will now work across all installed versions of Postgres.
Created a function to list all supported versions.  Now used for all version checking.
2015-04-02 22:07:23 -04:00
David Steele
984fceea67 Moved archive functions from pg_backrest.pl and Backup.pl to Archive.pm.
Moved Remote code from pg_backrest.pl to Config.pm.
Added version specific code to regression tests and Db.pm.
archive-push checks for duplicate WAL in the archive.
archive-push reads the db sys id to match up WAL to the correct archive.
2015-04-01 15:58:33 -04:00
David Steele
882f068254 All config options can now be provided on the command-line.
Lots of name changes, but more thought needed there.
Still needs cleanup.
2015-03-12 12:15:19 -04:00
David Steele
ae6bdecfaf Split command-line parameter processing out into a separate file. This is in preparation allowing all parameters to be specified/overridden on the command line, with pg_backrest.conf being option. 2015-03-08 13:26:09 -04:00
David Steele
5d10a18b25 Added thread-max parameter to test.pl to allow backup threads to be specified. 2015-02-28 10:21:36 -05:00
David Steele
d6205d9501 Looks like all unit tests pass - now for a long test run to see if that is really true. And to see if the old lockup is gone. 2015-02-27 23:31:39 -05:00
David Steele
e20f93ef1b Cleaned up test usage. 2015-01-12 09:05:26 -05:00
David Steele
84abcd7d61 Minor fixes. 2015-01-11 11:53:03 -05:00
David Steele
34116d3816 Added usage to test.pl and automatic search for psql-bin. 2015-01-11 11:52:16 -05:00
David Steele
06ad947102 Using ' instead of " where appropriate. 2014-09-16 11:22:55 -04:00
David Steele
6b4a8fbc21 Unit tests now working on Ubuntu 12.04 2014-08-10 21:22:17 -04:00
David Steele
06b5ee3f75 Working on new config load/save functions and unit tests. 2014-07-16 23:38:38 -04:00
David Steele
9d9dae418b Added archive-put unit tests.
Lots of other unit test improvements.
2014-07-15 22:32:41 -04:00
David Steele
375545320d Added tests for large files (16MB archive log).
Fixed a sync error in binary_xfer().
2014-07-13 09:13:19 -04:00
David Steele
ed4bcf3e46 Whitespace is now cleaned in test.pl 2014-07-12 19:36:12 -04:00
David Steele
88bbbb4222 Fighting with a relative path issue in archive logging. 2014-06-22 20:19:13 -04:00
David Steele
8fff74ccfb Backup unit test able to create and drop clusters. 2014-06-22 11:56:01 -04:00
David Steele
3c748c4b10 More work on unit tests. 2014-06-22 10:54:31 -04:00
David Steele
0b597d8da9 Working on getting backup running again. 2014-06-22 10:30:17 -04:00
David Steele
7bc464a44b Working on BackRest::File methods. 2014-06-21 15:17:09 -04:00
David Steele
976bafca9e Added new tests. 2014-06-07 17:29:11 -04:00
David Steele
c2dd53f908 Changes to allow specific tests to be run. 2014-06-05 10:20:03 -04:00
David Steele
1b9782cb9e Converted path_create(), improved unit tests. 2014-06-03 21:22:07 -04:00
David Steele
0338369193 Working on more robust file functions. 2014-06-01 17:23:33 -04:00
David Steele
3e12f9230b Working on unit tests for file_copy. Still need to add specific error tests, timestamp, and permissions. 2014-05-14 15:07:37 -04:00
David Steele
194696f674 Working on unit tests. 2014-04-02 17:25:37 -04:00
David Steele
27b820cfb4 Backup handles files that were removed by the db 2014-02-18 15:42:51 -05:00
David Steele
a78d677b1e Wait for archive files. 2014-02-13 20:29:42 -05:00
David Steele
219120ba81 Archiving and flag changes. 2014-02-12 07:39:42 -05:00
David Steele
9d08f2a644 More robust config. Retention is read from config. 2014-02-06 12:49:54 -05:00
David Steele
7ffade2453 Fixed file mod time and permissions 2014-02-05 11:35:09 -05:00
David Steele
c1e5904880 Fixed xlog copy - working on correctness 2014-01-12 01:46:27 -05:00
David Steele
ba5075a3e0 New implementation for archive logging 2014-01-03 21:19:41 -05:00
David Steele
2ba26ffc26 More refactoring. 2013-12-14 13:14:59 -05:00
David Steele
0dba2ab8c7 Added backup command 2013-12-10 20:16:43 -05:00
David Steele
0bdccd2982 Reversed order of tests. 2013-12-10 19:18:41 -05:00
David Steele
f09505f60e Working on unit tests for archiving. 2013-12-10 19:11:54 -05:00
David Steele
39989dd01c Working on manifest. 2013-12-09 17:26:47 -05:00
David Steele
02a9a693b8 Working on backup. 2013-12-06 20:24:14 -05:00
David Steele
a2c98391df Starting on configuration. 2013-12-02 14:34:37 -05:00
David Steele
2be8c5d9e7 Working on unit tests. 2013-11-23 20:05:04 -05:00
David Steele
5d0251be2f Working on archive-local and beefing up unit tests. 2013-11-23 19:16:09 -05:00
David Steele
8b37786e5b Now connecting to db. 2013-11-22 23:24:37 -05:00
David Steele
8db52c115a Fixed whitespace. 2013-11-22 17:29:01 -05:00
David Steele
d86e7eb627 Added checksums. 2013-11-20 22:30:26 -05:00
David Steele
bc46aefe61 Fixed for OSX. Do not every use TextEditor on code! 2013-11-20 22:24:30 -05:00
David Steele
e67821a230 Working on OSX port. 2013-11-20 21:49:07 -05:00
David Steele
548578c8c9 Working on unit test. 2013-11-17 13:58:21 -05:00