## 2.02 2018-05-06

Parallel Asynchronous Archive Get and Configuration Includes

Fix directory syncs running recursively when only the specified directory should be synced.

Fix archive-copy throwing path not found error for incr/diff backups.

Fix failure in manifest build when two or more files in PGDATA are linked to the same directory.

Fix delta restore failing when a linked file is missing.

Fix rendering of key/value and list options in help.

Add asynchronous, parallel archive-get.

This feature maintains a queue of WAL segments to help reduce latency when requests a WAL segment with restore_command.

Add support for additional configuration files.

The directory is specified by the --config-include-path option. Add --config-path option for overriding the default base path of the --config and --config-include-path option.

Add repo-s3-token option to allow temporary credentials tokens to be configured.

currently has no way to request new credentials so the entire command (e.g. backup, restore) must complete before the credentials expire.

Update the archive-push-queue-max, manifest-save-threshold, and buffer-size options to accept values in KB, MB, GB, TB, or PB where the multiplier is a power of 1024.

Make backup/restore path sync more efficient.

Scanning the entire directory can be very expensive if there are a lot of small tables. The backup manifest contains the path list so use it to perform syncs instead of scanning the backup/restore path.

Show command parameters as well as command options in initial info log message.

Rename archive-queue-max option to archive-push-queue-max.

This is consistent with the new archive-get-queue-max option. The old option name will continue to be accepted.

Make backup.history sync more efficient. Only the backup.history/[year] directory was being synced, so check if the backup.history is newly created and sync it as well.

Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.

Improved lock implementation written in C. Now only two types of locks can be taken: archive and backup. Most commands use one or the other but the stanza-* commands acquire both locks. This provides better protection than the old command-based locking scheme.

Storage object improvements. Convert all functions to variadic functions. Enforce read-only storage. Add `storageLocalWrite()` helper function. Add `storageCopy()`, `storageExists()`, `storageMove()`, `storageNewRead()`/`storageNewWrite()`, `storagePathCreate()`, `storagePathRemove()`, `storagePathSync()`, and `storageRemove()`. Add `StorageFileRead` and `StorageFileWrite` objects. Abstract Posix driver code into a separate module. Call `storagePathRemove()` from the Perl Posix driver.

Improve `String` and `StringList` objects. Add `strUpper()`, `strLower()`, `strLstExists()`, `strLstExistsZ()`, `strChr()`, `strSub()`, `strSubN()`, and `strTrunc()`.

Improve `Buffer` object. Add `bufNewC()`, `bufEq()` and `bufCat()`. Only reallocate buffer when the size has changed.

Add `pgControlInfo()` to read pg_control and determine the version.

Add `walSegmentNext()` and `walSegmentRange()`.

Error handling improvements. Add `THROWP_`* macro variants for error handling. These macros allow an `ErrorType` pointer to be passed and are required for functions that may return different errors based on a parameter. Add `_FMT` variants for all `THROW` macros so format types are checked by the compiler.

Split `cfgLoad()` into multiple functions to make testing easier. Mainly this helps with unit tests that need to do log expect testing.

Allow `MemContext` objects to be copied to a new parent. This makes it easier to create objects and then copy them to another context when they are complete without having to worry about freeing them on error. Update `List`, `StringList`, and `Buffer` to allow moves. Update `Ini` and `Storage` to take advantage of moves.

Full branch coverage in C code.

Refactor usec to msec in `common/time.c`. The implementation provides usec resolution but this is not needed in practice and it makes the interface more complicated due to the extra zeros.

Replace `THROW_ON_SYS_ERROR()` with `THROW_SYS_ERROR()`. The former macro was hiding missing branch coverage for critical error handling.

Start work on C handle io object and use it to output help.

Don't copy CFGDEF_NAME_ALT or CFGDEF_INHERIT when processing config option inheritance.

Split debug and assert code into separate headers. Assert can be used earlier because it only depends on the error-handler and not logging. Add `ASSERT()` macro which is preserved in production builds.

Cleanup C types. Remove `typec.h`. Order all typdefs above local includes.

Fix header exclusion defines that do not match the general pattern.

Update docs with 32-bit support and caveats.

32-bit support was added in v1.26.

Add monitoring examples using and jq.

Add example of command section usage to archiving configuration.

Remove documentation describing `info --output=json` as experimental.

Update out-of-date description for the spool-path option.

Add logic to find the real oid of the test1 database during restore testing.

Document build improvements. Perform `apt-get update` to ensure packages are up to date before installing. Add -p to the repository mkdir so it won't fail if the directory already exists, handy for testing packages.

Use lcov for C unit test coverage reporting.

Switch from Devel::Cover because it would not report on branch coverage for reports converted from gcov. Incomplete branch coverage for a module now generates an error. Coverage of unit tests is not displayed in the report unless they are incomplete for either statement or branch coverage.

Move test definitions to test/define.yaml. The location is better because it is no longer buried in the Perl test libs. Also, the data can be easily accessed from C.

Move help/version integration tests to mock/all. Help and version are covered by unit tests, so we really just to need to make sure there is output when called from the command line.

Move archive-stop and expire tests to the mock module. These are mock integration tests so they should be grouped with the other mock integration tests.

Add `harnessCfgLoad()` test function, which allows a new config to be loaded for unit testing without resetting log functions, opening a log file, or taking locks.

Add `HARNESS_FORK` macros for tests that require fork(). A standard pattern for tests makes fork() easier to use and should help prevent some common mistakes.

Add `TEST_ERROR_FMT` macro to simplify testing of formatted error messages.

Generate code counts for all source files. The source files are also classified by type and purpose.

Include VM type in gcov path to avoid conflicts between VMs with different architectures.

Improve logic for smart builds to include version changes. Skip version checks when testing in --dev mode.

Use pip 9.03 in test VMs. pip 10 drops support for Python 2.6 which is still used by the older test VMs.

Allow `-DDEBUG_UNIT` to be suppressed to test how debug macros behave.

Rename Perl tests so they don't conflict with their C counterparts.

Divide tests into three types (unit, integration, performance). Many options that were set per test can instead be inferred from the types, i.e. container, c, expect, and individual.

Try tweaking time sync settings to prevent clock drift rather than restarting VBoxService on every test run.