Additional Notesbut changes in this section are only to documentation or the test suite and have no direct impact on the
Fix resume after partial delete of backup by prior resume.
Improve
Partial multi-repository implementation.
Add backup verification to internal verify command.
Add
Allow option validity to be determined by command role.
Add
Replace
Allow [, #, and space as the first character in database names.
Create
Expire history files.
Report page checksum errors in
Add
Add
Improve
Improve performance of large file lists in
Add retries to
Prepare configuration module for multi-repository support.
Optimize small reads in IoRead.
Allow multiple remote locks from the same main process.
Conform retry in lockAcquireFile() to the common retry pattern.
Assert when buffer used is greater than size limit.
Replace
Error with hints when backup user cannot read
Improve
Improve working directory error message.
Add hint about starting the stanza when WAL segment not found.
Add hint for protocol version mismatch.
Add internal verify command.
Allow ProtocolParallel to complete with no jobs.
Add note that
Move info command text to the reference and link to user guide.
Update yum repository path for
Suppress errors when closing
Since the command has completed it is counterproductive to throw an error but still warn to indicate that something unusual happened.
Fix issue with
Automatically retrieve temporary
Add
Changes to the control/catalog/WAL versions in subsequent betas may break compatibility but
Asynchronous
Improve memory usage of unlogged relation detection in manifest build.
Proactively close file descriptors after forking async process.
Delay backup remote connection close until after archive check.
Improve detailed error output.
Improve TLS error reporting.
Allow HttpClient/HttpSession to work on plain sockets.
Add support for
Add general-purpose statistics collector.
Add
Add
Add missing
Fix typo in
Clarify that
Fix
This caused
Azure support for repository storage.
Add
This allows automatic expiration after a successful backup to be disabled.
Asynchronous S3 multipart upload.
Automatic retry for
Disable query parallelism in
Changes to the control/catalog/WAL versions in subsequent betas may break compatibility but
Improve handling of invalid HTTP response status.
Improve error when
Add hint when checksum delta is enabled after a timeline switch.
Use
Automatically determine cipher passphrase in
Fix expression when recursion enabled in storageInfoListP().
Improve behavior of the
Inline strPtr() to increase profiling accuracy.
Add pgLsnFromWalSegment().
Fix incorrect example for
Remove internal commands from
Update
Also add version ranges to indicate that a user guide is accurate for a range of
Update FAQ for expiring a specific backup set.
Update FAQ to clarify default PITR behavior.
Remove
Rename most instances of master to primary in tests.
Fix issue checking if file links are contained in path links.
Allow
The
Handle missing reason phrase in HTTP response.
Increase buffer size for
Ignore
Fix possibly missing
Time-based retention for full backups.
The
Ad hoc backup expiration.
Allow the user to remove a specified backup regardless of retention settings.
Note that setting
Note that setting
Add
Expire WAL archive only when
WAL prior to the first full backup was previously expired after the first full backup. Now it is preserved according to retention settings.
Add local
Changes to the control/catalog/WAL versions in subsequent betas may break compatibility but
Reduce
Throw user-friendly error if
Enforce non-null for most string options.
Remove empty subexpression from manifest regular expression.
Non-blocking TLS implementation.
Only limit backup copy size for WAL-logged files.
The prior behavior could possibly lead to
Add
Simplify storage driver info and list functions.
Split session functionality of SocketClient out into SocketSession.
Split session functionality of TlsClient out into TlsSession.
Use poll() instead of select() for monitoring socket read/write ready.
Use SocketSession/TlsSession for test servers.
Always throw error when SSL_ERROR_SYSCALL.
Use __noreturn_ on error functions when coverage testing.
Add
Note that setting
Add
Use dry-run to see which backups/archive would be removed by the
Improve performance of remote manifest build.
Fix detection of keepalive options on
Add configure host detection to set standards flags correctly.
Remove
These commands (e.g.
Limit backup file copy size to size reported at backup start.
If a file grows during the backup it will be reconstructed by WAL replay during recovery so there is no need to copy the additional data.
Add infrastructure for multiple compression type support.
Improve performance of MEM_CONTEXT*() macros.
Allow storage reads to be limited by bytes.
Prevent defunct processes in asynchronous archive commands.
Error when
Read HTTP content to eof when size/encoding not specified.
Fix resume when the resumable backup was created by
In this case the resumable backup should be ignored, but the
Auto-select backup set on restore when time target is specified.
Auto-selection is performed only when
Skip
Add more validations to the manifest on
Prevent
Fix missing files corrupting the manifest.
If a file was removed by
Use
Validate checksums are set in the manifest on
Fix error in timeline conversion.
The timeline is required to verify WAL segments in the archive after a backup. The conversion was performed base
Fix options being ignored by asynchronous commands.
The asynchronous
Fix handling of
Add
Specifies the database user name when connecting to
Allow path-style URIs in
The
Add basic time management functions.
Add httpLastModifiedToTime() to parse HTTP
Parse dates in storageS3InfoList() and storageS3Info().
Fix archive-push/archive-get when
These commands tried to use cwd() as cwd() does not match the chdir() to the path and make sure the next cwd() matches the result from the first call.
Fix reference list when
Since the
Fix segfault on unexpected EOF in
Add manifest build for new backups.
Fix remote timeout in delta restore.
When performing a delta restore on a largely unchanged cluster the remote could timeout if no files were fetched from the repository within
Fix handling of repeated HTTP headers.
When HTTP headers are repeated they should be considered equivalent to a single comma-separated header rather than generating an error, which was the prior behavior.
Monitoring systems can more easily ingest the jq can be used to pretty-print if desired.
The
Document how to contribute to
Document maximum version for
Fix container test path being used when
Fix mismatched timezone in expect test.
Don't autogenerate embedded libc code by default.
Add
The additional details include databases that can be used for selective restore and a list of tablespaces and symlinks with their default destinations.
Add
This restore type automatically adds
The
Migrate backup manifest load/save to C.
Improve performance of info file load/save.
Add helper function for adding CipherBlock filters to groups.
Document the relationship between
Add documentation clarifications regarding standby repositories.
Add FAQ for time-based Point-in-Time Recovery.
Improve slow manifest build for very large quantities of tables/segments.
Fix exclusions for special files.
The
The
Create log directories/files with
Move info file checksum to the end of the file.
Fix
Build
Retry
Fix incorrect handling of
Fix scoping violations exposed by optimizations in
Add
The
The
Add Perl interface to C storage layer.
Add Db object to encapsulate
Add PostgreSQL query client.
Fix archive retention expiring too aggressively.
The
The
Remove hard-coded
Honor
Rename
The new name is preferred because pgBackRest does not support any SSL protocol versions (they are all considered to be insecure). The old name will continue to be accepted.
Add most unimplemented functions to the remote storage driver.
Rename info*New() functions to info*NewLoad().
Add backup management functions to InfoBackup.
Add FAQ to the documentation.
Use
Fix segfault when
Bypass database checks when
Add
Filter improvements. Only process next filter in IoFilterGroup when input buffer is full or flushing. Improve filter's notion of done
to optimize filter processing.
Improve performance of non-blocking reads by using maximum buffer size.
Add storageInfoList() to get detailed info about all entries in a path.
Allow storageInfo() to follow links.
Allow StorageFileWrite to set user, group, and modification time.
Add pathExists() to Storage object.
Improve zero-length content handling in HttpClient object.
Don't append strerror() to error message when errno is 0.
Improve type safety of interfaces and drivers.
Add
Add
Add macros for object free functions.
Various MemContext callback improvements.
Various Buffer improvements.
Simplify storage object names.
Add ioWriteStr() and ioWriteStrLine().
Add separate functions to encode/decode each JSON type.
Add macros to create constant Buffer objects.
Add missing httpUriEncode() in S3 request.
Add unsigned int Variant type and update code to use it.
Expose handle (file descriptor) from IoWrite when applicable.
Add iniSave() and iniMove() to Ini object.
Add *Save() functions to most Info objects.
Extern infoHash() so it can be used by other modules.
varNewKv() accepts a KeyValue object rather than creating one.
Add constant for maximum buffer sizes required by cvt*() functions.
Add String constants.
Refactor Ini interface to expose String values instead of Variant.
Refactor main() as a switch() statement.
Add cfgOptionUInt() and cfgOptionUInt64() and update code to use them.
Improve log performance, simplify macros, rename logWill() to logAny().
Improve coverage in
Remove
Error on multiple option alternate names and simplify help command.
Use THROW_ON_SYS_ERROR*() to improve code coverage.
Improve macros and coverage rules that were hiding missing coverage.
Improve efficiency of FUNCTION_LOG*() macros.
Add user guides for
Automate coverage summary report generation.
Add
Restore index menu url default lost in b85e51d6.
Add harnessInfoChecksum/Z() to ease creation of test info files.
Update containers with
Add testUser() and testGroup().
Add
Reduce
Update
Fix zero-length reads causing problems for IO filters that did not expect them.
Fix reliability of error reporting from
Fix
Harden IO filters against zero input and optimize zero output case.
Move lockRelease() to the end of exitSafe().
Add CHECK() macro for production assertions.
Automatically generate constants for command and option names.
Use a macro instead of a nested struct to create common String variables.
Add STR() macro to create constant String objects from runtime strings.
Add macros to create constant Variant types.
Migrate backupRegExp() to C.
Option to build documentation from current
IMPORTANT NOTE: The new
Fix issues when a path option is / terminated.
Fix issues when
Fix C code to recognize
Fix issues with remote/local command logging options.
The
Increase
Improve error message when an
Add separate
CryptoHash improvements and fixes. Fix incorrect buffer size used in cryptoHashOne(). Add missing const to cryptoHashOne() and cryptoHashOneStr(). Add hash size constants. Extern hash type constant.
Add
Add file write to the remote and S3 storage drivers.
Add storageRepoWrite() and storagePg()/storagePgWrite() to storage helper.
Use a single file to handle global errors in async archiving.
Add document creation to XML objects.
Remove redundant documentation from
Add WAL info to
Refactor
Logging improvements. Allow three-digit process IDs in logging. Allow process id in C logging.
Add process id to ProtocolParallelJob.
Add process id to C
Close log file before exec().
Allow warnings to be written by archiveAsyncStatusOkWrite().
Move WAL path prefix logic into walPath().
Make notion of current
Add locking capability to the remote command.
Add forkSafe() to handle fork errors.
Add httpHeaderDup().
httpClientRequest() accepts a body parameter.
Add protocolKeepAlive() to send
Make strLstDup() null-tolerant.
Add strLstMergeAnti() for merge anti-joins.
Add cvtSSizeToZ() and debug macros.
Remove unused infoArchiveCheckPg() function.
Add constants for
Clarify that S3-compatible object stores are supported.
Build test harness with the same warnings as code being tested.
Add TEST_64BIT() macro to detect 64-bit platforms.
Skip coverage for macros with numbers in their name.
Use
Fix possible truncated WAL segments when an error occurs mid-write.
Fix info command missing WAL min/max when stanza specified.
Fix non-compliant JSON for options passed from C to Perl.
The
Enable socket keep-alive on older
Error when parameters are passed to a command that does not accept parameters.
Add hints when unable to find a WAL segment in the archive.
Improve error when hostname cannot be found in a certificate.
Add additional options to
Migrate
Add separate
Add ProtocolParallel* objects for parallelizing commands.
Add ProtocolCommand object.
Add exists() to remote storage.
Resolve storage path expressions before passing to remote.
Expose handle (file descriptor) from IoRead when applicable.
Add storageHelperFree() to storage helper.
Add kvKeyExists() to KeyValue object.
Add lstRemove() to List object.
Allow cfgExecParam() to exclude options.
MemContext improvements. Improve performance of context and memory allocations. Use contextTop/contextCurrent instead of memContextTop()/memContextCurrent(). Don't make a copy of the context name.
Make
Add note for
Add
Change execRead() to return a size_t.
Prevent option warning from being output when running help command.
Improve null-handling of strToLog() and varToLog().
Increase per-call stack trace size to
Move compress module to common/compress.
Move crypto module to common/crypto.
Update default documentation version to
Add instructions for building the coverage report.
Documentation builds on
Create test matrix for
Improve fork harness to allow multiple children and setup pipes automatically.
Reduce expect log level in
Rename test modules for consistency.
Only run test-level stack trace by default for unit-tested modules.
Add missing ToLog() coverage to String, List, and PgControl.
Create aliases for test VMs ordered by age.
Add unimplemented
Fix check for improperly configured
JSON improvements. Optimize parser implementation. Make the renderer more null tolerant.
Automatically adjust
Fix issue with multiple async status files causing a hard error.
The
Simplify
Replace the repository path with the repository
.
Add
Update address lookup in C TLS client to use modern methods.
Include Posix-compliant header for strcasecmp() and fd_set.
Add remote storage objects.
Add ProtocolClient object and helper functions.
Add Exec object.
Add IoHandleRead and IoHandleWrite objects.
Add cfgExecParam() to generate parameters for executing commands.
Ignore
Function log macro improvements. Rename FUNCTION_DEBUG_* and consolidate ASSERT_* macros for consistency. Improve CONST and P/PP type macro handling. Move MACRO_TO_STR() to FUNCTION_TEST_RETURN().
Make the
Improve accuracy of strSizeFormat().
Add ioReadBuf() to easily read into a buffer.
JSON improvements. Allow empty arrays in JSON parser. Fix null output in JSON renderer. Fix escaping in JSON string parser/renderer.
Allocate extra space for concatenations in the String object.
Return UnknownError from errorTypeFromCode() for invalid error codes.
Update Perl repo rules to work when stanza is not specified.
Update Storage::Local->list() to accept an undefined path.
Null-terminate list returned by strLstPtr().
Add kvMove() and varLstMove().
Replace FileOpenError with HostConnectError in TlsClient.
Allow string Variant objects to contain null.
Rename common/io/handle module to common/io/handleWrite.
Add const VariantList * debug type.
Fix hard-coded repository path.
Clarify that encryption is always performed client-side.
Add examples for building a documentation host.
Allow if in manifest variables, lists, and list items.
Move C module include in
Allow primary
Increase timeout in
Remove request for S3 object info directly after putting it.
Correct
Add error message when current user
Error when
Set TCP keepalives on
Reorder
Change file ownership only when required.
Redact
Enable S3 storage and encryption for
Migrate local
Add
Add HttpClient object.
Add TlsClient object.
Add interface objects for
Add encryption capability to Info* objects.
Add IoFilter interface to CipherBlock object.
Allow arbitrary InOut filters to be chained in IoFilterGroup.
Add infoBackup object to encapsulate the
Improve JSON to Variant conversion and add Variant to JSON conversion.
Storage helper improvements. Allow NULL stanza in storage helper. Add path expression for repository backup.
Info module improvements. Rename constants in Info module for consistency. Remove #define statements in the InfoPg module to conform with newly-adopted coding standards. Use cast to make for loop more readable in InfoPg module. Add infoArchiveIdHistoryMatch() to the InfoArchive object.
Allow I/O read interface to explicitly request blocking reads.
Improve error messages when info files are missing/corrupt.
Add strSizeFormat() to String object.
Add strLstInsert() and strLstInsertZ() to StringList object.
Rename PGBACKREST/BACKREST constants to PROJECT.
Require
Explicitly compile with Posix 2001 standard.
Add ServiceError for errors from a service that can be retried.
Conditional compilation of Perl logic in
Merge cipher.h into crypto.h.
Remove extraneous use/include statements.
Remove embedded semicolon from String constant macros.
Reduce debug level for infoIni() to test.
Return IoFilterGroup * from ioFilterGroupAdd().
Add coding standards for String constants.
Add missing LOG_DEBUG() macro.
Clarify when
Clarify that region/endpoint must be configured correctly for the bucket.
Add documentation for building the documentation.
Add admonitions to all documentation renderers (HTML, PDF, Markdown and help text) and update
Add HTML table rendering and update PDF/Markdown renderers to support header-less tables. Add optional table captions.
Escape special characters in latex when not in a code block.
Base menu ordering on natural ordering in the manifest.
Replace keywords with more flexible if statements.
Pre-build containers for any
Documentation may be built with user-specified packages.
Add Centos/RHEL 7 option to documentation build.
Allow custom logo for PDF documentation.
Modify general document elements to allow any child element.
Use absolute paths so that
Pick pg_switch_wal()/pg_switch_xlog() based on
Add configuration to the standby so it works as a primary when promoted.
Create common
Add
New test containers with static test certificates.
Fix test binary name for
Allow arbitrary multiplier and flush character in IoTestFilterMultiply.
Update URL for Docker install.
Add testRepoPath() to let C unit tests know where the code repository is located.
Merge
Merge
Fix issue with
Fix static WAL segment size used to determine if
Fix error after log file open failure when processing should continue.
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected.
Retry all S3
Correct current history item in InfoPg to always be in position 0.
Make ioReadLine() read less aggressively.
Add ioWriteFlush() to flush pending output.
Add destructors to IoRead and IoWrite objects.
Add
Add lstInsert() to List object.
Add strCatChr(), strEmpty(), and constant macros to String object.
Add regExpPrefix() to aid in static prefix searches.
Correct cfgDefDataFind() to use UINTP instead of VOIDPP.
Change infoArchiveCheckPg() to display the
Allow storage path and file mode to be 0.
Limit usable Buffer size without changing allocated size.
Construct Wait object in milliseconds instead of fractional seconds.
Add THROW*_ON_SYS_ERROR* macros to test and throw system errors.
Add KernelError to report miscellaneous kernel errors.
Use THROW_ON_SYS_ERROR macro to improve fork code coverage.
Storage interface methods no longer declare the driver as const.
Add memContextCallbackClear() to prevent double free() calls.
Merge
Add cryptoError() and update crypto code to use it.
Rename CipherError to CryptoError.
Reword misleading message in stack trace when parameter buffer is full.
Add logging macros for TimeMSec type.
Modify value of PERL_EMBED_ERROR macro.
Add new HTML tags and strExtra to DocHtmlElement.
Remove error suppression for
New test containers. Add libxml2 library needed for S3 development. Include new minor version upgrades for
Test speed improvements. Mount
New, concise coverage report for C.
Add TEST_LOG() and TEST_LOG_FMT() macros.
Improve alignment of expected vs. actual error test results.
Add time since the beginning of the run to each test statement.
Fix missing URI encoding in S3 driver.
Fix incorrect error message for duplicate options in configuration files.
Fix incorrectly reported error return in
A return code of 1 from the
Add checksum delta for incremental backups.
Checksum delta backups uses checksums rather than timestamps to determine if files have changed.
Ignore all files in a linked tablespace directory except the subdirectory for the current version of
Previously an error would be generated if other files were present and not owned by the
Improve
Improve support for special characters in filenames.
Allow
Migrate local, unencrypted, non-S3
Storage refactoring. Posix file functions now differentiate between open and missing errors. Don't use negations in objects below Storage. Rename posix driver files/functions for consistency. Full abstraction of storage driver interface. Merge protocol storage helper into storage helper. Add CIFS driver to storage helper for read-only repositories.
Update all interfaces to use variable parameter constructors.
Info objects now parse JSON and use specified storage.
Add ioReadLine()/ioWriteLine() to IoRead/IoWrite objects.
Add helper for repository storage.
Add cryptoHmacOne() for HMAC support.
Add cfgDefOptionMulti() to identify multi-value options.
Add bufNewZ() and bufHex() to Buffer object.
Allow hashSize() to run on remote storage.
Restore bIgnoreMissing flag in backupFile() lost in storage refactor.
Migrate walIsPartial(), walIsSegment(), and walSegmentFind() from Perl to C.
Migrate control functions to detect stop files to C.
Make archive-get info messages consistent between C and Perl implementations.
Change locking around async process forking to be more test friendly.
Simplify debug logging by allowing log functions to return String objects.
Improve documentation in filter.h and filter.internal.h.
Use
List allowable values for the
Update introduction to be pithy.
Install
Disable flapping
Move test expect log out of the regular test directory.
Fix buffer underrun in configuration test harness.
Fix missing test caused by a misplaced YAML tag.
Make Valgrind return an error even when a non-fatal issue is detected. Update some minor issues discovered in the tests as a result.
Add
Clear test directory between test runs.
Allow C or Perl coverage to run on more than one VM.
Don't perform valgrind when requested.
Remove compiler warnings that are not valid for u16.
Merge all posix storage tests into a single unit.
Add
Fix typo in unit test error messages, EXECTED => EXPECTED.
Make comment blocks consistent across all tests.
Fix issue where relative links in
This issue did not affect absolute links and relative tablespace links were caught by other checks.
Remove incompletely implemented
Offline operation runs counter to the purpose of this command, which is to check if archiving and backups are working correctly.
Fix issue where errors raised in C were not logged when called from Perl.
Fix issue when a boolean option (e.g.
Allow any option to be set in an environment variable.
This includes options that previously could only be specified on the command line, e.g.
Exclude temporary and unlogged relation (table/index) files from backup.
Implemented using the same logic as the patches adding this feature to
Allow arbitrary directories and/or files to be excluded from a backup.
Misuse of this feature can lead to inconsistent backups so read the
Add
Allow zero-size files in backup manifest to reference a prior manifest regardless of timestamp delta.
Improve asynchronous
Improve error message when a command is missing the
Validate configuration options in a single pass. By pre-calculating and storing the option dependencies in
Add gzip compression/decompression filters for C.
Improve performance of string to int conversion. Use strtoll() instead of sprintf() for conversion. Also use available integer min/max constants rather than hard-coded values.
Add uint64 variant type and supporting conversion functions.
Add basic C JSON parser.
Migrate minimum set of code for reading
Allow Buffer object used size
to be different than allocated size
. Add functions to manage used size and remaining size and update automatically when possible.
Abstract IO layer out of the storage layer. This allows the routines to be used for IO objects that do not have a storage representation. Implement buffer read and write IO objects. Implement filters and update cryptoHash to use the new interface. Implement size and buffer filters.
storageFileRead() accepts a buffer for output rather than creating one. This is more efficient overall and allows the caller to specify how many bytes will be read on each call. Reads are appended if the buffer already contains data but the buffer size will never increase.
Add iniSectionList() to Ini object and remove dead code.
Manifest improvements. Require Manifest object (and not loading it from disk). Prevent manifest from being built more than once. Limit manifest build recursion (i.e. links followed) to sixteen levels to detect link loops.
Do nothing in memContextMove() when the context is already in the specified parent.
Allow command/option constants to autonumber in both C and Perl to reduce churn when a new command/option is added.
Show exact log level required for stack trace param output instead of just debug
.
Update Archive::Info->archiveIdList() to return a valid error code instead of unknown.
Add cvtBoolToConstZ() to simplify conversion of boolean to string.
Add cvtZToUInt() to convert string to unsigned int.
Enable
Rename error-handling variables in Main.pm to conform to standard.
Remove redundant lines from embedded Perl by combining blank lines.
Define cipher magic size with sizeof() rather than using a constant.
Add cvtCharToZ() and macro for debugging char params.
Add strReplaceChr() to String object.
Correct
Fix invalid log level in
Stop trying to arrange contributors in
Contributor names have always been presented in the release notes exactly as given, but we tried to assign internal IDs based on last/first name which can be hard to determine and ultimately doesn't make sense. Inspired by Christophe's Human Beings Do Not Have a Primary Key
.
Allow containers to be defined in a document. The defined containers are built before the document build begins which allows them to be reused.
Move most host setup to containers defined in the documentation. This includes
Error if LibC build is performed outside the test environment.
LibC is no longer required for production builds.
Use pre-built images from Docker Hub when the container definition has not changed. Downloading an image is quite a bit faster than building a new image from scratch and saves minutes per test run in CI.
Refactor the common/log tests to not depend on common/harnessLog. common/harnessLog was not ideally suited for general testing and made all the tests quite awkward. Instead, move all code used to test the common/log module into the logTest module and repurpose common/harnessLog to do log expect testing for all other tests in a cleaner way. Add a few exceptions for config testing since the log levels are reset by default in config/parse.
Add
Improve error reporting for TEST_ASSIGN() and TEST_RESULT_VOID() macros.
Update code count for new file types and exclusions.
IMPORTANT NOTE: This release fixes a critical bug in the backup resume feature. All resumed backups prior to this release should be considered inconsistent. A backup will be resumed after a prior backup fails, unless aborted backup of same type exists, will be cleaned to remove invalid files and resumed
. If the message exists, do not use this backup or any backup in the same set for a restore and check the restore logs to see if a resumed backup was restored. If so, there may be inconsistent data in the cluster.
Fix critical bug in resume that resulted in inconsistent backups.
A regression in
Fix error in selective restore when only one user database exists in the cluster.
Fix non-compliant ISO-8601 timestamp format in S3 authorization headers.
AWS and some gateways were tolerant of space rather than zero-padded hours while others were not.
Improve the HTTP client to set
S3 (and gateways) always set
Set search_path = 'pg_catalog' on
Move cryptographic hash functions to C using
Split log levels into separate header file. Many modules that use debug.h do not need to do logging so this reduces dependencies for those modules.
Auto-generate Makefile with dependencies.
Rename cipher module to the more general crypto.
Update Debian package to add debug symbols to
Convert the not very portable uint type to unsigned int.
Create a new section to describe building
Add sample S3 policy to restrict bucket privileges.
Fix default location of
Preliminary documentation for
Remove call to
Build containers from scratch for more accurate testing. Use a prebuilt s3 server container.
Document generator improvements. Allow parameters to be passed when a container is created. Allow
Add zero-length file to
Update primary test environment (Vagrant and Docker) to Ubuntu 18.04.
Improve efficiency of C library builds now that they are used only for testing.
Remove RHEL and Debian package patches since they have been committed upstream.
Update parameters for
Make
Fix potential buffer overrun in error message handling.
Fix archive write lock being taken for the synchronous
Embed exported C functions and Perl modules directly into the
Use time_t instead of __time_t for better portability.
Print total runtime in milliseconds at command end.
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e.
Build
Test harness improvements. Allow more than one test to provide coverage for the same module. Add option to disable valgrind. Add option to disabled coverage. Add option to disable debug build. Add option to disable compiler optimization. Add
Update SSL error message test on CentOS 7.
Set
Fix directory syncs running recursively when only the specified directory should be synced.
Fix path not found
error for incr/diff backups.
Fix failure in manifest build when two or more files in
Fix delta restore failing when a linked file is missing.
Fix rendering of key/value and list options in help.
Add asynchronous, parallel
This feature maintains a queue of WAL segments to help reduce latency when
Add support for additional
The directory is specified by the
Add
Update the
Make backup/restore path sync more efficient.
Scanning the entire directory can be very expensive if there are a lot of small tables. The backup manifest contains the path list so use it to perform syncs instead of scanning the backup/restore path.
Show command parameters as well as command options in initial info log message.
Rename archive-queue-max option to archive-push-queue-max.
This is consistent with the new
Make
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
Improved lock implementation written in C. Now only two types of locks can be taken:
Storage object improvements. Convert all functions to variadic functions. Enforce read-only storage. Add storageLocalWrite() helper function. Add storageCopy(), storageExists(), storageMove(), storageNewRead()/storageNewWrite(), storagePathCreate(), storagePathRemove(), storagePathSync(), and storageRemove(). Add StorageFileRead and StorageFileWrite objects. Abstract Posix driver code into a separate module. Call storagePathRemove() from the Perl Posix driver.
Improve String and StringList objects. Add strUpper(), strLower(), strLstExists(), strLstExistsZ(), strChr(), strSub(), strSubN(), and strTrunc().
Improve Buffer object. Add bufNewC(), bufEq() and bufCat(). Only reallocate buffer when the size has changed.
Add pgControlInfo() to read
Add walSegmentNext() and walSegmentRange().
Error handling improvements. Add THROWP_* macro variants for error handling. These macros allow an ErrorType pointer to be passed and are required for functions that may return different errors based on a parameter. Add _FMT variants for all THROW macros so format types are checked by the compiler.
Split cfgLoad() into multiple functions to make testing easier. Mainly this helps with unit tests that need to do log expect testing.
Allow MemContext objects to be copied to a new parent. This makes it easier to create objects and then copy them to another context when they are complete without having to worry about freeing them on error. Update List, StringList, and Buffer to allow moves. Update Ini and Storage to take advantage of moves.
Full branch coverage in C code.
Refactor common/time.c. The implementation provides
Replace THROW_ON_SYS_ERROR() with THROW_SYS_ERROR(). The former macro was hiding missing branch coverage for critical error handling.
Start work on C handle io object and use it to output help.
Don't copy
Split debug and assert code into separate headers. Assert can be used earlier because it only depends on the error-handler and not logging. Add ASSERT() macro which is preserved in production builds.
Cleanup C types. Remove typec.h. Order all typdefs above local includes.
Fix header exclusion defines that do not match the general pattern.
Update docs with 32-bit support and caveats.
32-bit support was added in
Add monitoring examples using
Add example of command section usage to archiving configuration.
Remove documentation describing info --output=json as experimental.
Update out-of-date description for the
Add logic to find the real oid of the
Document build improvements. Perform apt-get update to ensure packages are up to date before installing. Add
Use
Switch from
Move test definitions to
Move help/version integration tests to
Move
Add harnessCfgLoad() test function, which allows a new config to be loaded for unit testing without resetting log functions, opening a log file, or taking locks.
Add HARNESS_FORK macros for tests that require fork(). A standard pattern for tests makes fork() easier to use and should help prevent some common mistakes.
Add TEST_ERROR_FMT macro to simplify testing of formatted error messages.
Generate code counts for all source files. The source files are also classified by type and purpose.
Include VM type in
Improve logic for smart builds to include version changes. Skip version checks when testing in
Use
Allow -DDEBUG_UNIT to be suppressed to test how debug macros behave.
Rename Perl tests so they don't conflict with their C counterparts.
Divide tests into three types (
Try tweaking time sync settings to prevent clock drift rather than restarting
Fix
Immediately error when a secure option (e.g.
Since
Fix issue passing
Fix issue where specifying
The max value for a
Fix manifest test in the
Error when multiple arguments are set in the config file for an option that does not accept multiple arguments.
Remove extraneous sudo commands from
Improve Perl configuration. Set config before Main::main() call to avoid secrets being exposed in a stack trace. Move logic for setting defaults to C.
Improve logging. Move command begin to C except when it must be called after another command in Perl (e.g.
Port most of Config::Config::configLoad() from Perl to C.
Fix incorrect enum types in
Enable
Improve code documentation in config module.
Improve debugging. Add ASSERT_DEBUG() macro for debugging and replace all current assert() calls except in tests that can't use the debug code. Replace remaining NDEBUG blocks with the more granular DEBUG_UNIT. Remove some debug memset() calls in MemContext since valgrind is more useful for these checks.
Add cfgOptionTest() and update cfgOption() calls that are better implemented as cfgOptionTest().
Build with
Check int size in common/type.h. This ensures that integers are at least 32-bits without having to run the test suite.
Improve conversion of C exceptions to Exception objects. Colons in the message would prevent all of the message from being loaded into the Exception object.
Show index in examples for indexed options, i.e.
Simplify table of contents on command page by only listing commands.
Remove references to the C library being optional.
Add CentOS/RHEL package builds.
Use clang for static code analysis.
Nothing found initially except for some functions that should have been marked __noreturn__.
Build performance improvements. Improve bin and libc build performance. Improve code generation performance.
Config test code writes secure options to a file instead of passing on the command-line.
Disable console display of coverage for C files since Devel::Cover does not handle it well.
Add new test for Common::Io::Process to show that output on stderr will raise an exception on close() even if the exit code is 0.
Update
Remove
Remove Debian package patch now that it has been merged upstream.
The
Improve
Improve performance of HTTPS client. Buffering now takes the pending bytes on the socket into account (when present) rather than relying entirely on select(). In some instances the final bytes would not be flushed until the connection was closed.
Improve S3 delete performance. The constant
Allow any non-command-line option to be reset to default on the command-line. This allows options in
The C library is now required. This eliminates conditional loading and eases development of new library features.
The
Rename
Implement
Implement
Config parsing implemented in C and passed to Perl as JSON.
Add Buffer, Ini, KeyValue, List, RegExp, Storage, String, StringList, Variant, VariantList, and Wait objects.
Add command, exit, log, and time modules.
Remove deprecated
Improve MemContext module. Add temporary context blocks and refactor allocation arrays to include allocation size.
Improve error module. Add functions to convert error codes to C errors and handle system errors.
Create a master list of errors in Build.pm so they stay up to date.
Move lock release later in exitSafe() to reduce the chance of a new process starting and acquiring a lock before the old process has exited.
Add 30 second wait loop to lockAcquire() when fail on no lock enabled. This should help prevent processes that are shutting down from interfering with processes that are starting up.
Replace cfgCommandTotal()/cfgOptionTotal() functions with constants. The constants are applicable in more cases and allow the compiler to optimize certain loops more efficiently.
Cleanup usage of internal options. Apply internal to options that need to be read to determine locality but should not appear in the help.
Refactor code to make valgrind happy.
Fix non-compliant formatting for function declarations.
All clusters in the documentation are initialized with checksums.
List deprecated option names in documentation and command-line help.
Clarify that S3 buckets must be created by the user.
Add coding standards document.
Improve section source feature to not require a title or content. The title will be pulled from the source document.
Allow code blocks to have a type. Currently this is only rendered in Markdown.
Add table render for Markdown format.
PDF rendering improvements. Check both code-block title.
Add host-add. The
Deploy historical documentation to
Run valgrind on all C unit tests.
Only build C binary/library for Perl unit/integration tests or C unit tests that require Perl.
Improve speed of C unit tests. Preserve object files between tests and use a Makefile to avoid rebuilding object files.
Report coverage errors via the console. This helps with debugging coverage issues on remote services like Travis.
No longer run
Rename Perl tests so they don't conflict with their C counterparts.
Update URL for Debian package repository.
IMPORTANT NOTE: This release fixes a critical bug in the backup resume feature. All resumed backups prior to this release should be considered inconsistent. A backup will be resumed after a prior backup fails, unless aborted backup of same type exists, will be cleaned to remove invalid files and resumed
. If the message exists, do not use this backup or any backup in the same set for a restore and check the restore logs to see if a resumed backup was restored. If so, there may be inconsistent data in the cluster.
Fix critical bug in resume that resulted in inconsistent backups. A regression in
Fix non-compliant ISO-8601 timestamp format in S3 authorization headers. AWS and some gateways were tolerant of space rather than zero-padded hours while others were not.
Fix directory syncs running recursively when only the specified directory should be synced.
Fix
Fix path not found
error for incr/diff backups.
Fix failure in manifest build when two or more files in
Fix delta restore failing when a linked file was missing.
Fix error in selective restore when only one user database exists in the cluster.
Improve the HTTP client to set
Improve performance of HTTPS client. Buffering now takes the pending bytes on the socket into account (when present) rather than relying entirely on select(). In some instances the final bytes would not be flushed until the connection was closed.
Improve S3 delete performance. The constant
Make backup/restore path sync more efficient. Scanning the entire directory can be very expensive if there are a lot of small tables. The backup manifest contains the path list so use it to perform syncs instead of scanning the backup/restore path. Remove recursive path sync functionality since it is no longer used.
Make
Add log-level-stderr option for stanza-* commands.
Update docs with 32-bit support and caveats. 32-bit support was added in
Clarify that S3 buckets must be created by the user.
Update out-of-date description for the
Remove call to
Add new test for Common::Io::Process to show that output on stderr will raise an exception on close() even if the exit code is 0.
Add zero-length file to
Disable package build tests since
Update SSL error message test on CentOS 7.
Update URL for Debian package repository.
Make
Change backup test user from
Fixed inability to restore a single database contained in a tablespace using --db-include.
Ensure latest
Fixed overly chatty error message when reporting an invalid command.
Add
Improve
Minor changes to Manifest module, mostly for test reproducibility.
Fix non-compliant formatting for function declarations.
Update
Add unit tests for the Manifest module.
Fixed an issue that suppressed locality errors for
Fixed an issue where WAL was not expired on
Fixed an issue where the
Fixed "db (prior)" output if no backups or archives exist for a prior version of the cluster.
Add memGrowRaw() to memory context module.
Document the relationship between the
Improve
Relax permissions set by
Split refactor
sections into improvements
and development
in the release notes. Many development notes are not relevant to users and simply clutter the release notes, so they are no longer shown on the website.
Allow internal options that do not show up in the documentation. Used for test options initially but other use cases are on the horizon.
Update CI branches to
No longer run
Move restore test infrastructure to HostBackup.pm. Required to test restores on the backup server, a fairly common scenario. Improve the restore function to accept optional parameters rather than a long list of parameters. In passing, clean up extraneous use of strType and strComment variables.
Sync time to prevent build failures when running on VirtualBox.
Fixed an issue that could cause copying large manifests to fail during restore.
Fixed incorrect WAL offset for 32-bit architectures.
Fixed an issue retrieving WAL for old database versions. After a
Fixed an issue with invalid backup groups being set correctly on restore. If the backup cannot map a group to a name it stores the group in the manifest as
Fixed an issue passing parameters to remotes. When more than one db was specified the path, port, and socket path would for db1 were passed no matter which db was actually being addressed.
Repository encryption support.
Disable gzip filter when
Inflate performance improvement for gzip filter.
Refactor protocol param generation into a new function. This allows the code to be tested more precisely and doesn't require executing a remote process.
Add
Remove configurable option hints.
Convert configuration definitions from auto-generated functions to auto-generated data structures.
Add
Enable additional warnings for C builds.
Simplify try..catch..finally names. Also wrap in a do...while loop to make sure that no random else is attached to the main if block.
Improve base64 implementation. Different encoded strings could be generated based on compiler optimizations. Even though decoding was still successful the encoded strings did not match the standard.
Disable
Add template to improve initial information gathered for issue submissions.
Clarify usage of the
Update
Automated tests for 32-bit i386/i686 architecture.
Update Debian/Ubuntu containers to download latest version of
Full unit test coverage for gzip filter.
Only check expect logs on CentOS 7. Variations in distros cause false negatives in tests but don't add much value.
Fix flapping protocol timeout test. It only matters that the correct error code is returned, so disable logging to prevent message ordering from failing the expect test.
Designate a single distro (Ubuntu 16.04) for coverage testing. Running coverage testing on multiple distros takes time but doesn't add significant value. Also ensure that the distro designated to run coverage tests is one of the default test distros. For C tests, enable optimizations on the distros that don't do coverage testing.
Automate generation of WAL and
Refactor C unit test macros so they compile with
Refactor C page checksum unit test to compile with
Fix custom settings for
Remove error when overlapping timelines are detected. Overlapping timelines are valid in many Point-in-Time-Recovery (PITR) scenarios.
Fix instances where
Improve performance of list requests on S3. Any beginning literal portion of a filter expression is used to generate a search prefix which often helps keep the request small enough to avoid rate limiting.
Improve protocol error handling. In particular, stop
errors are no longer reported as unexpected
.
Allow functions with sensitive options to be logged at debug level with redactions. Previously, functions with sensitive options had to be logged at trace level to avoid exposing them. Trace level logging may still expose secrets so use with caution.
Replace dynamically built class hierarchies in I/O layer with fixed parent() calls.
Improve labeling for errors in helper processes.
Update C naming conventions.
Use
Better separation of C source from Perl interface.
Add
Add C error handler.
Perl error handler recognizes errors thrown from the C library.
Page checksum module uses new C error handler.
Add C memory contexts.
Add base64 encode/decode.
Add I/O performance tests.
Add C unit test infrastructure.
Add test macros for C results and errors.
Warnings in C builds treated as errors.
Run all tests on tempfs rather than local disk.
Improve performance of test code. Wait when all tests have been assigned to reduce CPU load.
Remove Debian test repo after PostgreSQL 10 release.
Convert config and page checksum tests into C unit tests.
Add
Fixed an issue where warnings were being emitted in place of lower priority log messages during backup from standby initialization.
Fixed an issue where some
Exclude contents of
Exclude
Open log file after async process is completely separated from the main process to prevent the main process from also logging to the file.
Dynamically generate list of files for C library build.
Break up
Add passwordless SSH configuration.
Rename
Add full installation where required and remove doc containers that included parts of the installation.
Improve C library smart build by ignoring changes outside of
Fixed an issue that could cause compression to abort on growing files.
Fixed an issue with keep-alives not being sent to the remote from the local process.
Up to seven standbys can be configured for backup from standby.
Allow
Increase HTTP timeout for S3.
Add HTTP retries to harden against transient S3 network errors.
Configuration definitions are now pulled from the C library when present.
Fixed document generation to include section summaries on the Configuration page.
Move contributor list to the end of
Change log test order to ignore unimportant log errors while shutting down
Drain
Update Docker build in
Update containers to support C library builds in the documentation.
Simplify smart logic for C Library and package builds.
Fixed authentication issue in S3 retry.
The
Fixed an issue that could cause the empty archive directory for an old
Modified the
Added
Retry when S3 returns an internal error (500).
Add Local->manifest().
Fix description of
Add creation of
Move repository options into a separate section in command/command-line help.
Reduce log verbosity when building documentation by only logging sections that contain an execute list directly or in a child section.
Debian/Ubuntu documentation now builds on Ubuntu 16.
Remove vestigial repository options from
Fix log checking after
Use Google DNS in test environment for consistency.
Use new Travis Trusty image.
Generate global fake cert in containers for testing.
Consolidate
IMPORTANT NOTE:
Fixed an issue that prevented tablespaces from being backed up on
Fixed missing flag in C library build that resulted in a mismatched binary on 32-bit systems.
Add IO::Socket::SSL, i.e.
Harden protocol handshake to handle race conditions.
Fixed misleading error message when a file was opened for write in a missing directory.
Change log level of hardlink logging to
Cast size in S3 manifest to integer.
Rename Archive modules to remove redundancy.
Improve
Minor optimizations to package loads and ordering for
Remove exhaustive version list from Stable Releases TOC.
Improve
Update
Remove cache-key because it can vary by system.
Add documentation builds to CI.
Fix timeouts in ExecuteTest to speed multi-process testing.
Remove patch directory before Debian package builds.
Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage.
Split
Consolidate
Eliminate redundancy in
Install
More optimized container suite that greatly improves build time.
Added static Debian packages for Devel::Cover to reduce build time.
Add
Remove
Add
Remove
Balance database versions between VMs to minimize test duration.
Automatically check that all supported
Add
Fixed the
Fixed the
Fixed config warnings raised from a remote process causing errors in the master process.
Refactor storage layer to allow for new repository filesystems using drivers.
Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters.
Move modules in
Move backup modules into
Changed invalid
Fixed missing sudo in installation section.
Fixed an undefined variable when a module had no uncoverable code exceptions.
Fixed issue with
Moved test and env modules to new directories to avoid namespace conflicts with common tests.
Set
Remove flapping protocol timeout test that will be replaced in the upcoming storage patch.
Fixed an issue where read-only operations that used local worker processes (i.e.
Added the stanza-upgrade command to provide a mechanism for upgrading a stanza after upgrading to a new major version of
Added validation of
Simplify locking scheme. Now, only the master process will hold write locks (for
Do not set timestamps of files in the backup directories to match timestamps in the cluster directory. This was originally done to enable backup resume, but that process is now implemented with checksums.
Improved error message when the
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions.
Refactor Ini.pm to facilitate testing.
The
Allow functions to accept optional parameters as a hash.
Refactor File->list() and fileList() to accept optional parameters.
Refactor backupLabel() and add unit tests.
Silence some perl critic warnings.
Update wording for release note sections.
Ignore clock skew in container libc/package builds using make. It is common for containers to have clock skew so the build process takes care of this issue independently.
Complete statement/branch coverage for Ini.pm.
Improved functions used to test/munge manifest and info files.
Coverage testing always enabled on Debian-based containers.
Require description in every call to testResult().
Make iWaitSeconds an optional parameter for testResult().
Updated vagrant to new version and image.
Fixed flapping archive stop tests.
Added ability to test warning messages.
Fixed an issue where newly initialized (but unused) pages would cause page checksum warnings.
Fixed an issue where tables over 1GB would report page checksum warnings after the first segment.
Fixed an issue where databases created with a non-default tablespace would raise bogus warnings about
Improved the code and tests for fileManifest() to prevent a possible race condition when files are removed by the database while the manifest is being built.
Container executions now load the user's environment.
Continuous integration using
Automated builds of Debian packages for all supported distributions.
Added
Added
Added
C library and packages are built by default, added
The
Improved formatting of testResult() output.
Improved truncation when outputting errors logs in the ExecuteTest module.
Fixed flapping archive-stop test with testResult() retries.
Added final test of archive contents to archive-push test.
Temporarily disable flapping keep-alive test.
Fixed a regression introduced in
Refactor FileCommon::fileManifest() and FileCommon::fileStat to be more modular to allow complete branch/statement level coverage testing.
Complete branch/statement level coverage testing for FileCommon::fileManifest() and FileCommon::fileStat functions and helper functions.
Fixed an issue where an archive-push error would not be retried and would instead return errors to
Fixed a race condition in parallel archiving where creation of new paths generated an error when multiple processes attempted to do so at the same time.
Improved performance of
Updated async archiving documentation to more accurately describe how the new method works and how it differs from the old method.
Documentation can now be built with reusable blocks to reduce duplication.
Improved support for
Added ability to pass options to containers within the documentation.
Add proper tag to slightly emphasize proper nouns.
IMPORTANT NOTE: The new implementation of asynchronous archiving no longer copies WAL to a separate queue. If there is any WAL left over in the old queue after upgrading to pgbackrest --stanza=[stanza-name] archive-push and wait for the process to complete. Check that the queue in 1.13 and restore the original
Fixed const assignment giving compiler warning in C library.
Fixed a few directory syncs that were missed for the
Fixed an issue where a missing user/group on restore could cause an uninitialized value
error in File->owner().
Fixed an issue where protocol mismatch errors did not output the expected value.
Fixed a spurious
Improved, multi-process implementation of asynchronous archiving.
Improved
Improved pg_switch_xlog() cannot be executed on a replica.
Added archive and backup WAL ranges to the
Added warning to update pg_tablespace.spclocation when remapping tablespaces in
Remove remote lock requirements for the
Log file banner is not output until the first log entry is written.
Reduced the likelihood of torn pages causing a false positive in page checksums by filtering on start backup LSN.
Remove Intel-specific optimization from C library build flags.
Remove
Added
Return proper error code when unable to convert a relative path to an absolute path.
Refactor File and BackupCommon modules to improve test coverage.
Moved File->manifest() into the FileCommon.pm module.
Moved the Archive modules to the
Split the Archive.pm module.
Allow logging to be suppressed via logDisable() and logEnable().
Allow for locks to be taken more than once in the same process without error.
Lock directories can be created when more than one directory level is required.
Clean up optionValid()/optionTest() logic in Lock.pm.
Added Exception::exceptionCode() and Exception::exceptionMessage() to simplify error handling logic.
Represent
Allow empty files to be created with FileCommon::fileStringWrite() and use temp files to avoid partial reads.
Refactor process IO and process master/minion code out from the common protocol code.
Fixed alignment issues with multiline logging.
Added documentation to the User Guide for the
Update LICENSE.txt for 2017.
Fixed
Added integration for testing coverage with Devel::Cover.
Added unit tests for low-level functions in the File and BackupCommon modules.
C Library builds only run when C library has actually changed.
Added more flexibility in initializing and cleaning up after modules and tests.
testResult() suppresses logging and reports exceptions.
testException() allows messages to be matched with regular expressions.
Split test modules into separate files to make the code more maintainable. Tests are dynamically loaded by name rather than requiring an if-else block.
Allow multiple
Added expect log expression to replace year subdirectories in
Refactor name/locations of common modules that setup test environments.
IMPORTANT NOTE: In prior releases it was possible to specify options on the command-line that were invalid for the current command without getting an error. An error will now be generated for invalid options so it is important to carefully check command-line options in your environment to prevent disruption.
Fixed an issue where options that were invalid for the specified command could be provided on the command-line without generating an error. The options were ignored and did not cause any change in behavior, but it did lead to some confusion. Invalid options will now generate an error.
Fixed an issue where internal symlinks were not being created for tablespaces in the repository. This issue was only apparent when trying to bring up clusters in-place manually using filesystem snapshots and did not affect normal backup and restore.
Fixed an issue that prevented errors from being output to the console before the logging system was initialized, i.e. while parsing options. Error codes were still being returned accurately so this would not have made a process look like it succeeded when it did not.
Fixed an issue where the
Added the
Added the
Added the
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
For simplicity, the
Abstracted code to determine which database cluster is the primary and which are standbys.
Improved consistency and flexibility of the protocol layer by using JSON for all messages.
File copy protocol now accepts a function that can do additional processing on the copy buffers and return a result to the calling process.
Improved IO->bufferRead to always return requested number of bytes until EOF.
Simplified the result hash of File->manifest(), Db->tablespaceMapGet(), and Db->databaseMapGet().
Improved errors returned from child processes by removing redundant error level and code.
Code cleanup in preparation for improved
Improved parameter/result logging in debug/trace functions.
Fixed an issue that suppressed exceptions in PDF builds.
Fixed regression in section links introduced in
Added Retention to QuickStart section.
Allow a source to be included as a section so large documents can be broken up.
Added section link support to Markdown output.
Added list support to PDF output.
Added
Added
Added
Added
Builds in
Improvements to markdown rendering.
Remove code dependency on
Removed erroneous
Update control and WAL test files to
Improved exception handling in file unit tests.
Changed the
Various improvements to validation of backup and restore.
Add more realistic data files to synthetic backup and restore tests.
Fixed an issue where asynchronous archiving was transferring one file per execution instead of transferring files in batches. This regression was introduced in
Fixed an issue where a backup could error if no changes were made to a database between backups and only
Fixed an issue where tablespace paths with the same prefix would cause an invalid link error.
Added the
Removed extraneous use lib directives from Perl modules.
Fixed missing variable replacements.
Removed hard-coded host names from configuration file paths.
Allow command-line length to be configured using
Added
Added
Only generate HTML menu when required and don't require index page.
Assign numbers to sections by default.
VM mount points are now optional.
Fixed the
Fixed error message to properly display the archive command when an invalid archive command is detected.
Fixed an issue where the async archiver would not be started if
Fixed a remote timeout that occurred when a local process generated checksums (during resume or restore) but did not copy files, allowing the remote to go idle.
Non-exclusive backups will automatically be used on
Added the
Added the
Set
Check that archive_mode is enabled when
Clarified error message when unable to acquire
Process ID logged for local process start/stop INFO log output.
Added
Added
Update CentOS/Debian package definitions.
Fixed missing expect output for help module.
Fixed broken
Regression tests can now be run as any properly-configured user, not just vagrant.
Minimize TeXLive package list to save time during VM builds.
Fixed an issue where local processes were not disconnecting when complete and could later timeout.
Fixed an issue where the protocol layer could timeout while waiting for WAL segments to arrive in the archive.
Cache file log output until the file is created to create a more complete log.
Show Process ID in output instead of filtering it out with the timestamp.
Suppress dpkg-reconfigure: unable to re-open stdin: No file or directory
warning in Vagrant VM build.
Show Process ID in expect logs instead of filtering it out with the timestamp.
Fixed an issue where tablespaces were copied from the primary during standby backup.
Fixed the
Fixed an issue where
Converted Perl threads to processes to improve compatibility and performance.
Exclude contents of
The
Additional warnings when archive retention settings may not have the intended effect or would allow indefinite retention.
Experimental support for non-exclusive backups in
Refactor of protocol minions in preparation for the new local minion.
Remove obsolete thread index variable from File() module.
Changed temporary file names to consistently use the
Improve ASSERT error handling, safely check eval blocks, and convert $@ to $EVAL_ERROR.
Fixed minor documentation reproducibility issues related to binary paths.
Documentation for archive retention.
Suppress TOC for unsupported versions of
New vagrant base box and make uid/gid selection for containers dynamic.
Fixed an issue where a tablespace link that referenced another link would not produce an error, but instead skip the tablespace entirely.
Fixed an issue where options that should not allow multiple values could be specified multiple times in
Fixed an issue where the
Backup from a standby cluster. A connection to the primary cluster is still required to start/stop the backup and copy files that are not replicated, but the vast majority of files are copied from the standby in order to reduce load on the primary.
More flexible configuration for databases. Master and standby can both be configured on the backup server and
Exclude directories during backup that are cleaned, recreated, or zeroed by
Experimental support for non-exclusive backups in
Improve error message for links that reference links in manifest build.
Added hints to error message when relative paths are detected in
Improve backup log messages to indicate which host the files are being copied from.
Simplify protocol creation and identifying which host is local/remote.
Removed all OP_* function constants that were used only for debugging, not in the protocol, and replaced with __PACKAGE__.
Improvements in Db module: separated out connect() function, allow executeSql() calls that do not return data, and improve error handling.
Improve host tag rendering.
Refactor db version constants into a separate module.
Update synthetic backup tests to
Fixed an issue where tablespace paths that had $PGDATA as a substring would be identified as a subdirectories of $PGDATA even when they were not. Also hardened relative path checking a bit.
Added documentation for scheduling backups with cron.
Moved the backlog from the
Improved rendering of spaces in code blocks.
Fixed an issue an where an extraneous remote was created causing threaded backup/restore to possibly timeout and/or throw a lock conflict.
Fixed an issue where db-path was not required for the
Fixed
Fixed an issue where a remote could try to start its own remote when the
Fixed an issue where the contents of
Fixed an invalid log() call in lock routines.
Experimental support for non-exclusive backups in
Suppress banners on SSH protocol connections.
Improved remote error messages to identify the host where the error was raised.
All remote types now take locks. The exceptions date to when the test harness and
Enhancements to the protocol layer for improved reliability and error handling.
Exceptions are now passed back from threads as messages when possible rather than raised directly.
Temp files created during backup are now placed in the same directory as the target file.
Output lock file name when a lock cannot be acquired to aid in debugging.
Reduce calls to protocolGet() in backup/restore.
Added clarification on why the default for the
Updated information about package availability on supported platforms.
Added
HTML footer dates are statically created in English in order to be reproducible.
Fixed a version checking issue in
Fixed an issue where multi-threaded tests were not being run when requested.
Reduce the frequency that certain tests are run to save time in regression.
Disable control master for older OS versions where it is less stable.
Fixed an issue where
Fixed an issue where the
Fixed an issue where validation was being running on
Added
Added the
Failure to shutdown remotes at the end of the backup no longer throws an exception. Instead a warning is generated that recommends a higher
Experimental support for non-exclusive backups in
Improved handling of users/groups captured during backup that do not exist on the restore host. Also explicitly handle the case where user/group is not mapped to a name.
Option handling is now far more strict. Previously it was possible for a command to use an option that was not explicitly assigned to it. This was especially true for the
The pg_xlogfile_name() function is no longer used to construct WAL filenames from LSNs. While this function is convenient it is not available on a standby. Instead, the archive is searched for the LSN in order to find the timeline. If due to some misadventure the LSN appears on multiple timelines then an error will be thrown, whereas before this condition would have passed unnoticed.
Changed version variable to a constant. It had originally been designed to play nice with a specific packaging tool but that tool was never used.
Allow a static date to be used for documentation to generate reproducible builds.
Added documentation for asynchronous archiving to the user guide.
Recommended install location for
Added instructions for removing prior versions of
Fixed DTD search path that did not work properly when
Fixed
Fixed section names being repeated in the info output when multiple
Fixed
Allow hidden options to be added to a command. This allows certain commands (like
Allow command summaries to be inserted anywhere in the documentation to avoid duplication.
Update TeX Live to 2016 version.
New, consolidated implementation for link rendering.
Obsolete containers are removed by the
Major refactor of the test suite to make it more modular and object-oriented. Multiple Docker containers can now be created for a single test to simulate more realistic environments. Tests paths have been renamed for clarity.
Greatly reduced the quantity of Docker containers built by default. Containers are only built for
Fix usage of sprintf() due to new constraints in Perl 5.22. Parameters not referenced in the format string are no longer allowed.
Log directory create and file open now using FileCommon functions which produce more detailed error messages on failure.
Fixed syntax that was not compatible with Perl 5.2X.
Fixed absolute paths that were used for the PDF logo.
Release notes are now broken into sections so that bugs, features, and refactors are clearly delineated. An Additional Notes
section has been added for changes to documentation and the test suite that do not affect the core code.
Added man page generation.
The change log was the last piece of documentation to be rendered in Markdown only. Wrote a converter so the document can be output by the standard renderers. The change log will now be located on the website and has been renamed to Releases
.
Added an execution cache so that documentation can be generated without setting up the full container environment. This is useful for packaging, keeps the documentation consistent for a release, and speeds up generation when no changes are made in the execution list.
Remove function constants and pass strings directly to logDebugParam(). The function names were only used once so creating constants for them was wasteful.
Lists can now be used outside of
Replaced overzealous perl -cW check which failed on Perl 5.22 with perl -cw.
Added Ubuntu 16.04 (Xenial) and Debian 8 (Jessie) to the regression suite.
Upgraded doc/test VM to Ubuntu 16.04. This will help catch Perl errors in the doc code since it is not run across multiple distributions like the core and test code. It is also to be hoped that a newer kernel will make Docker more stable.
Test release version against the executable using
Enhanced text output of
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
Experimental support for non-exclusive backups in
IMPORTANT NOTE: This flag day release breaks compatibility with older versions of
Implemented a new configuration scheme which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the
The
The default configuration filename is now
The default repository name was changed from
Lock files are now stored in
Log files are now stored in
Executable filename changed from
All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The
Removed
Added
Fixed an issue where the master process was passing
IMPORTANT BUG FIX FOR TABLESPACES: A change to the repository format was accidentally introduced in 0.90 which means the on-disk backup was no longer a valid
Fixed repository incompatibility introduced in
Copy
Write
Rename
Static source analysis using Perl-Critic, currently passes on gentle.
Fixed an issue where specifying
Fixed an issue where a temp WAL file left over after a well-timed system crash could cause the next
The
When backing up and restoring tablespaces
Generate an error when
Improved error message when
Added checks for
When restore
Support for
Support for
The following tablespace checks have been added: paths or files in pg_tblspc, relative links in pg_tblspc, tablespaces in $PGDATA. All three will generate errors.
Fixed an issue where document generation failed because some OSs are not tolerant of having multiple installed versions of
Fixed an issue where longer-running backups/restores would timeout when remote and threaded. Keepalives are now used to make sure the remote for the main process does not timeout while the thread remotes do all the work. The error message for timeouts was also improved to make debugging easier.
Allow restores to be performed on a read-only repository by using
Minor styling changes, clarifications and rewording in the user guide.
The dev branch has been renamed to master and for the time being the master branch has renamed to release, though it will probably be removed at some point {[dash]}- thus ends the gitflow experiment for
Fixed an issue where the
Fixed an issue where log files were being overwritten instead of appended.
Fixed an issue where
Symlinks are no longer created in backup directories in the repository. These symlinks could point virtually anywhere and potentially be dangerous. Symlinks are still recreated during a restore.
Added better messaging for backup expiration. Full and differential backup expirations are logged on a single line along with a list of all dependent backups expired.
Archive retention is automatically set to full backup retention if not explicitly configured.
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping
The
Added a new user guide that covers
The website, markdown, and command-line help are now all generated from the same XML source.
Fixed an issue where an error could be returned after a backup or restore completely successfully.
Fixed an issue where a resume would fail if temp files were left in the root backup directory when the backup failed. This scenario was likely if the backup process got terminated during the copy phase.
Added
Experimental support for
Removed dependency on IO::String module.
Fixed an issue where resumed compressed backups were not preserving existing files.
Fixed an issue where resume and incr/diff would not ensure that the prior backup had the same compression and hardlink settings.
Fixed an issue where a cold backup using
Fixed an issue where a thread could be started even when none were requested.
Fixed an issue where the
Fixed an issue where the
Fixed an issue where the pg_stop_backup() NOTICEs were being output to
Experimental support for
Renamed
Added dynamic module loading to speed up commands, especially asynchronous archiving.
Code cleanup and refactoring to standardize on patterns that have evolved over time.
Command-line help is now extracted from the same XML source that is used for the other documentation and includes much more detail.
Expiration tests are now synthetic rather than based on actual backups. This will allow development of more advanced expiration features.
Fixed an issue that caused the formatted timestamp for both the oldest and newest backups to be reported as the current time by the
Fixed protocol issue that was preventing ssh errors (especially on connection) from being logged.
The repository is now created and updated with consistent directory and file modes. By default
Added the
Added the pg_start_backup() and pg_stop_backup() to return.
Remove
Added checks to be sure the
Experimental support for
Now using Perl DBI and DBD::Pg for connections to DBD::Pg on your operating system.
Major refactoring of the protocol layer to support future development.
Split most of database
in the user guide to database cluster
for clarity.
Changed most references to database
in the user guide to database cluster
for clarity.
Added vagrant test configurations for Ubuntu 14.04 and CentOS 7.
Removed dependency on CPAN packages for multi-threaded operation. While it might not be a bad idea to update the threads and Thread::Queue packages, it is no longer necessary.
Modified wait backoff to use a Fibonacci rather than geometric sequence. This will make wait time grow less aggressively while still giving reasonable values.
Added vagrant test configurations for Ubuntu 12.04 and CentOS 6.
More options for regression tests and improved code to run in a variety of environments.
Added file and directory syncs to the File object for additional safety during backup/restore and archiving.
Added support for Perl 5.10.1 and OpenSSH 5.3 which are default for CentOS/RHEL 6.
Improved error message when backup is run without
Removed
Moved version number out of the
Replaced IPC::System::Simple and Net::OpenSSH with IPC::Open3 to eliminate CPAN dependency for multiple operating systems.
IMPORTANT NOTE: This flag day release breaks compatibility with older versions of
Added the
Logging now uses unbuffered output. This should make log files that are being written by multiple threads less chaotic.
Experimental support for
More efficient file ordering for
Fixed an issue where
Fixed an issue in async archiving where
Fixed a locking issue that could allow multiple operations of the same type against a single stanza. This appeared to be benign in terms of data integrity but caused spurious errors while archiving and could lead to errors in backup/restore.
Allow duplicate WAL segments to be archived when the checksum matches. This is necessary for some recovery scenarios.
Allow comments/disabling in
Better logging before
Various command behavior and logging fixes.
Replaced JSON module with JSON::PP which ships with core Perl.
Various help fixes.
Fixed an issue where an absolute path was not written into
Better resume support. Resumed files are checked to be sure they have not been modified and the manifest is saved more often to preserve checksums as the backup progresses. More unit tests to verify each resume case.
Resume is now optional. Use the
More info messages during restore. Previously, most of the restore messages were debug level so not a lot was output in the log.
Added
Fixed a buffering error that could occur on large, highly-compressible files when copying to an uncompressed remote destination. The error was detected in the decompression code and resulted in a failed backup rather than corruption so it should not affect successful backups made with previous versions.
Pushing duplicate WAL now generates an error. This worked before only if checksums were disabled.
Database System IDs are used to make sure that all WAL in an archive matches up. This should help prevent misconfigurations that send WAL from multiple clusters to the same archive.
Improved threading model by starting threads early and terminating them late.
Regression tests working back to
Fixed broken checksums and now they work with normal and resumed backups. Finally realized that checksums and checksum deltas should be functionally separated and this simplified a number of things. Issue #28 has been created for checksum deltas.
Fixed an issue where a backup could be resumed from an aborted backup that didn't have the same type and prior backup.
Added restore functionality.
All options can now be set on the command-line making
De/compression is now performed without threads and checksum/size is calculated in stream. That means file checksums are no longer optional.
Added option
Checksum for
Link
Removed dependency on Moose. It wasn't being used extensively and makes for longer startup times.
More comprehensive unit tests in all areas.
Complete rewrite of BackRest::File module to use a custom protocol for remote operations and Perl native GZIP and SHA operations. Compression is performed in threads rather than forked processes.
Removed dependency on Storable and replaced with a custom ini file implementation.
Numerous other changes that can only be identified with a diff.
Added much needed documentation
Fairly comprehensive unit tests for all the basic operations. More work to be done here for sure, but then there is always more work to be done on unit tests.
Found and squashed a nasty bug where file_copy() was defaulted to ignore errors. There was also an issue in file_exists() that was causing the test to fail when the file actually did exist. Together they could have resulted in a corrupt backup with no errors, though it is very unlikely.
Worked on improving error handling in the File object. This is not complete, but works well enough to find a few errors that have been causing us problems (notably, find is occasionally failing building the archive async manifest when system is under load).
The
If an archive directory which should be empty could not be deleted backrest was throwing an error. There's a good fix for that coming, but for the time being it has been changed to a warning so processing can continue. This was impacting backups as sometimes the final archive file would not get pushed if the first archive file had been in a different directory (plus some bad luck).
Added
Added
Added option to force a checkpoint when starting the backup,
Removed
Changed lock file conflicts on
No restore functionality, but the backup directories are consistent
Archiving is single-threaded. This has not posed an issue on our multi-terabyte databases with heavy write volume. Recommend a large WAL volume or to use the async option with a large volume nearby.
Backups are multi-threaded, but the Net::OpenSSH library does not appear to be 100% thread-safe so it will very occasionally lock up on a thread. There is an overall process timeout that resolves this issue by killing the process. Yes, very ugly.
Checksums are lost on any resumed backup. Only the final backup will record checksum on multiple resumes. Checksums from previous backups are correctly recorded and a full backup will reset everything.
The Storable because Config::IniFile does not seem to handle large files well. Would definitely like to save these as human-readable text.
Absolutely no documentation (outside the code). Well, excepting these release notes.