db/pgbackrest
1
0
Fork 0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2024-12-12 10:04:14 +02:00
Code Issues Releases Activity
4ec51cdb2f
pgbackrest/CONTRIBUTING.md

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

714 lines
37 KiB
Markdown
Raw Normal View History

Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
# pgBackRest <br/> Contributing to pgBackRest
Update contributing documentation and add pull request template.
2021-07-21 19:14:06 +02:00
## Table of Contents
[Introduction](#introduction)
[Building a Development Environment](#building-a-development-environment)
[Coding](#coding)
[Testing](#testing)
[Submitting a Pull Request](#submitting-a-pull-request)
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
## Introduction
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
This documentation is intended to assist contributors to pgBackRest by outlining some basic steps and guidelines for contributing to the project.
Code fixes or new features can be submitted via pull requests. Ideas for new features and improvements to existing functionality or documentation can be [submitted as issues](https://github.com/pgbackrest/pgbackrest/issues). You may want to check the [Project Boards](https://github.com/pgbackrest/pgbackrest/projects) to see if your suggestion has already been submitted.
Bug reports should be [submitted as issues](https://github.com/pgbackrest/pgbackrest/issues). Please provide as much information as possible to aid in determining the cause of the problem.
You will always receive credit in the [release notes](http://www.pgbackrest.org/release.html) for your contributions.
Update documentation/links to main branch.
2021-10-13 18:01:53 +02:00
Coding standards are defined in [CODING.md](https://github.com/pgbackrest/pgbackrest/blob/main/CODING.md) and some important coding details and an example are provided in the [Coding](#coding) section below. At a minimum, unit tests must be written and run and the documentation generated before [submitting a Pull Request](#submitting-a-pull-request); see the [Testing](#testing) section below for details.
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Add building a development environment to contributing documentation. This documentation shows how to build a development environment on Ubuntu 19.04 and should work for other Debian-based distros. Note that this document is not included in automated testing due to some unresolved issues with Docker in Docker on Travis CI. We'll address this in the future when we add contributing documentation to the website.
2019-11-07 20:11:01 +02:00
## Building a Development Environment
Update test container OS versions. Update Ubuntu 12.04 to 16.04. Version 16.04 is recently EOL but testing on an old version is beneficial. Update Ubuntu 18.04 to 20.04. Update Fedora 32 to 33. Version 34 would have been preferred but there were some build issues, i.e. the default shell did not work with configure, and after ksh was installed configure locked up. Add --no-install-recommends to apt-get commands to save a bit of time and space. Update test Dockerfile to run in multiple steps. This makes the container larger but also makes rebuilding after changes faster. The --squash option may be used to keep the container small. Remove obsolete casts in protocol/parallel module. These casts were included in the original migration because Ubuntu 12.04 32-bit gcc required them, but Ubuntu 16.04 32-bit gcc complains. There is no production issue here since at this point in the code the file descriptors are guaranteed to be >= 0.
2021-07-18 23:29:20 +02:00
This example is based on Ubuntu 20.04, but it should work on many versions of Debian and Ubuntu.
Add building a development environment to contributing documentation. This documentation shows how to build a development environment on Ubuntu 19.04 and should work for other Debian-based distros. Note that this document is not included in automated testing due to some unresolved issues with Docker in Docker on Travis CI. We'll address this in the future when we add contributing documentation to the website.
2019-11-07 20:11:01 +02:00
pgbackrest-dev => Install development tools
```
Remove custom lcov build from contributing documentation. Using the stock lcov no longer seems to be a problem, probably due to a better implementation in CoverageTest.pm.
2020-03-22 01:08:27 +02:00
sudo apt-get install rsync git devscripts build-essential valgrind lcov autoconf \
Add missing package to contributing documentation.
2020-03-21 15:51:12 +02:00
autoconf-archive libssl-dev zlib1g-dev libxml2-dev libpq-dev pkg-config \
v2.32: Repository Commands Bug Fixes: * Fix resume after partial delete of backup by prior resume. (Reviewed by Cynthia Shang. Reported by Tom Swartz.) Features: * Add repo-ls command. (Reviewed by Cynthia Shang, Stefan Fercot.) * Add repo-get command. (Contributed by Stefan Fercot, David Steele. Reviewed by Cynthia Shang.) * Add archive-mode-check option. (Contributed by Stefan Fercot. Reviewed by David Steele, Michael Banck.) Improvements: * Improve archive-get performance. (Reviewed by Cynthia Shang.)
2021-02-08 16:08:16 +02:00
libxml-checker-perl libyaml-perl libdbd-pg-perl liblz4-dev liblz4-tool \
Add experimental Meson build. Meson is a new build system that offers simpler syntax and superior performance to autoconf/make. In addition, Windows is supported natively. The Meson build appears complete, but currently is used only for auto-generation of code and the host build of pgbackrest. Some container upgrades will be required before Meson can be used for container builds. Also patch the Debian package to force autoconf/make rather than Meson.
2022-06-03 20:13:56 +02:00
zstd libzstd-dev bzip2 libbz2-dev libyaml-dev ccache meson
Add building a development environment to contributing documentation. This documentation shows how to build a development environment on Ubuntu 19.04 and should work for other Debian-based distros. Note that this document is not included in automated testing due to some unresolved issues with Docker in Docker on Travis CI. We'll address this in the future when we add contributing documentation to the website.
2019-11-07 20:11:01 +02:00
```
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
Some unit tests and all the integration tests require Docker. Running in containers allows us to simulate multiple hosts, test on different distributions and versions of PostgreSQL, and use sudo without affecting the host system.
Add building a development environment to contributing documentation. This documentation shows how to build a development environment on Ubuntu 19.04 and should work for other Debian-based distros. Note that this document is not included in automated testing due to some unresolved issues with Docker in Docker on Travis CI. We'll address this in the future when we add contributing documentation to the website.
2019-11-07 20:11:01 +02:00
pgbackrest-dev => Install Docker
```
curl -fsSL https://get.docker.com | sudo sh
sudo usermod -aG docker `whoami`
```
This clone of the pgBackRest repository is sufficient for experimentation. For development, create a fork and clone that instead.
pgbackrest-dev => Clone pgBackRest repository
```
git clone https://github.com/pgbackrest/pgbackrest.git
```
Use Rocky Linux for documentation builds instead of CentOS. Since CentOS 8 will be EOL at the end of the year it makes sense to do this now. The centos:8 image is still used in documentation.xml because changes there require manual testing, which will need to be done at a later date. The changes are not user-facing, however, and can be done at any time. Also update CentOS references to RHEL since that is what we are emulating for testing purposes.
2021-10-28 21:15:49 +02:00
If using a RHEL-based system, the CPAN XML parser is required to run `test.pl` and `doc.pl`. Instructions for installing Docker and the XML parser can be found in the `README.md` file of the pgBackRest [doc](https://github.com/pgbackrest/pgbackrest/blob/main/doc) directory in the section "The following is a sample RHEL 7 configuration that can be used for building the documentation". NOTE that the "Install latex (for building PDF)" section is not required since testing of the docs need only be run for HTML output.
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
## Coding
The following sections provide information on some important concepts needed for coding within pgBackRest.
### Memory Contexts
Update documentation/links to main branch.
2021-10-13 18:01:53 +02:00
Memory is allocated inside contexts and can be long lasting (for objects) or temporary (for functions). In general, use `OBJ_NEW_BEGIN(MyObj)` for objects and `MEM_CONTEXT_TEMP_BEGIN()` for functions. See [memContext.h](https://github.com/pgbackrest/pgbackrest/blob/main/src/common/memContext.h) for more details and the [Coding Example](#coding-example) below.
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
### Logging
Update documentation/links to main branch.
2021-10-13 18:01:53 +02:00
Logging is used for debugging with the built-in macros `FUNCTION_LOG_*()` and `FUNCTION_TEST_*()` which are used to trace parameters passed to/returned from functions. `FUNCTION_LOG_*()` macros are used for production logging whereas `FUNCTION_TEST_*()` macros will be compiled out of production code. For functions where no parameter is valuable enough to justify the cost of debugging in production, use `FUNCTION_TEST_BEGIN()/FUNCTION_TEST_END()`, else use `FUNCTION_LOG_BEGIN(someLogLevel)/FUNCTION_LOG_END()`. See [debug.h](https://github.com/pgbackrest/pgbackrest/blob/main/src/common/debug.h) for more details and the [Coding Example](#coding-example) below.
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
Move error modules to common/error directory. There are already enough files to warrant a separate directory and more are coming. Also remove extraneous includes.
2023-04-06 08:38:49 +02:00
Logging is also used for providing information to the user via the `LOG_*()` macros, such as `LOG_INFO("some informational message")` and `LOG_WARN_FMT("no prior backup exists, %s backup has been changed to full", strZ(cfgOptionDisplay(cfgOptType)))` and also via `THROW_*()` macros for throwing an error. See [log.h](https://github.com/pgbackrest/pgbackrest/blob/main/src/common/log.h) and [error.h](https://github.com/pgbackrest/pgbackrest/blob/main/src/common/error/error.h) for more details and the [Coding Example](#coding-example) below.
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
### Coding Example
Update documentation/links to main branch.
2021-10-13 18:01:53 +02:00
The example below is not structured like an actual implementation and is intended only to provide an understanding of some of the more common coding practices. The comments in the example are only here to explain the example and are not representative of the coding standards. Refer to the Coding Standards document ([CODING.md](https://github.com/pgbackrest/pgbackrest/blob/main/CODING.md)) and sections above for an introduction to the concepts provided here. For an actual implementation, see [db.h](https://github.com/pgbackrest/pgbackrest/blob/main/src/db/db.h) and [db.c](https://github.com/pgbackrest/pgbackrest/blob/main/src/db/db.c).
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
#### Example: hypothetical basic object construction
```c
/*
* HEADER FILE - see db.h for a complete implementation example
*/
// Typedef the object declared in the C file
typedef struct MyObj MyObj;
// Constructor, and any functions in the header file, are all declared on one line
MyObj *myObjNew(unsigned int myData, const String *secretName);
// Declare the publicly accessible variables in a structure with Pub appended to the name
typedef struct MyObjPub // First letter upper case
{
unsigned int myData; // Contents of the myData variable
} MyObjPub;
// Declare getters and setters inline for the publicly visible variables
// Only setters require "Set" appended to the name
Add FN_INLINE_ALWAYS macro. Eliminate a lot of useless repetition for a commonly used pattern.
2022-09-09 02:36:03 +02:00
FN_INLINE_ALWAYS unsigned int
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
myObjMyData(const MyObj *const this)
{
return THIS_PUB(MyObj)->myData; // Use the built-in THIS_PUB macro
}
// Destructor
Add FN_INLINE_ALWAYS macro. Eliminate a lot of useless repetition for a commonly used pattern.
2022-09-09 02:36:03 +02:00
FN_INLINE_ALWAYS void
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
myObjFree(MyObj *const this)
{
objFree(this);
}
// TYPE and FORMAT macros for function logging
#define FUNCTION_LOG_MY_OBJ_TYPE \
MyObj *
#define FUNCTION_LOG_MY_OBJ_FORMAT(value, buffer, bufferSize) \
Refactor logging functions to never allocate memory. Allocating memory made these functions simpler but it meant that memory was leaking into the calling context when logging was enabled. It is not clear that this was an issue but it seems that trace level logging could result it a lot of memory usage depending on the use case. This also makes it possible to audit allocations returned to the calling context, which will be done in a followup commit. Also rename objToLog() to objNameToLog() since it seemed logical to name the new function objToLog().
2023-01-12 12:14:36 +02:00
FUNCTION_LOG_OBJECT_FORMAT(value, myObjToLog, buffer, bufferSize)
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
/*
* C FILE - see db.c for a more complete and actual implementation example
*/
// Declare the object type
struct MyObj
{
MyObjPub pub; // Publicly accessible variables must be first and named "pub"
const String *name; // Pointer to lightweight string object - see string.h
};
// Object constructor, and any functions in the C file, have the return type and function signature on separate lines
MyObj *
myObjNew(unsigned int myData, const String *secretName)
{
FUNCTION_LOG_BEGIN(logLevelDebug); // Use FUNCTION_LOG_BEGIN with a log level for displaying in production
FUNCTION_LOG_PARAM(UINT, myData); // When log level is debug, myData variable will be logged
FUNCTION_TEST_PARAM(STRING, secretName); // FUNCTION_TEST_PARAM will not display secretName value in production logging
FUNCTION_LOG_END();
ASSERT(secretName != NULL || myData > 0); // Development-only assertions (will be compiled out of production code)
Allow additional memory to be allocated with a mem context. The primary benefit is that objects can allocate memory for their struct with the context, which saves an additional allocation and makes it easier to read context/allocation dumps. Also, the memory context does not need to be stored with the object since it can be determined using the object pointer. Object pointers cannot be moved, so this means whatever additional memory is allocated cannot be resized. That makes the additional memory ideal for object structs, but not so much for allocating a list that might change size. Mem contexts can no longer be reused since they will probably be the wrong size so their memory is freed on memContextFree(). This still means fewer allocations and frees overall. Interfaces still need to be freed by mem context so the old objMove() and objFree() have been preserved as objMoveContext() and objFreeContext(). This will be addressed in a future commit.
2021-09-01 17:10:35 +02:00
OBJ_NEW_BEGIN(MyObj) // Create a long lasting memory context with the name of the object
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
{
*this = (MyObj) // Initialize the object
{
.pub =
{
.myData = myData, // Copy the simple data type to this object
},
.name = strDup(secretName), // Duplicate the String data type to the this object's memory context
};
}
Allow additional memory to be allocated with a mem context. The primary benefit is that objects can allocate memory for their struct with the context, which saves an additional allocation and makes it easier to read context/allocation dumps. Also, the memory context does not need to be stored with the object since it can be determined using the object pointer. Object pointers cannot be moved, so this means whatever additional memory is allocated cannot be resized. That makes the additional memory ideal for object structs, but not so much for allocating a list that might change size. Mem contexts can no longer be reused since they will probably be the wrong size so their memory is freed on memContextFree(). This still means fewer allocations and frees overall. Interfaces still need to be freed by mem context so the old objMove() and objFree() have been preserved as objMoveContext() and objFreeContext(). This will be addressed in a future commit.
2021-09-01 17:10:35 +02:00
OBJ_NEW_END();
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
FUNCTION_LOG_RETURN(MyObj, this);
}
// Function using temporary memory context
String *
myObjDisplay(unsigned int myData)
{
FUNCTION_TEST_BEGIN(); // No parameters passed to this function will be logged in production
FUNCTION_TEST_PARAM(UINT, myData);
FUNCTION_TEST_END();
String *result = NULL; // Result is created in the caller's memory context (referred to as "prior context" below)
MEM_CONTEXT_TEMP_BEGIN() // Begin a new temporary context
{
Rename strNew() to strNewZ() and add parameter-less strNew(). Replace all instances of strNew("") with strNew() and use strNewZ() for non-empty zero-terminated strings. Besides saving a useless parameter, this will allow smarter memory allocation in a future commit by signaling intent, in general, to append or not. In the tests use STRDEF() or VARSTRDEF() where more appropriate rather than blindly replacing with strNewZ(). Also replace strLstAdd() with strLstAddZ() where appropriate for the same reason.
2021-05-21 23:36:43 +02:00
String *resultStr = strNewZ("Hello"); // Allocate a string in the temporary memory context
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
if (myData > 1)
resultStr = strCatZ(" World"); // Append a value to the string still in the temporary memory context
else
LOG_WARN("Am I not your World?"); // Log a warning to the user
MEM_CONTEXT_PRIOR_BEGIN() // Switch to the prior context so the string duplication is in the caller's context
{
result = strDup(resultStr); // Create a copy of the string in the caller's context
}
MEM_CONTEXT_PRIOR_END(); // Switch back to the temporary context
}
MEM_CONTEXT_TEMP_END(); // Free everything created inside this temporary memory context - i.e resultStr
FUNCTION_TEST_RETURN(STRING, result); // Return result but do not log the value in production
}
// Create the logging function for displaying important information from the object
String *
myObjToLog(const MyObj *this)
{
return strNewFmt(
"{name: %s, myData: %u}", this->name == NULL ? NULL_Z : strZ(this->name), myObjMyData(this));
}
```
## Testing
A list of all possible test combinations can be viewed by running:
```
pgbackrest/test/test.pl --dry-run
```
While some files are automatically generated during `make`, others are generated by running the test harness as follows:
```
pgbackrest/test/test.pl --gen-only
```
Update documentation/links to main branch.
2021-10-13 18:01:53 +02:00
Prior to any submission, the html version of the documentation should also be run and the output checked by viewing the generated html on the local file system under `pgbackrest/doc/output/html`. More details can be found in the pgBackRest [doc/README.md](https://github.com/pgbackrest/pgbackrest/blob/main/doc/README.md) file.
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
```
pgbackrest/doc/doc.pl --out=html
```
> **NOTE:** `ERROR: [028]` regarding cache is invalid is OK; it just means there have been changes and the documentation will be built from scratch. In this case, be patient as the build could take 20 minutes or more depending on your system.
Add building a development environment to contributing documentation. This documentation shows how to build a development environment on Ubuntu 19.04 and should work for other Debian-based distros. Note that this document is not included in automated testing due to some unresolved issues with Docker in Docker on Travis CI. We'll address this in the future when we add contributing documentation to the website.
2019-11-07 20:11:01 +02:00
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
### Running Tests
Add building a development environment to contributing documentation. This documentation shows how to build a development environment on Ubuntu 19.04 and should work for other Debian-based distros. Note that this document is not included in automated testing due to some unresolved issues with Docker in Docker on Travis CI. We'll address this in the future when we add contributing documentation to the website.
2019-11-07 20:11:01 +02:00
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
Examples of test runs are provided in the following sections. There are several important options for running a test:
- `--dry-run` - without any other options, this will list all the available tests
- `--module` - identifies the module in which the test is located
- `--test` - the actual test set to be run
- `--run` - a number identifying the run within a test if testing a single run rather than the entire test
- `--vm-out` - displays the test output (helpful for monitoring the progress)
Update documentation/links to main branch.
2021-10-13 18:01:53 +02:00
- `--vm` - identifies the pre-built container when using Docker, otherwise the setting should be `none`. See [test.yml](https://github.com/pgbackrest/pgbackrest/blob/main/.github/workflows/test.yml) for a list of valid vm codes noted by `param: test`.
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
For more options, run the test or documentation engine with the `--help` option:
```
pgbackrest/test/test.pl --help
pgbackrest/doc/doc.pl --help
```
#### Without Docker
If Docker is not installed, then the available tests can be listed using `--vm=none`, and each test must then be run with `--vm=none`.
Add building a development environment to contributing documentation. This documentation shows how to build a development environment on Ubuntu 19.04 and should work for other Debian-based distros. Note that this document is not included in automated testing due to some unresolved issues with Docker in Docker on Travis CI. We'll address this in the future when we add contributing documentation to the website.
2019-11-07 20:11:01 +02:00
pgbackrest-dev => List tests that don't require a container
```
pgbackrest/test/test.pl --vm=none --dry-run
--- output ---
Update contributing documentation and add pull request template.
2021-07-21 19:14:06 +02:00
P00 INFO: test begin on x86_64 - log level info
v2.40: OpenSSL 3 Support NOTE TO PACKAGERS: An experimental meson build has been added but packagers should continue to use the autoconf/make build for the foreseeable future. Improvements: * OpenSSL 3 support. (Reviewed by Stephen Frost.) * Create snapshot when listing contents of a path. (Reviewed by John Morris, Stephen Frost.) * Force target-timeline=current when restore type=immediate. (Reviewed by Stephen Frost.) * Truncate files during delta restore when they are larger than expected. (Reviewed by Stephen Frost.) * Disable incremental manifest save when resume=n. (Contributed by Reid Thompson. Reviewed by David Steele.) * Set backup percent complete to zero before copy start. (Contributed by Reid Thompson. Reviewed by David Steele.) * Use S3 IsTruncated flag to determine list continuation. (Reviewed by John Morris, Soulou. Suggested by Christian Montagne.) Documentation Bug Fixes: * Skip internal options in the configuration reference. (Reported by Francisco Miguel Biete.) Documentation Improvements: * Add link to PostgreSQL configuration in repository host section. (Reviewed by Stefan Fercot. Suggested by Julien Cigar.) Test Suite Improvements: * Add experimental Meson build. (Reviewed by Eli Schwartz, Sam Bassaly.) * Allow any path to be passed to the --test-path option. (Contributed by Andrey Sokolov. Reviewed by David Steele.) * Fix compile error when DEBUG_EXEC_TIME is defined without DEBUG. (Contributed by Andrey Sokolov. Reviewed by David Steele.)
2022-07-18 15:32:30 +02:00
P00 INFO: clean autogenerate code
Suppress timestamps in contributing documentation. Timestamps cause a lot of churn and make it hard to tell if important changes are working.
2020-03-22 16:17:03 +02:00
P00 INFO: builds required: bin
v2.46: Block Incremental Backup and SFTP Storage Features: * Block incremental backup. (Reviewed by John Morris, Stephen Frost, Stefan Fercot.) * SFTP support for repository storage. (Contributed by Reid Thompson. Reviewed by Stephen Frost, David Steele.) * PostgreSQL 16 support. (Reviewed by Stefan Fercot.) Improvements: * Allow page header checks to be skipped. (Reviewed by David Christensen. Suggested by David Christensen.) * Avoid chown() on recovery files during restore. (Reviewed by Stefan Fercot, Marcelo Henrique Neppel. Suggested by Marcelo Henrique Neppel.) * Add error retry detail for HTTP retries. Documentation Improvements: * Add warning about using recovery type=none. (Reviewed by Stefan Fercot.) * Add note about running stanza-create on already-created repositories.
2023-05-22 10:13:13 +02:00
--> P00 INFO: 79 tests selected
Add experimental Meson build. Meson is a new build system that offers simpler syntax and superior performance to autoconf/make. In addition, Windows is supported natively. The Meson build appears complete, but currently is used only for auto-generation of code and the host build of pgbackrest. Some container upgrades will be required before Meson can be used for container builds. Also patch the Debian package to force autoconf/make rather than Meson.
2022-06-03 20:13:56 +02:00
v2.46: Block Incremental Backup and SFTP Storage Features: * Block incremental backup. (Reviewed by John Morris, Stephen Frost, Stefan Fercot.) * SFTP support for repository storage. (Contributed by Reid Thompson. Reviewed by Stephen Frost, David Steele.) * PostgreSQL 16 support. (Reviewed by Stefan Fercot.) Improvements: * Allow page header checks to be skipped. (Reviewed by David Christensen. Suggested by David Christensen.) * Avoid chown() on recovery files during restore. (Reviewed by Stefan Fercot, Marcelo Henrique Neppel. Suggested by Marcelo Henrique Neppel.) * Add error retry detail for HTTP retries. Documentation Improvements: * Add warning about using recovery type=none. (Reviewed by Stefan Fercot.) * Add note about running stanza-create on already-created repositories.
2023-05-22 10:13:13 +02:00
P00 INFO: P1-T01/79 - vm=none, module=common, test=error
[filtered 76 lines of output]
P00 INFO: P1-T78/79 - vm=none, module=performance, test=type
P00 INFO: P1-T79/79 - vm=none, module=performance, test=storage
Suppress timestamps in contributing documentation. Timestamps cause a lot of churn and make it hard to tell if important changes are working.
2020-03-22 16:17:03 +02:00
--> P00 INFO: DRY RUN COMPLETED SUCCESSFULLY
Add building a development environment to contributing documentation. This documentation shows how to build a development environment on Ubuntu 19.04 and should work for other Debian-based distros. Note that this document is not included in automated testing due to some unresolved issues with Docker in Docker on Travis CI. We'll address this in the future when we add contributing documentation to the website.
2019-11-07 20:11:01 +02:00
```
pgbackrest-dev => Run a test
```
Remove test.pl --smart, --dev, and --dev-test options. --smart is now the default mode. Since --dev is now just an alias for --no-optimize, remove it. --dev-test has been a noop for a while, so this seems like a good time to remove it. Also make the C auto-generator skip writing files that have not changed to avoid updating the timestamp.
2021-07-20 18:01:10 +02:00
pgbackrest/test/test.pl --vm=none --vm-out --module=common --test=wait
Add building a development environment to contributing documentation. This documentation shows how to build a development environment on Ubuntu 19.04 and should work for other Debian-based distros. Note that this document is not included in automated testing due to some unresolved issues with Docker in Docker on Travis CI. We'll address this in the future when we add contributing documentation to the website.
2019-11-07 20:11:01 +02:00
--- output ---
Update contributing documentation and add pull request template.
2021-07-21 19:14:06 +02:00
P00 INFO: test begin on x86_64 - log level info
v2.41: Backup Annotations Bug Fixes: * Fix incorrect time expiration being used for non-default repositories. (Reviewed by Stefan Fercot. Reported by Adam Brusselback.) * Fix issue when listing directories recursively with a filter. (Reviewed by Stephen Frost. Reported by Efremov Egor.) Features: * Backup key/value annotations. (Contributed by Stefan Fercot. Reviewed by David Steele. Suggested by Adam Berlin.) Improvements: * Support --set in JSON output for info command. (Contributed by Stefan Fercot. Reviewed by David Steele. Suggested by Anton Kurochkin.) * Update archive.info timestamps after a successful backup. (Reviewed by Stefan Fercot. Suggested by Alex Richman.) * Move standby timeline check after checkpoint. (Reviewed by Stefan Fercot, Keith Fiske. Suggested by Keith Fiske.) * Improve warning message on backup resume. (Suggested by Cynthia Shang.) Documentation Improvements: * Add absolute path for kill in pgbackrest.service. (Suggested by Don Seiler.)
2022-09-19 19:08:10 +02:00
P00 INFO: cleanup old data
Remove test.pl --smart, --dev, and --dev-test options. --smart is now the default mode. Since --dev is now just an alias for --no-optimize, remove it. --dev-test has been a noop for a while, so this seems like a good time to remove it. Also make the C auto-generator skip writing files that have not changed to avoid updating the timestamp.
2021-07-20 18:01:10 +02:00
P00 INFO: autogenerate configure
P00 INFO: autogenerated version in configure.ac script: no changes
P00 INFO: autogenerated configure script: no changes
v2.40: OpenSSL 3 Support NOTE TO PACKAGERS: An experimental meson build has been added but packagers should continue to use the autoconf/make build for the foreseeable future. Improvements: * OpenSSL 3 support. (Reviewed by Stephen Frost.) * Create snapshot when listing contents of a path. (Reviewed by John Morris, Stephen Frost.) * Force target-timeline=current when restore type=immediate. (Reviewed by Stephen Frost.) * Truncate files during delta restore when they are larger than expected. (Reviewed by Stephen Frost.) * Disable incremental manifest save when resume=n. (Contributed by Reid Thompson. Reviewed by David Steele.) * Set backup percent complete to zero before copy start. (Contributed by Reid Thompson. Reviewed by David Steele.) * Use S3 IsTruncated flag to determine list continuation. (Reviewed by John Morris, Soulou. Suggested by Christian Montagne.) Documentation Bug Fixes: * Skip internal options in the configuration reference. (Reported by Francisco Miguel Biete.) Documentation Improvements: * Add link to PostgreSQL configuration in repository host section. (Reviewed by Stefan Fercot. Suggested by Julien Cigar.) Test Suite Improvements: * Add experimental Meson build. (Reviewed by Eli Schwartz, Sam Bassaly.) * Allow any path to be passed to the --test-path option. (Contributed by Andrey Sokolov. Reviewed by David Steele.) * Fix compile error when DEBUG_EXEC_TIME is defined without DEBUG. (Contributed by Andrey Sokolov. Reviewed by David Steele.)
2022-07-18 15:32:30 +02:00
P00 INFO: autogenerate code
Suppress timestamps in contributing documentation. Timestamps cause a lot of churn and make it hard to tell if important changes are working.
2020-03-22 16:17:03 +02:00
P00 INFO: builds required: none
P00 INFO: 1 test selected
Add experimental Meson build. Meson is a new build system that offers simpler syntax and superior performance to autoconf/make. In addition, Windows is supported natively. The Meson build appears complete, but currently is used only for auto-generation of code and the host build of pgbackrest. Some container upgrades will be required before Meson can be used for container builds. Also patch the Debian package to force autoconf/make rather than Meson.
2022-06-03 20:13:56 +02:00
Suppress timestamps in contributing documentation. Timestamps cause a lot of churn and make it hard to tell if important changes are working.
2020-03-22 16:17:03 +02:00
P00 INFO: P1-T1/1 - vm=none, module=common, test=wait
Add experimental Meson build. Meson is a new build system that offers simpler syntax and superior performance to autoconf/make. In addition, Windows is supported natively. The Meson build appears complete, but currently is used only for auto-generation of code and the host build of pgbackrest. Some container upgrades will be required before Meson can be used for container builds. Also patch the Debian package to force autoconf/make rather than Meson.
2022-06-03 20:13:56 +02:00
Add missed --no-log-timestamp in unit tests and improved formatting. The --no-log-timestamp option was missed when unit test building was migrated to C, which caused test timings to show up in the contributing guide. This caused no harm but did create churn in this file during releases. Also improve the formatting when test timing is disabled.
2023-05-23 07:25:17 +02:00
P00 INFO: test command begin 2.47dev: [common/wait] --log-level=info --no-log-timestamp --repo-path=/home/vagrant/test/repo --test-path=/home/vagrant/test --vm=none --vm-id=0
P00 INFO: test command end: completed successfully
v2.32: Repository Commands Bug Fixes: * Fix resume after partial delete of backup by prior resume. (Reviewed by Cynthia Shang. Reported by Tom Swartz.) Features: * Add repo-ls command. (Reviewed by Cynthia Shang, Stefan Fercot.) * Add repo-get command. (Contributed by Stefan Fercot, David Steele. Reviewed by Cynthia Shang.) * Add archive-mode-check option. (Contributed by Stefan Fercot. Reviewed by David Steele, Michael Banck.) Improvements: * Improve archive-get performance. (Reviewed by Cynthia Shang.)
2021-02-08 16:08:16 +02:00
run 1 - waitNew(), waitMore, and waitFree()
Add missed --no-log-timestamp in unit tests and improved formatting. The --no-log-timestamp option was missed when unit test building was migrated to C, which caused test timings to show up in the contributing guide. This caused no harm but did create churn in this file during releases. Also improve the formatting when test timing is disabled.
2023-05-23 07:25:17 +02:00
L0018 expect AssertError: assertion 'waitTime <= 999999000' failed
Add experimental Meson build. Meson is a new build system that offers simpler syntax and superior performance to autoconf/make. In addition, Windows is supported natively. The Meson build appears complete, but currently is used only for auto-generation of code and the host build of pgbackrest. Some container upgrades will be required before Meson can be used for container builds. Also patch the Debian package to force autoconf/make rather than Meson.
2022-06-03 20:13:56 +02:00
Add missed --no-log-timestamp in unit tests and improved formatting. The --no-log-timestamp option was missed when unit test building was migrated to C, which caused test timings to show up in the contributing guide. This caused no harm but did create churn in this file during releases. Also improve the formatting when test timing is disabled.
2023-05-23 07:25:17 +02:00
run 1/1 ----- L0021 0ms wait
L0025 new wait
L0026 check remaining time
L0027 check wait time
L0028 check sleep time
L0029 check sleep prev time
L0030 no wait more
L0033 new wait = 0.2 sec
L0034 check remaining time
L0035 check wait time
L0036 check sleep time
L0037 check sleep prev time
L0038 check begin time
L0044 lower range check
L0045 upper range check
L0047 free wait
L0052 new wait = 1.1 sec
L0053 check wait time
L0054 check sleep time
L0055 check sleep prev time
L0056 check begin time
L0062 lower range check
L0063 upper range check
L0065 free wait
Add experimental Meson build. Meson is a new build system that offers simpler syntax and superior performance to autoconf/make. In addition, Windows is supported natively. The Meson build appears complete, but currently is used only for auto-generation of code and the host build of pgbackrest. Some container upgrades will be required before Meson can be used for container builds. Also patch the Debian package to force autoconf/make rather than Meson.
2022-06-03 20:13:56 +02:00
Add building a development environment to contributing documentation. This documentation shows how to build a development environment on Ubuntu 19.04 and should work for other Debian-based distros. Note that this document is not included in automated testing due to some unresolved issues with Docker in Docker on Travis CI. We'll address this in the future when we add contributing documentation to the website.
2019-11-07 20:11:01 +02:00
TESTS COMPLETED SUCCESSFULLY
Add experimental Meson build. Meson is a new build system that offers simpler syntax and superior performance to autoconf/make. In addition, Windows is supported natively. The Meson build appears complete, but currently is used only for auto-generation of code and the host build of pgbackrest. Some container upgrades will be required before Meson can be used for container builds. Also patch the Debian package to force autoconf/make rather than Meson.
2022-06-03 20:13:56 +02:00
Suppress timestamps in contributing documentation. Timestamps cause a lot of churn and make it hard to tell if important changes are working.
2020-03-22 16:17:03 +02:00
P00 INFO: P1-T1/1 - vm=none, module=common, test=wait
v2.28: Azure Repository Storage Bug Fixes: * Fix restore --force acting like --force --delta. This caused restore to replace files based on timestamp and size rather than overwriting, which meant some files that should have been updated were left unchanged. Normal restore and restore --delta were not affected by this issue. (Reviewed by Cynthia Shang.) Features: * Azure support for repository storage. (Reviewed by Cynthia Shang, Don Seiler.) * Add expire-auto option. This allows automatic expiration after a successful backup to be disabled. (Contributed by Stefan Fercot. Reviewed by Cynthia Shang, David Steele.) Improvements: * Asynchronous S3 multipart upload. (Reviewed by Stephen Frost.) * Automatic retry for backup, restore, archive-get, and archive-push. (Reviewed by Cynthia Shang.) * Disable query parallelism in PostgreSQL sessions used for backup control. (Reviewed by Stefan Fercot.) * PostgreSQL 13 beta2 support. Changes to the control/catalog/WAL versions in subsequent betas may break compatibility but pgBackRest will be updated with each release to keep pace. * Improve handling of invalid HTTP response status. (Reviewed by Cynthia Shang.) * Improve error when pg1-path option missing for archive-get command. (Reviewed by Cynthia Shang.) * Add hint when checksum delta is enabled after a timeline switch. (Reviewed by Matt Bunter, Cynthia Shang.) * Use PostgreSQL instead of postmaster where appropriate. (Reviewed by Cynthia Shang.) Documentation Bug Fixes: * Fix incorrect example for repo-retention-full-type option. (Reported by Höseyin Sönmez.) * Remove internal commands from HTML and man command references. (Reported by Cynthia Shang.) Documentation Improvements: * Update PostgreSQL versions used to build user guides. Also add version ranges to indicate that a user guide is accurate for a range of PostgreSQL versions even if it was built for a specific version. (Reviewed by Stephen Frost.) * Update FAQ for expiring a specific backup set. (Contributed by Cynthia Shang. Reviewed by David Steele.) * Update FAQ to clarify default PITR behavior. (Contributed by Cynthia Shang. Reviewed by David Steele.)
2020-07-20 14:57:22 +02:00
P00 INFO: tested modules have full coverage
Suppress timestamps in contributing documentation. Timestamps cause a lot of churn and make it hard to tell if important changes are working.
2020-03-22 16:17:03 +02:00
P00 INFO: writing C coverage report
P00 INFO: TESTS COMPLETED SUCCESSFULLY
Add building a development environment to contributing documentation. This documentation shows how to build a development environment on Ubuntu 19.04 and should work for other Debian-based distros. Note that this document is not included in automated testing due to some unresolved issues with Docker in Docker on Travis CI. We'll address this in the future when we add contributing documentation to the website.
2019-11-07 20:11:01 +02:00
```
An entire module can be run by using only the `--module` option.
pgbackrest-dev => Run a module
```
Remove test.pl --smart, --dev, and --dev-test options. --smart is now the default mode. Since --dev is now just an alias for --no-optimize, remove it. --dev-test has been a noop for a while, so this seems like a good time to remove it. Also make the C auto-generator skip writing files that have not changed to avoid updating the timestamp.
2021-07-20 18:01:10 +02:00
pgbackrest/test/test.pl --vm=none --module=postgres
Add building a development environment to contributing documentation. This documentation shows how to build a development environment on Ubuntu 19.04 and should work for other Debian-based distros. Note that this document is not included in automated testing due to some unresolved issues with Docker in Docker on Travis CI. We'll address this in the future when we add contributing documentation to the website.
2019-11-07 20:11:01 +02:00
--- output ---
Update contributing documentation and add pull request template.
2021-07-21 19:14:06 +02:00
P00 INFO: test begin on x86_64 - log level info
v2.41: Backup Annotations Bug Fixes: * Fix incorrect time expiration being used for non-default repositories. (Reviewed by Stefan Fercot. Reported by Adam Brusselback.) * Fix issue when listing directories recursively with a filter. (Reviewed by Stephen Frost. Reported by Efremov Egor.) Features: * Backup key/value annotations. (Contributed by Stefan Fercot. Reviewed by David Steele. Suggested by Adam Berlin.) Improvements: * Support --set in JSON output for info command. (Contributed by Stefan Fercot. Reviewed by David Steele. Suggested by Anton Kurochkin.) * Update archive.info timestamps after a successful backup. (Reviewed by Stefan Fercot. Suggested by Alex Richman.) * Move standby timeline check after checkpoint. (Reviewed by Stefan Fercot, Keith Fiske. Suggested by Keith Fiske.) * Improve warning message on backup resume. (Suggested by Cynthia Shang.) Documentation Improvements: * Add absolute path for kill in pgbackrest.service. (Suggested by Don Seiler.)
2022-09-19 19:08:10 +02:00
P00 INFO: cleanup old data
Remove test.pl --smart, --dev, and --dev-test options. --smart is now the default mode. Since --dev is now just an alias for --no-optimize, remove it. --dev-test has been a noop for a while, so this seems like a good time to remove it. Also make the C auto-generator skip writing files that have not changed to avoid updating the timestamp.
2021-07-20 18:01:10 +02:00
P00 INFO: autogenerate configure
P00 INFO: autogenerated version in configure.ac script: no changes
P00 INFO: autogenerated configure script: no changes
P00 INFO: autogenerate code
Suppress timestamps in contributing documentation. Timestamps cause a lot of churn and make it hard to tell if important changes are working.
2020-03-22 16:17:03 +02:00
P00 INFO: builds required: none
P00 INFO: 2 tests selected
Add experimental Meson build. Meson is a new build system that offers simpler syntax and superior performance to autoconf/make. In addition, Windows is supported natively. The Meson build appears complete, but currently is used only for auto-generation of code and the host build of pgbackrest. Some container upgrades will be required before Meson can be used for container builds. Also patch the Debian package to force autoconf/make rather than Meson.
2022-06-03 20:13:56 +02:00
Suppress timestamps in contributing documentation. Timestamps cause a lot of churn and make it hard to tell if important changes are working.
2020-03-22 16:17:03 +02:00
P00 INFO: P1-T1/2 - vm=none, module=postgres, test=client
P00 INFO: P1-T2/2 - vm=none, module=postgres, test=interface
v2.28: Azure Repository Storage Bug Fixes: * Fix restore --force acting like --force --delta. This caused restore to replace files based on timestamp and size rather than overwriting, which meant some files that should have been updated were left unchanged. Normal restore and restore --delta were not affected by this issue. (Reviewed by Cynthia Shang.) Features: * Azure support for repository storage. (Reviewed by Cynthia Shang, Don Seiler.) * Add expire-auto option. This allows automatic expiration after a successful backup to be disabled. (Contributed by Stefan Fercot. Reviewed by Cynthia Shang, David Steele.) Improvements: * Asynchronous S3 multipart upload. (Reviewed by Stephen Frost.) * Automatic retry for backup, restore, archive-get, and archive-push. (Reviewed by Cynthia Shang.) * Disable query parallelism in PostgreSQL sessions used for backup control. (Reviewed by Stefan Fercot.) * PostgreSQL 13 beta2 support. Changes to the control/catalog/WAL versions in subsequent betas may break compatibility but pgBackRest will be updated with each release to keep pace. * Improve handling of invalid HTTP response status. (Reviewed by Cynthia Shang.) * Improve error when pg1-path option missing for archive-get command. (Reviewed by Cynthia Shang.) * Add hint when checksum delta is enabled after a timeline switch. (Reviewed by Matt Bunter, Cynthia Shang.) * Use PostgreSQL instead of postmaster where appropriate. (Reviewed by Cynthia Shang.) Documentation Bug Fixes: * Fix incorrect example for repo-retention-full-type option. (Reported by Höseyin Sönmez.) * Remove internal commands from HTML and man command references. (Reported by Cynthia Shang.) Documentation Improvements: * Update PostgreSQL versions used to build user guides. Also add version ranges to indicate that a user guide is accurate for a range of PostgreSQL versions even if it was built for a specific version. (Reviewed by Stephen Frost.) * Update FAQ for expiring a specific backup set. (Contributed by Cynthia Shang. Reviewed by David Steele.) * Update FAQ to clarify default PITR behavior. (Contributed by Cynthia Shang. Reviewed by David Steele.)
2020-07-20 14:57:22 +02:00
P00 INFO: tested modules have full coverage
Suppress timestamps in contributing documentation. Timestamps cause a lot of churn and make it hard to tell if important changes are working.
2020-03-22 16:17:03 +02:00
P00 INFO: writing C coverage report
P00 INFO: TESTS COMPLETED SUCCESSFULLY
Add building a development environment to contributing documentation. This documentation shows how to build a development environment on Ubuntu 19.04 and should work for other Debian-based distros. Note that this document is not included in automated testing due to some unresolved issues with Docker in Docker on Travis CI. We'll address this in the future when we add contributing documentation to the website.
2019-11-07 20:11:01 +02:00
```
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
#### With Docker
Add building a development environment to contributing documentation. This documentation shows how to build a development environment on Ubuntu 19.04 and should work for other Debian-based distros. Note that this document is not included in automated testing due to some unresolved issues with Docker in Docker on Travis CI. We'll address this in the future when we add contributing documentation to the website.
2019-11-07 20:11:01 +02:00
Update test container OS versions. Update Ubuntu 12.04 to 16.04. Version 16.04 is recently EOL but testing on an old version is beneficial. Update Ubuntu 18.04 to 20.04. Update Fedora 32 to 33. Version 34 would have been preferred but there were some build issues, i.e. the default shell did not work with configure, and after ksh was installed configure locked up. Add --no-install-recommends to apt-get commands to save a bit of time and space. Update test Dockerfile to run in multiple steps. This makes the container larger but also makes rebuilding after changes faster. The --squash option may be used to keep the container small. Remove obsolete casts in protocol/parallel module. These casts were included in the original migration because Ubuntu 12.04 32-bit gcc required them, but Ubuntu 16.04 32-bit gcc complains. There is no production issue here since at this point in the code the file descriptors are guaranteed to be >= 0.
2021-07-18 23:29:20 +02:00
Build a container to run tests. The vm must be pre-configured but a variety are available. A vagrant file is provided in the test directory as an example of running in a virtual environment. The vm names are all three character abbreviations, e.g. `u20` for Ubuntu 20.04.
Add building a development environment to contributing documentation. This documentation shows how to build a development environment on Ubuntu 19.04 and should work for other Debian-based distros. Note that this document is not included in automated testing due to some unresolved issues with Docker in Docker on Travis CI. We'll address this in the future when we add contributing documentation to the website.
2019-11-07 20:11:01 +02:00
pgbackrest-dev => Build a VM
```
Update test container OS versions. Update Ubuntu 12.04 to 16.04. Version 16.04 is recently EOL but testing on an old version is beneficial. Update Ubuntu 18.04 to 20.04. Update Fedora 32 to 33. Version 34 would have been preferred but there were some build issues, i.e. the default shell did not work with configure, and after ksh was installed configure locked up. Add --no-install-recommends to apt-get commands to save a bit of time and space. Update test Dockerfile to run in multiple steps. This makes the container larger but also makes rebuilding after changes faster. The --squash option may be used to keep the container small. Remove obsolete casts in protocol/parallel module. These casts were included in the original migration because Ubuntu 12.04 32-bit gcc required them, but Ubuntu 16.04 32-bit gcc complains. There is no production issue here since at this point in the code the file descriptors are guaranteed to be >= 0.
2021-07-18 23:29:20 +02:00
pgbackrest/test/test.pl --vm-build --vm=u20
Add building a development environment to contributing documentation. This documentation shows how to build a development environment on Ubuntu 19.04 and should work for other Debian-based distros. Note that this document is not included in automated testing due to some unresolved issues with Docker in Docker on Travis CI. We'll address this in the future when we add contributing documentation to the website.
2019-11-07 20:11:01 +02:00
--- output ---
Update contributing documentation and add pull request template.
2021-07-21 19:14:06 +02:00
P00 INFO: test begin on x86_64 - log level info
Build u20 image to speed contributing document generation. This image was left out of the last round of builds, which forced the contributing document to build it from scratch.
2023-05-23 09:14:00 +02:00
P00 INFO: Using cached pgbackrest/test:u20-base-20230523A image (31c2124ab0db03d97eb6324e12e22eb64eb4a3b8) ...
Update test container OS versions. Update Ubuntu 12.04 to 16.04. Version 16.04 is recently EOL but testing on an old version is beneficial. Update Ubuntu 18.04 to 20.04. Update Fedora 32 to 33. Version 34 would have been preferred but there were some build issues, i.e. the default shell did not work with configure, and after ksh was installed configure locked up. Add --no-install-recommends to apt-get commands to save a bit of time and space. Update test Dockerfile to run in multiple steps. This makes the container larger but also makes rebuilding after changes faster. The --squash option may be used to keep the container small. Remove obsolete casts in protocol/parallel module. These casts were included in the original migration because Ubuntu 12.04 32-bit gcc required them, but Ubuntu 16.04 32-bit gcc complains. There is no production issue here since at this point in the code the file descriptors are guaranteed to be >= 0.
2021-07-18 23:29:20 +02:00
P00 INFO: Building pgbackrest/test:u20-test image ...
Suppress timestamps in contributing documentation. Timestamps cause a lot of churn and make it hard to tell if important changes are working.
2020-03-22 16:17:03 +02:00
P00 INFO: Build Complete
Add building a development environment to contributing documentation. This documentation shows how to build a development environment on Ubuntu 19.04 and should work for other Debian-based distros. Note that this document is not included in automated testing due to some unresolved issues with Docker in Docker on Travis CI. We'll address this in the future when we add contributing documentation to the website.
2019-11-07 20:11:01 +02:00
```
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
> **NOTE:** to build all the vms, just omit the `--vm` option above.
Add building a development environment to contributing documentation. This documentation shows how to build a development environment on Ubuntu 19.04 and should work for other Debian-based distros. Note that this document is not included in automated testing due to some unresolved issues with Docker in Docker on Travis CI. We'll address this in the future when we add contributing documentation to the website.
2019-11-07 20:11:01 +02:00
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
pgbackrest-dev => Run a Specific Test Run
Add building a development environment to contributing documentation. This documentation shows how to build a development environment on Ubuntu 19.04 and should work for other Debian-based distros. Note that this document is not included in automated testing due to some unresolved issues with Docker in Docker on Travis CI. We'll address this in the future when we add contributing documentation to the website.
2019-11-07 20:11:01 +02:00
```
Remove test.pl --smart, --dev, and --dev-test options. --smart is now the default mode. Since --dev is now just an alias for --no-optimize, remove it. --dev-test has been a noop for a while, so this seems like a good time to remove it. Also make the C auto-generator skip writing files that have not changed to avoid updating the timestamp.
2021-07-20 18:01:10 +02:00
pgbackrest/test/test.pl --vm=u20 --module=mock --test=archive --run=2
Add building a development environment to contributing documentation. This documentation shows how to build a development environment on Ubuntu 19.04 and should work for other Debian-based distros. Note that this document is not included in automated testing due to some unresolved issues with Docker in Docker on Travis CI. We'll address this in the future when we add contributing documentation to the website.
2019-11-07 20:11:01 +02:00
--- output ---
Update contributing documentation and add pull request template.
2021-07-21 19:14:06 +02:00
P00 INFO: test begin on x86_64 - log level info
v2.41: Backup Annotations Bug Fixes: * Fix incorrect time expiration being used for non-default repositories. (Reviewed by Stefan Fercot. Reported by Adam Brusselback.) * Fix issue when listing directories recursively with a filter. (Reviewed by Stephen Frost. Reported by Efremov Egor.) Features: * Backup key/value annotations. (Contributed by Stefan Fercot. Reviewed by David Steele. Suggested by Adam Berlin.) Improvements: * Support --set in JSON output for info command. (Contributed by Stefan Fercot. Reviewed by David Steele. Suggested by Anton Kurochkin.) * Update archive.info timestamps after a successful backup. (Reviewed by Stefan Fercot. Suggested by Alex Richman.) * Move standby timeline check after checkpoint. (Reviewed by Stefan Fercot, Keith Fiske. Suggested by Keith Fiske.) * Improve warning message on backup resume. (Suggested by Cynthia Shang.) Documentation Improvements: * Add absolute path for kill in pgbackrest.service. (Suggested by Don Seiler.)
2022-09-19 19:08:10 +02:00
P00 INFO: cleanup old data and containers
Remove test.pl --smart, --dev, and --dev-test options. --smart is now the default mode. Since --dev is now just an alias for --no-optimize, remove it. --dev-test has been a noop for a while, so this seems like a good time to remove it. Also make the C auto-generator skip writing files that have not changed to avoid updating the timestamp.
2021-07-20 18:01:10 +02:00
P00 INFO: autogenerate configure
P00 INFO: autogenerated version in configure.ac script: no changes
P00 INFO: autogenerated configure script: no changes
v2.41: Backup Annotations Bug Fixes: * Fix incorrect time expiration being used for non-default repositories. (Reviewed by Stefan Fercot. Reported by Adam Brusselback.) * Fix issue when listing directories recursively with a filter. (Reviewed by Stephen Frost. Reported by Efremov Egor.) Features: * Backup key/value annotations. (Contributed by Stefan Fercot. Reviewed by David Steele. Suggested by Adam Berlin.) Improvements: * Support --set in JSON output for info command. (Contributed by Stefan Fercot. Reviewed by David Steele. Suggested by Anton Kurochkin.) * Update archive.info timestamps after a successful backup. (Reviewed by Stefan Fercot. Suggested by Alex Richman.) * Move standby timeline check after checkpoint. (Reviewed by Stefan Fercot, Keith Fiske. Suggested by Keith Fiske.) * Improve warning message on backup resume. (Suggested by Cynthia Shang.) Documentation Improvements: * Add absolute path for kill in pgbackrest.service. (Suggested by Don Seiler.)
2022-09-19 19:08:10 +02:00
P00 INFO: clean autogenerate code
Suppress timestamps in contributing documentation. Timestamps cause a lot of churn and make it hard to tell if important changes are working.
2020-03-22 16:17:03 +02:00
P00 INFO: builds required: bin, bin host
Add experimental Meson build. Meson is a new build system that offers simpler syntax and superior performance to autoconf/make. In addition, Windows is supported natively. The Meson build appears complete, but currently is used only for auto-generation of code and the host build of pgbackrest. Some container upgrades will be required before Meson can be used for container builds. Also patch the Debian package to force autoconf/make rather than Meson.
2022-06-03 20:13:56 +02:00
P00 INFO: bin build for u20 (/home/vagrant/test/bin/u20)
Remove test.pl --smart, --dev, and --dev-test options. --smart is now the default mode. Since --dev is now just an alias for --no-optimize, remove it. --dev-test has been a noop for a while, so this seems like a good time to remove it. Also make the C auto-generator skip writing files that have not changed to avoid updating the timestamp.
2021-07-20 18:01:10 +02:00
P00 INFO: bin dependencies have changed, rebuilding
v2.41: Backup Annotations Bug Fixes: * Fix incorrect time expiration being used for non-default repositories. (Reviewed by Stefan Fercot. Reported by Adam Brusselback.) * Fix issue when listing directories recursively with a filter. (Reviewed by Stephen Frost. Reported by Efremov Egor.) Features: * Backup key/value annotations. (Contributed by Stefan Fercot. Reviewed by David Steele. Suggested by Adam Berlin.) Improvements: * Support --set in JSON output for info command. (Contributed by Stefan Fercot. Reviewed by David Steele. Suggested by Anton Kurochkin.) * Update archive.info timestamps after a successful backup. (Reviewed by Stefan Fercot. Suggested by Alex Richman.) * Move standby timeline check after checkpoint. (Reviewed by Stefan Fercot, Keith Fiske. Suggested by Keith Fiske.) * Improve warning message on backup resume. (Suggested by Cynthia Shang.) Documentation Improvements: * Add absolute path for kill in pgbackrest.service. (Suggested by Don Seiler.)
2022-09-19 19:08:10 +02:00
P00 INFO: bin build for none (/home/vagrant/test/build/none)
Suppress timestamps in contributing documentation. Timestamps cause a lot of churn and make it hard to tell if important changes are working.
2020-03-22 16:17:03 +02:00
P00 INFO: 1 test selected
Add experimental Meson build. Meson is a new build system that offers simpler syntax and superior performance to autoconf/make. In addition, Windows is supported natively. The Meson build appears complete, but currently is used only for auto-generation of code and the host build of pgbackrest. Some container upgrades will be required before Meson can be used for container builds. Also patch the Debian package to force autoconf/make rather than Meson.
2022-06-03 20:13:56 +02:00
Update test container OS versions. Update Ubuntu 12.04 to 16.04. Version 16.04 is recently EOL but testing on an old version is beneficial. Update Ubuntu 18.04 to 20.04. Update Fedora 32 to 33. Version 34 would have been preferred but there were some build issues, i.e. the default shell did not work with configure, and after ksh was installed configure locked up. Add --no-install-recommends to apt-get commands to save a bit of time and space. Update test Dockerfile to run in multiple steps. This makes the container larger but also makes rebuilding after changes faster. The --squash option may be used to keep the container small. Remove obsolete casts in protocol/parallel module. These casts were included in the original migration because Ubuntu 12.04 32-bit gcc required them, but Ubuntu 16.04 32-bit gcc complains. There is no production issue here since at this point in the code the file descriptors are guaranteed to be >= 0.
2021-07-18 23:29:20 +02:00
P00 INFO: P1-T1/1 - vm=u20, module=mock, test=archive, run=2
Suppress timestamps in contributing documentation. Timestamps cause a lot of churn and make it hard to tell if important changes are working.
2020-03-22 16:17:03 +02:00
P00 INFO: no code modules had all tests run required for coverage
P00 INFO: TESTS COMPLETED SUCCESSFULLY
Add building a development environment to contributing documentation. This documentation shows how to build a development environment on Ubuntu 19.04 and should work for other Debian-based distros. Note that this document is not included in automated testing due to some unresolved issues with Docker in Docker on Travis CI. We'll address this in the future when we add contributing documentation to the website.
2019-11-07 20:11:01 +02:00
```
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
### Writing a Unit Test
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
The goal of unit testing is to have 100 percent code coverage. Two files will usually be involved in this process:
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
- **define.yaml** - defines the number of tests to be run for each module and test file. There is a comment at the top of the file that provides more information about this file.
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
- **src/module/somefileTest.c** - where "somefile" is the path and name of the test file where the unit tests are located for the code being updated (e.g. `src/module/command/expireTest.c`).
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
#### define.yaml
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
Each module is separated by a line of asterisks (*) and each test within is separated by a line of dashes (-). In the example below, the module is `command` and the unit test is `check`. The number of calls to `testBegin()` in a unit test file will dictate the number following `total:`, in this case 4. Under `coverage:`, the list of files that will be tested.
```
# ********************************************************************************************************************************
- name: command
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
test:
# ----------------------------------------------------------------------------------------------------------------------------
- name: check
total: 4
containerReq: true
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
coverage:
- command/check/common
- command/check/check
```
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
#### somefileTest.c
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
Unit test files are organized in the `test/src/module` directory with the same directory structure as the source code being tested. For example, if new code is added to src/**command/expire**.c then test/src/module/**command/expire**Test.c will need to be updated.
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
Assuming that a test file already exists, new unit tests will either go in a new `testBegin()` section or be added to an existing section. Each such section is a test run. The comment string passed to `testBegin()` should reflect the function(s) being tested in the test run. Tests within a run should use `TEST_TITLE()` with a comment string describing the test.
```
// *****************************************************************************************************************************
if (testBegin("expireBackup()"))
{
Update contributing documentation and add pull request template.
2021-07-21 19:14:06 +02:00
// -------------------------------------------------------------------------------------------------------------------------
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
TEST_TITLE("manifest file removal");
```
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
#### Setting up the command to be run
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update documentation/links to main branch.
2021-10-13 18:01:53 +02:00
The [harnessConfig.h](https://github.com/pgbackrest/pgbackrest/blob/main/test/src/common/harnessConfig.h) describes a list of functions that should be used when configuration options are required for a command being tested. Options are set in a `StringList` which must be defined and passed to the `HRN_CFG_LOAD()` macro with the command. For example, the following will set up a test to run `pgbackrest --repo-path=test/test-0/repo info` command on multiple repositories, one of which is encrypted:
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
```
StringList *argList = strLstNew(); // Create an empty string list
Remove TEST_PATH_REPO, TEST_PATH_PG, and TEST_PATH_SPOOL constants. It seems better to use TEST_PATH in combination with a constant string rather than have a number of different path constants. This improves readability and reduces confusion about which constant should be used.
2021-06-28 19:28:28 +02:00
hrnCfgArgRawZ(argList, cfgOptRepoPath, TEST_PATH "/repo"); // Add the --repo-path option
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
hrnCfgArgKeyRawZ(argList, cfgOptRepoPath, 2, TEST_PATH "/repo2"); // Add the --repo2-path option
hrnCfgArgKeyRawStrId(argList, cfgOptRepoCipherType, 2, cipherTypeAes256Cbc); // Add the --repo2-cipher-type option
hrnCfgEnvKeyRawZ(cfgOptRepoCipherPass, 2, TEST_CIPHER_PASS); // Set environment variable for the --repo2-cipher-pass option
Replace harnessCfgLoad*() functions with HRN_CFG_LOAD() macro. HRN_CFG_LOAD() handles the majority of test configuration loads and has various options for special cases. It was not clear when to use harnessCfgLoadRaw() vs harnessCfgLoad(). Now "raw" functionality is granular and enabled by parameters, e.g. noStd.
2021-06-01 15:03:44 +02:00
HRN_CFG_LOAD(cfgCmdInfo, argList); // Load the command and option list into the test harness
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
```
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
#### Storing a file
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update documentation/links to main branch.
2021-10-13 18:01:53 +02:00
Sometimes it is desirable to store or manipulate files before or during a test and then confirm the contents. The [harnessStorage.h](https://github.com/pgbackrest/pgbackrest/blob/main/test/src/common/harnessStorage.h) file contains macros (e.g. `HRN_STORAGE_PUT` and `TEST_STORAGE_GET`) for doing this. In addition, `HRN_INFO_PUT` is convenient for writing out info files (archive.info, backup.info, backup.manifest) since it will automatically add header and checksum information.
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
```
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
HRN_STORAGE_PUT_EMPTY(
storageRepoWrite(), STORAGE_REPO_ARCHIVE "/10-1/000000010000000100000001-abcdabcdabcdabcdabcdabcdabcdabcdabcdabcd.gz");
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
```
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
#### Testing results
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update documentation/links to main branch.
2021-10-13 18:01:53 +02:00
Tests are run and results confirmed via macros that are described in [harnessTest.h](https://github.com/pgbackrest/pgbackrest/blob/main/test/src/common/harnessTest.h). With the exception of TEST_ERROR, the third parameter is a short description of the test. Some of the more common macros are:
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
- `TEST_RESULT_STR` - Test the actual value of the string returned by the function.
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
- `TEST_RESULT_UINT` / `TEST_RESULT_INT` - Test for an unsigned integer / integer.
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
- `TEST_RESULT_BOOL` - Test a boolean value.
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
- `TEST_RESULT_PTR` / `TEST_RESULT_PTR_NE` - Test a pointer: useful for testing if the pointer is `NULL` or not equal (`NE`) to `NULL`.
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
- `TEST_RESULT_VOID` - The function being tested returns a `void`. This is then usually followed by tests that ensure other actions occurred (e.g. a file was written to disk).
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
- `TEST_ERROR` / `TEST_ERROR_FMT` - Test that a specific error code was raised with specific wording.
Update contributing documentation and add pull request template.
2021-07-21 19:14:06 +02:00
> **NOTE:** `HRN_*` macros should be used only for test setup and cleanup. `TEST_*` macros must be used for testing results.
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
#### Testing a log message
If a function being tested logs something with `LOG_WARN`, `LOG_INFO` or other `LOG_*()` macro, then the logged message must be cleared before the end of the test by using the `TEST_RESULT_LOG()/TEST_RESULT_LOG_FMT()` macros.
```
TEST_RESULT_LOG(
"P00 WARN: WAL segment '000000010000000100000001' was not pushed due to error [25] and was manually skipped: error");
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
```
Update contributing documentation and add pull request template.
2021-07-21 19:14:06 +02:00
In the above, `Pxx` indicates the process (P) and the process number (xx), e.g. P00, P01.
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
#### Testing using child process
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update documentation/links to main branch.
2021-10-13 18:01:53 +02:00
Sometimes it is useful to use a child process for testing. Below is a simple example. See [harnessFork.h](https://github.com/pgbackrest/pgbackrest/blob/main/test/src/common/harnessFork.h) for more details.
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
```
Rename HARNESS_FORK*() macros to HRN_FORK*(). This matches the new pattern for harness macro naming and is shorter.
2021-07-13 17:58:23 +02:00
HRN_FORK_BEGIN()
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
{
Improve HRN_FORK*() macros. Simplify HRN_FORK_CHILD_BEGIN() by adding optional parameters with the common defaults. Add _FD() to macros that retrieve file descriptors to make their purpose clearer.
2021-07-13 20:22:53 +02:00
HRN_FORK_CHILD_BEGIN()
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
{
Add lock module initialization. Each call to lockAcquireP() passed enough information to initialize the lock system. This was somewhat inefficient and as locks become more complicated it will lead to more code duplication. Since a process can only take one type of lock it makes sense to do most of the initialization up front. Also reduce the log level of lockRelease() since it is only called at exit and the lock will be released in any case.
2023-03-25 09:07:31 +02:00
TEST_RESULT_INT_NE(lockAcquireP(), -1, "create backup/expire lock");
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Fix links and update child process example. Removed colon from example titles to fix links, fixed test.yml link, and updated the example for the parent/child test process to use the latest macros instead of sleep().
2021-08-09 22:56:06 +02:00
// Notify parent that lock has been acquired
HRN_FORK_CHILD_NOTIFY_PUT();
// Wait for parent to allow release lock
HRN_FORK_CHILD_NOTIFY_GET();
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
lockRelease(true);
}
Rename HARNESS_FORK*() macros to HRN_FORK*(). This matches the new pattern for harness macro naming and is shorter.
2021-07-13 17:58:23 +02:00
HRN_FORK_CHILD_END();
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
Rename HARNESS_FORK*() macros to HRN_FORK*(). This matches the new pattern for harness macro naming and is shorter.
2021-07-13 17:58:23 +02:00
HRN_FORK_PARENT_BEGIN()
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
{
Fix links and update child process example. Removed colon from example titles to fix links, fixed test.yml link, and updated the example for the parent/child test process to use the latest macros instead of sleep().
2021-08-09 22:56:06 +02:00
// Wait for child to acquire lock
HRN_FORK_PARENT_NOTIFY_GET(0);
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
Replace harnessCfgLoad*() functions with HRN_CFG_LOAD() macro. HRN_CFG_LOAD() handles the majority of test configuration loads and has various options for special cases. It was not clear when to use harnessCfgLoadRaw() vs harnessCfgLoad(). Now "raw" functionality is granular and enabled by parameters, e.g. noStd.
2021-06-01 15:03:44 +02:00
HRN_CFG_LOAD(cfgCmdInfo, argListText);
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
TEST_RESULT_STR_Z(
infoRender(),
"stanza: stanza1\n"
" status: error (no valid backups, backup/expire running)\n"
" cipher: none\n"
"\n"
" db (current)\n"
" wal archive min/max (9.4): none present\n",
"text - single stanza, no valid backups, backup/expire lock detected");
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Fix links and update child process example. Removed colon from example titles to fix links, fixed test.yml link, and updated the example for the parent/child test process to use the latest macros instead of sleep().
2021-08-09 22:56:06 +02:00
// Notify child to release lock
HRN_FORK_PARENT_NOTIFY_PUT(0);
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
}
Rename HARNESS_FORK*() macros to HRN_FORK*(). This matches the new pattern for harness macro naming and is shorter.
2021-07-13 17:58:23 +02:00
HRN_FORK_PARENT_END();
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
}
Rename HARNESS_FORK*() macros to HRN_FORK*(). This matches the new pattern for harness macro naming and is shorter.
2021-07-13 17:58:23 +02:00
HRN_FORK_END();
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
```
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
#### Testing using a shim
Update documentation/links to main branch.
2021-10-13 18:01:53 +02:00
A PostgreSQL libpq shim is provided to simulate interactions with PostgreSQL. Below is a simple example. See [harnessPq.h](https://github.com/pgbackrest/pgbackrest/blob/main/test/src/common/harnessPq.h) for more details.
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
```
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
// Set up two standbys but no primary
harnessPqScriptSet((HarnessPq [])
{
Remove support for PostgreSQL 9.0/9.1/9.2. Our new policy is to support ten versions of PostgreSQL, the five supported releases and the last five EOL releases. As of PostgreSQL 15, that means 9.0/9.1/9.2 are no longer supported by pgBackRest. Remove all logic associated with 9.0/9.1/9.2 and update the tests. Document the new support policy. Update InfoPg to read/write control versions for the history in backup.info, since we can no longer rely on the mappings being available. In theory this could have been an issue after removing 8.3/8.4 if anybody was using a version that old.
2022-12-20 07:20:47 +02:00
HRNPQ_MACRO_OPEN_GE_96(1, "dbname='postgres' port=5432", PG_VERSION_96, "/pgdata", true, NULL, NULL),
HRNPQ_MACRO_OPEN_GE_96(8, "dbname='postgres' port=5433", PG_VERSION_96, "/pgdata", true, NULL, NULL),
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
// Close the "inner" session first (8) then the outer (1)
HRNPQ_MACRO_CLOSE(8),
HRNPQ_MACRO_CLOSE(1),
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
HRNPQ_MACRO_DONE()
});
TEST_ERROR(cmdCheck(), ConfigError, "primary database not found\nHINT: check indexed pg-path/pg-host configurations");
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
```
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
### Running a Unit Test
**Code Coverage**
Unit tests are run for all files that are listed in `define.yaml` and a coverage report generated for each file listed under the tag `coverage:`. Note that some files are listed in multiple `coverage:` sections for a module; in this case, each test for the file being modified should be specified for the module in which the file exists (e.g. `--module=storage --test=posix --test=gcs`, etc.) or, alternatively, simply run the module without the `--test` option. It is recommended that a `--vm` be specified since running the same test for multiple vms is unnecessary for coverage. The following example would run the test set from the **define.yaml** section detailed above.
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
```
Remove test.pl --smart, --dev, and --dev-test options. --smart is now the default mode. Since --dev is now just an alias for --no-optimize, remove it. --dev-test has been a noop for a while, so this seems like a good time to remove it. Also make the C auto-generator skip writing files that have not changed to avoid updating the timestamp.
2021-07-20 18:01:10 +02:00
pgbackrest/test/test.pl --vm-out --module=command --test=check --vm=u20
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
```
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
> **NOTE:** Not all systems perform at the same speed, so if a test is timing out, try rerunning with another vm.
Because a test run has not been specified, a coverage report will be generated and written to the local file system under the pgBackRest directory `test/result/coverage/lcov/index.html` and a file with only the highlighted code that has not been covered will be written to `test/result/coverage/coverage.html`.
If 100 percent code coverage has not been achieved, an error message will be displayed, for example: `ERROR: [125]: c module command/check/check is not fully covered`
**Debugging with files**
Sometimes it is useful to look at files that were generated during the test. The default for running any test is that, at the start/end of the test, the test harness will clean up all files and directories created. To override this behavior, a single test run must be specified and the option `--no-cleanup` provided. Again, continuing with the check command, from **define.yaml** above, there are four tests. Below, test one will be run and nothing will be cleaned up so that the files and directories in `test/test-0` can be inspected.
```
Remove test.pl --smart, --dev, and --dev-test options. --smart is now the default mode. Since --dev is now just an alias for --no-optimize, remove it. --dev-test has been a noop for a while, so this seems like a good time to remove it. Also make the C auto-generator skip writing files that have not changed to avoid updating the timestamp.
2021-07-20 18:01:10 +02:00
pgbackrest/test/test.pl --vm-out --module=command --test=check --run=1 --no-cleanup
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
```
Update contributing documentation and add pull request template.
2021-07-21 19:14:06 +02:00
### Understanding Test Output
The following is a small sample of a typical test output.
```
run 8 - expireTimeBasedBackup()
run 8/1 ------------- L2285 no current backups
000.002s L2298 empty backup.info
000.009s 000.007s L2300 no backups to expire
```
**run 8 - expireTimeBasedBackup()** - indicates the run number (8) within the module and the parameter provided to testBegin, e.g. `testBegin("expireTimeBasedBackup()")`
**run 8/1 ------------- L2285 no current backups** - this is the first test (1) in run 8 which is the `TEST_TITLE("no current backups");` at line number 2285.
**000.002s L2298 empty backup.info** - the first number, 000.002s, is the time in seconds that the test started from the beginning of the run. L2298 is the line number of the test and `empty backup.info` is the test comment.
**000.009s 000.007s L2300 no backups to expire** - again, 000.009s, is the time in seconds that the test started from the beginning of the run. The second number, 000.007s, is the run time of the **previous** test (i.e. `empty backup.info` test took 000.007 seconds to execute). L2300 is the line number of the test and `no backups to expire` is the test comment.
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
## Adding an Option
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Add .inc extension to C files included in other C files. These files were never intended to be compiled on their own so the .c extension was a bit misleading. In particular Meson does not like .c files that are not intended to be compiled independently. Leave header files as is since they are already protected against being included more than once and are never expected to be compiled.
2022-05-31 22:06:41 +02:00
Options can be added to a command or multiple commands. Options can be configuration file only, command-line only or valid for both. Once an option is successfully added, the `config.auto.h` and `parse.auto.c.inc` files will automatically be generated by the build system.
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
To add an option, two files need be to be modified:
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
- `src/build/config/config.yaml`
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update contributing.xml with rendering changes from def7d513. Also update help.xml path missed in f4e1babf.
2021-10-13 17:43:14 +02:00
- `src/build/help/help.xml`
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
These files are discussed in the following sections along with how to verify the `help` command output.
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
### config.yaml
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
There are detailed comment blocks above each section that explain the rules for defining commands and options. Regarding options, there are two types: 1) command line only, and 2) configuration file. With the exception of secrets, all configuration file options can be passed on the command line. To configure an option for the configuration file, the `section:` key must be present.
Update documentation/links to main branch.
2021-10-13 18:01:53 +02:00
The `option:` section is broken into sub-sections by a simple comment divider (e.g. `# Repository options`) under which the options are organized alphabetically by option name. To better explain this section, two hypothetical examples will be discussed. For more details, see [config.yaml](https://github.com/pgbackrest/pgbackrest/blob/main/src/build/config/config.yaml).
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
Fix links and update child process example. Removed colon from example titles to fix links, fixed test.yml link, and updated the example for the parent/child test process to use the latest macros instead of sleep().
2021-08-09 22:56:06 +02:00
#### EXAMPLE 1 hypothetical command line only option
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
```
set:
type: string
command:
backup:
depend:
option: stanza
required: false
restore:
default: latest
command-role:
main: {}
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
```
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
Note that `section:` is not present thereby making this a command-line only option defined as follows:
- `set` - the name of the option
- `type` - the type of the option. Valid values for types are: `boolean`, `hash`, `integer`, `list`, `path`, `size`, `string`, and `time`
- `command` - list each command for which the option is valid. If a command is not listed, then the option is not valid for the command and an error will be thrown if it is attempted to be used for that command. In this case the valid commands are `backup` and `restore`.
- `backup` - details the requirements for the `--set` option for the `backup` command. It is dependent on the option `--stanza`, meaning it is only allowed to be specified for the `backup` command if the `--stanza` option has been specified. And `required: false` indicates that the `--set` option is never required, even with the dependency.
- `restore` - details the requirements for the `--set` option for the `restore` command. Since `required:` is omitted, it is not required to be set by the user but it is required by the command and will default to `latest` if it has not been specified by the user.
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
- `command-role` - defines the processes for which the option is valid. `main` indicates the option will be used by the main process and not be passed on to other local/remote processes.
Fix links and update child process example. Removed colon from example titles to fix links, fixed test.yml link, and updated the example for the parent/child test process to use the latest macros instead of sleep().
2021-08-09 22:56:06 +02:00
#### EXAMPLE 2 hypothetical configuration file option
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
```
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
repo-test-type:
section: global
type: string
group: repo
default: full
allow-list:
- full
- diff
- incr
command:
backup: {}
restore: {}
command-role:
main: {}
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
```
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
- `repo-test-type` - the name of the option
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Fix links and update child process example. Removed colon from example titles to fix links, fixed test.yml link, and updated the example for the parent/child test process to use the latest macros instead of sleep().
2021-08-09 22:56:06 +02:00
- `section` - the section of the configuration file where this option is valid (omitted for command line only options, see [Example 1](#example-1-hypothetical-command-line-only-option) above)
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
- `type` - the type of the option. Valid values for types are: `boolean`, `hash`, `integer`, `list`, `path`, `size`, `string`, and `time`
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
- `group` - indicates that this option is part of the `repo` group of indexed options and therefore will follow the indexing rules e.g. `repo1-test-type`.
Fix links and update child process example. Removed colon from example titles to fix links, fixed test.yml link, and updated the example for the parent/child test process to use the latest macros instead of sleep().
2021-08-09 22:56:06 +02:00
- `default` - sets a default for the option if the option is not provided when the command is run. The default can be global (as it is here) or it can be specified for a specific command in the command section (as in [Example 1](#example-1-hypothetical-command-line-only-option) above).
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
- `allow-list` - lists the allowable values for the option for all commands for which the option is valid.
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
- `command` - list each command for which the option is valid. If a command is not listed, then the option is not valid for the command and an error will be thrown if it is attempted to be used for that command. In this case the valid commands are `backup` and `restore`.
- `command-role` - defines the processes for which the option is valid. `main` indicates the option will be used by the main process and not be passed on to other local/remote processes.
At compile time, the `config.auto.h` file will be generated to contain the constants used for options in the code. For the C enums, any dashes in the option name will be removed, camel-cased and prefixed with `cfgOpt`, e.g. `repo-path` becomes `cfgOptRepoPath`.
Migrate command-line help generation to C. Command-line help is now generated at build time so it does not need to be committed. This reduces churn on commits that add configuration and/or update the help. Since churn is no longer an issue, help.auto.c is bzip2 compressed to save space in the binary. The Perl config parser (Data.pm) has been moved to doc/lib since the Perl build path is no longer required. Likewise doc/xml/reference.xml has been moved to src/build/help/help.xml since it is required at build time.
2021-09-09 00:16:06 +02:00
### help.xml
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
All options must be documented or the system will error during the build. To add an option, find the command section identified by `command id="COMMAND"` section where `COMMAND` is the name of the command (e.g. `expire`) or, if the option is used by more than one command and the definition for the option is the same for all of the commands, the `operation-general title="General Options"` section.
To add an option, add the following to the `<option-list>` section; if it does not exist, then wrap the following in `<option-list>` `</option-list>`. This example uses the boolean option `force` of the `restore` command. Simply replace that with your new option and the appropriate `summary`, `text` and `example`.
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
```
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
<option id="force" name="Force">
<summary>Force a restore.</summary>
<text>By itself this option forces the <postgres/> data and tablespace paths to be completely overwritten. In combination with <br-option>--delta</br-option> a timestamp/size delta will be performed instead of using checksums.</text>
<example>y</example>
</option>
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
```
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
> **IMPORTANT:** A period (.) is required to end the `summary` section.
### Testing the help
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
It is important to run the `help` command unit test after adding an option in case a change is required:
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
```
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
pgbackrest/test/test.pl --module=command --test=help --vm-out
```
To verify the `help` command output, build the pgBackRest executable:
```
pgbackrest/test/test.pl --vm=none --build-only
```
Use the pgBackRest executable to test the help output:
```
test/bin/none/pgbackrest help backup repo-type
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
```
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
### Testing the documentation
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
To quickly view the HTML documentation, the `--no-exe` option can be passed to the documentation generator in order to bypass executing the code elements:
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
```
Fix typo in contributing guide. Not sure how this got broken but it was probably an errant search and replace.
2021-10-28 19:28:49 +02:00
pgbackrest/doc/doc.pl --out=html --no-exe
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
```
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
The generated HTML files will be placed in the `doc/output/html` directory where they can be viewed locally in a browser.
Update contributing documentation and add pull request template.
2021-07-21 19:14:06 +02:00
If Docker is installed, it will be used by the documentation generator to execute the code elements while building the documentation, therefore, the `--no-exe` should be omitted, (i.e. `pgbackrest/doc/doc.pl --output=html`). `--no-cache` may be used to force a full build even when no code elements have changed since the last build. `--pre` will reuse the container definitions from the prior build and saves time during development.
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
The containers created for documentation builds can be useful for manually testing or trying out new code or features. The following demonstrates building through just the `quickstart` section of the `user-guide` without encryption.
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
```
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
pgbackrest/doc/doc.pl --out=html --include=user-guide --require=/quickstart --var=encrypt=n --no-cache --pre
Document how to contribute to pgBackRest. There's a lot more to be done here, but this is a good start.
2019-10-08 21:27:17 +02:00
```
Update contributing documentation. Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 19:21:25 +02:00
The resulting Docker containers can be listed with `docker ps` and the container can be entered with `docker exec doc-pg-primary bash`. Additionally, the `-u` option can be added for entering the container as a specific user (e.g. `postgres`).
Update contributing documentation and add pull request template.
2021-07-21 19:14:06 +02:00
## Submitting a Pull Request
Before submitting a Pull Request:
Update documentation/links to main branch.
2021-10-13 18:01:53 +02:00
- Does it meet the [coding standards](https://github.com/pgbackrest/pgbackrest/blob/main/CODING.md)?
Update contributing documentation and add pull request template.
2021-07-21 19:14:06 +02:00
- Have [Unit Tests](#writing-a-unit-test) been written and [run](#running-a-unit-test) with 100% coverage?
- If your submission includes changes to the help or online documentation, have the [help](#testing-the-help) and [documentation](#testing-the-documentation) tests been run?
- Has it passed continuous integration testing? Simply renaming your branch with the appendix `-cig` and pushing it to your GitHub account will initiate GitHub Actions to run CI tests.
When submitting a Pull Request:
- Provide a short submission title.
- Write a detailed comment to describe the purpose of your submission and any issue(s), if any, it is resolving; a link to the GitHub issue is also helpful.
Document required base branch for pull requests. Be explicit when submitting a PR about which branch to use as the base.
2022-05-10 00:07:11 +02:00
- Select the `integration` branch as the base for your PR, do not select `main` nor any other branch.
Update contributing documentation and add pull request template.
2021-07-21 19:14:06 +02:00
After submitting a Pull Request:
- One or more reviewers will be assigned.
- Respond to any issues (conversations) in GitHub but do not resolve the conversation; the reviewer is responsible for ensuring the issue raised has been resolved and marking the conversation resolved. It is helpful to supply the commit in your reply if one was submitted to fix the issue.
Lastly, thank you for contributing to pgBackRest!
Reference in New Issue Copy Permalink