1
0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2024-12-14 10:13:05 +02:00
Commit Graph

41 Commits

Author SHA1 Message Date
David Steele
430efff98a Update documentation/links to main branch. 2021-10-13 12:01:53 -04:00
David Steele
1212668d5e Update contributing.xml with rendering changes from def7d513.
Also update help.xml path missed in f4e1babf.
2021-10-13 11:43:14 -04:00
David Steele
f4e1babf6b Migrate command-line help generation to C.
Command-line help is now generated at build time so it does not need to be committed. This reduces churn on commits that add configuration and/or update the help.

Since churn is no longer an issue, help.auto.c is bzip2 compressed to save space in the binary.

The Perl config parser (Data.pm) has been moved to doc/lib since the Perl build path is no longer required.

Likewise doc/xml/reference.xml has been moved to src/build/help/help.xml since it is required at build time.
2021-09-08 18:16:06 -04:00
David Steele
475b57c89b Allow additional memory to be allocated with a mem context.
The primary benefit is that objects can allocate memory for their struct with the context, which saves an additional allocation and makes it easier to read context/allocation dumps. Also, the memory context does not need to be stored with the object since it can be determined using the object pointer.

Object pointers cannot be moved, so this means whatever additional memory is allocated cannot be resized. That makes the additional memory ideal for object structs, but not so much for allocating a list that might change size.

Mem contexts can no longer be reused since they will probably be the wrong size so their memory is freed on memContextFree(). This still means fewer allocations and frees overall.

Interfaces still need to be freed by mem context so the old objMove() and objFree() have been preserved as objMoveContext() and objFreeContext(). This will be addressed in a future commit.
2021-09-01 11:10:35 -04:00
Cynthia Shang
71b654fc29
Fix links and update child process example.
Removed colon from example titles to fix links, fixed test.yml link, and updated the example for the parent/child test process to use the latest macros instead of sleep().
2021-08-09 16:56:06 -04:00
David Steele
b47a07b8b9 Remove generated config.auto.c file.
This file duplicated the command list that already exists in parse.auto.c.

Combine the data from config.auto.c into parse.auto.c and adjust the interface functions as needed. Quite a few were able to be moved to parse.c as static.
2021-07-29 14:40:30 -04:00
Cynthia Shang
e3d05e2b4a
Update contributing documentation and add pull request template. 2021-07-21 13:14:06 -04:00
David Steele
9ee9b1fad6 Remove test.pl --smart, --dev, and --dev-test options.
--smart is now the default mode. Since --dev is now just an alias for --no-optimize, remove it. --dev-test has been a noop for a while, so this seems like a good time to remove it.

Also make the C auto-generator skip writing files that have not changed to avoid updating the timestamp.
2021-07-20 12:01:10 -04:00
David Steele
c5ae047e76 Partial migration of config code generation to C.
Parse enough of config.yaml to auto-generate config.auto.h and config.auto.c.

This commit implements most of the infrastructure needed to migrate the rest of the build code to C, but each set of auto-generated files will present its own challenges.

The build is now dependent on libyaml. At this point there is no need for a hard requirement, but that will come soon so it seems better to add the dependency now.
2021-07-18 19:02:01 -04:00
David Steele
6397d73535 Update test container OS versions.
Update Ubuntu 12.04 to 16.04. Version 16.04 is recently EOL but testing on an old version is beneficial.
Update Ubuntu 18.04 to 20.04.
Update Fedora 32 to 33. Version 34 would have been preferred but there were some build issues, i.e. the default shell did not work with configure, and after ksh was installed configure locked up.

Add --no-install-recommends to apt-get commands to save a bit of time and space.

Update test Dockerfile to run in multiple steps. This makes the container larger but also makes rebuilding after changes faster. The --squash option may be used to keep the container small.

Remove obsolete casts in protocol/parallel module. These casts were included in the original migration because Ubuntu 12.04 32-bit gcc required them, but Ubuntu 16.04 32-bit gcc complains. There is no production issue here since at this point in the code the file descriptors are guaranteed to be >= 0.
2021-07-18 17:29:20 -04:00
David Steele
1ace1ac938 Improve HRN_FORK*() macros.
Simplify HRN_FORK_CHILD_BEGIN() by adding optional parameters with the common defaults.

Add _FD() to macros that retrieve file descriptors to make their purpose clearer.
2021-07-13 14:22:53 -04:00
David Steele
76cfbf833d Rename HARNESS_FORK*() macros to HRN_FORK*().
This matches the new pattern for harness macro naming and is shorter.
2021-07-13 11:58:23 -04:00
David Steele
39c1c10120 Remove TEST_PATH_REPO, TEST_PATH_PG, and TEST_PATH_SPOOL constants.
It seems better to use TEST_PATH in combination with a constant string rather than have a number of different path constants. This improves readability and reduces confusion about which constant should be used.
2021-06-28 13:28:28 -04:00
David Steele
8250990afb Replace harnessCfgLoad*() functions with HRN_CFG_LOAD() macro.
HRN_CFG_LOAD() handles the majority of test configuration loads and has various options for special cases.

It was not clear when to use harnessCfgLoadRaw() vs harnessCfgLoad(). Now "raw" functionality is granular and enabled by parameters, e.g. noStd.
2021-06-01 09:03:44 -04:00
David Steele
ccac75e7de
Consolidate RHEL/CentOS user guide into a single document.
The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference.

Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions.

Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 16:03:15 -04:00
David Steele
aed3d468a1 Rename strNew() to strNewZ() and add parameter-less strNew().
Replace all instances of strNew("") with strNew() and use strNewZ() for non-empty zero-terminated strings. Besides saving a useless parameter, this will allow smarter memory allocation in a future commit by signaling intent, in general, to append or not.

In the tests use STRDEF() or VARSTRDEF() where more appropriate rather than blindly replacing with strNewZ(). Also replace strLstAdd() with strLstAddZ() where appropriate for the same reason.
2021-05-21 17:36:43 -04:00
Cynthia Shang
68faf1482a
Update contributing documentation.
Add more examples of how to write code, add configuration options, test, etc.
2021-05-21 13:21:25 -04:00
David Steele
fe4ba455ed Move configuration definition to src/build/config/config.yaml.
Moving to YAML allows the configuration data to be read by C programs.

Also go back to using YAML::XS since it is the only implementation that has proper boolean support.
2021-03-08 16:01:05 -05:00
David Steele
d2057c53bd Use YAML::Any module instead of YAML::XS in Perl.
YAML::XS requires libyaml so it not as portable as pure Perl versions of YAML.

Instead of using YAML:PP just use the general YAML::Any module which uses whatever is installed. We are not concerned about performance for YAML so whatever works is fine.
2021-01-24 15:06:38 -05:00
David Steele
d01669aa58 Move most tests to Github Actions.
Testing on Travis-CI has been getting slower (from ~18 minutes to 3-6 hours) and the travis-ci.org service will be terminated at the end of the year. Moving to travis-ci.com is an option but the quotas are too low for our purposes.

Instead use Github Actions, which does not currently have quotas, and runs our current tests with just a few tweaks.

This still leaves multi-architecture tests on Travis-CI but we may be able to run those and stay within the new quotas.

Also fix a minor bug in restoreTest.c exposed by Github Actions using a different name for the user and group.
2020-12-09 15:19:01 -05:00
David Steele
87996558d2
Replace double type with time in config module.
The C code does not use doubles to represent seconds like the Perl code did so time can be represented as an integer which reduces the number of data types that config has to understand.

Also remove Variant doubles since they are no longer used.

Note that not all double code was removed since we still need to display times to the user in seconds and it is possible for the times to be fractional. In the future this will likely be simplified by storing the original user input and using that value when the time needs to be displayed.
2020-12-09 08:59:51 -05:00
David Steele
74151d57ba Add apt-get update to contributing guide.
This allows apt-get to work correctly even if the base container is a bit out of date.
2020-10-05 08:42:18 -04:00
David Steele
3e9dce0d76 Rename strPtr()/strPtrNull() to strZ()/strZNull().
We use the Z suffix in many functions to indicate that we are expecting a zero-terminated string so make this function conform to the pattern.

As a bonus the new name is a bit shorter, which is a good quality in a commonly-used function.
2020-07-30 07:49:06 -04:00
Stephen Frost
a021c9fe05
Add bzip2 compression support.
bzip2 is a widely available, high-quality data compressor. It typically compresses files to within 10% to 15% of the best available techniques (the PPM family of statistical compressors), while being around twice as fast at compression and six times faster at decompression.

bzip2 is currently available on all supported platforms.
2020-05-05 16:49:01 -04:00
David Steele
47aa765375 Add Zstandard compression support.
Zstandard is a fast lossless compression algorithm targeting real-time compression scenarios at zlib-level and better compression ratios. It's backed by a very fast entropy stage, provided by Huff0 and FSE library.

Zstandard version >= 1.0 is required, which is generally only available on newer distributions.
2020-05-04 15:25:27 -04:00
David Steele
c9481bb95f Update contributing.xml to use Ubuntu 18.04.
Ubuntu 19.04 was EOL in January and has been removed from the repository servers.

It may be best to stick to LTS versions from now on.
2020-04-14 14:51:14 -04:00
Cynthia Shang
63b9f46a34 Update contributing documentation and remove test/README.md.
When the Vagrant file was updated to use pgbackrest/ vs /backrest/ as the location for executing tests and building the documentation, parts of the contributing.xml (and hence the CONTRIBUTING.md) were not updated since some parts of the document are not actually executed when the CONTRIBUTING.md is built from contributing.xml: those parts that are executed were updated but those parts that are not executed were not.

This commit fixes the contributing.xml issue but also removes test/README.md as its contents were out of date and redundant given that they are covered in CONTRIBUTING.md.
2020-04-09 18:25:25 -04:00
David Steele
4c831d8e83 Use --clean-only for reproducible builds in contributing documentation.
If the work or result directories already contain data then the docs might be generated slightly differently.  Doing a clean ensures they will always produce the same output (provided the code does not change).
2020-03-22 13:50:31 -04:00
David Steele
d70ca2592f Suppress timestamps in contributing documentation.
Timestamps cause a lot of churn and make it hard to tell if important changes are working.
2020-03-22 10:17:03 -04:00
David Steele
6a9fbccf76 Remove custom lcov build from contributing documentation.
Using the stock lcov no longer seems to be a problem, probably due to a better implementation in CoverageTest.pm.
2020-03-21 19:08:27 -04:00
David Steele
56fb399373 Build contributing documentation on Travis CI.
Building the contributing document has some special requirements because it runs Docker in Docker so the repo path must align on the host and all Docker containers.  Run `pgbackrest/doc/doc.pl` from within the home directory of the user that will do the doc build, e.g. `home/vagrant`.  If the repo is not located directly in the home directory, e.g. `/home/vagrant/pgbackrest`, then a symlink may be used, e.g. `ln -s /path/to/repo /home/vagrant/pgbackrest`.

Mount the repo in the Vagrantfile at /home/vagrant/pgbackrest but provide a link from the old location at /backrest to make the transition less painful.
2020-03-21 18:45:58 -04:00
David Steele
ee2e15bf55 Add missing package to contributing documentation. 2020-03-21 09:51:12 -04:00
Cynthia Shang
f538da0571 Updates to contributing documentation. 2020-03-18 17:30:36 -04:00
David Steele
4cd060b7fe Generate src/build/aclocal.m4 automatically.
This file is required when macros from the autoconf archive are used in configure.ac
2020-03-14 12:48:08 -04:00
Cynthia Shang
d68771a4a5 Fix incorrect lcov version in contributing guide. 2020-02-26 20:40:24 -05:00
David Steele
d41eea685a Change meaning of TEST_RESULT_STR() macro.
This macro was created before the String object existed so subsequent usage with String always included a lot of strPtr() wrapping.

TEST_RESULT_STR_Z() had already been introduced but a wholesale replacement of TEST_RESULT_STR() was not done since the priority was on the C migration.

Update all calls to (old) TEST_RESULT_STR() with one of the following variants: (new) TEST_RESULT_STR(), TEST_RESULT_STR_Z(), TEST_RESULT_Z(), TEST_RESULT_Z_STR().
2019-12-26 18:08:27 -07:00
Stephen Frost
3394a50170 Remove unused Perl library missed in f0ef73db. 2019-12-17 21:25:54 -05:00
David Steele
1db9e3b144 Remove *MP() macros variants.
Adding a dummy column which is always set by the P() macro allows a single macro to be used for parameters or no parameters without violating C's prohibition on the {} initializer.

-Wmissing-field-initializers remains disabled because it still gives wildly different results between versions of gcc.
2019-11-17 15:10:40 -05:00
David Steele
eca00e0be0 Add building a development environment to contributing documentation.
This documentation shows how to build a development environment on Ubuntu 19.04 and should work for other Debian-based distros.

Note that this document is not included in automated testing due to some unresolved issues with Docker in Docker on Travis CI.  We'll address this in the future when we add contributing documentation to the website.
2019-11-07 13:11:01 -05:00
Cynthia Shang
82baf1e521 Fix typo. 2019-10-10 08:33:40 -04:00
Cynthia Shang
38b72eded4 Document how to contribute to pgBackRest.
There's a lot more to be done here, but this is a good start.
2019-10-08 15:27:17 -04:00