pgbackrest/doc/xml/contributing.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE doc SYSTEM "doc.dtd">
<doc title="{[project]}" subtitle="Contributing to {[project]}" toc="y" cmd-line-len="132">
    <description>{[project]} Contributing Guidelines.</description>

    <!-- Variables used by the rest of the script ============================================================================== -->
    <variable-list>
        <variable key="host-user-id" eval="y">use English; $UID</variable>

        <!-- Options to pass to test.pl so log output is reproducible -->
        <variable key="test-cmd-extra">--no-log-timestamp</variable>

        <variable key="host-contrib-id">contrib</variable>
        <variable key="host-contrib">pgbackrest-dev</variable>
        <variable key="host-contrib-user">{[host-user]}</variable>
        <variable key="host-contrib-image">pgbackrest/doc:contrib</variable>

        <variable key="github-url-test">{[github-url-main]}/test/src</variable>
        <variable key="github-url-test-common">{[github-url-main]}/test/src/common</variable>
        <variable key="github-url-src">{[github-url-main]}/src</variable>
        <variable key="github-url-src-common">{[github-url-src]}/common</variable>

        <variable key="cwd" eval="y">use Cwd qw(cwd); cwd()</variable>
    </variable-list>

    <!-- Setup hosts used to build the documentation =========================================================================== -->
    <host-define if="{[os-type-is-debian]}" image="{[host-contrib-image]}" from="ubuntu:20.04">
        {[copy-ca-cert]}

        # Fix root tty
        RUN sed -i 's/^mesg n/tty -s \&amp;\&amp; mesg n/g' /root/.profile &amp;&amp; \

        # Suppress dpkg interactive output
            rm /etc/apt/apt.conf.d/70debconf

        # Install base packages
        RUN apt-get update &amp;&amp; \
            apt-get install -y sudo ssh curl vim 2>&amp;1

        # Add test user with sudo privileges
        RUN adduser --disabled-password --uid={[host-user-id]} --gecos "" {[host-user]} &amp;&amp; \
            echo '%{[host-user]} ALL=(ALL) NOPASSWD: ALL' >> /etc/sudoers

        WORKDIR /home/{[host-user]}

        ENTRYPOINT service ssh restart &amp;&amp; bash
    </host-define>

    <!-- ======================================================================================================================= -->
    <section id="contents">
        <title>Table of Contents</title>

        <p><link section="/introduction">Introduction</link></p>
        <p><link section="/environment">Building a Development Environment</link></p>
        <p><link section="/coding">Coding</link></p>
        <p><link section="/testing">Testing</link></p>
        <p><link section="/pr">Submitting a Pull Request</link></p>
    </section>

    <!-- ======================================================================================================================= -->
    <section id="introduction">
        <title>Introduction</title>

        <p>This documentation is intended to assist contributors to <backrest/> by outlining some basic steps and guidelines for contributing to the project.</p>

        <p>Code fixes or new features can be submitted via pull requests. Ideas for new features and improvements to existing functionality or documentation can be <link url="{[github-url-issues]}">submitted as issues</link>. You may want to check the <link url="{[github-url-projects]}">Project Boards</link> to see if your suggestion has already been submitted.</p>

        <p>Bug reports should be <link url="{[github-url-issues]}">submitted as issues</link>. Please provide as much information as possible to aid in determining the cause of the problem.</p>

        <p>You will always receive credit in the <link page="{[backrest-page-release]}">release notes</link> for your contributions.</p>

        <p>Coding standards are defined in <link url="{[github-url-main]}/CODING.md">CODING.md</link> and some important coding details and an example are provided in the <link section="/coding">Coding</link> section below. At a minimum, unit tests must be written and run and the documentation generated before <link section="/pr">submitting a Pull Request</link>; see the <link section="/testing">Testing</link> section below for details.</p>
    </section>

    <!-- ======================================================================================================================= -->
    <section id="environment">
        <title>Building a Development Environment</title>

        <p>This example is based on <proper>Ubuntu 20.04</proper>, but it should work on many versions of <proper>Debian</proper> and <proper>Ubuntu</proper>.</p>

        <host-add id="{[host-contrib-id]}" name="{[host-contrib]}" user="{[host-contrib-user]}" image="{[host-contrib-image]}" os="u20" option="-v /var/run/docker.sock:/var/run/docker.sock -v {[cwd]}/test:{[cwd]}/test" mount="/home/{[host-contrib-user]}/pgbackrest:/home/{[host-contrib-user]}/pgbackrest"/>

        <execute-list host="{[host-contrib]}">
            <title>Install development tools</title>

            <execute user="root" pre="y" show="n">
                <exe-cmd>
                    apt-get update
                </exe-cmd>
            </execute>

            <execute user="root" pre="y">
                <exe-cmd>
                    apt-get install rsync git devscripts build-essential valgrind lcov autoconf
                    autoconf-archive libssl-dev zlib1g-dev libxml2-dev libpq-dev pkg-config
                    libxml-checker-perl libyaml-perl libdbd-pg-perl liblz4-dev liblz4-tool
                    zstd libzstd-dev bzip2 libbz2-dev libyaml-dev ccache meson
                </exe-cmd>
                <exe-cmd-extra>-y 2>&amp;1</exe-cmd-extra>
            </execute>

            <!-- Clean so the tests give consistent output -->
            <execute show="n">
                <exe-cmd>
                    pgbackrest/test/test.pl --clean-only
                </exe-cmd>
            </execute>
        </execute-list>

        <p>Some unit tests and all the integration tests require <proper>Docker</proper>.  Running in containers allows us to simulate multiple hosts, test on different distributions and versions of <postgres/>, and use sudo without affecting the host system.</p>

        <execute-list host="{[host-contrib]}">
            <title>Install Docker</title>

            <execute pre="y">
                <exe-cmd>curl -fsSL https://get.docker.com | sudo sh</exe-cmd>
                <exe-cmd-extra>2>&amp;1</exe-cmd-extra>
            </execute>

            <execute user="root">
                <exe-cmd>
                    usermod -aG docker `whoami`
                </exe-cmd>
            </execute>

            <!-- Hack permissions on the docker socket so the local user can run commands -->
            <execute user="root" show="n">
                <exe-cmd>
                    chmod 666 /var/run/docker.sock
                </exe-cmd>
            </execute>
        </execute-list>

        <p>This clone of the <backrest/> repository is sufficient for experimentation.  For development, create a fork and clone that instead.</p>

        <execute-list host="{[host-contrib]}">
            <title>Clone <backrest/> repository</title>

            <execute skip="y">
                <exe-cmd>
                    git clone https://github.com/pgbackrest/pgbackrest.git
                </exe-cmd>
            </execute>
        </execute-list>

        <p>If using a RHEL-based system, the CPAN XML parser is required to run <file>test.pl</file> and <file>doc.pl</file>. Instructions for installing Docker and the XML parser can be found in the <file>README.md</file> file of the <backrest/> <link url="{[github-url-main]}/doc">doc</link> directory in the section <quote>The following is a sample RHEL 7 configuration that can be used for building the documentation</quote>. NOTE that the <quote>Install latex (for building PDF)</quote> section is not required since testing of the docs need only be run for HTML output.</p>
    </section>

    <!-- ======================================================================================================================= -->
    <section id="coding">
        <title>Coding</title>

        <p>The following sections provide information on some important concepts needed for coding within <backrest/>.</p>

        <!-- =================================================================================================================== -->
        <section id="memory-context">
            <title>Memory Contexts</title>

            <p>Memory is allocated inside contexts and can be long lasting (for objects) or temporary (for functions). In general, use <code>OBJ_NEW_BEGIN(MyObj)</code> for objects and <code>MEM_CONTEXT_TEMP_BEGIN()</code> for functions. See <link url="{[github-url-src-common]}/memContext.h">memContext.h</link> for more details and the <link section="/coding/coding-example">Coding Example</link> below.</p>
        </section>

        <!-- =================================================================================================================== -->
        <section id="message-logging">
            <title>Logging</title>

            <p>Logging is used for debugging with the built-in macros <code>FUNCTION_LOG_*()</code> and <code>FUNCTION_TEST_*()</code> which are used to trace parameters passed to/returned from functions. <code>FUNCTION_LOG_*()</code> macros are used for production logging whereas <code>FUNCTION_TEST_*()</code> macros will be compiled out of production code. For functions where no parameter is valuable enough to justify the cost of debugging in production, use <code>FUNCTION_TEST_BEGIN()/FUNCTION_TEST_END()</code>, else use <code>FUNCTION_LOG_BEGIN(someLogLevel)/FUNCTION_LOG_END()</code>. See <link url="{[github-url-src-common]}/debug.h">debug.h</link> for more details and the <link section="/coding/coding-example">Coding Example</link> below.</p>

            <p>Logging is also used for providing information to the user via the <code>LOG_*()</code> macros, such as <code>LOG_INFO("some informational message")</code> and <code>LOG_WARN_FMT("no prior backup exists, %s backup has been changed to full", strZ(cfgOptionDisplay(cfgOptType)))</code> and also via <code>THROW_*()</code> macros for throwing an error. See <link url="{[github-url-src-common]}/log.h">log.h</link> and <link url="{[github-url-src-common]}/error/error.h">error.h</link> for more details and the <link section="/coding/coding-example">Coding Example</link> below.</p>
        </section>

        <!-- =================================================================================================================== -->
        <section id="coding-example">
            <title>Coding Example</title>

            <p>The example below is not structured like an actual implementation and is intended only to provide an understanding of some of the more common coding practices. The comments in the example are only here to explain the example and are not representative of the coding standards. Refer to the Coding Standards document (<link url="{[github-url-main]}/CODING.md">CODING.md</link>) and sections above for an introduction to the concepts provided here. For an actual implementation, see <link url="{[github-url-src]}/db/db.h">db.h</link> and <link url="{[github-url-src]}/db/db.c">db.c</link>.</p>

            <!-- =============================================================================================================== -->
            <section id="coding-ex1">
                <title>Example: hypothetical basic object construction</title>

                <code-block type="c">
/*
 *  HEADER FILE - see db.h for a complete implementation example
 */

// Typedef the object declared in the C file
typedef struct MyObj MyObj;

// Constructor, and any functions in the header file, are all declared on one line
MyObj *myObjNew(unsigned int myData, const String *secretName);

// Declare the publicly accessible variables in a structure with Pub appended to the name
typedef struct MyObjPub         // First letter upper case
{
    unsigned int myData;        // Contents of the myData variable
} MyObjPub;

// Declare getters and setters inline for the publicly visible variables
// Only setters require "Set" appended to the name
FN_INLINE_ALWAYS unsigned int
myObjMyData(const MyObj *const this)
{
    return THIS_PUB(MyObj)->myData;    // Use the built-in THIS_PUB macro
}

// Destructor
FN_INLINE_ALWAYS void
myObjFree(MyObj *const this)
{
    objFree(this);
}

// TYPE and FORMAT macros for function logging
#define FUNCTION_LOG_MY_OBJ_TYPE                                            \
    MyObj *
#define FUNCTION_LOG_MY_OBJ_FORMAT(value, buffer, bufferSize)               \
    FUNCTION_LOG_OBJECT_FORMAT(value, myObjToLog, buffer, bufferSize)

/*
 * C FILE - see db.c for a more complete and actual implementation example
 */

// Declare the object type
struct MyObj
{
    MyObjPub pub;               // Publicly accessible variables must be first and named "pub"
    const String *name;         // Pointer to lightweight string object - see string.h
};

// Object constructor, and any functions in the C file, have the return type and function signature on separate lines
MyObj *
myObjNew(unsigned int myData, const String *secretName)
{
    FUNCTION_LOG_BEGIN(logLevelDebug);              // Use FUNCTION_LOG_BEGIN with a log level for displaying in production
        FUNCTION_LOG_PARAM(UINT, myData);           // When log level is debug, myData variable will be logged
        FUNCTION_TEST_PARAM(STRING, secretName);    // FUNCTION_TEST_PARAM will not display secretName value in production logging
    FUNCTION_LOG_END();

    ASSERT(secretName != NULL || myData > 0);       // Development-only assertions (will be compiled out of production code)

    OBJ_NEW_BEGIN(MyObj)                // Create a long lasting memory context with the name of the object
    {
        *this = (MyObj)                 // Initialize the object
        {
            .pub =
            {
                .myData = myData,                       // Copy the simple data type to this object
            },
            .name = strDup(secretName),     // Duplicate the String data type to the this object's memory context
        };
    }
    OBJ_NEW_END();

    FUNCTION_LOG_RETURN(MyObj, this);
}

// Function using temporary memory context
String *
myObjDisplay(unsigned int myData)
{
    FUNCTION_TEST_BEGIN();                      // No parameters passed to this function will be logged in production
        FUNCTION_TEST_PARAM(UINT, myData);
    FUNCTION_TEST_END();

    String *result = NULL;     // Result is created in the caller's memory context (referred to as "prior context" below)

    MEM_CONTEXT_TEMP_BEGIN()   // Begin a new temporary context
    {
        String *resultStr = strNewZ("Hello");    // Allocate a string in the temporary memory context

        if (myData > 1)
            resultStr = strCatZ(" World");      // Append a value to the string still in the temporary memory context
        else
            LOG_WARN("Am I not your World?");   // Log a warning to the user

        MEM_CONTEXT_PRIOR_BEGIN()           // Switch to the prior context so the string duplication is in the caller's context
        {
            result = strDup(resultStr);     // Create a copy of the string in the caller's context
        }
        MEM_CONTEXT_PRIOR_END();            // Switch back to the temporary context
    }
    MEM_CONTEXT_TEMP_END();      // Free everything created inside this temporary memory context - i.e resultStr

    FUNCTION_TEST_RETURN(STRING, result);    // Return result but do not log the value in production
}

// Create the logging function for displaying important information from the object
String *
myObjToLog(const MyObj *this)
{
    return strNewFmt(
        "{name: %s, myData: %u}", this->name == NULL ? NULL_Z : strZ(this->name), myObjMyData(this));
}
                </code-block>
            </section>
        </section>
    </section>

    <!-- ======================================================================================================================= -->
    <section id="testing">
        <title>Testing</title>

        <p>A list of all possible test combinations can be viewed by running:</p>
        <code-block>
            pgbackrest/test/test.pl --dry-run
        </code-block>

        <p>While some files are automatically generated during <code>make</code>, others are generated by running the test harness as follows:</p>

        <code-block>
            pgbackrest/test/test.pl --gen-only
        </code-block>

        <p>Prior to any submission, the html version of the documentation should also be run and the output checked by viewing the generated html on the local file system under <code>pgbackrest/doc/output/html</code>. More details can be found in the <backrest/> <link url="{[github-url-main]}/doc/README.md">doc/README.md</link> file.</p>

        <code-block>
            pgbackrest/doc/doc.pl --out=html
        </code-block>

        <admonition type="note"><code>ERROR: [028]</code> regarding cache is invalid is OK; it just means there have been changes and the documentation will be built from scratch. In this case, be patient as the build could take 20 minutes or more depending on your system.</admonition>

        <!-- =================================================================================================================== -->
        <section id="running">
            <title>Running Tests</title>

            <p>Examples of test runs are provided in the following sections. There are several important options for running a test:</p>
            <list>
                <list-item><setting>--dry-run</setting> - without any other options, this will list all the available tests</list-item>
                <list-item><setting>--module</setting> - identifies the module in which the test is located</list-item>
                <list-item><setting>--test</setting> - the actual test set to be run</list-item>
                <list-item><setting>--run</setting> - a number identifying the run within a test if testing a single run rather than the entire test</list-item>
                <list-item><setting>--vm-out</setting> - displays the test output (helpful for monitoring the progress)</list-item>
                <list-item><setting>--vm</setting> - identifies the pre-built container when using Docker, otherwise the setting should be <code>none</code>. See <link url="{[github-url-main]}/.github/workflows/test.yml">test.yml</link> for a list of valid vm codes noted by <code>param: test</code>.</list-item>
            </list>

            <p>For more options, run the test or documentation engine with the <setting>--help</setting> option:</p>

            <code-block>
pgbackrest/test/test.pl --help
pgbackrest/doc/doc.pl --help
            </code-block>

            <!-- =============================================================================================================== -->
            <section id="without-docker">
                <title>Without Docker</title>

                <p>If <proper>Docker</proper> is not installed, then the available tests can be listed using <setting>--vm=none</setting>, and each test must then be run with <setting>--vm=none</setting>.</p>

                <execute-list host="{[host-contrib]}">
                    <title>List tests that don't require a container</title>

                    <execute output="y">
                        <exe-cmd>pgbackrest/test/test.pl --vm=none --dry-run</exe-cmd>
                        <exe-cmd-extra>{[test-cmd-extra]}</exe-cmd-extra>
                        <exe-highlight>[0-9]+ tests selected|DRY RUN COMPLETED SUCCESSFULLY</exe-highlight>
                    </execute>
                </execute-list>

                <execute-list host="{[host-contrib]}">
                    <title>Run a test</title>

                    <execute output="y">
                        <exe-cmd>pgbackrest/test/test.pl --vm=none --vm-out --module=common --test=wait</exe-cmd>
                        <exe-cmd-extra>{[test-cmd-extra]}</exe-cmd-extra>
                    </execute>
                </execute-list>

                <p>An entire module can be run by using only the <setting>--module</setting> option.</p>

                <execute-list host="{[host-contrib]}">
                    <title>Run a module</title>

                    <execute output="y">
                        <exe-cmd>pgbackrest/test/test.pl --vm=none --module=postgres</exe-cmd>
                        <exe-cmd-extra>{[test-cmd-extra]}</exe-cmd-extra>
                    </execute>
                </execute-list>
            </section>

            <!-- =============================================================================================================== -->
            <section id="with-docker" depend="/environment">
                <title>With Docker</title>

                <p>Build a container to run tests. The vm must be pre-configured but a variety are available. A vagrant file is provided in the test directory as an example of running in a virtual environment. The vm names are all three character abbreviations, e.g. <id>u20</id> for <proper>Ubuntu 20.04</proper>.</p>

                <execute-list host="{[host-contrib]}">
                    <title>Build a VM</title>

                    <execute output="y">
                        <exe-cmd>pgbackrest/test/test.pl --vm-build --vm=u20</exe-cmd>
                        <exe-cmd-extra>{[test-cmd-extra]}</exe-cmd-extra>
                    </execute>
                </execute-list>

                <admonition type="note">to build all the vms, just omit the <setting>--vm</setting> option above.</admonition>

                <execute-list host="{[host-contrib]}">
                    <title>Run a Specific Test Run</title>

                    <execute output="y">
                        <exe-cmd>pgbackrest/test/test.pl {[dash]}-vm=u20 {[dash]}-module=mock {[dash]}-test=archive {[dash]}-run=2</exe-cmd>
                        <exe-cmd-extra>{[test-cmd-extra]}</exe-cmd-extra>
                    </execute>
                </execute-list>
            </section>
        </section>

        <!-- =================================================================================================================== -->
        <section id="unit-test">
            <title>Writing a Unit Test</title>

            <p>The goal of unit testing is to have 100 percent code coverage. Two files will usually be involved in this process:</p>

            <list>
                <list-item><b>define.yaml</b> - defines the number of tests to be run for each module and test file. There is a comment at the top of the file that provides more information about this file.</list-item>
                <list-item><b>src/module/somefileTest.c</b> - where <quote>somefile</quote> is the path and name of the test file where the unit tests are located for the code being updated (e.g. <file>src/module/command/expireTest.c</file>).</list-item>
            </list>

            <!-- =============================================================================================================== -->
            <section id="define-yaml">
                <title>define.yaml</title>

                <p>Each module is separated by a line of asterisks (*) and each test within is separated by a line of dashes (-). In the example below, the module is <code>command</code> and the unit test is <code>check</code>. The number of calls to <code>testBegin()</code> in a unit test file will dictate the number following <code>total:</code>, in this case 4. Under <code>coverage:</code>, the list of files that will be tested.</p>

                <code-block>
  # ********************************************************************************************************************************
  - name: command

    test:
      # ----------------------------------------------------------------------------------------------------------------------------
      - name: check
        total: 4
        containerReq: true

        coverage:
          - command/check/common
          - command/check/check
                </code-block>
            </section>

            <!-- =============================================================================================================== -->
            <section id="test-file">
                <title>somefileTest.c</title>

                <p>Unit test files are organized in the <code>test/src/module</code> directory with the same directory structure as the source code being tested. For example, if new code is added to src/<b>command/expire</b>.c then test/src/module/<b>command/expire</b>Test.c will need to be updated.</p>

                <p>Assuming that a test file already exists, new unit tests will either go in a new <code>testBegin()</code> section or be added to an existing section. Each such section is a test run. The comment string passed to <code>testBegin()</code> should reflect the function(s) being tested in the test run. Tests within a run should use <code>TEST_TITLE()</code> with a comment string describing the test.</p>

                <code-block>
// *****************************************************************************************************************************
if (testBegin("expireBackup()"))
{
    // -------------------------------------------------------------------------------------------------------------------------
    TEST_TITLE("manifest file removal");
                </code-block>
            </section>

            <!-- =============================================================================================================== -->
            <section id="test-command">
                <title>Setting up the command to be run</title>

                <p>The <link url="{[github-url-test-common]}/harnessConfig.h">harnessConfig.h</link> describes a list of functions that should be used when configuration options are required for a command being tested. Options are set in a <code>StringList</code> which must be defined and passed to the <code>HRN_CFG_LOAD()</code> macro with the command. For example, the following will set up a test to run <cmd>pgbackrest --repo-path=test/test-0/repo info</cmd> command on multiple repositories, one of which is encrypted:</p>

                <code-block>
StringList *argList = strLstNew();                                  // Create an empty string list
hrnCfgArgRawZ(argList, cfgOptRepoPath, TEST_PATH "/repo");          // Add the --repo-path option
hrnCfgArgKeyRawZ(argList, cfgOptRepoPath, 2, TEST_PATH "/repo2");   // Add the --repo2-path option
hrnCfgArgKeyRawStrId(argList, cfgOptRepoCipherType, 2, cipherTypeAes256Cbc);  // Add the --repo2-cipher-type option
hrnCfgEnvKeyRawZ(cfgOptRepoCipherPass, 2, TEST_CIPHER_PASS);        // Set environment variable for the --repo2-cipher-pass option
HRN_CFG_LOAD(cfgCmdInfo, argList);                                  // Load the command and option list into the test harness
                </code-block>
            </section>

            <!-- =============================================================================================================== -->
            <section id="test-store">
                <title>Storing a file</title>

                <p>Sometimes it is desirable to store or manipulate files before or during a test and then confirm the contents. The <link url="{[github-url-test-common]}/harnessStorage.h">harnessStorage.h</link> file contains macros (e.g. <code>HRN_STORAGE_PUT</code> and <code>TEST_STORAGE_GET</code>) for doing this. In addition, <code>HRN_INFO_PUT</code> is convenient for writing out info files (archive.info, backup.info, backup.manifest) since it will automatically add header and checksum information.</p>

                <code-block>
HRN_STORAGE_PUT_EMPTY(
    storageRepoWrite(), STORAGE_REPO_ARCHIVE "/10-1/000000010000000100000001-abcdabcdabcdabcdabcdabcdabcdabcdabcdabcd.gz");
                </code-block>
            </section>

            <!-- =============================================================================================================== -->
            <section id="test-results">
                <title>Testing results</title>

                <p>Tests are run and results confirmed via macros that are described in <link url="{[github-url-test-common]}/harnessTest.h">harnessTest.h</link>. With the exception of TEST_ERROR, the third parameter is a short description of the test. Some of the more common macros are:</p>

                <list>
                    <list-item><id>TEST_RESULT_STR</id> - Test the actual value of the string returned by the function.</list-item>
                    <list-item><id>TEST_RESULT_UINT</id> / <id>TEST_RESULT_INT</id> - Test for an unsigned integer / integer.</list-item>
                    <list-item><id>TEST_RESULT_BOOL</id> - Test a boolean value.</list-item>
                    <list-item><id>TEST_RESULT_PTR</id> / <id>TEST_RESULT_PTR_NE</id> - Test a pointer: useful for testing if the pointer is <id>NULL</id> or not equal (<id>NE</id>) to <id>NULL</id>.</list-item>
                    <list-item><id>TEST_RESULT_VOID</id> - The function being tested returns a <code>void</code>. This is then usually followed by tests that ensure other actions occurred (e.g. a file was written to disk).</list-item>
                    <list-item><id>TEST_ERROR</id> / <id>TEST_ERROR_FMT</id> - Test that a specific error code was raised with specific wording.</list-item>
                </list>

                <admonition type="note"><code>HRN_*</code> macros should be used only for test setup and cleanup. <code>TEST_*</code> macros must be used for testing results.</admonition>
            </section>

            <!-- =============================================================================================================== -->
            <section id="test-log">
                <title>Testing a log message</title>

                <p>If a function being tested logs something with <code>LOG_WARN</code>, <code>LOG_INFO</code> or other <code>LOG_*()</code> macro, then the logged message must be cleared before the end of the test by using the <code>TEST_RESULT_LOG()/TEST_RESULT_LOG_FMT()</code> macros.</p>

                <code-block>
TEST_RESULT_LOG(
    "P00   WARN: WAL segment '000000010000000100000001' was not pushed due to error [25] and was manually skipped: error");
                </code-block>

                <p>In the above, <code>Pxx</code> indicates the process (P) and the process number (xx), e.g. P00, P01.</p>
            </section>

            <!-- =============================================================================================================== -->
            <section id="test-child">
                <title>Testing using child process</title>

                <p>Sometimes it is useful to use a child process for testing. Below is a simple example. See <link url="{[github-url-test-common]}/harnessFork.h">harnessFork.h</link> for more details.</p>

                <code-block>
HRN_FORK_BEGIN()
{
    HRN_FORK_CHILD_BEGIN()
    {
        TEST_RESULT_INT_NE(lockAcquireP(), -1, "create backup/expire lock");

        // Notify parent that lock has been acquired
        HRN_FORK_CHILD_NOTIFY_PUT();

        // Wait for parent to allow release lock
        HRN_FORK_CHILD_NOTIFY_GET();

        lockRelease(true);
    }
    HRN_FORK_CHILD_END();

    HRN_FORK_PARENT_BEGIN()
    {
        // Wait for child to acquire lock
        HRN_FORK_PARENT_NOTIFY_GET(0);

        HRN_CFG_LOAD(cfgCmdInfo, argListText);
        TEST_RESULT_STR_Z(
            infoRender(),
            "stanza: stanza1\n"
            "    status: error (no valid backups, backup/expire running)\n"
            "    cipher: none\n"
            "\n"
            "    db (current)\n"
            "        wal archive min/max (9.4): none present\n",
            "text - single stanza, no valid backups, backup/expire lock detected");

        // Notify child to release lock
        HRN_FORK_PARENT_NOTIFY_PUT(0);
    }
    HRN_FORK_PARENT_END();
}
HRN_FORK_END();
                </code-block>
            </section>

            <!-- =============================================================================================================== -->
            <section id="test-shim">
                <title>Testing using a shim</title>

                <p>A <postgres/> libpq shim is provided to simulate interactions with <postgres/>. Below is a simple example. See <link url="{[github-url-test-common]}/harnessPq.h">harnessPq.h</link> for more details.</p>

                <code-block>
// Set up two standbys but no primary
harnessPqScriptSet((HarnessPq [])
{
    HRNPQ_MACRO_OPEN_GE_96(1, "dbname='postgres' port=5432", PG_VERSION_96, "/pgdata", true, NULL, NULL),
    HRNPQ_MACRO_OPEN_GE_96(8, "dbname='postgres' port=5433", PG_VERSION_96, "/pgdata", true, NULL, NULL),

    // Close the "inner" session first (8) then the outer (1)
    HRNPQ_MACRO_CLOSE(8),
    HRNPQ_MACRO_CLOSE(1),

    HRNPQ_MACRO_DONE()
});

TEST_ERROR(cmdCheck(), ConfigError, "primary database not found\nHINT: check indexed pg-path/pg-host configurations");
                </code-block>
            </section>
        </section>

        <!-- =================================================================================================================== -->
        <section id="unit-test-run">
            <title>Running a Unit Test</title>

            <p><b>Code Coverage</b></p>
            <p>Unit tests are run for all files that are listed in <file>define.yaml</file> and a coverage report generated for each file listed under the tag <code>coverage:</code>. Note that some files are listed in multiple <code>coverage:</code> sections for a module; in this case, each test for the file being modified should be specified for the module in which the file exists (e.g. <code>--module=storage --test=posix --test=gcs</code>, etc.) or, alternatively, simply run the module without the <code>--test</code> option. It is recommended that a <code>--vm</code> be specified since running the same test for multiple vms is unnecessary for coverage. The following example would run the test set from the <b>define.yaml</b> section detailed above.</p>

            <code-block>
pgbackrest/test/test.pl --vm-out --module=command --test=check --vm=u20
            </code-block>

<admonition type="note">Not all systems perform at the same speed, so if a test is timing out, try rerunning with another vm.</admonition>

            <p>Because a test run has not been specified, a coverage report will be generated and written to the local file system under the <backrest/> directory <file>test/result/coverage/lcov/index.html</file> and a file with only the highlighted code that has not been covered will be written to <file>test/result/coverage/coverage.html</file>.</p>

            <p>If 100 percent code coverage has not been achieved, an error message will be displayed, for example: <code>ERROR: [125]: c module command/check/check is not fully covered</code></p>

            <p><b>Debugging with files</b></p>

            <p>Sometimes it is useful to look at files that were generated during the test. The default for running any test is that, at the start/end of the test, the test harness will clean up all files and directories created. To override this behavior, a single test run must be specified and the option <code>--no-cleanup</code> provided. Again, continuing with the check command, from <b>define.yaml</b> above, there are four tests. Below, test one will be run and nothing will be cleaned up so that the files and directories in <path>test/test-0</path> can be inspected.</p>

            <code-block>
pgbackrest/test/test.pl --vm-out --module=command --test=check --run=1 --no-cleanup
            </code-block>
        </section>

        <!-- =================================================================================================================== -->
        <section id="test-output">
            <title>Understanding Test Output</title>

            <p>The following is a small sample of a typical test output.</p>

            <code-block>
run 8 - expireTimeBasedBackup()

run 8/1 ------------- L2285 no current backups
    000.002s          L2298     empty backup.info
    000.009s 000.007s L2300     no backups to expire
            </code-block>

            <p><b>run 8 - expireTimeBasedBackup()</b> - indicates the run number (8) within the module and the parameter provided to testBegin, e.g. <code>testBegin("expireTimeBasedBackup()")</code></p>

            <p><b>run 8/1 ------------- L2285 no current backups</b> - this is the first test (1) in run 8 which is the <code>TEST_TITLE("no current backups");</code> at line number 2285.</p>

            <p><b>000.002s          L2298     empty backup.info</b> -  the first number, 000.002s, is the time in seconds that the test started from the beginning of the run. L2298 is the line number of the test and <code>empty backup.info</code> is the test comment.</p>

            <p><b>000.009s 000.007s L2300     no backups to expire</b> -  again, 000.009s, is the time in seconds that the test started from the beginning of the run. The second number, 000.007s, is the run time of the <b>previous</b> test (i.e. <code>empty backup.info</code> test took 000.007 seconds to execute). L2300 is the line number of the test and <code>no backups to expire</code> is the test comment.</p>
        </section>
    </section>

    <!-- ======================================================================================================================= -->
    <section id="option">
        <title>Adding an Option</title>

        <p>Options can be added to a command or multiple commands. Options can be configuration file only, command-line only or valid for both. Once an option is successfully added, the <file>config.auto.h</file> and <file>parse.auto.c.inc</file> files will automatically be generated by the build system.</p>

        <p>To add an option, two files need be to be modified:</p>
        <list>
            <list-item><file>src/build/config/config.yaml</file></list-item>
            <list-item><file>src/build/help/help.xml</file></list-item>
        </list>

        <p>These files are discussed in the following sections along with how to verify the <code>help</code> command output.</p>

        <!-- =================================================================================================================== -->
        <section id="config-file">
            <title>config.yaml</title>

            <p>There are detailed comment blocks above each section that explain the rules for defining commands and options. Regarding options, there are two types: 1) command line only, and 2) configuration file. With the exception of secrets, all configuration file options can be passed on the command line. To configure an option for the configuration file, the <id>section:</id> key must be present.</p>

            <p>The <id>option:</id> section is broken into sub-sections by a simple comment divider (e.g. <code># Repository options</code>) under which the options are organized alphabetically by option name. To better explain this section, two hypothetical examples will be discussed. For more details, see <link url="{[github-url-src]}/build/config/config.yaml">config.yaml</link>.</p>

            <!-- =============================================================================================================== -->
            <section id="option-ex1">
                <title>EXAMPLE 1 hypothetical command line only option</title>

                <code-block>
  set:
    type: string
    command:
      backup:
        depend:
          option: stanza
        required: false
      restore:
        default: latest
    command-role:
      main: {}
                </code-block>
            </section>

            <p>Note that <id>section:</id> is not present thereby making this a command-line only option defined as follows:</p>

            <list>
                <list-item><id>set</id> - the name of the option</list-item>
                <list-item><id>type</id> - the type of the option. Valid values for types are: <id>boolean</id>, <id>hash</id>, <id>integer</id>, <id>list</id>, <id>path</id>, <id>size</id>, <id>string</id>, and <id>time</id></list-item>
                <list-item><id>command</id> - list each command for which the option is valid. If a command is not listed, then the option is not valid for the command and an error will be thrown if it is attempted to be used for that command. In this case the valid commands are <cmd>backup</cmd> and <cmd>restore</cmd>.</list-item>
                <list-item><id>backup</id> - details the requirements for the <code>--set</code> option for the <cmd>backup</cmd> command. It is dependent on the option <code>--stanza</code>, meaning it is only allowed to be specified for the <cmd>backup</cmd> command if the <code>--stanza</code> option has been specified. And <code>required: false</code> indicates that the <code>--set</code> option is never required, even with the dependency.</list-item>
                <list-item><id>restore</id> - details the requirements for the <code>--set</code> option for the <cmd>restore</cmd> command. Since <code>required:</code> is omitted, it is not required to be set by the user but it is required by the command and will default to <code>latest</code> if it has not been specified by the user.</list-item>
                <list-item><id>command-role</id> - defines the processes for which the option is valid. <code>main</code> indicates the option will be used by the main process and not be passed on to other local/remote processes.</list-item>
            </list>

            <!-- =============================================================================================================== -->
            <section id="option-ex2">
                <title>EXAMPLE 2 hypothetical configuration file option</title>

                <code-block>
  repo-test-type:
    section: global
    type: string
    group: repo
    default: full
    allow-list:
      - full
      - diff
      - incr
    command:
      backup: {}
      restore: {}
    command-role:
      main: {}
                </code-block>
            </section>

            <list>
                <list-item><id>repo-test-type</id> - the name of the option</list-item>
                <list-item><id>section</id> - the section of the configuration file where this option is valid (omitted for command line only options, see <link section="/option/config-file/option-ex1">Example 1</link> above)</list-item>
                <list-item><id>type</id> - the type of the option. Valid values for types are: <id>boolean</id>, <id>hash</id>, <id>integer</id>, <id>list</id>, <id>path</id>, <id>size</id>, <id>string</id>, and <id>time</id></list-item>
                <list-item><id>group</id> - indicates that this option is part of the <code>repo</code> group of indexed options and therefore will follow the indexing rules e.g. <code>repo1-test-type</code>.</list-item>
                <list-item><id>default</id> - sets a default for the option if the option is not provided when the command is run. The default can be global (as it is here) or it can be specified for a specific command in the command section (as in <link section="/option/config-file/option-ex1">Example 1</link> above).</list-item>
                <list-item><id>allow-list</id> - lists the allowable values for the option for all commands for which the option is valid.</list-item>
                <list-item><id>command</id> - list each command for which the option is valid. If a command is not listed, then the option is not valid for the command and an error will be thrown if it is attempted to be used for that command. In this case the valid commands are <cmd>backup</cmd> and <cmd>restore</cmd>.</list-item>
                <list-item><id>command-role</id> - defines the processes for which the option is valid. <code>main</code> indicates the option will be used by the main process and not be passed on to other local/remote processes.</list-item>
            </list>

            <p>At compile time, the <file>config.auto.h</file> file will be generated to contain the constants used for options in the code. For the C enums, any dashes in the option name will be removed, camel-cased and prefixed with <code>cfgOpt</code>, e.g. <code>repo-path</code> becomes <code>cfgOptRepoPath</code>.</p>
        </section>

        <!-- =================================================================================================================== -->
        <section id="help-file">
            <title>help.xml</title>

            <p>All options must be documented or the system will error during the build. To add an option, find the command section identified by <code>command id="COMMAND"</code> section where <id>COMMAND</id> is the name of the command (e.g. <cmd>expire</cmd>) or, if the option is used by more than one command and the definition for the option is the same for all of the commands, the <code>operation-general title="General Options"</code> section.</p>

            <p>To add an option, add the following to the <code>&lt;option-list&gt;</code> section; if it does not exist, then wrap the following in <code>&lt;option-list&gt;</code> <code>&lt;/option-list&gt;</code>. This example uses the boolean option <code>force</code> of the <cmd>restore</cmd> command. Simply replace that with your new option and the appropriate <code>summary</code>, <code>text</code> and <code>example</code>.</p>

            <code-block>
&lt;option id="force" name="Force"&gt;
    &lt;summary&gt;Force a restore.&lt;/summary&gt;

    &lt;text&gt;By itself this option forces the &lt;postgres/&gt; data and tablespace paths to be completely overwritten.  In combination with &lt;br-option&gt;--delta&lt;/br-option&gt; a timestamp/size delta will be performed instead of using checksums.&lt;/text&gt;

    &lt;example>y&lt;/example&gt;
&lt;/option&gt;
            </code-block>
            <admonition type="important">A period (.) is required to end the <code>summary</code> section.</admonition>
        </section>

        <!-- =================================================================================================================== -->
        <section id="help-test">
            <title>Testing the help</title>

            <p>It is important to run the <cmd>help</cmd> command unit test after adding an option in case a change is required:</p>

            <code-block>
pgbackrest/test/test.pl --module=command --test=help --vm-out
            </code-block>

            <p>To verify the <cmd>help</cmd> command output, build the <backrest/> executable:</p>

            <code-block>
pgbackrest/test/test.pl --vm=none --build-only
            </code-block>

            <p>Use the <backrest/> executable to test the help output:</p>

            <code-block>
test/bin/none/pgbackrest help backup repo-type
            </code-block>
        </section>

        <!-- =================================================================================================================== -->
        <section id="doc-test">
            <title>Testing the documentation</title>

            <p>To quickly view the HTML documentation, the <code>--no-exe</code> option can be passed to the documentation generator in order to bypass executing the code elements:</p>

            <code-block>
pgbackrest/doc/doc.pl --out=html --no-exe
            </code-block>

            <p>The generated HTML files will be placed in the <file>doc/output/html</file> directory where they can be viewed locally in a browser.</p>

            <p>If Docker is installed, it will be used by the documentation generator to execute the code elements while building the documentation, therefore, the <setting>--no-exe</setting> should be omitted, (i.e. <code>pgbackrest/doc/doc.pl --output=html</code>). <setting>--no-cache</setting> may be used to force a full build even when no code elements have changed since the last build. <setting>--pre</setting> will reuse the container definitions from the prior build and saves time during development.</p>

            <p>The containers created for documentation builds can be useful for manually testing or trying out new code or features. The following demonstrates building through just the <code>quickstart</code> section of the <file>user-guide</file> without encryption.</p>

            <code-block>
pgbackrest/doc/doc.pl --out=html --include=user-guide --require=/quickstart --var=encrypt=n --no-cache --pre
            </code-block>

            <p>The resulting Docker containers can be listed with <code>docker ps</code> and the container can be entered with <code>docker exec doc-pg-primary bash</code>.  Additionally, the <code>-u</code> option can be added for entering the container as a specific user (e.g. <code>postgres</code>).</p>
        </section>
    </section>

    <!-- ======================================================================================================================= -->
    <section id="pr">
        <title>Submitting a Pull Request</title>

        <p>Before submitting a Pull Request:</p>

        <list>
            <list-item>Does it meet the <link url="{[github-url-main]}/CODING.md">coding standards</link>?</list-item>
            <list-item>Have <link section="/testing/unit-test">Unit Tests</link> been written and <link section="/testing/unit-test-run">run</link> with 100% coverage?</list-item>
            <list-item>If your submission includes changes to the help or online documentation, have the <link section="/option/help-test">help</link> and <link section="/option/doc-test">documentation</link> tests been run?</list-item>
            <list-item>Has it passed continuous integration testing? Simply renaming your branch with the appendix <code>-cig</code> and pushing it to your GitHub account will initiate GitHub Actions to run CI tests.</list-item>
        </list>

        <p>When submitting a Pull Request:</p>
        <list>
            <list-item>Provide a short submission title.</list-item>
            <list-item>Write a detailed comment to describe the purpose of your submission and any issue(s), if any, it is resolving; a link to the GitHub issue is also helpful.</list-item>
            <list-item>Select the <code>integration</code> branch as the base for your PR, do not select <code>main</code> nor any other branch.</list-item>
        </list>

        <p>After submitting a Pull Request:</p>

        <list>
            <list-item>One or more reviewers will be assigned.</list-item>
            <list-item>Respond to any issues (conversations) in GitHub but do not resolve the conversation; the reviewer is responsible for ensuring the issue raised has been resolved and marking the conversation resolved. It is helpful to supply the commit in your reply if one was submitted to fix the issue.</list-item>
        </list>

        <p>Lastly, thank you for contributing to <backrest/>!</p>
    </section>
</doc>