1
0
mirror of https://github.com/postgrespro/pg_probackup.git synced 2024-12-05 11:00:22 +02:00
Commit Graph

2893 Commits

Author SHA1 Message Date
Zhuravlev Uriy aka stalkerg
c9c943b2fb Relax version test. 2016-03-02 12:11:25 +03:00
Zhuravlev Uriy aka stalkerg
b3ecb19377 Fix docs for ptrack and multithreads. 2016-02-29 22:00:57 +03:00
Zhuravlev Uriy aka stalkerg
07d4b8a030 Add threads for restore. 2016-02-29 21:49:53 +03:00
Zhuravlev Uriy aka stalkerg
fed13c298e Add multithread beckup. 2016-02-29 20:23:48 +03:00
Zhuravlev Uriy aka stalkerg
6d51bb10d9 Fix copy bitmap. 2016-02-28 02:10:27 +03:00
Zhuravlev Uriy aka stalkerg
9c475eccbf First version of ptrack support. 2016-02-27 21:07:55 +03:00
Michael Paquier
7cbbdf0fc4 Update README regarding oldest version of Postgres supported
Noticed by Uriy Zhuravlev.
2016-02-11 22:10:35 +09:00
Michael Paquier
fa248a0507 Remove custom version of readlink in pgut_port
Postgres has its own version in lpgport relying on junction points,
so if we consider a WIN32 port this is actually not necessary.
2016-01-20 13:19:59 +09:00
Michael Paquier
1ae2feb38f Fix inconsistency in call of readlink()
If the target path is too long, an error needs to be emitted as well.
The buffer is correctly null-terminated, this will just avoid running
into weird problems should the target have a too long name.
2016-01-20 13:13:32 +09:00
Michael Paquier
b84334b8f3 Remove unportable code of IsDir
IsDir was somewhat optimized on systems where DT_DIR (dirent.d_type
being not part of the POSIX spec) is present on some systems leading
to more complex logic depending on the file system used, particularly
on XFS this routine was actually broken. Having a call to stat()
should not be that expensive to check if a path is a directory or
not and this is proving to be far more stable coding, so just rely
on that, that's more portable anyway and will avoid future surprises.

Report by Yury Zhuravlev, though I did not use his patch.
2016-01-19 20:51:38 +09:00
Michael Paquier
7254d30cc9 Correct incorrect project name in README
The upstream project is pg_rman.
2016-01-19 13:06:34 +09:00
Michael Paquier
05601aa8f6 Correct mention that a full backup is needed after database creation
This restriction has been inherited from the upstream project pg_rman, however
the new differential facility is proving to be able to address that need quite
nicely after more testing regarding that.

Mentioned by Thomas Reiss.
2016-01-19 13:01:40 +09:00
Michael Paquier
5c37daca69 Sanitize error checks
All the ERROR_* fields are removed in favor of a more simple layer
made of ERROR, FATAL, PANIC. The two last ones are not actually used
yet, thought there should be some code paths that would need more
polishing on this matter.

The error message emitted before leaving should be fine to let the
user know what is happening.
2016-01-19 12:41:30 +09:00
Michael Paquier
3a9cc02d71 Update README with new instructions to use source code
This has been forgotten 04834e73 that added a dependency to the Postgres
source tree when building.
2016-01-19 09:26:16 +09:00
Michael Paquier
2882954b95 Remove dead code xlog_is_complete_wal
This routine is not used anywhere now.
2016-01-19 09:21:54 +09:00
Michael Paquier
f2c523b1ee Remove un-needed WAL segments from archive at backup deletion
During the execution of the "delete" command, the oldest start LSN is saved
and used to determine what is the oldest WAL segment that needs to be kept
for the existing backups. Like pg_archivecleanup, the implemented logic
ignores the timeline part of the WAL segment name, ensuring that a segment
from a parent timeline will not be removed prematurely. This logic could
be made more complicated regarding this matter, but in order to take backups
from a node on a given timeline this is far enough.
2016-01-18 16:11:47 +09:00
Michael Paquier
a5afc8758e Improve a couple of comments in backup.c 2016-01-18 15:24:28 +09:00
Michael Paquier
bee3489584 Update some copyright lines to 2016 2016-01-16 00:02:09 +09:00
Michael Paquier
0106bf9c5a Remove NOT_USED section 2016-01-15 23:52:31 +09:00
Michael Paquier
04834e73c7 Page-level backup using block tracking in WAL records
This commit improves the performance of page-level, or differential
backup, by not having to scan anymore all the pages of a relation file,
something that can be very long on large data sets, but by scanning the
list of blocks changed by WAL records since the last full or differential
backup.

As a restriction and to avoid potential data corruption should hint-bit
updates occur on a page, backups can only be taken from a server that has
wal_log_hints or data checksums enabled.

Base patch by Yury Zhuravlev, heavily modified by me.
2016-01-15 23:47:38 +09:00
Michael Paquier
47d0b60ebd Fix some code indentation 2016-01-15 15:56:56 +09:00
Michael Paquier
ea89ca3145 Remove remaining LVM-snapshot logic
Extracted from a patch by Yury Zhuravlev, and visibly this portion was
missed in last cleanup that occurred in b92d722.
2016-01-15 15:26:05 +09:00
Michael Paquier
e2bbf69403 Remove code duplication to get a node's current timeline
The same code was duplicated between restore and backup. At the same
time this commit introduces routines to fetch the control data file.
2016-01-15 14:09:31 +09:00
Michael Paquier
a5a76c14c5 Fix one-off error when waiting for necessary segment to be archived
The .ready flag file of the segment forcibly switched to when stopping the
backup needs to be tracked, and not the next one.
2016-01-15 13:37:20 +09:00
Michael Paquier
04c4c7b865 Make regression tests more verbose
The tests now rely on --verbose to print out useful information in the
logs that can be used for debugging purposes. In case of a failure in
some of the tests an overall cleanup is not done, this is particularly
useful for PGDATA which could still be used for extra failure analysis.
2016-01-15 13:29:19 +09:00
Michael Paquier
f94c5ab447 Sanitize logging facility
--debug and --verbose had actually the same meaning as they were aimed
at giving to the user information regarding how the process is running,
hence both options are merged into --verbose and use elog(LOG) to decide
if a given message should be printed out depending on the verbosity of
the call. This makes a couple of routines more readable as they do not
depend on any boolean checks.

The "_()" have been removed from the code, those are aimed at being used
for translation but having them mandatorily in each log message is just
useless noise. If needed, pgut.c should be updated in consequence to
have a more portable facility.

At the same time this commit takes care of putting into correct shape
some code paths more in-line with PostgreSQL policy. There are surely
more of this kind of ugly stuff but at this stage things are more simple
and more manageable.
2016-01-14 16:36:39 +09:00
Michael Paquier
3ac6d13329 Remove --compress-data/-Z
Performance of compression is quite questionable on many objects like
that and makes the routines aimed at managing file copy, backup and restore
more complicated than they should be.

This commit results in a largely simplified code in data.c, which will
be helpful when integrating differential backup using WAL file lookup.
2016-01-14 15:19:38 +09:00
Michael Paquier
49d37ecd3e Rework Makefile and remove incoherent definitions
-DFRONTEND should be overridden as part of CPPFLAGS and it is not really
necessary to have the source list. The definition of pg_arman.h was rather
obscure as well.
2016-01-13 22:47:49 +09:00
Michael Paquier
dc27aab59d Remove unneeded filters to lxml2 and lxslt
Those are dependent with how Postgres is compiled, so it makes little
sense to actually have that.
2016-01-13 22:25:09 +09:00
Michael Paquier
84d7ec5958 Add comment telling why enforced pg_switch_xlog() is needed
Without that, there is no way to be sure that the segments needed
up to the target recovery will be here.
2016-01-13 21:39:54 +09:00
Michael Paquier
9f1424bc87 Remove use of xlog_fname in code
Let's rely on Postgres stuff instead.

Based on a patch by YUriy Zhuravlev for the backup part, updated by
me for the restore part.
2016-01-13 16:28:51 +09:00
Michael Paquier
c5c07d8613 Fix failure in a couple of regression tests
The method used here is to enforce the archiving of the last segment
available to ensure that recovery works up to the point defined by
the tests.
2016-01-13 16:06:17 +09:00
Michael Paquier
751519d373 Fix typo 2016-01-13 15:26:17 +09:00
Michael Paquier
6de8b16403 Incorporate new test facility in pg_arman
This new facility has the advantage on not relying on static data when
generating the tests making the whole facility more robust. This is
basically taken from the upstream project pg_rman and adapted for the
sake of this pet project, so most of the credit go to Kyotaro Horiguchi
and Amit Langote regarding this facility. However I have adapted a bunch
of things and fixed a lot of redundancy and code duplication.
2016-01-13 14:57:51 +09:00
Michael Paquier
8559733103 Remove USE_PGXS usage
Let's be honest, this is not going to be integrated into Postgres core,
so let's make things simpler.
2016-01-13 10:57:59 +09:00
Michael Paquier
25ebddb02b Update CRC32 algorithm to CRC32C
This is to be compatible with Postgres >= 9.5 that switched to a more
performant algorithm.

Per reminder from Zhuravlev Uriy.
2015-12-07 21:44:33 +09:00
Michael Paquier
55db450c6f Fix typos in docs
Report and patch by Zhuravlev Uriy.
2015-12-07 21:41:16 +09:00
Michael Paquier
db249c800e Replace incremental backup by page-level differential backup
This will allow the introduction of file-level differential backup.
2014-01-30 15:58:55 +09:00
Michael Paquier
a1de8d6145 Remove mention to archive recovery in README 2014-01-30 15:36:22 +09:00
Michael Paquier
4d586dda9c Add a note in README about the origin of pg_arman 2014-01-27 12:19:00 +09:00
Michael Paquier
3fc8d00bb4 Fix regression test failures caused by recent project renaming
Just a configuration file that missed the shot as well as an outdated
output for options.
2014-01-27 12:14:58 +09:00
Michael Paquier
5429f70807 Fix whitespaces and typos in documentation
The output of "show" command was incorrect as well.
2014-01-27 12:10:54 +09:00
Michael Paquier
c8ebcc9c41 Rename project to pg_arman
This is officially a fork!
2014-01-27 12:02:56 +09:00
Michael Paquier
820485d225 Strip off archive backup mode
This mode is not actually necessary if we consider that the core of
pg_rman is the obtention of differential and full backups, the server
being afterwards in charge to recover necessary WAL segments from the
archive.

Regression tests and documentation are updated in accordance to the
changes.
2014-01-24 22:37:55 +09:00
Michael Paquier
5988e6bd10 Remove server backup feature
In order to keep only the core of pg_rman for incremental/differential
backup, this looks necessary and makes the code more simple. Including
server log files in a backup could be subject to discussion as well,
as for example a Postgres base backup does not include them, just
because in this case server instance is not aware of the log files.
2014-01-24 20:37:13 +09:00
Michael Paquier
f9a3d799af Correct some typos and variable placements 2014-01-17 22:46:39 +09:00
Michael Paquier
7eb06cbc64 Add backup option --validate
This permits to validate a backup that has just been taken. Other
backups taken previously are ignored.
2014-01-17 22:42:27 +09:00
Michael Paquier
342496e331 Fix incremental and archive backup removal on delete command
The previous algorithm was smart enough to remove full backups older
than the given number of generations, but not enough to remove
incremental and archive backups. This resulted in keeping in the backup
list a set of incremental and archive backups older than the latest
full backup allowed. As it is useless to keep them, the deletion
algorithm is made smarter to take that into account and remove all of
them cleanly only when necessary.
2014-01-17 22:02:27 +09:00
Michael Paquier
c812fa3953 Refactor code for file list creation
A lot of code in this area was duplicated, so better to remove that.
2014-01-13 13:04:43 +09:00
Michael Paquier
09745505aa Make regression tests pass on OSX
Diffs were generated because of wc that puts some spaces before the
output which is in this case a number of lines. This does not impact
regression tests on Linux/Unix.
2014-01-13 12:46:58 +09:00