Feature #5295
Implementation of multi-threaded rsync for base backups
0%
Description
Due to the single-threaded nature of pg_basebackup, performing a base backup can be a rather slow process.
Multi-threading the interface in Python would resolve this.
Files
Updated by Joshua Drake about 10 years ago
- File threaded_rsync.py threaded_rsync.py added
On 04/17/2014 09:08 AM, pitrtools-tickets@lists.commandprompt.com wrote:
----------------------------------------
Due to the single-threaded nature of pg_basebackup, performing a base backup can be a rather slow process.
Multi-threading the interface in Python would resolve this.
Here is the code to implement.
--
Command Prompt, Inc. - http://www.commandprompt.com/ 509-416-6579
PostgreSQL Support, Training, Professional Services and Development
High Availability, Oracle Conversion, Postgres-XC, @cmdpromptinc
Political Correctness is for cowards.
{{attachment(204)}}
Updated by Joshua A. Drake about 10 years ago
Development plan:
- Add value rsync_threads (int) to cmd_standby.ini; Default 1
- Corresponds to --num_threads options passed to threaded_rsync.py. Defines amount of threads used in parallel rsync for base backups.
- Update doc/cmd_standby.README accordingly
- In cmd_standby: CMDStandby.base_backup_func, insert threaded_rsync.py with appropriate options
Updated by Joshua A. Drake almost 10 years ago
threaded_rsync.py is not always located correctly as it is referred to by a relative path
Fix: https://github.com/commandprompt/PITRTools/pull/41
Updated by Joshua A. Drake almost 10 years ago
threaded_rsync.py is not invoked "properly" in subprocess.call(), and thus only works sometimes.
Fix: https://github.com/commandprompt/PITRTools/pull/43