Backup-mariadb
backup-mariadb (previously backup_mariadb.py) is a Python 3 script that allows generating MariaDB backups of different formats (right now mydumper and xtrabackup (mariabackup) are implemented, but can be easily extended with others.
Note that while backup-mariadb can call xtrabackup, it can only do so for a local mariadb instance, so normally it is used in combination with transfer.py, which uses cumin to transfer backups from a remote host, and later postprocess it with backup-mariadb.
backup-mariadb can be used with options on command line, but it is intended to be used more easily with a configuration file.
Usage
backup_mariadb.py is installed on PATH on dbprov* hosts (via Puppet), and it is used to generate weekly dumps, as well as postprocess remote snapshots (xtrabackup) generated by transfer.py.
dbprov1001$ backup-mariadb -h
usage: backup_mariadb.py [-h] [--config-file CONFIG_FILE] [--host HOST]
[--port PORT] [--user USER] [--password PASSWORD]
[--threads THREADS] [--type {dump,snapshot}]
[--only-postprocess] [--rotate]
[--retention RETENTION] [--backup-dir BACKUP_DIR]
[--rows ROWS] [--archive] [--compress]
[--regex REGEX] [--stats-host STATS_HOST]
[--stats-port STATS_PORT] [--stats-user STATS_USER]
[--stats-password STATS_PASSWORD]
[--stats-database STATS_DATABASE]
[section]
Create a mysql/mariadb logical backup using mydumper or a snapshot using
mariabackup.It has 2 modes: Interactive, where options are received from the
command line and non-interactive, where it reads from a config file and
performs several backups
positional arguments:
section Section name of the backup. E.g.: "s3", "tendril". If
section is set, --config-file is ignored. If it is
empty, only config-file options will be used and other
command line options will be ignored. For --only-
postprocess work, you can also provide an absolute
path of the directory
optional arguments:
-h, --help show this help message and exit
--config-file CONFIG_FILE
Config file to use. By default, /etc/wmfbackups/backups.cnf
.
--host HOST Host to generate the backup from. Default: localhost.
--port PORT Port to connect to. Default: 3306.
--user USER User to connect for backup. Default: root.
--password PASSWORD Password used to connect. Default: empty password.
--threads THREADS Number of threads to use for exporting. Default: 18
concurrent threads.
--type {dump,snapshot}
Backup type: dump or snapshot. Default: dump
--only-postprocess If present, only postprocess and perform the metadata
gathering metrics for the given ongoing section
backup, skipping the actual backup. Default: Do the
whole process.
--rotate If present, run the rotation process, by moving it to
the standard."latest" backup. Default: Do not rotate.
--retention RETENTION
If rotate is set, purge backups of this section older
than the given value, in days. Default: 18 days.
--backup-dir BACKUP_DIR
Directory where the backup will be stored. Default:
/srv/backups.
--rows ROWS Max number of rows to dump per file. Default: 20000000
--archive If present, archive each db on its own tar file.
--compress If present, compress everything into a tar.gz.Default:
Do not compress.
--regex REGEX Only backup tables matching this regular
expression,with format: database.table. Default: all
tables
--stats-file STATS_FILE
Separate file where the statistics options are
defined. This option and stats-host are exclusive.
--stats-host STATS_HOST
Host where the statistics database is. This option and
stats-file are exclusive
--stats-port STATS_PORT
Port where the statistics database is. Default: 3306
--stats-user STATS_USER
User for the statistics database.
--stats-password STATS_PASSWORD
Password used for the statistics database.
--stats-database STATS_DATABASE
MySQL schema that contains the statistics database.