.. _installation-production-backup: Backing up the application ========================== Aside from backing up the installed application itself, or at least all of the important configuration files, it is important to regularly create backups of the database and the persisted file data in production environments. The following sections highlight some useful tools and the relevant configuration values of |kadi| in order to get started. .. warning:: In order to keep the database and file system used to store local files in sync, both backups should always be performed as simultaneously as possible. .. _installation-production-backup-database: Database backups ---------------- Since PostgreSQL is used as database system, the ``pg_dump`` utility provided by PostgreSQL can be used to create backups of the database, which can later be restored again using ``pg_restore``. The concrete backup command to run depends on the specific way the backup should be performed and on the installed version of PostgreSQL. The following commands show just one example of how to use ``pg_dump``, including some initial setup, assuming the directory ``/mnt/backups/kadi`` exists and has the correct permissions to be used by the ``kadi`` user: .. code-block:: bash sudo su - kadi # Switch to the kadi user echo "::::" > ~/.pgpass # Create a PostgreSQL password file chmod 600 ~/.pgpass # Give the password file the correct permissions pg_dump -Fc -h -U -f /mnt/backups/kadi/db.dump # Perform the actual backup The PostgreSQL password file and the actual backup command require some parameters that depend on the :confval:`SQLALCHEMY_DATABASE_URI` configuration value of |kadi|. Note that specifying the port, which defaults to ``5432``, is mandatory in the password file. Based on the example above, the following command could be used to restore the backed up database contents, assuming the specified database has been created as usual and is still empty (otherwise, the ``--clean`` argument has to be supplied in addition): .. code-block:: bash pg_restore -h -U -d /mnt/backups/kadi/db.dump Note that the command will prompt for the password of the database user unless a corresponding ``.pgpass`` file is present like in the backup command above. .. warning:: When restoring from backup, either make sure that the |kadi| version that was used when creating the backup matches the one that is used when restoring it, or upgrade the database schema **afterwards** using: .. code-block:: bash sudo su - kadi # Switch to the kadi user kadi db upgrade # Upgrade the database schema For more details about ``pg_dump`` and ``pg_restore``, please see the official documentation, e.g. the `following `__ for the current PostgreSQL version. After restoring from backup, the search indices may need to be recreated based on the database contents, unless Elasticsearch is hosted or backed up separately. The following command can be used to perform this task: .. code-block:: bash sudo su - kadi # Switch to the kadi user kadi search reindex # Recreate and -populate the search indices In case any old indices still exist, these should still be searchable until the command finishes (in case the application is already running while performing the operation). Once the command finishes, the old indices are deleted and switched with the new ones afterwards. Note that interrupting the command may lead to orphaned indices being created, which can be deleted manually by using: .. code-block:: bash sudo su - kadi # Switch to the kadi user kadi search ls # List all search indices kadi search remove # Remove all search indices specified by File backups ------------ .. note:: This section focuses on the built-in local storage provider configured via the :confval:`STORAGE_PATH` configuration item (see also the complete :ref:`storage configuration `). However, note that different storage providers most likely require different backup and restore strategies. In order to backup all locally stored file data, namely all record files as well as miscellaneous uploads, such as profile or group images, the contents stored in the paths specified by the :confval:`STORAGE_PATH` and :confval:`MISC_UPLOADS_PATH` configuration values of |kadi| are relevant. After restoring from backup, to ensure that the application can handle the files stored in :confval:`STORAGE_PATH` correctly, the previous ownership and permissions of all files should be restored (if not already the case), depending on the user and group under which the application is running. For this, the following commands can be used, assuming the default user/group and storage directory are used: .. code-block:: bash sudo chown -R kadi:kadi /opt/kadi/storage sudo find /opt/kadi/storage -type d -exec chmod 750 {} \; sudo find /opt/kadi/storage -type f -exec chmod 640 {} \; The same process can be repeated for the configured :confval:`MISC_UPLOADS_PATH`, which usually corresponds to ``/opt/kadi/uploads``. Additionally, in order to verify that there are no inconsistencies between the file information stored in the database and the actual data stored by the corresponding storage provider(s), the following command may be used: .. code-block:: bash sudo su - kadi # Switch to the kadi user kadi files check # Check for any file inconsistencies Note that running this command might take a while to complete, depending on the amount of stored data. (Optional) Redis backups ------------------------ Generally speaking, it might also make sense to backup Redis, especially if your instance of |kadi| makes use of a lot of background tasks. Usually, Redis should already be configured by default to save a snapshot of its database to ``/var/lib/redis/dump.rdb`` in regular intervals, which can simply be backed up as usual. Restoring from backup can then be performed by running the following commands: .. code-block:: bash sudo systemctl stop redis-server sudo mv /var/lib/redis/dump.rdb sudo chown redis:redis /var/lib/redis/dump.rdb sudo chmod 660 /var/lib/redis/dump.rdb sudo systemctl start redis-server For more information about changing the snapshot intervals or performing snapshots on demand, please also refer to the official `Redis persistence `__ documentation.