Manual installation

Manual installation is currently the recommended installation method, as it offers more flexibility in terms of how the development environment is set up. If you are interested in running some of the listed dependencies in Docker containers, without losing too much flexibility, please refer to the hybrid installation as well.

Installing the dependencies

Python and Virtualenv

As the backend code of the application is based on the Flask web framework and multiple other Python libraries, Python 3 needs to be installed. This should generally be the case already, otherwise it can be installed using:

sudo apt install python3

Note

Note that a Python version >=3.7 and <3.11 is required. The currently installed version can be checked using:

python3 --version

If the currently installed Python version is not suitable, please see the instructions on how to install and use a different Python version for production or development installations. Please also note the section about using virtual environments, which will be relevant in a later step.

To create an isolated environment for the application and its dependencies, Virtualenv is used, which can be installed like this:

sudo apt install virtualenv

Libraries

Some external libraries and tools are required as additional dependencies, which can be installed using:

sudo apt install libmagic1 build-essential python3-dev python3-venv libpq-dev libpcre3-dev

PostgreSQL

The RDBMS used in the application is PostgreSQL. Any up to date version >=11 should work, which can be installed like this:

sudo apt install postgresql

Redis

Redis is an in-memory data structure that can be used for different purposes. Currently it is used as cache for request rate limiting and as a message broker for running asynchronous tasks with celery, a distributed task queue. Any up to date version >=5 should work, which can be installed like this:

sudo apt install redis-server

Node.js

Node.js and npm are used for managing the frontend dependencies and building/compiling the asset bundles. The current LTS version can be installed using:

sudo apt install curl
curl -fsSL https://deb.nodesource.com/setup_18.x | sudo bash -
sudo apt install nodejs

For more information, see also assets.

Elasticsearch

Note

As Elasticsearch requires quite a few system resources, especially memory, installation is optional for development. If you opt not to install it at the moment, this whole section may be skipped completely. Note that in this case, the full-text resource search functionality will not work.

If Elasticsearch is required, especially when developing search-related functionality, it can also be installed later on using the instructions outside of this box. Remember to initialize the search indices after the installation using the Kadi CLI, as explained in Setting up the application.

Alternatively, even if installed now, Elasticsearch can always be stopped until needed again using:

sudo systemctl disable elasticsearch # Disable Elasticsearch starting automatically
sudo systemctl stop elasticsearch    # Stop Elasticsearch
sudo systemctl start elasticsearch   # When needed, start it again

Once Elasticsearch is running again, using the Kadi CLI may be required to reindex existing data if the search indices went out of sync with the database, which happens when creating or updating resources while Elasticsearch is not running:

kadi search reindex

Elasticsearch is the full-text search engine used in the application. Currently, only version 7 is supported, which can be installed like this:

sudo apt install wget apt-transport-https gnupg
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-7.x.list
sudo apt update && sudo apt install elasticsearch

Some configuration values have to be set manually after installation by using the following command. These settings configure Elasticsearch to use a single-node cluster and disable the basic security features included in free Elasticsearch installations, which are not necessarily needed in this simple setup.

echo -e "discovery.type: single-node\nxpack.security.enabled: false" | sudo tee -a /etc/elasticsearch/elasticsearch.yml

To start Elasticsearch and also configure it to start automatically when the system boots, the following commands can be used:

sudo systemctl enable elasticsearch.service
sudo systemctl start elasticsearch.service

Note

If the Elasticsearch service has trouble starting in time, try increasing the start timeout from 90 seconds to 180 seconds:

sudo mkdir /etc/systemd/system/elasticsearch.service.d
echo -e "[Service]\nTimeoutStartSec=180" | sudo tee /etc/systemd/system/elasticsearch.service.d/startup-timeout.conf
sudo systemctl daemon-reload
sudo systemctl restart elasticsearch.service

Installing Kadi4Mat

To create and activate a new virtual environment for the application, the following commands can be used:

virtualenv -p python3 ${HOME}/.venvs/kadi
source ${HOME}/.venvs/kadi/bin/activate

This will create and activate a new virtual environment named kadi using Python 3 as interpreter. The environment is stored inside the .venvs directory in the current user’s home directory. This directory can of course be changed freely. For an easier way to manage virtual environments, a tool like virtualenvwrapper can also be helpful. For all following steps, the virtual environment is assumed to always be active.

Afterwards, the application can be installed via pip using the previously checked out source code:

pip install -e .[dev]

This will install the application in editable mode (-e), which simply creates a link to the sources so all changes are reflected in the installed package immediately. By specifying the [dev] modifier, all development dependencies will be installed as well.

At this point, it is also recommended to install the pre-commit hooks already by running:

pre-commit install

Configuration

PostgreSQL

To set up PostgreSQL, a user and a database belonging to that user have to be created. When prompted for a password, it is recommended to use kadi. This way, the default development configuration of the application does not need to be changed later on.

sudo -Hiu postgres createuser -P kadi
sudo -Hiu postgres createdb -O kadi kadi -E utf-8

Kadi4Mat

In order for the application (or more specifically, its command line interface, as explained in the next section) to know which environment it should run in, the KADI_ENV environment variable needs to be set accordingly. It is highly recommended to set this variable via a .env file. This file has to be created first, and should reside in the project’s root directory, i.e. ${HOME}/workspace/kadi/.env, when following along with the example directory structure. After creating it, the following content needs to be added:

KADI_ENV=development

For the development environment, all configuration values, e.g. regarding the database, have default values set, which correspond to the configuration described in this documentation, so no further changes should be necessary.

If necessary, customizing any configuration values is best done using the KADI_CONFIG_FILE environment variable. This variable needs to point to a valid configuration file, which also needs to be created first, in which the desired values can be specified. As before, the .env file should be used, e.g. assuming the config file resides in config/config.py relative to the project’s example root directory, the following content can be added:

KADI_CONFIG_FILE=${HOME}/workspace/kadi/config/config.py

Please see Configuration for more information about the configuration file itself and its values. Also, check out the manual production installation for an example and kadi/config.py to see all default configuration values for the different environments, grouped via different classes.

Setting up the application

Before the application can be used, some initialization steps have to be done using the Kadi command line interface (CLI). As the Kadi CLI needs to run in the context of the application, it needs access to the .env file, so the correct environment is used. Therefore, commands should always be run inside the directory that contains this file.

kadi assets dev   # Install the frontend dependencies and build the assets
kadi db init      # Initialize the database
kadi i18n compile # Compile the backend translations

If Elasticsearch was installed previously, the following command has to be run additionally to initialize the search indices:

kadi search init

Another useful command when setting up the application for the first time is the following one, which can be used to set up some sample users and resources:

kadi db sample-data

The command creates the following sample users:

Username

Password

System role

Sysadmin

Notes

user

user

Member

Yes

Main user for testing purposes.

admin

admin

Admin

No

Admin user. Can manage all resources.

member

member

Member

No

Regular user. Can create resources.

guest

guest

Guest

No

Guest user. Read only access.

Running the application

Flask includes a lightweight development server, which can be run using the Kadi CLI:

kadi run

Afterwards, the application should run locally on port 5000:

firefox http://localhost:5000

To be able to run asynchronous background tasks with celery (needed for example when processing chunked file uploads or sending emails), the following command can be run in a separate terminal:

kadi celery worker -B --loglevel=INFO

This will start the normal celery worker as well as celery beat, which is used for periodic tasks (e.g. for removing expired, deleted resources) in a single process, which is convenient for development.

To simulate sending emails without using an actual SMTP server, the following can be used to simply print the emails on the terminal instead (this debugging server is already configured as default when developing):

python -m smtpd -n -c DebuggingServer localhost:8025

Updating the application

After updating the application code via git, the following commands may have to be run again:

pip install -U pip    # (Optional) Make sure the newest version of pip is being used
pip install -e .[dev] # Install any new backend dependencies
kadi assets dev       # Install any new frontend dependencies and rebuild the assets
kadi db upgrade       # Upgrade the database schema
kadi i18n compile     # Recompile any new backend translations

While mainly intended for productive installations, it is also recommended to take a look at the update notes, since some of the listed changes may also be relevant for development environments.