Manual installation

Manual installation is currently the recommended installation method, as it offers more flexibility in terms of how the development environment is set up. If you are interested in running some of the listed dependencies in Docker containers, without losing too much flexibility, please refer to the hybrid installation as well.

Installing the dependencies

Python and Virtualenv

As the backend code of the application is based on the Flask web framework and multiple other Python libraries, Python 3 needs to be installed. This should generally be the case already, otherwise it can be installed using:

sudo apt install python3

Note

Note that a Python version >=3.7 and <3.11 is required. The currently installed version can be checked using:

python3 --version

If the currently installed Python version is not suitable, please see the instructions on how to install and use a different Python version for production or development installations.

To create an isolated environment for the application and its dependencies, Virtualenv is used, which can be installed like this:

sudo apt install virtualenv

Libraries

Some external libraries and tools are required as additional dependencies, which can be installed using:

sudo apt install libmagic1 build-essential python3-dev python3-venv libpq-dev libpcre3-dev

PostgreSQL

The RDBMS used in the application is PostgreSQL. Any up to date version >=11 should work, which can be installed like this:

sudo apt install postgresql

Redis

Redis is an in-memory data structure that can be used for different purposes. Currently it is used as cache for request rate limiting and as a message broker for running asynchronous tasks with Celery, a distributed task queue. Any up to date version >=5 should work, which can be installed like this:

sudo apt install redis-server

Node.js

Node.js and npm are used for managing the frontend dependencies and building/compiling the asset bundles. The current LTS version can be installed using:

sudo apt install curl

# When using Debian
curl -fsSL https://deb.nodesource.com/setup_18.x | sudo bash -
# When using Ubuntu
curl -fsSL https://deb.nodesource.com/setup_18.x | sudo -E bash -

sudo apt install nodejs

For more information, see also assets.

Elasticsearch

Note

As Elasticsearch requires quite a few system resources, especially memory, installation is optional for development. If you opt not to install it at the moment, this whole section may be skipped completely. Note that in this case, the full-text resource search functionality will not work.

If Elasticsearch is required, especially when developing search-related functionality, it can also be installed later on using the instructions outside of this box. Remember to initialize the search indices after the installation using the Kadi CLI, as explained in Setting up the application.

Alternatively, even if installed now, Elasticsearch can always be stopped until needed again using:

sudo systemctl disable elasticsearch # Disable Elasticsearch starting automatically
sudo systemctl stop elasticsearch    # Stop Elasticsearch
sudo systemctl start elasticsearch   # When needed, start it again

Once Elasticsearch is running again, using the Kadi CLI may be required to reindex existing data if the search indices went out of sync with the database, which happens when creating or updating resources while Elasticsearch is not running:

kadi search reindex

Elasticsearch is the full-text search engine used in the application. Currently, only version 7 is supported, which can be installed like this:

sudo apt install wget apt-transport-https gnupg
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-7.x.list
sudo apt update && sudo apt install elasticsearch

Some configuration values have to be set manually after installation by using the following command. These settings configure Elasticsearch to use a single-node cluster and disable the basic security features included in free Elasticsearch installations, which are not necessarily needed in this simple setup.

echo -e "discovery.type: single-node\nxpack.security.enabled: false" | sudo tee -a /etc/elasticsearch/elasticsearch.yml

To start Elasticsearch and also configure it to start automatically when the system boots, the following commands can be used:

sudo systemctl enable elasticsearch.service
sudo systemctl start elasticsearch.service

Note

If the Elasticsearch service has trouble starting in time, try increasing the start timeout from 90 seconds to 180 seconds:

sudo mkdir /etc/systemd/system/elasticsearch.service.d
echo -e "[Service]\nTimeoutStartSec=180" | sudo tee /etc/systemd/system/elasticsearch.service.d/startup-timeout.conf
sudo systemctl daemon-reload
sudo systemctl restart elasticsearch.service

Installing Kadi4Mat

To create and activate a new virtual environment for the application, the following commands can be used:

virtualenv -p python3 ${HOME}/.venvs/kadi
source ${HOME}/.venvs/kadi/bin/activate

Note

If not using the default Python version installed via APT, virtual environments need to be created in a slightly different way, since the virtualenv version installed via APT may not work properly with the newly installed Python version. Instead, the built-in venv module can be used:

python3.x -m venv <my_venv>   # Instead of "virtualenv -p python3 <my_venv>"
source <my_venv>/bin/activate
pip install wheel
pip install -U pip

Note that the wheel package needs to be installed separately after activating the virtual environment for the first time, while updating pip ensures that its most recent version is used, both of which virtualenv normally would have done automatically.

Tip

When using the virtualenvwrapper tool, the new environment can still be managed using the tool after initially creating it, as long as the new virtual environment was created in the directory specified by the WORKON_HOME environment variable.

This will create and activate a new virtual environment named kadi using Python 3 as interpreter. The environment is stored inside the .venvs directory in the current user’s home directory. This directory can of course be changed freely. For an easier way to manage virtual environments, a tool like virtualenvwrapper can also be helpful. For all following steps, the virtual environment is assumed to always be active (by sourcing the activate script).

Afterwards, the application can be installed via pip using the previously checked out source code:

pip install -e .[dev]

This will install the application in editable mode (-e), which simply creates a link to the sources so all changes are reflected in the installed package immediately. By specifying the [dev] modifier, all development dependencies will be installed as well.

At this point, it is also recommended to install the pre-commit hooks already by running:

pre-commit install

Configuration

PostgreSQL

To set up PostgreSQL, a user and a database belonging to that user have to be created. When prompted for a password, it is recommended to use kadi. This way, the default development configuration of the application does not need to be changed later on.

sudo -Hiu postgres createuser -P kadi
sudo -Hiu postgres createdb -O kadi kadi -E utf-8

Kadi4Mat

In order for the application (or more specifically, its command line interface, as explained in the next section) to know which environment it should run in, the KADI_ENV environment variable needs to be set accordingly. It is highly recommended to set this variable via a .env file. This file has to be created first, and should reside in the project’s root directory, i.e. ${HOME}/workspace/kadi/.env, when following along with the example directory structure. After creating it, the following content needs to be added:

KADI_ENV=development

For the development environment, all configuration values, e.g. regarding the database, have usable default values set. These correspond to the configuration described in this documentation, so no further changes should be necessary.

If necessary, customizing any configuration values is best done using the KADI_CONFIG_FILE environment variable. This variable needs to point to a valid configuration file, which also needs to be created first, in which the desired values can be specified. As before, the .env file should be used, e.g. assuming the config file resides in config/config.py relative to the project’s example root directory, the following content can be added:

KADI_CONFIG_FILE=${HOME}/workspace/kadi/config/config.py

Please see Configuration for more information about the configuration file itself and its values. Also, check out the manual production installation for an example and kadi/config.py to see all default configuration values for the different environments, grouped via different classes.

Setting up the application

Before the application can be used, some initialization steps have to be done using the Kadi command line interface (CLI). As the Kadi CLI needs to run in the context of the application, it needs access to the .env file, so the correct environment is used. Therefore, commands should always be run inside the directory that contains this file.

kadi assets dev   # Install the frontend dependencies and build the assets
kadi db init      # Initialize the database
kadi i18n compile # Compile the backend translations

If Elasticsearch was installed previously, the following command has to be run additionally to initialize the search indices:

kadi search init

Another useful command when setting up the application for the first time is the following one, which can be used to set up some sample users and resources:

kadi db sample-data

The command creates the following sample users:

Username	Password	System role	Sysadmin	Notes
user	user	Member	Yes	Main user for testing purposes.
admin	admin	Admin	No	Admin user. Can manage all resources.
member	member	Member	No	Regular user. Can create resources.
guest	guest	Guest	No	Guest user. Read only access.

Running the application

Flask includes a lightweight development server, which can be run using the Kadi CLI:

kadi run

Afterwards, the application should run locally on port 5000:

firefox http://localhost:5000

To be able to run asynchronous background tasks with Celery (needed for example when processing chunked file uploads or sending emails), the following command can be run in a separate terminal:

kadi celery worker -B --loglevel=INFO

This will start the normal Celery worker as well as Celery Beat, which is used for periodic tasks (e.g. for removing expired, deleted resources) in a single process, which is convenient for development.

To simulate sending emails without using an actual SMTP server, the following can be used to simply print the emails on the terminal instead (this debugging server is already configured as default when developing):

python -m smtpd -n -c DebuggingServer localhost:8025

Updating the application

After updating the application code via git, the following commands may have to be run again:

pip install -U pip    # (Optional) Make sure the newest version of pip is being used
pip install -e .[dev] # Install any new backend dependencies
kadi db upgrade       # Upgrade the database schema
kadi assets dev       # Install any new frontend dependencies and rebuild the assets
kadi i18n compile     # Recompile any new backend translations

While mainly intended for productive installations, it is also recommended to take a look at the update notes, since some of the listed changes may also be relevant for development environments.