.. _installation-development-installation: Initial installation ==================== Obtaining the source code ------------------------- When developing, the first step is always to obtain the source code of |kadi| using `git `__, which might have to be installed first using: .. code-block:: bash sudo apt install git Before actually obtaining the code, it is recommended to create a fork of the `main repository `__. Afterwards, the code can be cloned into a local directory via SSH (recommended) or HTTPS, the latter being shown in the following command. Note that the ```` placeholder needs to be substituted with the correct username/namespace that the new fork resides in: .. code-block:: bash git clone https://gitlab.com//kadi.git ${HOME}/workspace/kadi This will copy the code into the ``workspace/kadi`` directory in the current user's home directory. This directory can of course be changed freely, however, the rest of this documentation assumes that the source code resides there and that all commands are run within this directory: .. code-block:: bash cd ${HOME}/workspace/kadi To be able to update the code from the central repository, it can be added as an additional remote, often called ``upstream`` (note that the default remote after cloning, pointing to the new fork, is always called ``origin``): .. code-block:: bash git remote add upstream https://gitlab.com/iam-cms/kadi.git Installing the dependencies --------------------------- .. tip:: Some of the dependent services mentioned in this section, namely PostgreSQL, Redis and (optionally) Elasticsearch, can also be run via :ref:`Docker containers ` instead of installing and running them separately. .. include:: ../snippets/dependencies.rst Node.js ~~~~~~~ `Node.js `__ and `npm `__ are used for managing the frontend dependencies and building/compiling the asset bundles. The current LTS version can be installed using: .. code-block:: bash sudo apt install ca-certificates curl gnupg sudo install -m 0755 -d /etc/apt/keyrings curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | sudo gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_22.x nodistro main" | sudo tee /etc/apt/sources.list.d/nodesource.list sudo apt update && sudo apt install nodejs For more information, see also :ref:`assets ` in the project overview. Elasticsearch ~~~~~~~~~~~~~ .. note:: As Elasticsearch requires quite a few system resources, especially memory, installation is optional for development. **If you opt not to install it at the moment, this whole section may be skipped completely.** Note that in this case, the full-text resource search functionality will not work. If Elasticsearch is required, especially when developing search-related functionality, it can also be installed later on using the instructions below this box. Remember to initialize the search indices after the installation using the Kadi CLI, as explained later in the :ref:`initialization ` section. Alternatively, even if installed now, Elasticsearch can always be stopped until needed again using: .. code-block:: bash sudo systemctl disable elasticsearch # Disable Elasticsearch starting automatically sudo systemctl stop elasticsearch # Stop Elasticsearch sudo systemctl start elasticsearch # When needed, start it again Once Elasticsearch is running again, using the Kadi CLI may be required to reindex existing data if the search indices went out of sync with the database, which happens when creating or updating resources while Elasticsearch is not running: .. code-block:: bash kadi search reindex .. include:: ../snippets/elasticsearch.rst Installing |kadi| ----------------- To create and activate a new virtual environment for the application, the following commands can be used: .. code-block:: bash virtualenv -p python3 ${HOME}/.venvs/kadi source ${HOME}/.venvs/kadi/bin/activate .. include:: ../snippets/venv.rst .. tip:: When using the :ref:`virtualenvwrapper ` tool, new environments created like explained in the note above can still be managed using this tool after the initial creation, as long as the new virtual environment was created in the directory specified by the ``WORKON_HOME`` environment variable. This will create and activate a new virtual environment named *kadi* using Python 3 as interpreter. The environment is stored inside the ``.venvs`` directory in the current user's home directory. This directory can of course be changed freely. For an easier way to manage virtual environments, a tool like :ref:`virtualenvwrapper ` can also be helpful. **For all following steps, the virtual environment is assumed to always be active** (by sourcing the ``activate`` script). Afterwards, the application can be installed via `pip `__ using the previously checked out source code: .. code-block:: bash pip install -e .[dev] This will install the application in editable mode (``-e``), which simply creates a link to the sources so all changes are reflected in the installed package immediately. By specifying the ``[dev]`` modifier, all development dependencies will be installed as well. At this point, it is also recommended to install the :ref:`pre-commit ` hooks already by running: .. code-block:: bash pre-commit install Configuration ------------- .. _installation-development-installation-configuration-postgresql: PostgreSQL ~~~~~~~~~~ To set up PostgreSQL, a user and a database belonging to that user have to be created. **When prompted for a password, it is recommended to use** ``kadi``. This way, the default development configuration of the application does not need to be changed later on. .. code-block:: bash sudo -Hiu postgres createuser -P kadi sudo -Hiu postgres createdb -O kadi -E utf-8 -T template0 kadi .. _installation-development-installation-configuration-kadi4mat: |kadi| ~~~~~~ In order for the application (or more specifically, its command line interface, as explained in the next section) to know which environment it should run in, the ``KADI_ENV`` environment variable needs to be set accordingly. It is highly recommended to set this variable via a ``.env`` file. This file has to be created first, and should reside in the project's root directory, i.e. ``${HOME}/workspace/kadi/.env``, when following along with the example directory structure. After creating it, the following content needs to be added: .. code-block:: none KADI_ENV=development For the ``development`` environment, all configuration values, e.g. regarding the database, have usable default values set. These correspond to the configuration described in this documentation, so no further changes should be necessary. **If necessary**, customizing any configuration values is best done using the ``KADI_CONFIG_FILE`` environment variable. This variable needs to point to a valid configuration file, which also needs to be created first, in which the desired values can be specified. As before, the ``.env`` file should be used, e.g. assuming the config file resides in ``config/config.py`` relative to the project's example root directory, the following content can be added: .. code-block:: none KADI_CONFIG_FILE=${HOME}/workspace/kadi/config/config.py Please see :ref:`Configuration ` for more information about the configuration file itself and its values. Also, check out the manual :ref:`production ` installation for an example and ``kadi/config.py`` to see all default configuration values for the different environments, grouped via different classes. .. _installation-development-installation-init: Initializing the application ---------------------------- Before the application can be used, some initialization steps have to be done using the Kadi :ref:`command line interface ` (CLI). As the Kadi CLI needs to run in the context of the application, it needs access to the ``.env`` file, so the correct environment is used. **Therefore, commands should always be run inside the directory that contains this file.** .. code-block:: bash kadi assets dev # Install the frontend dependencies and build the assets kadi db init # Initialize the database kadi i18n compile # Compile the backend translations If Elasticsearch was installed previously, the following command has to be run additionally to initialize the search indices: .. code-block:: bash kadi search init Another useful command when setting up the application for the first time is the following one, which can be used to set up some sample users and resources: .. code-block:: bash kadi db sample-data The command creates the following sample users: ======== ======== =========== ======== ===================================== Username Password System role Sysadmin Notes ======== ======== =========== ======== ===================================== user user Member Yes Main user for testing purposes. admin admin Admin No Admin user. Can manage all resources. member member Member No Regular user. Can create resources. guest guest Guest No Guest user. Read only access. ======== ======== =========== ======== ===================================== Running the application ----------------------- Flask includes a lightweight development server, which can be run using the Kadi CLI: .. code-block:: bash kadi run Afterwards, the application should be reachable locally at ``__. To be able to run asynchronous background tasks with Celery (needed for example when processing chunked file uploads, purging records or sending emails), the following command needs to be run in a separate terminal: .. code-block:: bash kadi celery worker -B --loglevel=INFO This will start the normal Celery worker as well as Celery Beat in a single process for convenience when developing. The latter is used for periodic tasks (e.g. for removing expired, deleted resources).