Open Notificaties Documentation¶

Open Notificaties is a modern, open-source messaging service that implements the VNG API standard for the “Notificaties API” in line with the Common Ground model. Among others, it can be used in combination with Open Zaak which implements the VNG standards for “API’s voor Zaakgericht werken” that require a Notificaties API.

Getting Started¶

To get you started, you might find some of these links relevant:

New to Open Notificaties? Have a look at the Introduction
New to the VNG standards for “API’s voor Zaakgericht werken”? Read up on the API-specifications.
Want to get started with Open Notificaties yourself? See Installation.
Need help with Open Notificaties? Contact Support.
Are you a developer? Head over to Development (TODO)!

Open Notificaties is and only uses Open-source.

Introduction¶

Open Notificaties is a modern, open-source messaging service that implements the VNG API standard for the “Notificaties API” in line with the Common Ground model. Among others, it can be used in combination with Open Zaak which implements the VNG standards for “API’s voor Zaakgericht werken” that require a Notificaties API.

Open Notificaties exposes a Notificaties API (notifications) to route privacy-safe messages from publishers to subscribers.

Open Notificaties is based on the API reference implementations by VNG Realisatie to create a production-grade product that can be used by municipalities.

Architecture¶

Open Notificaties is based on the reference implementation of the “API’s voor Zaakgericht werken” made by VNG Realisatie. The overall architecture remains faithful to the Common Ground principles and all API specifications.

The architecture of Open Notificaties focusses on excellent performance, optimal stability and to guarantee data integrity. Under the hood, it uses RabbitMQ as message broker with just a light weight REST API on top of it.

It largely resembles the original reference implementation and remains its own component to scale separately from any other component.

Who’s behind Open Notificaties?¶

The Open Notificaties project was built for the Open Zaak project, initiated shortly before the 1.0 release of the VNG standards for “API’s voor Zaakgericht werken”, on August 1, 2019, by 9 municipalities:

Amsterdam
Rotterdam
Utrecht
Tilburg
Arnhem
Haarlem
‘s-Hertogenbosch
Delft
Hoorn, Medemblik, Stede Broec, Drechteland, Enkhuizen (SED)

Using Dimpact as a legal entity, they formed a project to develop a modern, open-source data- and services-layer to enable zaakgericht werken, in line with the Common Ground model.

Open-source¶

Open Notificaties is open-source and available under the EUPL license.

In addition, this project makes use of various open-source Python libraries and npm packages under the hood.

API-specifications¶

Open Notificaties adheres to the API-specifications as described by the VNG API standard for the “Notificaties API”.

Supported API versions¶

The following API’s are available in Open Notificaties:

API	Specification version(s)
Notificaties API	1.0

Installation¶

There are several ways to install Open Notificaties. A scalable solution is to use Kubernetes. You can also run the Docker containers on a single machine

Before you begin¶

Check the minimum system requirements for the target machine(s).
Make sure the target machine(s) have access to the Internet.
The target machine(s) should be reachable via at least a local DNS entry:
- Open Zaak: open-zaak.<organization.local>
- Open Notificaties: open-notificaties.<organization.local>
The machine(s) do not need to be publically accessible and do not need a public DNS entry. In some cases, you might want this but it’s not recommended. The same machine can be used for both Open Zaak and Open Notificaties.
If you want to use NLX, make sure you have a publicaly available domain name, for example nlx.<organization.com>, where your NLX-inway is accessible to the outside world.

Guides¶

Hardware¶

Based on our initial performance tests in both a Kubernetes environment and single machine setups, we can indicate some minimum system requirements to reach a certain performance.

Notifications are likely to increase over time, due to more external components makes use of it, or more subscribers want to receive notifications. Hence, we recommend a Kubernetes setup since it will scale more easy.

Determine what you need¶

Since notifications are typically only sent for update and create actions, the number of notifications will most likely not be that high. Some functional views might cause a series of create and/or update actions, so there is a short burst of notifications which you will need to handle.

It really depends on how many consumers and other components will make use of Open Notificaties but assuming 50 requests per second will probably be a good start for any environment.

Minimum system requirements¶

Platform: 64-bit
Processor(s): 4 - 12 CPUs (see below) at 2.0 GHz
RAM: 4 - 12 GB (see below)
Hard disk space: 20 GB

Based on the number of requests per second you need, you can see what kind of hardware you need to achieve this.

Requests per second	CPUs	Memory (GB)
50	4	4
100	6	6
200	12	12

With these specifications you can run everything on a single machine or divided over several instances.

General recommendations¶

Use a seperate database server with roughly a third of the CPUs and memory as the main server. The database is usually the limiting factor.

Kubernetes recommendations¶

Preferably use 2 load balancer (like Traefik) replica’s.
Use as many replica’s as available CPU’s taking into account you need to have a few replica’s for your load balancer, and possibly other services.

Deploying on Kubernetes (TODO)¶

Deploying on a single server¶

Open Notificaties can be deployed on a single machine - either a dedicated server (DDS) or virtual private server (VPS). The required hardware can be rented from a hosting provider or be provided in your environment. Please see Hardware to determine the hardware requirements.

This documentation describes the architecture, prerequisites and how to deploy Open Notificaties on a server. Additionally, it documents the possible configuration options.

Note

The default settings allow Open Notificaties to be deployed to the same machine as Open Zaak.

Architecture¶

The application is deployed as Docker containers, of which the images are available on docker hub. Traffic is routed to the server, where the web server (nginx) handles SSL termination and proxies the requests to the application containers.

Data is stored in a PostgreSQL database. By default, the database is installed on the same machine (running on the host), but you can make use of a hosted database (Google Cloud, AWS, Azure…). See the Configuration parameters for more information.

Prerequisites¶

Before you can deploy, you need:

A server¶

Ensure you have a server with root privileges. We assume you can directly ssh to the machine as root user. If that’s not the case, a user with sudo will also work. Python 3 must be available on the server.

Note

Make sure there is enough space in /var/lib/docker. You need at least 8 GB to download all Docker container images.

Supported operating systems

Support for different Linux flavours is maintained in the Ansible collection used for deployment.

Currently the following OS flavours are supported:

Debian: buster (10, actively supported), stretch (9, actively supported), jessie (8)
Ubuntu: eoan (EOL), disco (EOL), cosmic (EOL), bionic (18.04 LTS). focal (20.04 LTS) is not tested yet
SUSE Enterprise Linux: 15 (actively supported)
OpenSUSE: 15.1
Red Hat: 7, 8
CentOS: 7, 8 (actively supported)

A copy of the deployment configuration¶

You can either clone the https://github.com/open-zaak/open-notificaties repository, or download and extract the latest ZIP: https://github.com/open-zaak/open-notificaties/archive/master.zip

Python and a Python virtualenv¶

You will need to have at least Python 3.5 installed on your system. In the examples, we assume you have Python 3.6.

Create a virtualenv with:

[user@laptop]$ python3.6 -m venv env/
[user@laptop]$ source env/bin/activate

Make sure to install the deployment tooling. In your virtualenv, install the dependencies:

(env) [user@laptop]$ pip install -r deployment/requirements.txt
(env) [user@laptop]$ ansible-galaxy collection install -r requirements.yml
(env) [user@laptop]$ ansible-galaxy role install -r requirements.yml

Deployment¶

Deployment is done with an Ansible playbook, performing the following steps:

Install and configure PostgreSQL database server
Install the Docker runtime
Install the SSL certificate with Let’s Encrypt
Setup Open Notificaties with Docker
Install and configure nginx as reverse proxy

Initial steps¶

Make sure the virtualenv is activated:

[user@laptop]$ source env/bin/activate

Navigate to the correct deployment directory:

(env) [user@laptop]$ cd deployment/single-server

Create the vars/open-notificaties.yml file - you can find an example in vars/open-notificaties.yml.example. Generate a secret key using the django secret key generator and put the value between single quotes.

Configure the host by creating the hosts file from the example:

(env) [user@laptop]$ cp hosts.example hosts

Edit the open-notificaties.gemeente.nl to point to your actual domain name. You must make sure that the DNS entry for this domain points to the IP address of your server.

Warning

It’s important to use the correct domain name, as the SSL certificate will be generated for this domain and only this domain will be whitelisted by Open Notificaties! If you are using a private DNS name, then no SSL certificate can be created via Letsencrypt - make sure to disable it by setting certbot_create_if_missing=false.

Running the deployment¶

Execute the playbook by running:

(env) [user@laptop]$ ansible-playbook open-notificaties.yml

Hint

If you have your secrets Ansible vault encrypted, make sure you have either:
- set the ANSIBLE_VAULT_PASSWORD_FILE environment variable, or
- pass --ask-vault-pass flag to ansible-playbook.
If you need to override any deployment variables (see Configuration parameters), you can pass variables to ansible-playbook using the syntax: --extra-vars "some_var=some_value other_var=other_value".
If you want to run the deployment from the same machine as where it will run (ie. install to itself), you can pass --connection local to ansible-playbook.
If you cannot connect as root to the target machine, you can pass --user <user> --become --become-method=sudo --ask-become-pass which will connect as user <user> that needs sudo-rights on the target machine to install the requirements.

A full example might look like this:

(env) [user@laptop]$ ansible-playbook open-notificaties.yml \
    --user admin
    --inventory my-hosts \  # Use inventory file ``my-hosts`` instead of ``hosts``.
    --limit open-notificaties.gemeente.nl \  # Only pick open-notificaties.gemeente.nl from the inventory file.
    --extra-vars "certbot_create_if_missing=false app_db_name=opennotificaties-test app_db_user=opennotificaties-test" \
    --connection local \
    --become \
    --become-method=sudo \
    --ask-become-pass

Note

You can run the deployment multiple times, it will not affect the final outcome. If you decide to change configuration parameters, you do not have to start from scratch.

Environment configuration¶

After the initial deployment, some initial configuration is required. This configuration is stored in the database and is only needed once.

Create a superuser

A superuser allows you to perform all administrative tasks.

Log in to the server:

[user@laptop]$ ssh root@open-notificaties.gemeente.nl

Create the superuser (interactive on the shell). Note that the password you type in will not be visible - not even with asterisks. This is normal.

[root@open-notificaties.gemeente.nl]# docker exec -it opennotificaties-0 src/manage.py createsuperuser
Gebruikersnaam: demo
E-mailadres: admin@open-notificaties.gemeente.nl
Password:
Password (again):
Superuser created successfully.

Configure Open Notificaties Admin

See the Open Notificaties configuration on how to configure Open Notificaties post-installation.

Configuration parameters¶

At deployment time, you can configure a number of parts of the deployment by overriding variables. You can override variables on the command line (using the -e "..." syntax) or by overriding them in vars/secrets.yml.

Note

Tweaking configuration parameters is considered advanced usage.

Generic variables¶

certbot_admin_email: e-mail address to use to accept the Let’s Encrypt terms and conditions.
certbot_create_if_missing: whether to use Let’s Encrypt to create an SSL certificate for your domain. Set to false if you want to use an existing certificate.

Open Notificaties specific variables¶

The default values can be found in roles/opennotificaties/defaults/main.yml.

opennotificaties_db_port: database port. If you are running multiple PostgreSQL versions on the same machine, you’ll have to point to the correct port.
opennotificaties_db_host: specify the hostname if you’re using a cloud database or a database on a different server.
opennotificaties_db_name: specify a different database name.
opennotificaties_secret_key: A Django secret key. Used for cryptographic operations - this may NOT leak, ever. If it does leak, change it.

Scaling

The opennotificaties_replicas variable controls scaling on backend services. If your hardware allows it, you can create more replicas. By default, 3 replicas are running.

The format of each replica is:

name: opennotificaties-i
port: 900i

The port number must be available on the host - i.e. you may not have other services already listening on that port.

The opennotificaties_worker_replicas variable controls the scaling of the queue workers - these are responsible for actually distributing the notifications. By default, 3 replicas spin up.

The format of each replica is:

name: opennotificaties-worker-i

Updating an Open Notificaties installation¶

Make sure you have the deployment tooling installed - see the installation steps for more details.

If you have an existing environment (from the installation), make sure to update it:

# fetch the updates from Github
[user@host]$ git fetch origin

# checkout the tag of the version you wish to update to, e.g. 1.0.0
[user@host]$ git checkout X.Y.z

# activate the virtualenv
[user@host]$ source env/bin/activate

# ensure all (correct versions of the) dependencies are installed
(env) [user@host]$ pip install -r requirements.txt
(env) [user@host]$ ansible-galaxy install -r requirements.yml

Open Notificaties deployment code defines variables to specify the Docker image tag to use. This is synchronized with the git tag you’re checking out.

Next, to perform the upgrade, you run the open-notificaties.yml playbook just like with the installation in Running the deployment:

(env) [user@laptop]$ ansible-playbook open-notificaties.yml

Note

This will instruct the docker containers to restart using a new image. You may notice some brief downtime (order of seconds to minutes) while the new image is being downloaded and containers are being restarted.

Open Notificaties configuration¶

Before you can work with Open Notificaties after installation, a few settings need to be configured first.

Configure Notificaties API¶

Open Notificaties uses the Autorisaties API to check if the sender is authorized to send notifications. Open Zaak offers an Autorisaties API and below we assume you use this one.

Configure the Open Zaak Autorisaties API endpoint (so Open Notificaties knows where to check for the proper autorisations):
1. Navigate to Configuratie > Autorisatiecomponentconfiguratie
2. Fill out the form:
  - API root: The URL to the Notificaties API. For example: https://open-zaak.gemeente.local/autorisaties/api/v1/.
  - Component: Notificatierouteringcomponent
3. Click Opslaan.

Adding new component to send notifications¶

For a complete example of setting up Open Zaak to send notifications to Open Notificaties, see the documentation of Open Zaak.

Below are the general steps to allow an application to send notifications and using Open Zaak as the example.

Configure the credentials for the Open Zaak Autorisaties API (so Open Notificaties can access the Autorisaties API):
1. Navigate to API Autorisaties > Externe API credentials
2. Click Externe API credential toevoegen.
3. Fill out the form:
  - API root: The URL of the Open Zaak Autorisaties API endpoint
  - Label: For example: Open Zaak Autorisaties
  - Client ID: For example: open-notificaties
  - Secret: Some random string
  - User ID: Same as the Client ID
  - User representation: For example: Open Notificaties
  Make sure Open Notificaties is authorized in Open Zaak to access the Autorisaties API by using the same Client ID and Secret as given here.
4. Click Opslaan.
Finally, we need to allow Open Zaak to access Open Notificaties (for authentication purposes, so we can then check its authorizations):
1. Navigate to API Autorisaties > Client credentials
2. Click Client credential toevoegen.
3. Fill out the form:
  - Client ID: The same Client ID as given in Open Zaak step 2c
  - Secret: The same Secret as given in Open Zaak step 2c
  Make sure Open Zaak is configured to use this Client ID and secret to access Open Notificaties.
4. Click Opslaan.

All done!

Post-install checklist¶

After Open Notificaties has been installed successfully, go through the following checklist to see if the software works as expected:

Check configuration

Check the configuration page for Open Notificaties, accessible at the url https://open-notificaties.gemeente.nl/view-config/. This page will indicate whether certain settings are properly configured.

Manual¶

Note

The manual is currently only available in the Dutch language!

Once Open Notificaties is installed, it can be further configured via the management interface.

ADFS (On premise) Configureren¶

Open Notificaties ondersteunt Single Sign On (SSO) via ADFS voor de beheerinterface.

Gebruikers kunnen op die manier inloggen op Open Notificaties met hun ADFS account. In deze flow:

Klikt een gebruiker op het inlogscherm op Inloggen met ADFS
De gebruiker wordt naar de ADFS omgeving geleid waar ze inloggen met gebruikersnaam en wachtwoord (en eventuele Multi Factor Authorization)
De ADFS omgeving stuurt de gebruiker terug naar Open Notificaties (waar de account aangemaakt wordt indien die nog niet bestaat)
Een beheerder in Open Notificaties kent de juiste groepen toe aan deze gebruiker als deze voor het eerst inlogt.

Note

Standaard krijgen deze gebruikers geen toegang tot de beheerinterface. Deze rechten moeten door een (andere) beheerder ingesteld worden. De account is wel aangemaakt indien deze nog niet bestond.

Configureren van ADFS zelf¶

Contacteer de IAM/ADFS-beheerders in je organisatie om een Application Group aan te maken in de ADFS omgeving. Er zijn configuratiehandleidingen (Engels) beschikbaar.

Voor de Redirect URI vul je https://open-notificaties.gemeente.nl/adfs/callback in, waarbij je open-notificaties.gemeente.nl vervangt door het relevante domein.

Aan het eind van dit proces moet je de volgende gegevens hebben (on premise):

Server adres, bijvoorbeeld login.gemeente.nl
Client ID, bijvoorbeeld 3ae1852d-bf76-4731-9c41-1a31928cf6a6

Configureren van ADFS in Open Notificaties¶

Zorg dat je de volgende gegevens hebt:

Server adres
Client ID

Navigeer vervolgens in de admin naar Configuratie > ADFS Confiugration.

Vink Enable aan om ADFS in te schakelen.
Vul bij Server (on premise) het server adres in, bijvoorbeeld login.gemeente.nl.
Vul bij Client ID het Client ID in, bijvoorbeeld 3ae1852d-bf76-4731-9c41-1a31928cf6a6.
Vul bij Relying Party ID opnieuw het Client ID in, bijvoorbeeld 3ae1852d-bf76-4731-9c41-1a31928cf6a6.
Laat bij Claim mapping de standaardwaarden staan.
Vul bij Username claim de waarde winaccountname in.

Klik tot slot rechtsonder op Opslaan.

Je kan vervolgens het makkelijkst testen of alles werkt door in een incognitoscherm naar https://open-notificaties.gemeente.nl/admin/ te navigeren en op Inloggen met ADFS te klikken.

Support¶

Development is done separately from any support or service that you might need to install, use, implement or integrate Open Notificaties.

What kind of support do you need?

Need help with the installation? Contact a service provider!
Found a bug? Please create an issue on Github!
Want to contribute? See Development (TODO)!

Service providers¶

Below is a list of service providers.

Maykin Media

Want to be mentioned here? Please create a Pull Request on Github to add your company.

Development (TODO)¶

Open Notificaties is open-source software. We’d love to have you contribute!

Contributing¶

Development environment¶

Performance¶

Performance is measured using different types of scenarios.

Measuring performance¶

Goal¶

The purpose of performance measurements is to gain insight into the relationships between system requirements, number of users and response times of the API’s. From these relationships we can draw up the minimum system requirements, depending on the number of users, on which the API’s still perform acceptable.

A standardized performance measurement also provides insight into which effect various optimizations have.

Technical test scenarios¶

There’s basically just one API-call that will be called often and that’s to send a notification. The other API-calls are neglible. The API-call (request) is measured to gain insights in the technical performance.

Notificaties API

Create NOTIFICATIE (POST /api/v1/notificaties)

Test specification¶

Using scenarios¶

A scenario in this test specification is equal to an API-call (request). Each API resource is continuously without any delay, or waiting time, between each request. This way, we can determine the maximum number of requests per second and average response times.

Virtual users¶

We test with an increasing number of virtual users, from 1 to 100, that concurrently execute the test scenarios. A virtual user is technically a script that executes the different scenarios one after another. This way, we see the number of virtual users that concurrently access the API’s and its impact on performance.

Testdata¶

There’s not test data involved.

Changes¶

1.1.3 (2021-03-17)¶

Bugfix release fixing some deployment issues

Fixed broken STATIC_URL and MEDIA_URL settings derived from SUBPATH. This should fix CSS/Javascript assets not loading in
Removed single-server documentation duplication (which was outdated too)
Removed raven test command from documentation, it was removed.
Made CORS set-up opt-in

1.1.2 (2020-12-17)¶

Quality of life release, no functional changes.

Updated deployment tooling to version 0.10.0. This adds support for CentOS/RHEL 7 and 8.
Migrated CI from Travis CI to Github Actions
Made PostgreSQL 10, 11 and 12 support explicit through build matrix

1.1.1 (2020-11-09)¶

Small quality of life release.

Updated documentation links in API Schema documentation
Added missing Redis service to docker-compose.yml
Fixed docker-compose.yml (Postgres config, session cache…)
Fixed version var in deploy config
Fixed settings/config for hosting on a subpath
Added management command for initial Open Notificaties setup (setup_configuration)
Fixed broken links in docs
Bumped dev-tools isort, black and pip-tools to latest versions
Fixed tests by mocking HTTP calls that weren’t mocked yet
Fixed handling HTTP 401 responses on callback auth validation. Now both 403 and 401 are valid responses.

1.1.0 (2020-03-16)¶

Feature and small improvements release.

Note

The API remains unchanged.

Removed unnecessary sections in documentation
Updated deployment examples
Tweak deployment to not conflict (or at least less likely :-) ) with Open Zaak install Open Zaak and Open Notificaties on the same machine are definitely supported
Added support for ADFS Single Sign On (disabled by default)
Added documentation build to CI

1.0.0 final (2020-02-07)¶

🎉 First stable release of Open Notificaties.

Features:

Notificaties API implementation
Tested with Open Zaak integration
Admin interface to view data created via the APIs
Scalable notification delivery workers
NLX ready (can be used with NLX)
Documentation on https://open-notificaties.readthedocs.io/
Deployable on Kubernetes, single server and as VMware appliance
Automated test suite
Automated deployment