Purpose of This Document

BUSINESS TECHNICAL

A technical design document describes a part of a product (component/module/service/library) with a technical focus (architecture, design, domain etc.).

Roles and Responsibilities

BUSINESS

Role

Responsibilities

Contact

Team name

Product Owner

Prioritisation of features

Alexander Kirkaune - alkir@eg.dk 

Cross Team

Overview

BUSINESS TECHNICAL

Data Integration Service (called D-I-S on a daily basis) is a case flow coordination web service. It handles information about the flow of cases between Origin Systems and Consumer Systems. DIS handles the communication of case statuses between the sending system and the receiving systems(s). DIS facilitates the request when a receiving system requests data on a case from a sending system. As DIS only stores metadata of a case it will notify receiving systems when a sending system creates, updates, closes, or deletes a case. It's up to the receiving system to request the case through DIS.

Acronyms and Terminology

BUSINESS TECHNICAL 

Term

Description

CaseThe object being sent from an Origin System to a Consumer System.
Consumer System

Systems subscribing to an Origin System.

DIS

Data Integration Service.

NJ

NemJournalisering.

Origin System

The owner of a case.


Table of Contents:

Related pages:

External Environments for Shared Tech Products

Main Features and Functionalities

BUSINESS TECHNICAL

Origin Systems

An Origin System is the "owner" of a case. It is where the case is created, but it also holds the responsibility to update, close, or delete (if required). Every change made to the case after creation besides closing and deletion is an update.

Each Origin System has the responsibility to create a data model for the CaseDetails and therefore different Origin Systems can have different data models for CaseDetails. It is the responsibility of the Origin System to expose the endpoint for fetching CaseDetails by Consumer Systems. The endpoint should be secured by an access token from KeyCloak. The URL to the endpoint is set in the caseDetailsUri attribute on the Case object.

When publishing the four commands (CreateCaseCommand, DeleteCaseCommand, UpdateCaseCommand, CloseCaseCommand) these commands are pushed into four different queues. All commands in each queue will be processed sequentially. But the four queues will be processed in parallel!

Consumer Systems

Consumer Systems are the subscribers of cases and corresponding data delivered through DIS by the Origin Systems.

The structure of data delivered is dictated by each Origin System depending on the need for available data. Only EGDW.CaseMetadata is static and set by DIS. The content of CaseDetails and how it is mapped is a contract between the Origin System and the Consumer System. This means the Id, Created, and Updated are set by the Origin System. When a case is created and sent to DIS, DIS will save the metadata in a database (DB) and set the Updated attribute to the same value as Created. When the case, later on, is updated by the Origin System, the value in the DIS DB for metadata will be overwritten with the Updated value.

NemJournalisering (NJ)

When fully implemented, DIS will replace NemJournalisering entirely. They differ in functionality as NJ is considered a Data Warehouse System for cases, where DIS handles information and status about cases.

Notifications

When a case is created, updated, closed, or deleted by the Origin System each Consumer System is notified with a push notification, by utilizing the RabbitMQ technology. External consumers are not connecting to the same queues as the internal consumers.

Every client will have a separate queue with template queue name: dis_notifications_{CLIENT_NAME}.

Every time a case is created / updated / close / deleted a message is sent to each client's queue who have the permission to see that specific case.

To get notifications about cases you need to connect to the Rabbit instance (see the Environments, Endpoints, and Resources section).

To do that you'll need to do proper auth (see more in the Authentication and Authorization section). After connecting you can subscribe to your queue.
You will only get notifications for cases that your client has access to.

Persistence

To make sure that notifications will go out before sending the notifications DIS is checking on the health of the external RabbitMQ connection. To achieve this an additional consumer is introduced to external rabbit (dis-healthcheck-stub) only with the purpose of reporting connection health. If the RabbitMQ instance is down a message is redirected to the internal queue (dis-DelayedExternalNotifications) along with the list of clients that we should notify. Then consumers of this queue will try to send messages to the external RabbitMQ instanse.

Notifications model

The messages that will go into the notifications queues are:

DIS.Queue.Models.IExternalSystemCaseNotification

and the data inside:

    public interface IExternalSystemCaseNotification
    {
        Guid Id { get; }
        ExternalNotificationType NotificationType { get; }
        string? CaseDetailsUri { get; }
        DateTime? EventTime { get; }
    }

    public enum ExternalNotificationType
    {
        Created = 0,
        Updated = 1,
        Closed = 2,
        Deleted = 3
  }

How to Integrate

TECHNICAL BUSINESS

  • Obtain KeyCloak credential from the Cross team.
  • Connect to RabbitMQ:
    • Obtain credentials for RabbitMQ from EG DW.
    • Connect to RabbitMQ for push notifications (see the Authentication and Authorization section).

Environments, Endpoints, and Resources

TECHNICAL

Read about the usage for different environments: External Environments for Shared Tech Products.

Environment

URL for Rabbit

Swagger

ExtTestNANA
PreProdNAhttps://preprod-dis.egki.dk/swagger/index.html
PRODamqps://rabbit.egdw.dk:5671/disNA

The following is an overview of what endpoints will return which data. The same can be seen at the Swagger-endpoint.


Request typeEndpointResponseMedia typeRequires authorizationNote
GET/canarystringtext/plainNo

Used to validate that the application is running.

GET/api/Cases/{id}{
  "id": "Guid",
  "name": "string",
  "kle": "string",
  "entityCode": int,
  "organisationId": "string",
  "originSystemCode": "string",
  "caseDetailsUri": "string",
  "created": "DateTime",
  "updated": "DateTime",
  "caseDetails": "string"
}
text/plainYes

The response is a specific case and as an attribute on the object is caseDetailsUri which is a URI to the Origin System for CaseDetails. The same CaseDetails are provided in the caseDetails attribute.

The id in the parameter is the id of the case and is required in the request. The id is set by the Origin System.

An example of the Guid could look like this: 3fa85f64-5717-4562-b3fc-2c963f66afa6

An example of the DateTime would look like this: 2023-05-15T11:40:47.067Z

GET/api/Cases[
  {
    "id": "Guid",
    "name": "string",
    "kle": "string",
    "entityCode": int,
    "organisationId": "string",
    "originSystemCode": "string",
    "caseDetailsUri": "string",
    "created": "DateTime",
    "updated": "DateTime"
  }
]
text/plainYes

The response is a list of cases that the client has access to. A URI to the CaseDetails is provided in the object of each case in the list.

As a query parameter the following parameters are optional:

  • createdStartDate: string($date-time)
  • createdEndDate: string($date-time)
  • updatedStartDate: string($date-time)
  • updatedEndDate: string($date-time)
  • kle: string

The parameters are optional and can be used as a filter when fetching cases.

An example of the Guid could look like this: 3fa85f64-5717-4562-b3fc-2c963f66afa6

An example of the DateTime would look like this: 2023-05-15T11:40:47.067Z

HTTP Status Codes

BUSINESS TECHNICAL

When requesting a specific case the responses could be:

  • 200 - OK: The case is fetched.
  • 403 - Forbidden: The requesting system does not have permission to get the case.
  • 404 - Not Found: The requested case could not be found. If a system asks for case details and the response is HTTP 404, then system should assume that the case has already been deleted and skip processing of that message.

When requesting a list of cases the responses could be:

  • 200 - OK: The list of cases.
  • 204 - No Content: The request was successful but the list is empty.

Error Handling

BUSINESS TECHNICAL

Since this is a distributed system it is expected that in some cases a system could have some de-synchronized events / data. For example a system could be processing CaseCreatedEvent on a case that has already been deleted. If a system asks for case details and the response is HTTP 404, then system should assume that the case has already been deleted and skip processing of that message - that way we'll get eventual consistency.

Testing

TECHNICAL

For a Consumer System to test notifications from RabbitMQ the system need to subscribe to an Origin System and the Origin System would have to process a case.

To test if the DIS application is reachable use the Canary endpoint.

Privacy and Security

BUSINESS TECHNICAL

Authentication and Authorization

To enable connection to the RabbitMQ instance you're going to need a user and password with access to proper queue. Reach out to your contact at EG DW who will provide them to you.

For API endpoint security KeyCloak is used for auth.

To authorize a consuming system and make calls to DIS an access bearer token is needed. This token is generated using KeyCloak, with the use of clientId and clientSecret. To add a system to KeyCloak a realm manager (the Cross team) has to set up the client. Once the client has been created the clientId and clientSecret should be provided to the external integrating system. The integrating system should implement the call to KeyCloak to obtain the bearer token and place this in the header when sending requests to DIS and the Origin System (for CaseDetails).

FAQ

  • If /api/Cases is requested with the filter set to get cases which have been updated from xx to yy will you then receive cases created in the same set period?

Yes, when a case is created the Updated attribute is set to the same value as Created and therefore cases which have been created in the same period will be returned as a part of the response


  • Is there any pagination on /api/Cases?

No. If the response dies due to too big an amount of data, the response will be 404 - Not Found.


  • If a case is deleted, will it then still show up in the list of cases (/api/Cases)?

No. When the case is CaseDeleted command is sent to DIS, metadata for this case is deleted in DIS and therefore do not exists. If requested, the response will be 404 - Not Found

Change History

Title

Version

Applicability

Responsible

Revise date

Revise comment

DIS - TDD

1.0

DIS

 


  • No labels