Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Purpose of This Document

Status
colourGreen
titleBusiness
 
Status
subtletrue
colourYellow
titleTechnical

A technical design document describes a part of a product (component/module/service/library) with a technical focus (architecture, design, domain etc.).

Roles and Responsibilities

Status
colourGreen
titleBusiness

Role

Responsibilities

Contact

Team name

Product Owner

Prioritisation of features

Alexander Kirkaune - alkir@eg.dk 

Cross Team

Overview

Status
colourGreen
titleBusiness
 
Status
subtletrue
colourYellow
titleTECHNICAL

Data Integration Service (called D-I-S on a daily basis) is a case flow coordination web service. It handles information about the flow of cases between Origin Systems and Consumer Systems.

Acronyms and Terminology

Status
colourGreen
titleBusiness
 
Status
subtletrue
colourYellow
titleTECHNICAL
 

Term

Description

CaseThe object being sent from an Origin System to a Consumer System.
Consumer System

Systems subscribing to an Origin System.

DIS

Data Integration Service.

NJ

NemJournalisering.

Origin System

The owner of a case.


Table of Contents:

Table of Contents
styledisc

Related pages:None

Children Display

Main Features and Functionalities

Status
colourGreen
titleBusiness
 
Status
subtletrue
colourYellow
titleTECHNICAL

Origin Systems

An Origin System is the "owner" of a case. It is where the case is created, but it also holds the responsibility to update, close, or delete (if required). Every change made to the case after creation besides closing and deletion is an update.

Each Origin System has the responsibility to create a data model for the CaseDetails and therefore different Origin Systems can have different data models for CaseDetails. It is the responsibility of the Origin System to expose the endpoint for fetching CaseDetails by Consumer Systems. The endpoint should be secured by an access token from KeyCloak. The URL to the endpoint is set in the caseDetailsUri attribute on the Case object.

When publishing the four commands (CreateCaseCommand, DeleteCaseCommand, UpdateCaseCommand, CloseCaseCommand) these commands are pushed into four different queues. All commands in each queue will be processed sequentially. But the four queues will be processed in parallel!

Consumer Systems

Consumer Systems are the subscribers of cases and corresponding data delivered through DIS by the Origin Systems.

The structure of data delivered is dictated by each Origin System depending on the need for available data. Only EGDW.CaseMetadata is static and set by DIS. The content of CaseDetails and how it is mapped is a contract between the Origin System and the Consumer System. This means the Id, Created, and Updated are set by the Origin System. When a case is created and sent to DIS, DIS will save the metadata in a database (DB) and set the Updated attribute to the same value as Created. When the case, later on, is updated by the Origin System, the value in the DIS DB for metadata will be overwritten with the Updated value.

NemJournalisering (NJ)

When fully implemented, DIS will replace NemJournalisering entirely. They differ in functionality as NJ is considered a Data Warehouse System for cases, where DIS handles information and status about cases.

Expand
titleDIS and NJ syncronization model

draw.io Diagram
bordertrue
diagramNameDIS and NJ sync
simpleViewerfalse
width
linksauto
tbstyletop
diagramDisplayNameDIS and NJ synchronization
lboxtrue
diagramWidth761
height636
revision3

Notifications

When a case is created, updated, closed, or deleted by the Origin System each Consumer System is notified with a push notification, by utilizing the RabbitMQ technology. External consumers are not connecting to the same queues as the internal consumers.

Every client will have separate queue with template queue name: dis_notifications_{CLIENT_NAME}.

Every time case is created / updated / close / deleted message is sent to each client's queue who have permission to see that specific case.

To get notifications about cases you need to connect to Rabbit instance (see the Environments, Endpoints, and Resources section).

To do that you'll need to do proper auth (see more in Authentication part). After connection you can subscribe to your queue.
You will get notifications only about cases that your client have access to.

Persistence

To make sure that notifications will go out before sending notifications DIS is checking on health of external rabbit connection. To achieve this additional consumer is introduced to external rabbit (dis-healthcheck-stub) only for purpose of reporting connection health. If rabbit is down message is redirected to internal queue (dis-DelayedExternalNotifications) along with the list of clients that we should notify. Then consumer of this queue will try to send messages to external rabbit.

Notifications model

The messages that will go into the notifications queues are:

DIS.Queue.Models.IExternalSystemCaseNotification

and data inside:

    public interface IExternalSystemCaseNotification
    {
        Guid Id { get; }
        ExternalNotificationType NotificationType { get; }
        string? CaseDetailsUri { get; }
        DateTime? EventTime { get; }
    }

    public enum ExternalNotificationType
    {
        Created = 0,
        Updated = 1,
        Closed = 2,
        Deleted = 3
  }

How to Integrate

Status
subtletrue
colourYellow
titleTECHNICAL
Status
colourGreen
titleBusiness

  • Obtain KeyCloak credential from the Cross team.
  • Connect to RabbitMQ:
    • Obtain credentials for RabbitMQ from EG DW.
    • Connect to RabbitMQ for push notifications (see the Authentication section).

Environments, Endpoints, and Resources

Status
subtletrue
colourYellow
titleTECHNICAL

Read about usage for different environments: External Environments for Shared Tech Products.

Environment

URL for Rabbit

Swagger

ExtTestNANA
PreProdNAhttps://preprod-dis.egki.dk/swagger/index.html
PRODamqps://rabbit.egdw.dk:5671/disNA

The following is an overview of what endpoints will return which data. Can also be seen at the Swagger-endpoint.

Expand
titleEndpoints and return values


Request typeEndpointResponseMedia typeRequires authorizationNote
GET/canarystringtext/plainNo

Used to validate that the application is running.

GET/api/Cases/{id}{
  "id": "Guid",
  "name": "string",
  "kle": "string",
  "entityCode": int,
  "organisationId": "string",
  "originSystemCode": "string",
  "caseDetailsUri": "string",
  "created": "DateTime",
  "updated": "DateTime",
  "caseDetails": "string"
}
text/plainYes

The response is a specific case and as an attribute on the object is caseDetailsUri which is a URI to the Origin System for CaseDetails. The same CaseDetails are provided in the caseDetails attribute.

The id in the parameter is the id of the case and is required in the request. The id is set by the Origin System.

An example of the Guid could look like this: 3fa85f64-5717-4562-b3fc-2c963f66afa6

An example of the DateTime would look like this: 2023-05-15T11:40:47.067Z

GET/api/Cases[
  {
    "id": "Guid",
    "name": "string",
    "kle": "string",
    "entityCode": int,
    "organisationId": "string",
    "originSystemCode": "string",
    "caseDetailsUri": "string",
    "created": "DateTime",
    "updated": "DateTime"
  }
]
text/plainYes

The response is a list of cases that the client has access to. A URI to the CaseDetails is provided in the object of each case in the list.

As a query parameter the following parameters are optional:

  • createdStartDate: string($date-time)
  • createdEndDate: string($date-time)
  • updatedStartDate: string($date-time)
  • updatedEndDate: string($date-time)
  • kle: string

The parameters are optional and can be used as a filter when fetching cases.

An example of the Guid could look like this: 3fa85f64-5717-4562-b3fc-2c963f66afa6

An example of the DateTime would look like this: 2023-05-15T11:40:47.067Z

Expand
titleSchemas

HTTP Status Codes

Status
colourGreen
titleBusiness
 
Status
subtletrue
colourYellow
titleTECHNICAL

If one asks for case details and the response will be HTTP 404 then one should assume that the case has already been deleted and skip processing of that message

Error Handling

Status
colourGreen
titleBusiness
 
Status
subtletrue
colourYellow
titleTECHNICAL

Since this is a distributed system it is expected that in some cases one could have some de-synchronized events / data. For example one could be processing CaseCreatedEvent on a case that has already been deleted. If one asks for case details and the response will be HTTP 404 then one should assume that the case has already been deleted and skip processing of that message - that way we'll get eventual consistency.

Testing

Status
subtletrue
colourYellow
titleTECHNICAL

For a Consumer System to test notifications from RabbitMQ the system need to subscribe to an Origin System and the Origin System would have to process a case.

To test if the DIS application is reachable use the Canary endpoint.

Privacy and Security

Status
colourGreen
titleBusiness
 
Status
subtletrue
colourYellow
titleTECHNICAL

Authentication

To be able to connect to Rabbit instance you're going to need user and password with access to proper queue. Your contact at EG DW will provide them to you.

For API endpoint security KeyCloak is used for auth.

To authorize a consuming system and make calls to DIS an access bearer token is needed. This token is generated using KeyCloak, with the use of clientId and clientSecret. To add a system to KeyCloak a realm manager (the Cross team) has to set up the client. Once the client has been created the clientId and clientSecret should be provided to the external integrating system. The integrating system should implement the call to KeyCloak to obtain the bearer token and place this in the header when sending requests to DIS and the Origin System (for CaseDetails).

FAQ

  • If /api/Cases is requested with the filter set to get cases which have been updated from xx to yy will you then receive cases created in the same set period?
Expand
titleShow answer

Yes, when a case is created the Updated attribute is set to the same value as Created and therefore cases which have been created in the same period will be returned as a part of the response


  • Is there any pagination on /api/Cases?
Expand
titleShow answer

No. If the response dies due to too big an amount of data, the response will be 404 - Not Found.


  • If a case is deleted, will it then still show up in the list of cases (/api/Cases)?
Expand
titleShow answer

No. When the case is CaseDeleted command is sent to DIS, metadata for this case is deleted in DIS and therefore do not exists. If requested, the response will be 404 - Not Found

Change History

Page properties
idTDD

Title

Version

Applicability

Responsible

Revise date

Revise comment

DIS - TDD

1.0

DIS