Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Updating overview and HTTP status codes.

Purpose of This Document

Status
colourGreen
titleBusiness
 
Status
subtletrue
colourYellow
titleTechnical

A technical design document describes a part of a product (component/module/service/library) with a technical focus (architecture, design, domain etc.).

Roles and Responsibilities

Status
colourGreen
titleBusiness

Role

Responsibilities

Contact

Team name

Product Owner

Prioritisation of features

Alexander Kirkaune - alkir@eg.dk 

Cross Team

Overview

Status
colourGreen
titleBusiness
 
Status
subtletrue
colourYellow
titleTECHNICAL

Data Integration Service (called D-I-S on a daily basis) is a case flow coordination web service. It handles information about the flow of cases between Origin Systems and Consumer Systems. DIS handles the communication of case statuses between the sending system and the receiving systems(s). DIS facilitates the request when a receiving system requests data on a case from a sending system. As DIS only stores metadata of a case it will notify receiving systems when a sending system creates, updates, closes, or deletes a case. It's up to the receiving system to request the case through DIS.

Acronyms and Terminology

Status
colourGreen
titleBusiness
 
Status
subtletrue
colourYellow
titleTECHNICAL
 

Term

Description

CaseThe object being sent from an Origin System to a Consumer System.
Consumer System

Systems subscribing to an Origin System.

DIS

Data Integration Service.

NJ

NemJournalisering.

Origin System

The owner of a case.


Table of Contents:

Table of Contents
styledisc

Related pages:

Children Display

External Environments for Shared Tech Products

Main Features and Functionalities

Status
colourGreen
titleBusiness
 
Status
subtletrue
colourYellow
titleTECHNICAL

Origin Systems

An Origin System is the "owner" of a case. It is where the case is created, but it also holds the responsibility to update, close, or delete (if required). Every change made to the case after creation besides closing and deletion is an update.

Each Origin System has the responsibility to create a data model for the CaseDetails and therefore different Origin Systems can have different data models for CaseDetails. It is the responsibility of the Origin System to expose the endpoint for fetching CaseDetails by Consumer Systems. The endpoint should be secured by an access token from KeyCloak. The URL to the endpoint is set in the caseDetailsUri attribute on the Case object.

When publishing the four commands (CreateCaseCommand, DeleteCaseCommand, UpdateCaseCommand, CloseCaseCommand) these commands are pushed into four different queues. All commands in each queue will be processed sequentially. But the four queues will be processed in parallel!

Consumer Systems

Consumer Systems are the subscribers of cases and corresponding data delivered through DIS by the Origin Systems

. To authorize a consuming system and make calls to DIS an access bearer token is needed. This token is generated using KeyCloak, with the use of clientId and clientSecret

.

To add a system to KeyCloak a realm manager (the Cross team) has to set up the client. Once the client has been created the clientId and clientSecret should be provided to the external integrating system. The integrating system should implement the call to KeyCloak to obtain the bearer token and place this in the header when sending requests to DIS and the Origin System (for CaseDetails).

The structure of data delivered is dictated by each Origin System depending on the need for available data. Only EGDW.CaseMetadata is static and set by DIS. The content of CaseDetails and how it is mapped is a contract between the Origin System and the Consumer System. This means the Id, Created, and Updated are set by the Origin System. When a case is created and sent to DIS, DIS will save the metadata in a database (DB) and set the Updated attribute to the same value as Created. When the case, later on, is updated by the Origin System, the value in the DIS DB for metadata will be overwritten with the Updated value.

NemJournalisering (NJ)

When fully implemented, DIS will replace NemJournalisering entirely. They differ in functionality as NJ is considered a Data Warehouse System for cases, where DIS handles information and status about cases.

Expand
titleDIS and NJ syncronization model

draw.io Diagram
bordertrue
diagramNameDIS and NJ sync
simpleViewerfalse
width
linksauto
tbstyletop
diagramDisplayNameDIS and NJ synchronization
lboxtrue
diagramWidth761
height636
revision3

Notifications

When a case is created, updated, closed, or deleted by the Origin System each Consumer System is notified with a push notification, by utilizing the RabbitMQ technology. External consumers are not connecting to the same queues as the internal consumers.

Every client will have a separate queue with template queue name: dis_notifications_{CLIENT_NAME}.

Every time a case is created / updated / close / deleted a message is sent to each client's queue who have the permission to see that specific case.

To get notifications about cases you need to connect to the Rabbit instance (see the Environments, Endpoints, and Resources section).

To do that you'll need to do proper auth (see more in the Authentication

part

and Authorization section). After

connection

connecting you can subscribe to your queue.
You will only get notifications

only about

for cases that your client

have

has access to.

Persistence

To make sure that notifications will go out before sending the notifications DIS is checking on the health of the external

rabbit

RabbitMQ connection. To achieve this an additional consumer is introduced to external rabbit (dis-healthcheck-stub) only

for

with the purpose of reporting connection health. If

rabbit

the RabbitMQ instance is down a message is redirected to the internal queue (dis-DelayedExternalNotifications) along with the list of clients that we should notify. Then

consumer

consumers of this queue will try to send messages to the external

rabbit

RabbitMQ instanse.

Notifications model

The messages that will go into the notifications queues are:

DIS.Queue.Models.IExternalSystemCaseNotification

and the data inside:

    public interface IExternalSystemCaseNotification
    {
        Guid Id { get; }
        ExternalNotificationType NotificationType { get; }
        string? CaseDetailsUri { get; }
        DateTime? EventTime { get; }
    }

    public enum ExternalNotificationType
    {
        Created = 0,
        Updated = 1,
        Closed = 2,
        Deleted = 3
  }

How to Integrate

Status
subtletrue
colourYellow
titleTECHNICAL
Status
colourGreen
titleBusiness

As an Origin System expose endpoint for CaseDetails.

  • Secure the endpoint using KeyClaok.
    • Obtain KeyCloak credential from the Cross team.
    • Connect to RabbitMQ:
      • Obtain credentials for RabbitMQ from EG DW.
    As an external consumer connect
      • Connect to RabbitMQ for push notifications
    .As an external consumer obtain credentials for RabbitMQ from EG DW
      • (see the Authentication and Authorization section).

    Environments, Endpoints, and Resources

    Status
    subtletrue
    colourYellow
    titleTECHNICAL

    Read about the usage for different environments: External Environments for Shared Tech Products.

    Environment

    URL for Rabbit

    Swagger

    ExtTestNANA
    PreProdNAhttps://preprod-dis.egki.dk/swagger/index.html
    PRODamqps://rabbit.egdw.dk:5671/disNA

    The following is an overview of what endpoints will return which data.

    Can also

    The same can be seen at the Swagger-endpoint.

    Expand
    titleEndpoints and return values


    Request typeEndpointResponseMedia typeRequires authorizationNote
    GET/canarystringtext/plainNo

    Used to validate that the application is running.

    GET/api/Cases/{id}{
      "id": "Guid",
      "name": "string",
      "kle": "string",
      "entityCode": int,
      "organisationId": "string",
      "originSystemCode": "string",
      "caseDetailsUri": "string",
      "created": "DateTime",
      "updated": "DateTime",
      "caseDetails": "string"
    }
    text/plainYes

    The response is a specific case and as an attribute on the object is caseDetailsUri which is a URI to the Origin System for CaseDetails. The same CaseDetails are provided in the caseDetails attribute.

    The id in the parameter is the id of the case and is required in the request. The id is set by the Origin System.

    An example of the Guid could look like this: 3fa85f64-5717-4562-b3fc-2c963f66afa6

    An example of the DateTime would look like this: 2023-05-15T11:40:47.067Z

    GET/api/Cases[
      {
        "id": "Guid",
        "name": "string",
        "kle": "string",
        "entityCode": int,
        "organisationId": "string",
        "originSystemCode": "string",
        "caseDetailsUri": "string",
        "created": "DateTime",
        "updated": "DateTime"
      }
    ]
    text/plainYes

    The response is a list of cases that the client has access to. A URI to the CaseDetails is provided in the object of each case in the list.

    As a query parameter the following parameters are optional:

    • createdStartDate: string($date-time)
    • createdEndDate: string($date-time)
    • updatedStartDate: string($date-time)
    • updatedEndDate: string($date-time)
    • kle: string

    The parameters are optional and can be used as a filter when fetching cases.

    An example of the Guid could look like this: 3fa85f64-5717-4562-b3fc-2c963f66afa6

    An example of the DateTime would look like this: 2023-05-15T11:40:47.067Z

    Expand
    titleSchemas

    Image Modified

    HTTP Status Codes

    Status
    colourGreen
    titleBusiness
     
    Status
    subtletrue
    colourYellow
    titleTECHNICAL

    When requesting a specific case the responses could be:

    • 200 - OK: The case is fetched.
    If one
    • 403 - Forbidden: The requesting system does not have permission to get the case.
    • 404 - Not Found: The requested case could not be found. If a system asks for case details and the response
    will be
    • is HTTP 404, then
    one
    • system should assume that the case has already been deleted and skip processing of that message.

    When requesting a list of cases the responses could be:

    • 200 - OK: The list of cases.
    • 204 - No Content: The request was successful but the list is empty.

    Error Handling

    Status
    colourGreen
    titleBusiness
     
    Status
    subtletrue
    colourYellow
    titleTECHNICAL

    Since this is a distributed system it is expected that in some cases

    one

    a system could have some de-synchronized events / data. For example

    one

    a system could be

    processing 

    processing CaseCreatedEvent

     on

    on a case that has already been deleted. If

    one

    a system asks for case details and the response

    will be

    is HTTP 404, then

    one

    system should assume that the case has already been deleted and skip processing of that message - that way we'll get eventual consistency.

    Testing

    Status
    subtletrue
    colourYellow
    titleTECHNICAL

    For a Consumer System to test

    one

    notifications from RabbitMQ the system need to subscribe to an Origin System and the Origin System would have to process a case.

    To test if the DIS application is reachable use the Canary endpoint.

    Privacy and Security

    Status
    colourGreen
    titleBusiness
     
    Status
    subtletrue
    colourYellow
    titleTECHNICAL

    Authentication and Authorization

    To

    be able to connect to Rabbit

    enable connection to the RabbitMQ instance you're going to need a user and password with access to proper queue.

    Your

    Reach out to your contact at EG DW who will provide them to you.

    For API endpoint security KeyCloak is used for auth.

    To authorize a consuming system and make calls to DIS an access bearer token is needed. This token is generated using KeyCloak, with the use of clientId and clientSecret. To add a system to KeyCloak a realm manager (the Cross team) has to set up the client. Once the client has been created the clientId and clientSecret should be provided to the external integrating system. The integrating system should implement the call to KeyCloak to obtain the bearer token and place this in the header when sending requests to DIS and the Origin System (for CaseDetails).

    FAQ

    • If /api/Cases is requested with the filter set to get cases which have been updated from xx to yy will you then receive cases created in the same set period?
    Expand
    titleShow answer

    Yes, when a case is created the Updated attribute is set to the same value as Created and therefore cases which have been created in the same period will be returned as a part of the response


    • Is there any pagination on /api/Cases?
    Expand
    titleShow answer

    No. If the response dies due to too big an amount of data, the response will be 404 - Not Found.


    • If a case is deleted, will it then still show up in the list of cases (/api/Cases)?
    Expand
    titleShow answer

    No. When the case is CaseDeleted command is sent to DIS, metadata for this case is deleted in DIS and therefore do not exists. If requested, the response will be 404 - Not Found

    Change History

    Page properties
    idTDD

    Title

    Version

    Applicability

    Responsible

    Revise date

    Revise comment

    DIS - TDD

    1.0

    DIS