Functional Requirements
...
FR-001: Batch File Upload
...
Priority: HIGH
Related BR: BR-003
Description
Users shall upload batch invoice files through
...
RESTful API with automatic vendor format detection, file validation, and storage in organization-specific blob containers
...
.
Priority: HIGH
API Specification
...
Endpoint: POST /v1/organizations/{organizationId}/batches
Request
...
:
POST /v1/organizations/123e4567-e89b-12d3-a456-426614174000/batches...
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbG...
Content-Type: multipart/form-data...
file: [XML binary]
metadata: {
"batchName": "Invoice_November_2025",
"priority": "normal"
}
Response (201 Created):
{
"success": true,
"data": {
"...
Response Format (201 Created):
...
batchId": "550e8400-e29b-41d4-a716-446655440000",
...
"...
status": "uploaded",
"uploadedAt": "2025-11-21T10:30:00Z",
"fileInfo": {
"fileName": "invoices_nov.xml",
"fileSize": 15728640,
"...
detectedFormat": ...
"GASEL"
...
},
"blobPath": "acme-batches-2025/11/21/550e8400.../source.xml"
}...
...
}
Validation Rules
| Field | Rule | Error Code | Message |
|---|---|---|---|
| file | Required | VALIDATION_ERROR | File is required |
| file.size | 1KB ≤ size ≤ 100MB | FILE_TOO_LARGE | File must be 1KB-100MB |
| file.contentType | application/xml or text/xml | INVALID_CONTENT_TYPE | File must be XML |
| file.content | Well-formed XML | INVALID_XML | XML not well-formed. Line {line}, Column {column} |
| metadata.batchName | 1-255 chars, no / \ | VALIDATION_ERROR | Batch name: 1-255 chars, no path separators |
| metadata.priority | "normal" or "high" | VALIDATION_ERROR | Priority must be 'normal' or 'high' |
Processing Steps
- Validate authentication (OAuth 2.0 JWT token)
- Verify user has BatchOperator role
- Validate file size (1KB - 100MB)
- Validate content type (XML)
- Quick parse: Check XML well-formedness
- Detect vendor format (namespace analysis)
- Generate UUID batch ID
- Store XML in blob:
{org}-batches-{year}/{month}/{day}/{id}/source.xml - Calculate SHA-256 checksum
- Create batch metadata JSON
- Return 201 Created with batch details
Acceptance Criteria
| # | Criterion | Test | Expected Result |
|---|---|---|---|
| 1 | Accepts XML up to 100MB | Upload 100MB file | 201 Created |
| 2 | Validates well-formed XML | Upload malformed XML | 400 INVALID_XML |
| 3 | Stores in org-specific container | Verify blob path | {org}/2025/11/21/{id}/source.xml |
| 4 | Returns UUID batch ID | Check format | Valid UUID v4 |
| 5 | Detects GASEL format | Upload GASEL sample | detectedFormat: "GASEL" |
| 6 | Detects XELLENT format | Upload XELLENT sample | detectedFormat: "XELLENT" |
| 7 | Detects ZYNERGY format | Upload ZYNERGY sample | detectedFormat: "ZYNERGY" |
| 8 | Calculates SHA-256 checksum | Verify checksum | Matches file |
| 9 | Requires BatchOperator role | Upload without role | 403 ACCESS_DENIED |
| 10 | Rate limited (10/hour/org) | Upload 11 files | 11th returns 429 |
...
FR-002: Batch Processing Initiation
Priority: HIGH
Related BR: BR-003
Description
Users shall initiate batch processing through API, which enqueues the batch to batch-upload-queue for asynchronous processing by ParserService.
API Specification
Endpoint: POST /v1/organizations/{orgId}/batches/{batchId}/start
Request:
{
"validationMode": "strict"
}
Response (202 Accepted):
{
"success": true,
"data": {
"batchId": "550e8400-e29b-41d4-a716-446655440000",
"status": "queued",
"queuedAt": "2025-11-21T10:35:00Z",
"estimatedProcessingTime": "15-30 minutes",
"queuePosition": 2
}
}
Processing Flow
- Validate batch exists and status is "uploaded"
- Create message in
batch-upload-queue - Update batch status to "queued"
- Return 202 Accepted with queue position
- ParserService picks up asynchronously
Validation Rules
| Check | Rule | Error Code | HTTP | Action |
|---|---|---|---|---|
| Batch exists | Must exist in blob | RESOURCE_NOT_FOUND | 404 | Return error |
| Batch ownership | Must belong to org | ACCESS_DENIED | 403 | Return error |
| Batch status | Must be "uploaded" | PROCESSING_ERROR | 422 | Return current status |
| Organization active | isActive = true | ORGANIZATION_INACTIVE | 422 | Return error |
| Queue capacity | Depth < 10,000 | SERVICE_UNAVAILABLE | 503 | Return retry-after |
| User permission | BatchOperator+ role | ACCESS_DENIED | 403 | Return error |
Acceptance Criteria
| # | Criterion | Test | Expected Result |
|---|---|---|---|
| 1 | Only "uploaded" batches start | Start processing batch | 409 if already processing |
| 2 | Creates queue message | Verify message | Present in batch-upload-queue |
| 3 | Updates status to "queued" | Check metadata | status: "queued" |
| 4 | Returns estimated time | Empty vs full queue | Time varies |
| 5 | Idempotent (safe duplicates) | Call /start twice | Both return 202, process once |
| 6 | Returns queue position | Verify accuracy | Matches queue depth |
| 7 | Requires BatchOperator role | Call without role | 403 ACCESS_DENIED |
| 8 | Supports validation modes | Set lenient mode | Mode in queue message |
| 9 | Queue full returns 503 | Depth >10,000 | 503 with retry-after |
...
FR-003: Parser Service (XML → JSON)
Priority: CRITICAL
Related BR: BR-002, BR-003
Description
ParserService listens to batch-upload-queue, downloads XML, detects vendor format, validates against XSD, parses to canonical JSON, and enqueues 32-item batches to batch-items-queue.
Processing Steps
- Dequeue message from
batch-upload-queue - Download XML:
{org}-batches-{year}/{month}/{day}/{id}/source.xml - Detect vendor format (namespace + structure analysis)
- Load vendor-specific schema mapping
- Validate XML against vendor XSD schema
- Parse XML using XPath expressions
- Transform each invoice to canonical JSON
- Store JSON:
{org}-invoices-{year}/{month}/{day}/{invoice-id}.json - Group invoices into 32-item batches
- Enqueue to
batch-items-queue - Update batch metadata (totalItems, vendorCode, status="processing")
- Delete message from
batch-upload-queue
Canonical JSON Schema
{
"invoiceId": "uuid",
"invoiceNumber": "2025-11-001",
"invoiceDate": "2025-11-06",
"dueDate": "2025-11-20",
"currency": "SEK",
"periodStart": "2025-10-01",
"periodEnd": "2025-10-31",
"customer": {
"customerId": "020624-2380",
"fullName": "Medeni Schröder",
"email": "muntaser.af@zavann.net",
"phone": "09193538799",
"address": {
"street": "Strandbo 63B",
"city": "Växjö",
"postalCode": "352 58",
"country": "SE"
}
},
"invoiceDetails": {
"subTotal": 599.42,
"taxAmount": 149.86,
"totalAmount": 749.28,
"lineItems": [...]
},
"delivery": {
"meteringPointId": "735999756427205424",
"gridArea": "SE4",
"gridOwner": "Växjö Energi Elnät AB",
"consumption": 420
},
"sourceMetadata": {
"vendorCode": "GASEL",
"vendorVersion": "1.0",
"parsedAt": "2025-11-21T10:35:45Z"
}
}
Acceptance Criteria
| # | Criterion | Test | Expected Result |
|---|---|---|---|
| 1 | Listens to batch-upload-queue | Send message | Dequeued within 30s |
| 2 | Downloads batch XML | Verify download logged | File downloaded |
| 3 | Detects GASEL (100% accuracy) | 50 GASEL samples | All detected |
| 4 | Detects XELLENT (100% accuracy) | 50 XELLENT samples | All detected |
| 5 | Detects ZYNERGY (100% accuracy) | 50 ZYNERGY samples | All detected |
| 6 | Loads correct schema mapping | Verify mapping file | Correct vendor mapping |
| 7 | Validates XML against XSD | Invalid XML | Validation errors logged |
| 8 | Parses GASEL via XPath | Parse sample | All fields extracted |
| 9 | Transforms to canonical JSON | Verify schema | All fields present |
| 10 | Stores JSON in correct path | Check blob | {org}/invoices/2025/11/21/{id}.json |
| 11 | Groups into 32-item batches | 100 invoices | 4 messages (32+32+32+4) |
| 12 | Enqueues to batch-items-queue | Verify messages | Messages present |
| 13 | Updates batch metadata | Check metadata | totalItems, vendorCode set |
| 14 | Deletes from batch-upload-queue | Verify removal | Message gone |
| 15 | Retries 3× on errors | Force blob error | 3 retries logged |
| 16 | Moves to poison queue (3 fails) | Force permanent error | In poison queue |
| 17 | Performance: 10K in <2 min | Performance test | ≤ 120 seconds |
...
Blob Storage Path Pattern (Updated per Nov 21 decision):
{org-id}-batches-{year}/{month}/{day}/{batch-id}/source.xml
...
Acceptance Criteria:
| Criterion | Validation Method | Test Data | Expected Result |
|---|---|---|---|
| Accepts XML files up to 100MB | Upload 100MB XML file | gasel_100mb.xml | 201 Created |
| Validates file is well-formed XML | Upload malformed XML | invalid.xml | 400 INVALID_XML |
| Stores in org-specific container with month/day path | Upload, verify blob path | valid.xml | Path: {org}/2025/11/21/{id}/source.xml |
| Returns unique UUID batch ID | Upload, check batchId format | valid.xml | Valid UUID v4 |
| Detects GASEL format | Upload GASEL XML | gasel_sample.xml | detectedFormat: "GASEL" |
| Detects XELLENT format | Upload XELLENT XML | xellent_sample.xml | detectedFormat: "XELLENT" |
| Detects ZYNERGY format | Upload ZYNERGY XML | zynergy_sample.xml | detectedFormat: "ZYNERGY" |
| Calculates SHA-256 checksum | Upload, verify checksum | valid.xml | Checksum matches file |
| Requires Batch Operator role | Upload without role | valid.xml | 403 ACCESS_DENIED |
| Rate limited (10 uploads/hour/org) | Upload 11 files in 1 hour | valid.xml x11 | 11th returns 429 |
| File size > 100MB rejected | Upload 101MB file | large.xml | 413 FILE_TOO_LARGE |
| Non-XML files rejected | Upload PDF file | invoice.pdf | 415 UNSUPPORTED_FORMAT |
Validation Rules:
| Field | Rule | Error Code | Error Message |
|---|---|---|---|
| file | Required | VALIDATION_ERROR | File is required |
| file.size | 1KB ≤ size ≤ 100MB | FILE_TOO_LARGE | File must be between 1KB and 100MB |
| file.contentType | Must be application/xml or text/xml | INVALID_CONTENT_TYPE | File must be XML format |
| file.content | Well-formed XML (parseable) | INVALID_XML | XML file is not well-formed. Line {line}, Column {column}: {error} |
| metadata.batchName | 1-255 characters, no path separators | VALIDATION_ERROR | Batch name must be 1-255 characters without / or \ |
| metadata.priority | Must be "normal" or "high" | VALIDATION_ERROR | Priority must be 'normal' or 'high' |
Error Scenarios:
| Scenario | HTTP | Error Code | Message | Details |
|---|---|---|---|---|
| File too large | 413 | FILE_TOO_LARGE | File exceeds 100MB limit | { "fileSize": 105906176, "limit": 104857600 } |
| Invalid XML | 400 | INVALID_XML | XML file is not well-formed | { "line": 142, "column": 23, "error": "Unexpected end tag" } |
| Missing token | 401 | UNAUTHORIZED | Missing or invalid authentication token | { "suggestion": "Include Authorization: Bearer {token} header" } |
| Insufficient permissions | 403 | ACCESS_DENIED | User does not have Batch Operator role | { "requiredRole": "Batch Operator", "userRoles": ["Read-Only"] } |
| Rate limit exceeded | 429 | RATE_LIMIT_EXCEEDED | Too many batch uploads. Limit: 10 per hour | { "limit": 10, "window": "1 hour", "retryAfter": "2025-11-21T11:30:00Z" } |
3.2 FR-002: Batch Processing Initiation
Requirement: Users shall initiate batch processing through API, which enqueues the batch to batch-upload-queue for asynchronous processing by ParserService.
Priority: HIGH
API Endpoint: POST /organizations/{organizationId}/batches/{batchId}/start
Request Format:
POST /v1/organizations/{orgId}/batches/{batchId}/start HTTP/1.1 Authorization: Bearer {token} Content-Type: application/json { "validationMode": "strict" }
...
Response Format (202 Accepted):
{ "success": true, "data": { "batchId": "550e8400-e29b-41d4-a716-446655440000", "status": "queued", "queuedAt": "2025-11-21T10:35:00Z", "estimatedProcessingTime": "15-30 minutes", "queuePosition": 2, "queueName": "batch-upload-queue" } }
...
Processing Flow:
1. Validate batch exists and status is "uploaded"
2. Create message in batch-upload-queue with batch metadata
3. Update batch status to "queued" in blob metadata
4. Return 202 Accepted with queue position
5. ParserService picks up message asynchronously
...
Acceptance Criteria:
| Criterion | Validation Method | Test Data | Expected Result |
|---|---|---|---|
| Batch must be in "uploaded" status | Start already-processing batch | processing-batch-id | 409 CONFLICT |
| Creates message in batch-upload-queue | Verify queue message created | valid-batch-id | Message present |
| Updates batch status to "queued" | Check batch metadata after start | valid-batch-id | status: "queued" |
| Returns estimated time based on queue | Check with empty vs full queue | various | Time varies with queue depth |
| Idempotent (duplicate calls safe) | Call /start twice | valid-batch-id | Both return 202, one processes |
| Returns current queue position | Verify position accuracy | valid-batch-id | Position matches queue |
| Requires Batch Operator role | Call without role | valid-batch-id | 403 ACCESS_DENIED |
| Supports "strict" and "lenient" validation | Set validationMode=lenient | valid-batch-id | Mode stored in message |
| Queue full returns 503 | Start when queue depth >10000 | valid-batch-id | 503 SERVICE_UNAVAILABLE |
Validation Rules:
| Check | Rule | Error Code | HTTP | Action |
|---|---|---|---|---|
| Batch exists | Must exist in blob storage | RESOURCE_NOT_FOUND | 404 | Return error immediately |
| Batch ownership | Must belong to organization in path | ACCESS_DENIED | 403 | Return error immediately |
| Batch status | Must be "uploaded", not "queued"/"processing"/"completed" | PROCESSING_ERROR | 422 | Return error with current status |
| Organization active | Organization.isActive = true | ORGANIZATION_INACTIVE | 422 | Return error |
| Queue capacity | Queue depth < 10,000 | SERVICE_UNAVAILABLE | 503 | Return with retry-after |
| User permission | User has BatchOperator or higher role | ACCESS_DENIED | 403 | Return error |
| Validation mode | Must be "strict" or "lenient" | VALIDATION_ERROR | 400 | Return error with allowed values |
Error Scenarios:
| Scenario | HTTP | Error Code | Message | Details | User Action |
|---|---|---|---|---|---|
| Batch not found | 404 | RESOURCE_NOT_FOUND | Batch does not exist | { "batchId": "{id}" } | Verify batch ID |
| Already processing | 409 | CONFLICT | Batch is already processing | { "currentStatus": "processing", "startedAt": "2025-11-21T10:00:00Z" } | Wait for completion or cancel |
| Invalid status | 422 | PROCESSING_ERROR | Batch cannot be started from current status | { "currentStatus": "completed", "allowedStatuses": ["uploaded"] } | Re-upload batch |
| Queue at capacity | 503 | SERVICE_UNAVAILABLE | System at capacity, retry later | { "queueDepth": 10500, "retryAfter": "2025-11-21T11:00:00Z" } | Schedule for off-peak |
3.3 FR-003: Parser Service (XML → JSON Transformation)
Requirement: The ParserService shall listen to batch-upload-queue, download batch XML from blob storage, detect vendor format, validate against XSD schema, parse to individual invoices, transform to canonical JSON format, and enqueue to batch-items-queue in groups of 32.
Priority: CRITICAL
Service Specifications:
Trigger: Message in batch-upload-queue
Input: Batch ID from queue message
Output: Individual JSON files in blob + messages in batch-items-queue
Processing Steps:
1. Dequeue message from batch-upload-queue
2. Download batch XML from blob: {org}-batches-{year}/{month}/{day}/{batch-id}/source.xml
3. Detect vendor format (namespace + structure analysis):
- urn:ediel → GASEL
- oio.dk → XELLENT
- Zynergy → ZYNERGY
4. Load vendor-specific schema mapping from: {org}-data/schemas/{vendor}-mapping.json
5. Validate XML against vendor XSD schema
6. Parse XML using XPath expressions from mapping
7. Transform each invoice to canonical JSON format
8. Store individual JSON files: {org}-invoices-{year}/{month}/{day}/{invoice-id}.json
9. Group invoices into batches of 32
10. Enqueue each 32-item batch to batch-items-queue
11. Update batch metadata: totalItems, vendor info, status="processing"
12. Delete message from batch-upload-queue (on success)
13. On error: Retry (3x with backoff) or move to poison queue
...
Canonical JSON Schema (Output Format):
{ "invoiceId": "uuid", "invoiceNumber": "2025-11-001", "invoiceDate": "2025-11-06", "dueDate": "2025-11-20", "currency": "SEK", "periodStart": "2025-10-01", "periodEnd": "2025-10-31", "customer": { "customerId": "020624-2380", "fullName": "Medeni Schröder", "firstName": null, "lastName": null, "email": "muntaser.af@zavann.net", "phone": "09193538799", "address": { "street": "Strandbo 63B", "houseNumber": null, "apartment": null, "city": "Växjö", "postalCode": "352 58", "country": "SE" }, "taxIdentifier": "020624-2380", "customerType": "private" }, "invoiceDetails": { "subTotal": 599.42, "taxAmount": 149.86, "totalAmount": 749.28, "lineItems": [ { "lineNumber": 1, "description": "Elförbrukning - Fast pris", "quantity": 420, "unit": "KWH", "unitPrice": 1.026, "lineAmount": 430.92, "taxRate": 25.0, "taxAmount": 107.73, "category": "electricity" }, { "lineNumber": 2, "description": "Månadsavgift", "quantity": 1, "unit": "MON", "unitPrice": 15.20, "lineAmount": 15.20, "taxRate": 25.0, "taxAmount": 3.80, "category": "fee" } ] }, "delivery": { "meteringPointId": "735999756427205424", "gridArea": "SE4", "gridOwner": "Växjö Energi Elnät AB", "previousReading": { "date": "2025-09-30", "value": 10580, "type": "Actual" }, "currentReading": { "date": "2025-10-31", "value": 11000, "type": "Actual" }, "consumption": 420, "consumptionUnit": "kWh" }, "payment": { "paymentId": "202511001", "paymentMethod": "Bankgiro", "bankAccount": "168-6039", "dueDate": "2025-11-20" }, "sourceMetadata": { "vendorCode": "GASEL", "vendorVersion": "1.0", "originalBatchId": "BATCH2025110600001", "originalInvoiceId": "2025-11-001", "contractReference": "CON-2024-001", "customFields": { "productCode": "telinet_fixed", "productName": "Fast Pris Vintersäkra", "taxClassification": "Normal" }, "parsedAt": "2025-11-21T10:35:45Z" } }
...
Acceptance Criteria:
| Criterion | Validation Method | Test Data | Expected Result |
|---|---|---|---|
| Listens to batch-upload-queue | Send message, verify service picks up | Queue message | Message dequeued within 30s |
| Downloads batch XML from blob | Verify file download logged | Batch in blob | File downloaded successfully |
| Detects GASEL format (100% accuracy) | Test with 50 GASEL samples | gasel_*.xml | All detected as GASEL |
| Detects XELLENT format (100% accuracy) | Test with 50 XELLENT samples | xellent_*.xml | All detected as XELLENT |
| Detects ZYNERGY format (100% accuracy) | Test with 50 ZYNERGY samples | zynergy_*.xml | All detected as ZYNERGY |
| Loads correct schema mapping | Verify mapping file loaded | gasel_sample.xml | gasel-mapping.json loaded |
| Validates XML against XSD | Upload invalid GASEL XML | invalid_gasel.xml | Validation errors in batch metadata |
| Parses GASEL using XPath mappings | Parse GASEL, verify JSON fields | gasel_sample.xml | All fields extracted |
| Parses XELLENT with namespace handling | Parse XELLENT (com:, main: prefixes) | xellent_sample.xml | All fields extracted |
| Parses ZYNERGY nested structure | Parse ZYNERGY | zynergy_sample.xml | All fields extracted |
| Transforms to canonical JSON | Verify JSON schema compliance | All vendors | All pass schema validation |
| Stores JSON: {org}-invoices-{year}/{month}/{day}/{id}.json | Check blob path | Parsed invoice | Correct path used |
| Groups into 32-item batches | Parse 100 invoices, count queue messages | 100-invoice batch | 4 messages (32+32+32+4) |
| Enqueues to batch-items-queue | Verify messages in queue | Parsed batch | Messages present |
| Updates batch metadata | Check metadata after parsing | Parsed batch | totalItems, vendorCode set |
| Deletes from batch-upload-queue on success | Verify message removed | Successful parse | Message gone |
| Retries 3x on transient errors | Force blob download error | Failing batch | 3 retries logged |
| Moves to poison queue after 3 failures | Force permanent error | Failing batch | Message in poison queue |
| Parsing completes within 2 minutes for 10K batch | Performance test | 10K-invoice XML | ≤ 120 seconds |
Vendor-Specific Parsing Rules:
GASEL Format:
Required Elements (validation fails if missing):
- BatchHeader/InterchangeID
- BatchHeader/TotalInvoiceCount
- SupplierParty/PartyName
- Invoice/InvoiceHeader/InvoiceNumber
- Invoice/CustomerParty/PartyName
- Invoice/MonetarySummary/PayableAmount
Optional Elements (null if missing):
- Invoice/DeliveryLocation/MeteringPointID
- Invoice/ContractDetails/ContractID
- Invoice/CustomerParty/Contact/ElectronicMail
- Invoice/CustomerParty/Contact/Telephone
Date Format: ISO 8601 (YYYY-MM-DD)
Amount Format: Decimal with 2 places, period as separator
Namespace: urn:ediel:se:electricity:invoice:1.0
...
XELLENT Format:
Required Elements:
- BatchHeader/BatchID
- com:ID (invoice number)
- com:IssueDate
- com:BuyerParty/com:PartyName/com:Name
- com:LegalTotals/com:ToBePaidTotalAmount
Optional Elements:
- com:BuyerParty/com:ContactInformation/@E-Mail
- com:BuyerParty/com:Address
Special Handling:
- Multiple namespaces (com:, main:, fsv:)
- Amount format: "1 245,00" (space separator, comma decimal)
- Must normalize to standard decimal format
Namespace: http://rep.oio.dk/ubl/xml/schemas/0p71/pie/
...
ZYNERGY Format:
Required Elements:
- BatchHeader/BatchId
- InvoiceData/InvoiceNumber
- Customer/ReadOnlyFullName
- InvoiceData/InvoiceAmount
Optional Elements:
- Customer/FirstName and LastName (if ReadOnlyFullName empty)
- InvoiceAddress/EmailAddress
- VAT details
Special Handling:
- Nested structure (Invoice > Customer, InvoiceData, InvoiceAddress, VAT)
- Multiple company references (CompaniesId throughout)
- InvoiceAmount vs InvoiceBalance distinction
Namespace: http://eg.dk/Zynergy/1.0/invoice.xsd
...
Dependencies:
- Azure Storage Queue:
batch-upload-queue - Vendor schema mappings in blob storage
- XSD schema files for validation
- Canonical JSON schema definition
- Error handling and retry infrastructure
Risks & Mitigation:
| Risk | Likelihood | Impact | Mitigation Strategy | Owner |
|---|---|---|---|---|
| Large XML file memory issues (>50MB) | MEDIUM | HIGH | - Stream-based parsing (XmlReader, not XDocument) - Process invoices incrementally - Worker memory limit monitoring - File size alerts at 75MB | Technical Architect |
| Parsing performance bottleneck | MEDIUM | HIGH | - Parallel XPath evaluation where possible - Compiled XPath expressions cached - POC: parse 10K invoices in <2 minutes - Horizontal scaling of ParserService | Technical Architect |
| XSD validation performance | LOW | MEDIUM | - Cache compiled XSD schemas - Make validation optional in lenient mode - Async validation (don't block parsing) | Technical Architect |
| Vendor-specific edge cases | HIGH | MEDIUM | - Extensive test suite per vendor (50+ samples) - Error collection from production - Vendor liaison for unclear cases - Lenient mode for known variations | Product Owner |
3.4 FR-004: Document Generator Service (JSON → HTML → PDF)
Requirement: The DocumentGeneratorService (based on existing zyn-DocumentGenerator) shall listen to batch-items-queue, load Handlebars templates, render HTML with invoice data, generate PDFs using Playwright, and store documents in blob storage with month/day hierarchy.
Priority: CRITICAL
Service Specifications:
Trigger: Message in batch-items-queue (contains 32 invoice references)
Input: Batch of 32 invoice IDs and organization ID
Output: PDF + HTML files in blob storage, messages in distribution routing queues
Processing Steps (per 32-item batch):
1. Dequeue message from batch-items-queue
2. Acquire blob lease: {org}-batches-{year}/{month}/{day}/{batch-id}/locks/{worker-id}.lock
3. For each of 32 invoices:
a. Download JSON: {org}-invoices-{year}/{month}/{day}/{invoice-id}.json
b. Load organization config: {org}-data/organization.json
c. Determine template category from distribution type
d. Load Handlebars template: {org}-data/templates/{category}/active.html
e. Load organization branding (logo from blob URL, colors, fonts)
f. Compile Handlebars template (cache compiled version)
g. Render HTML with invoice data + branding
h. Generate PDF from HTML using Playwright (headless Chromium)
i. Store HTML: {org}-invoices-{year}/{month}/{day}/{invoice-id}.html
j. Store PDF: {org}-invoices-{year}/{month}/{day}/{invoice-id}.pdf
k. Update invoice JSON metadata with file paths and render timestamp
l. Determine distribution method from invoice data
m. Enqueue to appropriate queue:
- Mail (postal) → postal-bulk-queue (processed in bulk)
- Email → email-queue
- SMS (future) → sms-queue
- Kivra (future) → kivra-queue
- E-faktura (future) → efaktura-queue
4. Release blob lease
5. Update batch metadata: processedItems += 32
6. Delete message from batch-items-queue
7. On error: Retry (3x) or poison queue
...
Acceptance Criteria:
| Criterion | Validation Method | Test Data | Expected Result |
|---|---|---|---|
| Listens to batch-items-queue | Send message, verify pickup | Queue message | Dequeued within 30s |
| Processes 32 invoices per message | Send 32-item batch, count outputs | 32 invoices | 32 PDFs generated |
| Acquires blob lease before processing | Check lease on blob | Valid batch | Lease acquired |
| Downloads invoice JSON from correct path | Verify download logged | Invoice JSON | Correct path: {org}/2025/11/21/{id}.json |
| Loads organization template | Verify template file accessed | Org with template | Template loaded |
| Determines template category correctly | Invoice → "invoice" template, Letter → "confirmation" | Various types | Correct template used |
| Compiles Handlebars template | Render with variables | Template with {{invoiceNumber}} | Number inserted |
| Caches compiled templates (24h) | Render same template twice | Same template | Second render faster |
| Renders HTML with Swedish characters | Render with åäö | Swedish invoice | Characters correct |
| Generates PDF with Playwright | Convert HTML to PDF | Rendered HTML | PDF created, A4 format |
| PDF includes organization branding | Check PDF for logo, colors | Branded template | Branding visible |
| Stores HTML in correct blob path | Verify path | Generated HTML | {org}/invoices/2025/11/21/{id}.html |
| Stores PDF in correct blob path | Verify path | Generated PDF | {org}/invoices/2025/11/21/{id}.pdf |
| Updates invoice metadata JSON | Check metadata after render | Processed invoice | fileReferences populated |
| Determines distribution method | Check routing logic | Various configs | Correct queue selected |
| Enqueues to postal-bulk-queue for mail | Invoice with postal delivery | Mail invoice | Message in postal queue |
| Enqueues to email-queue for email | Invoice with email delivery | Email invoice | Message in email queue |
| Releases blob lease on completion | Verify lease released | Processed batch | Lease gone |
| Updates batch statistics | Check batch metadata | Processed batch | processedItems incremented |
| Rendering within 2 seconds per invoice (p95) | Performance test 1000 invoices | Various | p95 ≤ 2 seconds |
| PDF generation within 5 seconds per invoice (p95) | Performance test 1000 PDFs | Various | p95 ≤ 5 seconds |
| Retries on transient errors | Force blob error | Failing invoice | 3 retries attempted |
| Moves to poison queue after 3 failures | Force permanent error | Failing invoice | Poison queue message |
Handlebars Template Example:
<!DOCTYPE html> <html lang="sv"> <head> <meta charset="UTF-8"> <title>Faktura {{invoiceNumber}}</title> <style> body { font-family: {{organization.fontFamily}}; } .header { background-color: {{organization.primaryColor}}; color: white; padding: 20px; } .logo { max-width: 200px; } </style> </head> <body> <div class="header"> <img src="{{organization.logoUrl}}" alt="{{organization.displayName}}" class="logo" /> <h1>Faktura {{invoiceNumber}}</h1> </div> <div class="customer-info"> <h2>Kund</h2> <p><strong>{{customer.fullName}}</strong></p> <p>{{customer.address.street}}</p> <p>{{customer.address.postalCode}} {{customer.address.city}}</p> </div> <div class="invoice-details"> <p><strong>Fakturadatum:</strong> {{invoiceDate}}</p> <p><strong>Förfallodatum:</strong> {{dueDate}}</p> <p><strong>Period:</strong> {{periodStart}} - {{periodEnd}}</p> {{#if delivery.meteringPointId}} <p><strong>Mätpunkt:</strong> {{delivery.meteringPointId}}</p> <p><strong>Elområde:</strong> {{delivery.gridArea}}</p> <p><strong>Förbrukning:</strong> {{delivery.consumption}} {{delivery.consumptionUnit}}</p> {{/if}} </div> <table> <thead> <tr> <th>Beskrivning</th> <th>Antal</th> <th>Enhet</th> <th>Pris</th> <th>Belopp</th> </tr> </thead> <tbody> {{#each invoiceDetails.lineItems}} <tr> <td>{{this.description}}</td> <td>{{formatNumber this.quantity decimals=2}}</td> <td>{{this.unit}}</td> <td>{{formatCurrency this.unitPrice}}</td> <td>{{formatCurrency this.lineAmount}}</td> </tr> {{/each}} <tr class="total-row"> <td colspan="4"><strong>Delsumma:</strong></td> <td><strong>{{formatCurrency invoiceDetails.subTotal}}</strong></td> </tr> <tr> <td colspan="4"><strong>Moms (25%):</strong></td> <td><strong>{{formatCurrency invoiceDetails.taxAmount}}</strong></td> </tr> <tr class="total-row"> <td colspan="4"><strong>Att betala:</strong></td> <td><strong>{{formatCurrency invoiceDetails.totalAmount}} {{currency}}</strong></td> </tr> </tbody> </table> {{#if payment.paymentId}} <div class="payment-info"> <h3>Betalningsinformation</h3> <p><strong>OCR-nummer:</strong> {{payment.paymentId}}</p> <p><strong>Bankgiro:</strong> {{payment.bankAccount}}</p> <p><strong>Förfallodatum:</strong> {{payment.dueDate}}</p> </div> {{/if}} </body> </html>
...
Dependencies:
- Handlebars.Net library for template rendering
- Playwright library for PDF generation
- Azure Storage Queue:
batch-items-queue - Blob storage for templates, JSON, PDF, HTML
- Organization configuration in blob
- Custom Handlebars helpers (formatCurrency, formatNumber, formatDate)
Risks & Mitigation:
| Risk | Likelihood | Impact | Mitigation Strategy | Owner |
|---|---|---|---|---|
| Handlebars rendering performance | HIGH | HIGH | - Pre-compile templates on first use - Cache compiled templates (24h TTL) - Parallel rendering for 32 items - POC: 1000 renders in <30 seconds | Technical Architect |
| Playwright memory consumption | HIGH | HIGH | - Semaphore limit: max 10 concurrent PDFs - Worker instance memory monitoring - Graceful degradation if memory high - Browser instance pooling | Technical Architect |
| Swedish character encoding (åäö) | MEDIUM | MEDIUM | - UTF-8 throughout entire pipeline - Font embedding in PDF - Visual testing with Swedish content - Sample invoices with all Swedish special chars | QA Team |
| Template injection security | LOW | CRITICAL | - Handlebars safe mode (no eval) - Template sanitization on upload - No dynamic helper registration - Security code review | Security Officer |
| Missing template category | LOW | MEDIUM | - Fall back to default "invoice" template - Log warning for missing category - Template category validation | Product Owner |
3.5 FR-005: Email Delivery Service
Requirement: The EmailDeliveryService shall listen to email-queue, download PDF from blob storage, send via SendGrid with Swedish-localized email template, track delivery status, and retry on transient failures with fallback to postal queue.
Priority: HIGH
Service Specifications:
Trigger: Message in email-queue
Input: Invoice ID, recipient email, organization ID
Output: Email sent via SendGrid, delivery status updated
Processing Steps:
1. Dequeue message from email-queue
2. Download PDF: {org}-invoices-{year}/{month}/{day}/{invoice-id}.pdf
3. Load organization email configuration
4. Load email template (Swedish)
5. Create SendGrid message:
- From: noreply@{org-domain}.com
- To: {customer-email}
- Subject: "Faktura {invoiceNumber} från {orgName}"
- Body: HTML template with invoice summary
- Attachment: invoice-{invoiceNumber}.pdf
6. Send via SendGrid API
7. Handle response:
- Success (2xx): Update invoice status="delivered", log messageId
- Rate limit (429): Re-queue with Retry-After delay
- Transient error (5xx): Retry with exponential backoff (3x)
- Permanent error (4xx): Move to postal-bulk-queue (fallback)
8. Update invoice metadata with delivery attempt
9. Delete from email-queue (on success or permanent failure)
...
Email Template (Swedish):
<!DOCTYPE html> <html lang="sv"> <head> <meta charset="UTF-8"> <title>Faktura</title> </head> <body style="font-family: Arial, sans-serif; max-width: 600px; margin: 0 auto;"> <div style="background: #0066CC; color: white; padding: 20px; text-align: center;"> <h1>Faktura från {{organizationName}}</h1> </div> <div style="padding: 20px;"> <p>Hej {{customerName}},</p> <p>Din faktura för perioden {{periodStart}} till {{periodEnd}} är nu tillgänglig.</p> <table style="width: 100%; margin: 20px 0; border-collapse: collapse;"> <tr style="background: #f5f5f5;"> <td style="padding: 10px; border: 1px solid #ddd;"><strong>Fakturanummer:</strong></td> <td style="padding: 10px; border: 1px solid #ddd;">{{invoiceNumber}}</td> </tr> <tr> <td style="padding: 10px; border: 1px solid #ddd;"><strong>Förfallodatum:</strong></td> <td style="padding: 10px; border: 1px solid #ddd;">{{dueDate}}</td> </tr> <tr style="background: #f5f5f5;"> <td style="padding: 10px; border: 1px solid #ddd;"><strong>Att betala:</strong></td> <td style="padding: 10px; border: 1px solid #ddd;"><strong>{{totalAmount}} SEK</strong></td> </tr> </table> <p><strong>Betalningsinformation:</strong></p> <p>Bankgiro: {{bankAccount}}<br> OCR-nummer: {{ocrNumber}}</p> <p>Din faktura finns bifogad som PDF.</p> <p>Vid frågor, kontakta oss på {{supportEmail}} eller {{supportPhone}}.</p> <p>Med vänlig hälsning,<br> {{organizationName}}</p> </div> <div style="background: #f5f5f5; padding: 15px; text-align: center; font-size: 12px; color: #666;"> <p>Detta är ett automatiskt meddelande. Svara inte på detta e-postmeddelande.</p> </div> </body> </html>
...
Acceptance Criteria:
| Criterion | Validation Method | Test Data | Expected Result |
|---|---|---|---|
| Listens to email-queue | Send message, verify processing | Queue message | Message dequeued |
| Downloads PDF from correct blob path | Verify blob access logged | Invoice with PDF | PDF downloaded |
| Sends via SendGrid API | Mock SendGrid, verify API call | Email invoice | SendGrid API called |
| PDF attached to email | Receive test email, check attachment | Email invoice | PDF attached |
| Subject includes invoice number (Swedish) | Check email subject | Invoice 123 | "Faktura 123 från..." |
| From address uses org domain | Check email headers | Org config | From: noreply@acme.com |
| Reply-to set to org support | Check email headers | Org config | Reply-To: support@acme.com |
| Swedish email template used | Check email body | Email invoice | Swedish text |
| Retry 2x on transient failure (1min, 5min) | Force 500 error from SendGrid | Failing email | 2 retries logged |
| Fallback to postal on permanent failure | Force 400 error (invalid email) | Bad email | Postal queue message |
| Delivery status tracked in invoice metadata | Check metadata after send | Delivered invoice | deliveryAttempts array updated |
| SendGrid messageId logged | Check invoice metadata | Delivered invoice | providerMessageId present |
| Rate limit handling (429) | Simulate rate limit | Many emails | Re-queued with delay |
| Email size validation (<25MB) | Large PDF attachment | 30MB PDF | Error or compression |
SendGrid Configuration:
{ "sendgrid": { "apiKey": "{{from-azure-keyvault}}", "fromEmail": "noreply@{org-domain}.com", "fromName": "{organizationName}", "replyTo": "support@{org-domain}.com", "tracking": { "clickTracking": false, "openTracking": true, "subscriptionTracking": false }, "mailSettings": { "sandboxMode": false, "spamCheck": { "enable": true, "threshold": 5 } } } }
...
Dependencies:
- SendGrid account with Nordic IP reputation
- Azure Storage Queue:
email-queue - Email templates in Swedish (+ future: Norwegian, Danish, Finnish)
- Organization email configuration in blob
- Fallback queue routing to postal
Risks & Mitigation (Nordic Email Deliverability):
| Risk | Likelihood | Impact | Mitigation Strategy | Owner |
|---|---|---|---|---|
| Swedish ISP spam filtering (Telia, Tele2, Telenor) | MEDIUM | HIGH | - Dedicated IP warmup (2-week ramp) - SPF: include:sendgrid.net - DKIM signing enabled - DMARC p=quarantine policy - Monitor bounce rates by ISP - Request whitelisting from major ISPs | Operations Manager |
| SendGrid rate limits (enterprise plan needed) | MEDIUM | MEDIUM | - Enterprise plan: 2M emails/month - Queue-based pacing - Monitor daily send volume - Distribute sends over 24 hours - Priority queue for SLA customers | Product Owner |
| PDF attachment size (>25MB) | LOW | LOW | - Compress PDFs with Ghostscript - Target: <5MB per invoice - Alert if PDF >20MB - Fallback: send download link | Technical Architect |
| Email template rendering errors | LOW | MEDIUM | - Template validation on deployment - Fallback to plain text if HTML fails - Error monitoring - Sample sends for all templates | QA Team |
| Customer email address invalid | MEDIUM | LOW | - Email validation before send - Skip email, go directly to postal - Log invalid addresses for org to correct | Product Owner |
3.6 FR-006: Postal Delivery Service (21G Bulk Integration)
Requirement: The PostalDeliveryService shall listen to postal-bulk-queue, collect invoices for bulk processing, create ZIP archive with PDFs and XML metadata in 21G format, upload to 21G SFTP server at scheduled times (12:00 and 20:00 Swedish time), and track delivery confirmations.
Priority: HIGH
Service Specifications:
Trigger: Scheduled execution (12:00 and 20:00 CET/CEST)
Input: All messages in postal-bulk-queue accumulated since last run
Output: ZIP file uploaded to 21G SFTP, delivery confirmations tracked
Processing Steps:
1. Scheduled trigger (12:00 and 20:00 Swedish time)
2. Fetch all messages from postal-bulk-queue (batch retrieval)
3. For each invoice in queue:
a. Download PDF: {org}-invoices-{year}/{month}/{day}/{invoice-id}.pdf
b. Download invoice JSON for metadata
c. Validate recipient address is complete
d. Add to 21G batch collection
4. Group by organization (21G requires org-specific batches)
5. For each organization batch:
a. Create 21G XML metadata file with all invoice details
b. Create ZIP archive: {org-code}_{date}_{sequence}.zip
- Contains: invoice1.pdf, invoice2.pdf, ..., metadata.xml
c. Upload ZIP to 21G SFTP: /incoming/{org-code}/
d. Verify upload success
e. Update all invoice statuses: status="postal_sent"
f. Delete messages from postal-bulk-queue
6. Log bulk send statistics to Application Insights
7. Send notification email to organization (bulk send report)
...
21G ZIP Structure:
ACME_20251121_001.zip
├── metadata.xml (21G format)
├── invoice_001.pdf
├── invoice_002.pdf
├── invoice_003.pdf
└── ...
...
21G Metadata XML Format:
<?xml version="1.0" encoding="UTF-8"?> <PrintBatch xmlns="urn:21g:print:batch:1.0"> <BatchHeader> <BatchId>ACME_20251121_001</BatchId> <OrganizationCode>ACME</OrganizationCode> <CreationDate>2025-11-21T12:00:00</CreationDate> <TotalDocuments>150</TotalDocuments> <ServiceLevel>Economy</ServiceLevel> </BatchHeader> <Documents> <Document> <DocumentId>invoice_001.pdf</DocumentId> <DocumentType>Invoice</DocumentType> <Recipient> <Name>Medeni Schröder</Name> <Street>Strandbo 63B</Street> <PostalCode>352 58</PostalCode> <City>Växjö</City> <Country>SE</Country> </Recipient> <PrintOptions> <Format>A4</Format> <Color>false</Color> <Duplex>false</Duplex> </PrintOptions> </Document> <!-- ... more documents --> </Documents> </PrintBatch>
...
Acceptance Criteria:
| Criterion | Validation Method | Test Data | Expected Result |
|---|---|---|---|
| Scheduled execution at 12:00 Swedish time | Check execution logs | Scheduled time | Runs at 12:00 CET/CEST |
| Scheduled execution at 20:00 Swedish time | Check execution logs | Scheduled time | Runs at 20:00 CET/CEST |
| Fetches all messages from postal-bulk-queue | Queue 100 messages, verify all fetched | 100 postal invoices | All 100 fetched |
| Downloads PDFs from blob storage | Verify blob access | Postal invoices | All PDFs downloaded |
| Validates recipient address complete | Invoice with missing city | Incomplete address | Skipped with error log |
| Groups by organization | Mix of Org A and Org B invoices | Multi-org batch | Separate ZIPs per org |
| Creates 21G XML metadata | Verify XML structure | Postal batch | Valid 21G XML |
| Creates ZIP archive | Verify ZIP contents | Postal batch | PDFs + metadata.xml |
| Uploads to 21G SFTP | Mock SFTP, verify upload | ZIP file | File uploaded |
| Verifies upload success | Check SFTP confirmation | Uploaded ZIP | Confirmation received |
| Updates invoice status to "postal_sent" | Check invoice metadata | Sent invoices | Status updated |
| Deletes messages from queue | Check queue after processing | Processed batch | Queue empty |
| Logs bulk statistics | Check Application Insights | Processed batch | Statistics logged |
| Sends org notification email | Check email received | Processed batch | Email with counts |
| Handles SFTP connection errors | Simulate SFTP down | Postal batch | Retry logged, alert sent |
| Respects 21G batch size limits | Create large batch | 10,000 invoices | Split into multiple ZIPs |
21G Integration Specifications:
SFTP Connection:
- Host: sftp.21g.se (or provider-specific)
- Port: 22
- Authentication: SSH key (stored in Azure Key Vault)
- Directory structure: /incoming/{org-code}/
- File naming: {org-code}{YYYYMMDD}{sequence}.zip
21G SLA:
- Processing time: 24-48 hours
- Confirmation: Email notification when processed
- Tracking: Available via 21G portal
Dependencies:
- 21G SFTP account and credentials
- Azure Storage Queue:
postal-bulk-queue - Scheduled worker (Azure Container Apps with CRON)
- ZIP file creation library
- 21G XML schema compliance
- Email notification service
Risks & Mitigation (Nordic Postal Context):
| Risk | Likelihood | Impact | Mitigation Strategy | Owner |
|---|---|---|---|---|
| 21G SFTP connectivity issues | LOW | HIGH | - Retry logic (3 attempts with 5min delay) - Secondary SFTP credentials - Alert on connection failure - Manual upload procedure documented - 21G support contact documented | Operations Manager |
| Swedish postal delays (holidays, strikes) | MEDIUM | MEDIUM | - Set customer expectations (5-7 days) - Monitor 21G processing SLA - Track delivery confirmations - Escalation for >10 days - Alternative print partner identified | Product Owner |
| Incomplete recipient addresses | MEDIUM | LOW | - Address validation before queueing - Skip invalid addresses - Alert organization of invalid addresses - Provide address correction interface | Product Owner |
| 21G format specification changes | LOW | MEDIUM | - Version 21G XML schema - Monitor 21G API announcements - Test uploads to 21G staging - 21G account manager liaison | Technical Architect |
| ZIP file corruption | LOW | HIGH | - SHA-256 checksum in metadata - Verify ZIP integrity before upload - Keep ZIP in blob for 30 days - 21G confirms successful unzip | Technical Architect |
3.7 FR-007: Distribution Routing Logic
Requirement: The system shall determine the appropriate distribution queue for each invoice based on organization configuration, customer preferences, and distribution type (invoice vs document) following Swedish regulatory requirements.
Priority: HIGH
Routing Decision Tree:
Invoice Distribution Routing
│
├─ Check customer preference (if available)
│ ├─ Preference = "digital" → email-queue (or kivra-queue in future)
│ └─ Preference = "postal" → postal-bulk-queue
│
├─ Check organization default channels (from config)
│ ├─ Priority 1: email
│ │ └─ Has valid email? → email-queue
│ ├─ Priority 2: kivra (Phase 2)
│ │ └─ Kivra user? → kivra-queue
│ └─ Priority 3: postal
│ └─ postal-bulk-queue
│
├─ Document type consideration
│ ├─ Invoice (faktura) → All channels available
│ └─ Confirmation letter → Email/postal only
│
└─ Swedish regulatory compliance
└─ Customer always has right to postal ("rätt till pappersfaktura")
...
Queue Selection Logic:
public async Task<string> DetermineDistributionQueueAsync( InvoiceDistribution distribution, OrganizationConfig config) { // Customer preference overrides (if explicitly set) if (distribution.CustomerPreference == "postal") return "postal-bulk-queue"; // Try channels in priority order var priorities = config.DeliveryChannels.ChannelPriority .OrderBy(p => p.Priority); foreach (var channel in priorities) { switch (channel.Channel) { case "email": if (!string.IsNullOrEmpty(distribution.CustomerEmail) && IsValidEmail(distribution.CustomerEmail)) { return "email-queue"; } break; case "kivra": // Phase 2 if (await IsKivraUserAsync(distribution.CustomerPersonnummer)) { return "kivra-queue"; } break; case "efaktura": // Phase 2 (B2B only) if (distribution.CustomerType == "business" && !string.IsNullOrEmpty(distribution.OrganizationNumber)) { return "efaktura-queue"; } break; case "postal": if (distribution.IsCompleteAddress()) { return "postal-bulk-queue"; } break; } } // Ultimate fallback: postal (Swedish law requires paper option) return "postal-bulk-queue"; }
...
Acceptance Criteria:
| Criterion | Validation Method | Test Data | Expected Result |
|---|---|---|---|
| Customer preference honored | Set preference="postal" | Email-enabled invoice | Routes to postal queue |
| Organization priority followed | Priority: [email, postal] | Valid email | Routes to email queue |
| Email validated before routing | Invalid email address | bad-email@invalid | Routes to postal queue |
| Complete address required for postal | Missing postal code | Incomplete address | Error logged, skipped |
| Document type considered | Confirmation letter | Non-invoice doc | Only email/postal |
| Swedish postal fallback | All digital channels fail | Failed digital | postal-bulk-queue |
| Business invoices support e-faktura (future) | Organization number present | B2B invoice | efaktura-queue (Phase 2) |
| Routing decision logged | Check logs | Any invoice | Decision reason logged |
Dependencies:
- Organization delivery configuration
- Customer preference storage (future)
- Email validation library
- Address validation library (Swedish postal codes)
- Kivra user lookup API (Phase 2)
Risks & Mitigation:
| Risk | Likelihood | Impact | Mitigation Strategy | Owner |
|---|---|---|---|---|
| Invalid email addresses (>5% in Nordic utilities) | HIGH | LOW | - Email validation regex - Automatic postal fallback - Report invalid emails to organization - Customer data quality improvement program | Product Owner |
| Incomplete postal addresses | MEDIUM | MEDIUM | - Address validation against Swedish postal database - Skip invalid addresses with alert - Organization notification of incomplete addresses | Product Owner |
| Swedish "rätt till pappersfaktura" compliance | LOW | CRITICAL | - Always enable postal as fallback - Never force digital-only - Document compliance in privacy policy - Legal review of routing logic | Legal/Compliance |
3.8 FR-008: Blob Concurrency Control (Note: Read-Only, No Concurrent Updates)
Requirement: The system shall use blob leases to ensure exclusive access during batch processing, preventing concurrent worker instances from processing the same 32-item batch.
Priority: HIGH
Clarification: Based on your feedback, there are no concurrent updates to files - workers only read invoice JSON files. The blob lease is used to ensure only one worker processes a given 32-item batch from the queue.
Lease Implementation:
public async Task ProcessBatchItemsAsync(BatchItemsMessage message) { var lockBlobPath = $"{message.OrganizationId}-batches-{year}/{month}/{day}/{message.BatchId}/locks/{message.MessageId}.lock"; BlobLease lease = null; try { // Acquire lease (5-minute duration) lease = await _blobLockService.AcquireLockAsync( containerName: $"{message.OrganizationId}-batches-{year}", blobName: lockBlobPath, leaseDuration: TimeSpan.FromMinutes(5)); // Process 32 invoices (read-only operations) foreach (var invoiceId in message.InvoiceIds) { // Read invoice JSON from blob (no updates to JSON) var invoiceJson = await _blobStorage.DownloadJsonAsync(invoiceId); // Render and generate (creates new HTML/PDF blobs) await RenderAndGenerateAsync(invoiceJson); // No concurrent update risk - creating new blobs only } // Update batch metadata (ETag-based optimistic concurrency) await UpdateBatchMetadataAsync(message.BatchId, meta => { meta.Statistics.ProcessedItems += message.InvoiceIds.Count; }); } finally { if (lease != null) { await _blobLockService.ReleaseLockAsync(lease); } } }
...
Acceptance Criteria:
| Criterion | Validation Method | Test Data | Expected Result |
|---|---|---|---|
| Acquires blob lease before processing | Start processing, check lease | 32-item batch | Lease acquired |
| Lease duration is 5 minutes | Check lease properties | Any batch | Duration = 5 min |
| Only one worker processes batch | Send same message to 2 workers | Duplicate message | One succeeds, one waits |
| Lease renewed for long processing | Process 32 items slowly | Slow batch | Lease renewed |
| Lease released on completion | Check lease after processing | Completed batch | Lease released |
| Lease released on error | Force error during processing | Failing batch | Lease released |
| Different batches process in parallel | Queue 10 batches | 10 x 32 items | All process concurrently |
| Batch metadata updates use ETags | Concurrent metadata updates | 2 workers update stats | No lost updates |
Dependencies:
- Azure Blob Storage lease API
- ETag-based optimistic concurrency for metadata updates
- Retry logic for lease acquisition conflicts
3.9 FR-009: Queue Message Handling
Requirement: The system shall process queue messages with proper visibility timeouts, automatic retry on failure, poison queue handling, and message deduplication.
Priority: HIGH
Queue Configuration:
| Queue Name | Purpose | Visibility Timeout | Max Delivery Count | Dead Letter Queue |
|---|---|---|---|---|
batch-upload-queue | Triggers ParserService | 10 minutes | 3 | poison-queue |
batch-items-queue | Triggers DocumentGenerator (32 items) | 5 minutes | 3 | poison-queue |
email-queue | Triggers EmailService | 2 minutes | 3 | poison-queue |
postal-bulk-queue | Collected for 21G bulk send | N/A (batch retrieval) | 1 | poison-queue |
poison-queue | Failed messages for manual review | N/A | 0 | None |
Message Format Standard:
{ "messageId": "uuid", "messageType": "batch.upload|batch.items|email.delivery|postal.delivery", "version": "1.0", "timestamp": "2025-11-21T10:30:00Z", "data": { // Message-specific payload }, "metadata": { "correlationId": "uuid", "organizationId": "uuid", "retryCount": 0, "enqueuedAt": "2025-11-21T10:30:00Z" } }
...
Retry Policy (Exponential Backoff):
| Attempt | Delay | Total Elapsed |
|---|---|---|
| 1 | Immediate | 0 |
| 2 | 60 seconds | 1 minute |
| 3 | 300 seconds | 6 minutes |
| 4 | 900 seconds | 21 minutes |
| Failed | Poison queue | - |
Acceptance Criteria:
| Criterion | Validation Method | Test Data | Expected Result |
|---|---|---|---|
| Messages have proper visibility timeout | Check queue properties | Any message | Correct timeout set |
| Failed messages retry automatically | Force error, verify retry | Failing message | 3 retries attempted |
| Retry count incremented | Check message metadata | Retried message | retryCount incremented |
| Exponential backoff applied | Measure retry delays | Failing message | 1min, 5min, 15min |
| After 3 retries, moved to poison queue | Force permanent failure | Failing message | In poison queue |
| Poison queue triggers alert | Message in poison queue | Failed message | Alert email sent |
| Support team notified | Check alert recipients | Poison message | Support receives email |
| No duplicate processing (idempotent) | Send duplicate message | Same invoice ID | Processed once |
| Correlation ID traces through system | Follow message across queues | Any message | Same correlationId |
Poison Queue Handling:
{ "messageId": "uuid", "originalMessageType": "batch.items", "failedAt": "2025-11-21T10:50:00Z", "retryCount": 3, "lastError": { "code": "TEMPLATE_RENDERING_FAILED", "message": "Required variable 'customer.address.street' not found in template", "stackTrace": "...", "attemptTimestamps": [ "2025-11-21T10:35:00Z", "2025-11-21T10:36:00Z", "2025-11-21T10:41:00Z", "2025-11-21T10:56:00Z" ] }, "originalMessage": { // Full original message for debugging }, "metadata": { "correlationId": "uuid", "alertSent": true, "alertRecipients": ["support@egflow.com"], "manualReviewRequired": true } }
...
Dependencies:
- Azure Storage Queues with dead-letter queue support
- Alert service for poison queue notifications
- Monitoring dashboard for poison queue depth
3.10 FR-010: Health Check Endpoints
Requirement: All services shall expose health check endpoints that verify connectivity to dependencies (database, blob storage, queues) and return health status for Azure Traffic Manager and monitoring.
Priority: HIGH
API Endpoint: GET /health
Response Format (Healthy):
{ "status": "Healthy", "timestamp": "2025-11-21T10:30:00Z", "version": "1.0.0", "checks": { "blobStorage": { "status": "Healthy", "responseTime": "45ms", "lastChecked": "2025-11-21T10:30:00Z" }, "storageQueue": { "status": "Healthy", "responseTime": "32ms", "queueDepth": 150 }, "postgresql": { "status": "Healthy", "responseTime": "12ms", "activeConnections": 8 }, "keyVault": { "status": "Healthy", "responseTime": "67ms" } }, "environment": "production", "region": "westeurope" }
...
Response Format (Unhealthy):
{ "status": "Unhealthy", "timestamp": "2025-11-21T10:30:00Z", "checks": { "blobStorage": { "status": "Unhealthy", "error": "Connection timeout after 5000ms", "lastChecked": "2025-11-21T10:30:00Z" }, "postgresql": { "status": "Healthy", "responseTime": "15ms" } } }
...
Acceptance Criteria:
| Criterion | Validation Method | Test Data | Expected Result |
|---|---|---|---|
| Returns 200 when all checks healthy | All dependencies up | N/A | 200 OK, status="Healthy" |
| Returns 503 when any check unhealthy | Stop database | N/A | 503 Service Unavailable |
| Checks blob storage connectivity | Disconnect blob storage | N/A | blobStorage.status="Unhealthy" |
| Checks queue connectivity | Disable queue access | N/A | storageQueue.status="Unhealthy" |
| Checks PostgreSQL connectivity | Stop database | N/A | postgresql.status="Unhealthy" |
| Checks Key Vault access | Revoke Key Vault permissions | N/A | keyVault.status="Unhealthy" |
| Response time < 1 second | Performance test | N/A | Health check completes quickly |
| Traffic Manager uses for routing | Simulate region failure | N/A | Traffic routes to healthy region |
| Includes environment and region | Check response body | N/A | Environment and region present |
Dependencies:
- Health check library (ASP.NET Core HealthChecks)
- Azure Traffic Manager configuration
- Monitoring integration
3.11 FR-011: Template Category Management
Requirement: The system shall support template categories (e.g., "Invoice", "Confirmation Letter", "Reminder") to group related templates and enable dynamic template selection based on document type.
Priority: MEDIUM
API Endpoint: GET /organizations/{organizationId}/template-categories
Response Format:
{ "success": true, "data": { "categories": [ { "categoryId": "uuid", "categoryName": "invoice", "displayName": "Faktura", "description": "Standard invoice template", "activeTemplateId": "uuid", "activeTemplateVersion": "2.1.0", "templateCount": 3 }, { "categoryId": "uuid", "categoryName": "confirmation", "displayName": "Bekräftelsebrev", "description": "Contract confirmation letter", "activeTemplateId": "uuid", "activeTemplateVersion": "1.0.0", "templateCount": 1 }, { "categoryId": "uuid", "categoryName": "reminder", "displayName": "Påminnelse", "description": "Payment reminder", "activeTemplateId": null, "templateCount": 0 } ] } }
...
Template Category Determination Logic:
Document Type → Template Category Mapping:
Invoice (faktura) → "invoice" template
Confirmation letter (bekräftelsebrev) → "confirmation" template
Payment reminder (påminnelse) → "reminder" template
Termination notice (uppsägning) → "termination" template
Contract change (avtalsändring) → "contract_change" template
Default: If category not found → use "invoice" template
...
Acceptance Criteria:
| Criterion | Validation Method | Test Data | Expected Result |
|---|---|---|---|
| Lists all categories for organization | GET /template-categories | Org with 3 categories | 3 categories returned |
| Shows active template per category | Check activeTemplateId | Category with active template | Template ID present |
| Returns null for unused categories | Check reminder category | No reminder template | activeTemplateId: null |
| Includes template count | Verify count | Category with 3 versions | templateCount: 3 |
| Category names localized (Swedish) | Check displayName | All categories | Swedish names |
3.12 FR-012: Git Branching Strategy (Development Workflow)
Requirement: The development team shall follow a structured Git branching strategy with development, staging, and main branches, following industry best practices for continuous integration and deployment.
Priority: HIGH
Branch Structure (Updated per Nov 20 decision):
main (production)
↑
└── staging (acceptance testing)
↑
└── development (integration testing)
↑
└── feature/* (short-lived branches)
...
Branch Policies:
| Branch | Purpose | Merge From | Deploy To | Protection |
|---|---|---|---|---|
| feature/ | Individual features | N/A | N/A | None (local only) |
| development | Integration testing | feature/* | Dev environment | PR required, 1 approval |
| staging | Acceptance testing (internal) | development | Staging environment | PR required, 2 approvals, all tests pass |
| main | Production | staging | Production (West + North Europe) | PR required, 3 approvals, security scan, all tests pass |
Deployment Flow:
Developer → feature/GAS-12345-batch-upload
↓
PR to development
↓
CI/CD: Build, Test, Deploy to Dev
↓
Integration testing in Dev
↓
PR to staging (approved by Product Owner)
↓
CI/CD: Build, Test, Deploy to Staging
↓
UAT testing in Staging (European team only)
↓
PR to main (approved by Product Owner + Architect + Ops)
↓
CI/CD: Build, Security Scan, Deploy to Prod (West Europe)
↓
Deploy to Prod (North Europe) - manual approval
...
Acceptance Criteria:
| Criterion | Validation Method | Expected Result |
|---|---|---|
| Feature branches merge to development only | Attempt direct merge to staging | PR blocked |
| Development branch auto-deploys to dev env | Merge to development | Dev environment updated |
| Staging branch requires 2 approvals | Create PR to staging | Cannot merge with 1 approval |
| Main branch requires 3 approvals | Create PR to main | Cannot merge with 2 approvals |
| All tests must pass before staging merge | Failing test in PR | Merge blocked |
| Security scan required for main | PR to main | SAST scan runs |
| Feature branches deleted after merge | Merge feature branch | Branch auto-deleted |
Dependencies:
- Azure DevOps or GitHub repository
- CI/CD pipeline configuration
- Branch protection policies
- Code review requirements
3.13 FR-013: Synthetic Test Data Generation (GDPR Compliance)
Requirement: The system shall provide a synthetic data generation tool that creates realistic but fake Swedish invoice data (personnummer, addresses, names) for use in staging and development environments, with zero production data copying.
Priority: CRITICAL
Synthetic Data Requirements:
Swedish Personnummer Generation:
Format: YYMMDD-XXXX
- YY: Year (00-99)
- MM: Month (01-12)
- DD: Day (01-31)
- XXXX: Last 4 digits with Luhn checksum
Generation Rules:
- Must pass Luhn algorithm validation
- Must NOT match any real personnummer
- Use test ranges: 19000101-19991231 (obviously fake dates)
- Flag as test: personnummer starts with "19"
...
Swedish Address Generation:
Street names: Random from Swedish street database
- Storgatan, Kungsgatan, Vasagatan, Drottninggatan...
- House numbers: 1-150
- Apartment: A-Z (optional)
Cities: Top 50 Swedish cities
- Stockholm, Göteborg, Malmö, Uppsala, Västerås...
Postal codes: Valid format (XXX XX) but non-existent ranges
- Use: 00X XX range (invalid but correct format)
Examples:
- Storgatan 45, 001 23 Stockholm
- Vasagatan 12 A, 002 45 Göteborg
...
Synthetic Invoice Data:
Invoice numbers: Test prefix "TEST-" + sequential
Amounts: Random between 100-5000 SEK
Consumption: Random 100-2000 kWh (residential realistic)
Metering points: Test range 735999999999999XXX
Email addresses: {firstname}.{lastname}@example-test.se
Phone numbers: +46701234XXX (test range)
...
Acceptance Criteria:
| Criterion | Validation Method | Expected Result |
|---|---|---|
| Generates valid personnummer (Luhn check) | Validate 1000 generated numbers | All pass Luhn validation |
| Personnummer obviously fake | Review generated numbers | All start with "19" (invalid birth years) |
| Addresses realistic but invalid | Check against real postal database | No matches found |
| Email addresses use test domain | Check generated emails | All @example-test.se |
| Phone numbers in test range | Check generated phones | All +467012340XX |
| Can generate 10K invoice batch | Generate full batch | 10K valid invoices |
| Zero real data in output | Scan for real personnummer patterns | No real data found |
| Reproducible (seed-based) | Generate twice with same seed | Identical output |
Dependencies:
- Swedish personnummer validation library
- Swedish postal code database (for validation, not generation)
- Random data generation library
Risks & Mitigation:
- Visual "TEST DATA" watermark on PDFs
- Automated scanning for real personnummer
- Code review of generation logic
- Long names, special characters
- Missing optional fields
- Various consumption patterns
- Screen sharing for production issues
- Never copy production data
- Comprehensive logs without PII
...