Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • Kivra digital mailbox integration (planned for Phase 2)
  • e-Faktura/PEPPOL electronic invoicing (planned for Phase 2)
  • Customer self-service portal (planned for Phase 2)
  • Payment processing and reconciliation (future phase)
  • Invoice amendments and credit notes (future phase)
  • Advanced analytics and business intelligence (future phase)
  • SMS distribution via Wiraya (future phase)

Documentation Standards:

  • All technical diagrams created using Draw.io VSCode extension
  • Diagrams include source XML for version control
  • All findings documented in Confluence (link above)
  • Visual structures for flows and relationships
  • Searchable, structured format for all technical details

...

Table of Contents

  1. Project Overview
  2. Business Requirements
  3. Functional Requirements
  4. Non-Functional Requirements
  5. Data Flow Diagrams
  6. API Specifications
  7. Error Handling & Validation
  8. Glossary
  9. Appendices

1. Project Overview

1.1 Executive Summary

EG Flow is a cloud-based multi-tenant system designed for Nordic utility companies to automate invoice processing, rendering, and multi-channel delivery. The system supports multiple vendor XML formats from existing billing systems and delivers invoices through various channels including postal mail, email, and future digital mailboxes.

Strategic Context - Nordic Utilities Market:

The Nordic electricity and district heating market is characterized by:

  • High digitalization: 95%+ of Swedish households have internet access
  • Regulatory complexity: GDPR, Swedish Data Protection Law, energy market regulations
  • Seasonal peaks: Invoice volumes concentrate in heating season (Oct-Mar)
  • Multi-vendor landscape: Utilities use diverse billing systems (Telinet, Karlskoga, Zynergy, etc.)
  • Print partner ecosystem: Established relationships with 21G and other postal services
  • Customer expectations: Fast delivery, digital options, environmental consciousness

Primary Business Goals:

  1. Reduce time-to-market for new utility company onboarding (weeks → days)
  2. Automate invoice processing from batch upload to delivery (days → hours)
  3. Support multi-vendor integration without custom development per client
  4. Enable flexible delivery channels with automatic fallback mechanisms
  5. Ensure regulatory compliance (GDPR, data residency, audit requirements)
  6. Provide operational visibility with real-time batch status and metrics

Target Users:

  • Primary: Utility companies (electricity, water, district heating, gas)
  • Secondary: End customers receiving invoices
  • Tertiary: EG system administrators and support staff
  • Tertiary: Print partners (21G) receiving bulk distribution files

1.2 Project Scope

In Scope for Phase 1

 Multi-tenant batch invoice processing with organization data isolation
 Multi-vendor XML support: GASEL (Telinet), XELLENT (Karlskoga), ZYNERGY (EG Software)
 Template-based rendering: Handlebars templates → HTML → PDF
 Email delivery: SendGrid integration with retry logic
 Postal delivery: 21G bulk integration with SFTP file drop
 RESTful API: Batch management, status tracking, template management
 Role-based access control: 6 distinct roles with organization boundaries
 Real-time status tracking: Batch and invoice-level progress monitoring
 Application monitoring: Application Insights with structured logging (Serilog)
 Multi-region deployment: West Europe (Sweden/Denmark), North Europe (Norway/Finland)
 GDPR compliance: Data subject rights, 7-year retention, audit logging
 Test data strategy: Synthetic data for staging, no production data copying

Out of Scope for Phase 1

❌ Kivra digital mailbox integration (Phase 2)
❌ e-Faktura/PEPPOL electronic invoicing (Phase 2)
❌ SMS distribution via Wiraya (Phase 2)
❌ Customer self-service portal (Phase 2)
❌ Payment processing

1.3 Success Metrics

CategoryMetricTargetMeasurement Method
Processing EfficiencyBatch processing time (100K invoices)< 2 hoursTimestamp tracking in metadata
System PerformanceAPI response time (p95)< 500msApplication Insights
ReliabilitySystem uptime99.9%Azure Monitor
QualityBatch success rate> 99.5%(completed/total) × 100
DeliveryEmail delivery rate> 95%SendGrid analytics
DeliveryPostal delivery success> 98%21G confirmation tracking
ScaleMonthly processing capacity10M+ invoicesSystem throughput monitoring
User SatisfactionCustomer satisfaction score> 4.5/5Quarterly survey
SupportSupport ticket volume< 50/monthJira ticket tracking

2. Business Requirements

2.0 Priority Level Definitions

PriorityDefinitionGo-Live ImpactStakeholder ApprovalExample
CRITICALCore system functionality, data integrity, legal compliance, securityBLOCKING - Cannot go live without thisAll stakeholders requiredAuthentication, GDPR, data isolation
HIGHPrimary business value, significant operational impactBLOCKING - Should not go live without thisProduct Owner + Technical ArchitectBatch processing, multi-vendor XML
MEDIUMImportant for operations, workarounds exist temporarilyNON-BLOCKING - Can go live with plan to completeProduct Owner approvalTemplate management UI, postal integration
LOWNice to have, minimal business impact, future optimizationNON-BLOCKING - Defer to Phase 2Product Owner decisionAdvanced filtering, custom dashboards

2.1 BR-001: Multi-Tenant Organization Support

Requirement: The system shall support multiple independent utility companies (organizations) with complete data isolation at the blob storage, processing, and user access levels.

Business Value:

  • Enables SaaS business model for Nordic utilities market
  • Reduces per-customer infrastructure costs by 60-70%
  • Accelerates new customer onboarding (weeks → days)
  • Increases addressable market across Sweden, Norway, Denmark, Finland

Priority: CRITICAL

Nordic Market Context: Swedish utilities market has ~150 electricity suppliers, ~290 district heating companies. Multi-tenancy is essential for market penetration.

Acceptance Criteria:

CriterionMeasurement MethodTest CaseTarget
System supports 50+ concurrent organizationsLoad test with 50 organizations uploading batches simultaneouslyTC-001All batches process successfully
Each organization has isolated blob storageVerify container naming: {org-id}-invoices-{year}TC-002No cross-org container access
Users cannot access other organizations' dataAttempt API calls to different org-id with valid tokenTC-003All return 403 Forbidden
Each organization configures own brandingUpload logo, colors; verify in rendered PDFTC-004Branding applied correctly
Each organization defines delivery channelsConfigure email priority; verify email sent firstTC-005Channel priority honored
Organization data never shared in logsReview Application Insights logs for data leakageTC-006No PII in logs

Dependencies:

  • Azure Blob Storage with container-per-organization strategy
  • PostgreSQL schema with user_organization_roles table
  • Middleware for organization context extraction and validation
  • Role-based access control implementation

Risks & Mitigation (Nordic Context):

RiskLikelihoodImpactMitigation StrategyOwner
Data leakage between organizationsLOWCRITICAL- Middleware enforces org boundaries
- Automated cross-org access testing
- Penetration testing by external auditor
- GDPR-compliant architecture review
Security Officer
Performance degradation with 50+ tenantsMEDIUMHIGH- Blob storage auto-scales
- PostgreSQL connection pooling
- Indexed queries on organization_id
- Load testing with 100 organizations
Technical Architect
Swedish data residency requirementsLOWHIGH- West Europe primary region
- Data residency enforced in org config
- No cross-border data transfer
Compliance

2.2 BR-002: Multi-Vendor XML Format Support

Requirement: The system shall process invoice batch files from multiple vendor billing systems (GASEL/Telinet, XELLENT/Karlskoga, ZYNERGY/EG Software) with automatic format detection and transformation to canonical JSON.

Business Value:

  • Eliminates integration barrier for new customers (any billing system supported)
  • Reduces onboarding time by 80% (no custom integration development)
  • Increases market addressability to all Nordic utilities regardless of billing system
  • Enables vendor-agnostic downstream processing

Priority: HIGH

Nordic Market Context: Top 3 billing systems in Swedish utilities market: Telinet (EDIEL format), Karlskoga/Xellent (OIOXML), EG Zynergy (proprietary). Supporting these covers ~70% of market.

Acceptance Criteria:

CriterionMeasurement MethodTest CaseTarget
System auto-detects GASEL formatTest with 50 GASEL XML samplesTC-010100% accuracy
System auto-detects XELLENT formatTest with 50 XELLENT XML samplesTC-011100% accuracy
System auto-detects ZYNERGY formatTest with 50 ZYNERGY XML samplesTC-012100% accuracy
All formats transform to canonical JSONParse each format, verify JSON schemaTC-013All required fields present
XML validation against vendor XSDValidate sample files against schemasTC-014Schema validation passes
Clear error for unsupported formatsUpload unknown format XMLTC-015Returns 415 with vendor list
Detection within 1 secondPerformance test with 100MB filesTC-016< 1 second

Dependencies:

  • Schema registry with field mappings for each vendor
  • XML parsing library supporting complex namespaces
  • XSD schema files from vendors (Telinet, Karlskoga, EG)
  • Canonical JSON schema definition

Risks & Mitigation (Nordic Context):

RiskLikelihoodImpactMitigation StrategyOwner
Vendor XML schema changes without noticeMEDIUMHIGH- Version all schemas (v1.0, v1.1, etc.)
- Support multiple versions simultaneously
- 3-month deprecation notice in vendor contracts
- Automated schema change detection
- Direct vendor communication channels
Product Owner
EDIEL standard evolutionMEDIUMMEDIUM- Monitor Ediel.org for updates
- Participate in Nordic Ediel working groups
- Backward compatibility for 2 versions
- Phased schema migration
Technical Architect
Complex namespace handling (OIOXML)LOWMEDIUM- Use System.Xml.Linq for namespace support
- XmlNamespaceManager for prefixes
- Extensive unit testing per vendor
- Schema mapping documentation
Dev Team
Incomplete field mappingsMEDIUMMEDIUM- Comprehensive mapping validation
- Custom fields dictionary for unmapped data
- Vendor format validation during onboarding
- Optional field graceful degradation
Dev Team
Energiforsk format variationsLOWLOW- Document accepted variations
- Lenient parsing mode option
- Warning system for non-critical deviations
Product Owner

2.3 BR-003: Batch Invoice Processing

Requirement: The system shall process batch invoice files containing up to 100,000 invoices with parallel processing, retry logic, and granular status tracking.

Business Value:

  • Enables high-volume monthly invoice runs (typical Swedish utility: 50K-200K customers)
  • Reduces manual processing effort by 95%
  • Improves time-to-customer (days → hours)
  • Enables predictable SLA commitments to customers

Priority: HIGH

Nordic Market Context: Monthly invoice cycles concentrated in first/last week of month. Heating season (Oct-Mar) has 40% higher volumes. System must handle seasonal peaks.

Acceptance Criteria:

CriterionMeasurement MethodTest CaseTarget
Single batch supports 100,000 invoicesUpload 100K batch, verify all processedTC-020All invoices processed
Batches processed asynchronouslyAPI returns 202 immediately, processing continuesTC-021< 500ms API response
Real-time progress trackingPoll status API during processingTC-022Updates every 30 seconds
Failed items don't block batchIntroduce 10 errors in 1000-item batchTC-023990 succeed, 10 fail independently
Retry mechanism (3 attempts)Force temporary failure, verify retriesTC-0243 retries then poison queue
Processing completes within 2 hours (100K)Load test with 100K invoice batchTC-025<= 120 minutes
Supports XML, JSON, CSV formats (Phase 1: XML only)Upload each format typeTC-026XML fully supported

Dependencies:

  • Azure Storage Queues for message passing
  • Worker auto-scaling (Container Apps with KEDA)
  • Blob storage for batch source files and processed invoices
  • Batch metadata storage with real-time updates

Risks & Mitigation (Nordic Context):

RiskLikelihoodImpactMitigation StrategyOwner
Processing timeout during heating season peaks (Oct-Mar)MEDIUMHIGH- Pre-warm workers 1st/last week of month
- Priority queue for SLA customers
- Scheduled off-peak processing windows
- Customer communication about peak times
Operations Manager
Memory constraints with large XML files (>50MB)MEDIUMMEDIUM- Stream-based XML parsing (XmlReader)
- Chunk processing for large batches
- File size monitoring and alerts
- 100MB hard limit with split guidance
Technical Architect
Disk space exhaustion on workersLOWMEDIUM- Ephemeral storage cleanup after processing
- Blob storage only for persistence
- Worker instance monitoring
Operations Manager
Queue message 64KB limit exceededMEDIUMMEDIUM- Store invoice JSON in blob, queue has reference only
- Message payload validation
- Compression for large messages
Technical Architect

2.4 BR-004: Template-Based Invoice Rendering

Requirement: The system shall generate PDF and HTML invoices using organization-specific Handlebars templates with dynamic data binding and brand customization.

Business Value:

  • Enables brand consistency across all customer communications
  • Supports regulatory requirements (Swedish Energy Markets Inspectorate guidelines)
  • Allows per-organization customization without code changes
  • Future-proofs for multi-language support (Swedish, Norwegian, Danish, Finnish)

Priority: HIGH

Nordic Market Context: Swedish regulations require specific invoice information (consumption details, tax breakdown, grid owner, metering point). Templates must support regulatory compliance.

Acceptance Criteria:

CriterionMeasurement MethodTest CaseTarget
Organizations upload custom Handlebars templatesUpload HTML template via API (future) or blobTC-030Template stored successfully
Templates support dynamic data bindingRender with test invoice dataTC-031All fields populated
PDF generation from HTML with complianceGenerate PDF, verify format and contentTC-032A4, readable, complete
Template versioning supportedCreate v2.0.0, verify old batches use v1.0.0TC-033Version isolation working
Templates updateable without affecting in-flight batchesUpdate template while batch processesTC-034In-flight uses old version
Template validation before activationUpload template with missing variableTC-035Validation error returned
Organization branding appliedLogo, colors, fonts in rendered PDFTC-036Branding visible
Swedish regulatory fields requiredVerify mätpunkt, grid owner, consumption presentTC-037All required fields shown

Dependencies:

  • Handlebars.Net template engine library
  • PDF generation library (Playwright with headless Chromium or IronPDF)
  • Blob storage for template files
  • Template metadata storage with version tracking

Risks & Mitigation (Nordic Context):

RiskLikelihoodImpactMitigation StrategyOwner
Template rendering performance bottleneckHIGHHIGH- Compiled template caching (24-hour TTL)
- Parallel rendering within 32-item batches
- POC testing: render 10K invoices in < 5 min
- Horizontal scaling of BatchItemWorkers
Technical Architect
PDF generation quality issues (Swedish characters)MEDIUMMEDIUM- UTF-8 encoding throughout
- Font embedding for åäö characters
- Visual regression testing
- Sample PDF review with Swedish content
QA Team
Swedish Energy Markets Inspectorate complianceLOWCRITICAL- Legal review of template requirements
- Required fields checklist in template validation
- Compliance testing with regulatory samples
- Annual regulatory update review
Legal/Compliance
Template injection attacksLOWCRITICAL- Sandboxed Handlebars execution
- No {{eval}} or {{exec}} helpers
- Template sanitization
- Security code review
Security Officer

2.5 BR-005: Multi-Channel Delivery with Nordic Integration

Requirement: The system shall deliver invoices through multiple channels (email, postal, future: Kivra, e-Faktura, SMS) with configurable priority, automatic fallback, and integration with Nordic delivery partners.

Business Value:

  • Increases delivery success rate to >98% (vs ~92% email-only)
  • Reduces returned mail costs (postal fallback)
  • Meets customer preferences (digital-first, postal backup)
  • Enables compliance with Swedish "rätt till pappersfaktura" (right to paper invoice)
  • Future-proofs for digital mailbox mandate discussions

Priority: HIGH

Nordic Market Context: Swedish law requires postal option for all customers. Kivra has 4.2M users in Sweden (40% population). E-faktura standard for B2B. Multi-channel is regulatory and competitive necessity.

Acceptance Criteria:

CriterionMeasurement MethodTest CaseTarget
Email delivery via SendGridSend 1000 test invoicesTC-040>95% delivered
Postal delivery via 21G bulk SFTPCreate zip, upload to 21G SFTPTC-041File accepted by 21G
Channel priority configurable per orgSet priority [email, postal], verify orderTC-042Email attempted first
Automatic fallback on failureForce email failure, verify postal attemptedTC-043Postal triggered automatically
Delivery status tracked per invoiceCheck invoice metadata after deliveryTC-044Status and timestamps recorded
Retry logic for transient failuresSimulate SendGrid 429 rate limitTC-045Retries with backoff
Delivery confirmation loggedVerify audit log entriesTC-046All deliveries logged
21G bulk processing at scheduled timesVerify postal queue processed 12:00 and 20:00TC-047Batches sent on schedule

Dependencies:

  • SendGrid account with Nordic sender reputation
  • 21G SFTP server access and credentials
  • Azure Key Vault for credential storage
  • Postal queue processing service (scheduled workers)
  • ZIP file generation for 21G format

Risks & Mitigation (Nordic Context):

RiskLikelihoodImpactMitigation StrategyOwner
SendGrid Nordic deliverability issuesMEDIUMHIGH- Dedicated IP for Nordic sending
- SPF/DKIM/DMARC configuration
- Sender reputation monitoring
- Backup: Azure Communication Services
- Gradual ramp-up for new organizations
Operations Manager
21G SFTP connectivity issuesLOWHIGH- Retry logic with exponential backoff
- Dual SFTP credentials (primary/backup)
- Alert on connection failure
- 21G SLA monitoring
- Manual upload procedure documented
Operations Manager
Postal delivery delays (Swedish postal challenges)MEDIUMMEDIUM- Set customer expectations (5-7 business days)
- Track 21G processing confirmations
- Escalation process for delays>10 days
- Alternative print partner identified
Product Owner
Email spam filtering (Swedish ISPs)MEDIUMMEDIUM- Warm up sending IP gradually
- Monitor bounce rates by ISP
- Whitelist requests to major ISPs (Telia, Tele2, Telenor)
- Plain text + HTML multi-part emails
Operations Manager
Kivra API rate limits (future)MEDIUMMEDIUM- Queue-based sending
- Respect Kivra rate limits (documented in API)
- Fallback to email if Kivra unavailable
Technical Architect

2.6 BR-006: Real-Time Batch Status Visibility

Requirement: Users shall view batch processing status, progress statistics, and individual invoice statuses in real-time through the API with granular breakdown by processing stage and delivery channel.

Business Value:

  • Reduces support inquiries by 60% (self-service status checking)
  • Enables proactive issue resolution before customer complaints
  • Improves operational transparency and trust
  • Provides audit trail for service level agreement (SLA) tracking

Priority: MEDIUM

Nordic Market Context: Utility operations teams work standard hours (08:00-17:00). Real-time visibility enables remote monitoring and reduces after-hours escalations.

Acceptance Criteria:

CriterionMeasurement MethodTest CaseTarget
Batch status via API (queued/processing/completed/failed)GET /batches/{id} during processingTC-050Status reflects reality
Progress percentage accurateCompare reported % to actual processed countTC-051Within 1% accuracy
Individual invoice status queryableGET /batches/{id}/items with status filterTC-052Returns correct filtered list
Statistics include success/failure countsVerify statistics.successfulItems matches realityTC-053Exact match
Estimated completion time providedCheck estimatedCompletionAt during processingTC-054Within 20% of actual
Failed invoices listed with error detailsQuery items with status=failedTC-055Error messages present
Status updates within 30 secondsProcess batch, poll status APITC-056Updates every ≤30 seconds
Delivery channel breakdown shownVerify deliveryChannelBreakdown matches actualTC-057Accurate breakdown

Dependencies:

  • Batch metadata updates from all workers
  • Real-time statistics calculation (incremental updates)
  • Blob storage access for metadata reads
  • API caching strategy for frequently-polled batches

Risks & Mitigation:

RiskLikelihoodImpactMitigation StrategyOwner
Performance impact of frequent metadata updatesMEDIUMMEDIUM- ETag-based optimistic concurrency
- Batch statistics updates (not per-invoice)
- Caching layer for status API
- Throttle updates to max 1/minute
Technical Architect
Stale status data (eventual consistency)MEDIUMLOW- Document 30-second update SLA
- Timestamp on status response
- Retry-After header for frequent polls
- WebSocket push for Phase 2
Product Owner

2.7 BR-007: Security & Access Control

Requirement: The system shall implement OAuth 2.0 authentication with Microsoft Entra ID, role-based access control with 6 distinct roles, organization-level data isolation, and comprehensive audit logging compliant with Swedish and EU security standards.

Business Value:

  • Ensures GDPR Article 32 compliance (security of processing)
  • Prevents unauthorized data access and potential €20M GDPR fines
  • Enables customer trust and enterprise sales (security requirements in tenders)
  • Supports SOC 2 Type II compliance for future certification

Priority: CRITICAL

Nordic Market Context: Swedish utilities handle sensitive customer data (personnummer, consumption patterns). GDPR fines can reach 4% of annual turnover. Enterprise security is non-negotiable.

Acceptance Criteria:

CriterionMeasurement MethodTest CaseTarget
OAuth 2.0 with Entra ID authenticationAttempt API access with/without tokenTC-060401 without token, 200 with valid token
6 roles implementedVerify all roles in PostgreSQLTC-061All 6 roles present
Users access only their organization dataUser from Org A attempts Org B accessTC-062403 Forbidden
All data access logged to audit trailReview audit_logs table after operationsTC-063All actions logged
API authentication required (all endpoints except /health)Call endpoints without tokenTC-064401 on all except /health
MFA enforced for admin rolesAdmin login without MFATC-065MFA challenge presented
API key rotation every 90 daysCheck key age in Key VaultTC-066Alerts at 80 days
Failed login lockout (5 attempts = 15 min)Attempt 6 failed loginsTC-067Account locked

Dependencies:

  • Microsoft Entra ID tenant configuration
  • PostgreSQL schema for users, roles, user_organization_roles
  • Azure Key Vault for secrets management
  • Audit logging infrastructure
  • MFA provider integration

Risks & Mitigation (Nordic/EU Context):

RiskLikelihoodImpactMitigation StrategyOwner
GDPR non-compliance penaltyLOWCRITICAL- Annual GDPR compliance audit
- Privacy Impact Assessment (PIA) completed
- Data Protection Officer (DPO) review
- Datainspektionen (Swedish DPA) guidelines followed
- EU Standard Contractual Clauses for vendors
Legal/Compliance
Unauthorized personnummer accessLOWCRITICAL- Personnummer encrypted at rest
- Access logging for all PII fields
- Role-based field-level access control
- Swedish Personal Data Act compliance
Security Officer
Authentication complexity delays onboardingMEDIUMMEDIUM- Entra ID B2B invitation flow
- Self-service org admin provisioning
- Clear onboarding documentation
- SSO with customer Entra ID tenants
Product Owner
Audit log storage costs (7-year retention)MEDIUMLOW- PostgreSQL for hot logs (1 year)
- Archive to blob after 1 year
- Compressed JSON format
- Query optimization
Technical Architect

2.8 BR-008: GDPR & Swedish Data Protection Compliance

Requirement: The system shall comply with GDPR and Swedish Data Protection Law including all data subject rights, lawful processing basis, 7-year retention for invoices, data minimization, and Nordic data residency.

Business Value:

  • Avoids GDPR fines (up to €20M or 4% of annual turnover)
  • Enables enterprise sales (GDPR compliance is tender requirement)
  • Builds customer trust and brand reputation
  • Supports Swedish utility regulatory requirements (Energimarknadsinspektionen)

Priority: CRITICAL

Nordic Market Context: Swedish Datainspektionen actively enforces GDPR. Recent fines in telecom/energy sector for inadequate security. Data residency within EU required for many public utility contracts.

Acceptance Criteria:

CriterionMeasurement MethodTest CaseTarget
Right to access: Data export APIRequest customer data exportTC-070JSON export with all customer invoices
Right to erasure: AnonymizationRequest customer deletion, verify PII removedTC-071Personnummer, name, email redacted
7-year invoice retentionCheck lifecycle policy configTC-072Invoices retained 7 years
Audit logging for all data accessQuery audit_logs for customer data accessTC-073All accesses logged
Data residency (Nordic countries only)Verify blob storage regionsTC-074West/North Europe only
Consent management for digital deliveryTrack customer delivery channel consentTC-075Consent recorded
Privacy policy complianceLegal reviewTC-076Approved by legal
Lawful basis documentedReview data processing inventoryTC-077Contract basis for invoice data

Dependencies:

  • Legal review of GDPR compliance approach
  • Data anonymization procedures and testing
  • Azure Blob lifecycle policies for retention
  • Data Processing Agreement (DPA) with Azure
  • Privacy Impact Assessment (PIA) completion

Risks & Mitigation (Swedish/EU Context):

RiskLikelihoodImpactMitigation StrategyOwner
Datainspektionen (Swedish DPA) auditLOWCRITICAL- Annual self-assessment against GDPR
- Document all processing activities (ROPA)
- DPO appointed and consulted
- Privacy by design in architecture
- Regular employee training
Legal/Compliance
Cross-border data transfer violationsLOWCRITICAL- All data stays in EU (Azure West/North Europe)
- No backup to non-EU regions
- Vendor contracts include EU Standard Clauses
- Azure compliance certifications verified
Legal/Compliance
Personnummer handling violationsMEDIUMCRITICAL- Personnummer only in encrypted fields
- Access logging for all reads
- Minimal retention (7 years, then deletion)
- Swedish Personal Data Act guidelines
- No personnummer in URLs or logs
Security Officer
Right to erasure vs 7-year retention conflictMEDIUMHIGH- Anonymization instead of deletion (legal basis: accounting law)
- Legal opinion obtained
- Balance data subject rights with legal obligations
- Documented in privacy policy
Legal/Compliance
Third-party processor compliance (SendGrid, 21G)MEDIUMHIGH- Data Processing Agreements (DPA) signed
- Sub-processor list maintained
- Regular compliance audits
- EU-based data centers only
Legal/Compliance

2.9 BR-009: Operational Monitoring & Observability

Requirement: The system shall provide comprehensive monitoring through Application Insights, structured logging with Serilog, real-time dashboards, and automated alerting to enable 24/7 operational visibility and proactive issue resolution.

Business Value:

  • Reduces mean time to detection (MTTD) from hours to minutes
  • Enables proactive issue resolution before customer impact
  • Supports SLA compliance and reporting
  • Reduces on-call burden through automated diagnostics

Priority: HIGH

Nordic Market Context: Swedish customers expect high service quality. Many utilities have SLA commitments (99.9% uptime). Winter heating season requires 24/7 monitoring.

Acceptance Criteria:

CriterionMeasurement MethodTest CaseTarget
Application Insights integrated (all services)Verify telemetry in Azure portalTC-080All services sending data
Structured logging with correlation IDsTrace request across servicesTC-081Same correlation ID
Custom metrics trackedVerify metrics in dashboardTC-082Batch, delivery, queue metrics present
Dashboards refresh every 5 minutesCheck dashboard timestampTC-083≤ 5 minutes old
Dashboards accessible to authorized usersLogin as different rolesTC-084Access granted appropriately
Critical alerts trigger within 5 minutesSimulate high error rateTC-085Alert received within 5 min
Alert escalation after 15 min unacknowledgedDon't acknowledge alertTC-086Escalation triggered
PII masked in logsSearch logs for personnummerTC-087No personnummer found

Required Dashboards:

  1. Operations Dashboard: Active batches, queue depths, worker counts, error rate, health status
  2. Performance Dashboard: API latency, processing times, PDF generation, delivery latency
  3. Business Dashboard: Invoices processed, delivery breakdown, top organizations
  4. Vendor Dashboard: Batches by format, parsing success rates

Dependencies:

  • Application Insights workspace
  • Alert action groups (email, SMS)
  • Dashboard configuration and permissions
  • On-call rotation schedule

Risks & Mitigation (Nordic Context):

RiskLikelihoodImpactMitigation StrategyOwner
Alert fatigue from false positivesHIGHMEDIUM- Tuned thresholds based on baseline
- 2-week monitoring before alerts live
- Weekly alert review meetings
- Alert suppression during maintenance
Operations Manager
Insufficient monitoring during off-hoursMEDIUMHIGH- 24/7 on-call rotation
- Automated incident creation in Jira
- Runbook for common issues
- PagerDuty integration for escalation
Operations Manager
Log storage costs exceeding budgetLOWMEDIUM- 90-day log retention
- Sampling for high-volume logs
- Cost alerts at 80% budget
- Archive old logs to blob
Technical Architect
Missing critical metricsMEDIUMMEDIUM- Monthly metrics review
- Stakeholder feedback on dashboard usefulness
- Iterate dashboard based on incidents
Operations Manager

2.10 BR-010: Test Data Strategy (GDPR Compliance)

Requirement: The system shall use only synthetic, non-sensitive test data in all non-production environments, with zero copying of production data to staging, to ensure GDPR compliance and prevent data breaches.

Business Value:

  • Ensures GDPR Article 5(1)(b) compliance (purpose limitation)
  • Prevents catastrophic data breach from non-production environments
  • Reduces storage and transfer costs (no production data duplication)
  • Enables offshore development team participation (India)

Priority: CRITICAL

Nordic Market Context: Swedish Datainspektionen prohibits production data in test environments. Indian development team cannot access personnummer data. European team must lead production debugging.

Acceptance Criteria:

CriterionMeasurement MethodTest CaseTarget
Zero production data in stagingAudit staging blob storageTC-090No real personnummer found
Synthetic data generator creates realistic batchesGenerate 10K synthetic invoicesTC-091Valid structure, fake PII
European team handles production issuesProduction bug workflow documentedTC-092No PII sent to offshore team
Reproduction scenarios use synthetic dataCreate synthetic scenario from production bugTC-093Issue reproducible
Staging data clearly marked as testVisual indicators in UI, filename patternsTC-094Test data obvious
Production access restricted (European team only)Attempt production access from India VPNTC-095Access denied (geo-fenced)
Synthetic data follows Swedish patternsReview generated personnummer, addressesTC-096Realistic but invalid

Dependencies:

  • Synthetic data generation tool (Swedish personnummer, addresses, etc.)
  • Staging environment completely isolated from production
  • Network policies restricting production access
  • Documentation of production issue workflow

Risks & Mitigation (EU/India Context):

RiskLikelihoodImpactMitigation StrategyOwner
Accidental production data leak to stagingLOWCRITICAL- Automated scanning for real personnummer patterns
- No database copying tools
- Production data access audit logs
- Annual security training
Security Officer
Offshore team delays due to data restrictionsMEDIUMMEDIUM- European team creates reproduction scenarios
- Comprehensive synthetic data sets
- Well-documented issue reports
- Pair programming for production issues
Product Owner
Synthetic data unrealistic (testing gaps)MEDIUMMEDIUM- Generate from real data distributions (anonymized)
- Edge cases library (long names, special chars)
- Swedish address/personnummer validation
- Regular synthetic data quality review
QA Team
Production debugging bottleneck (European team only)MEDIUMMEDIUM- European team on-call 24/7
- Comprehensive monitoring reduces debugging needs
- Runbooks for common issues
- Knowledge transfer to European team
Operations Manager

2.11 BR-011: Invoice Download API

Requirement: Users shall be able to download generated invoices (PDF and HTML) through authenticated API endpoints with access control and audit logging.

Business Value:

  • Enables customer service to retrieve invoices for disputes
  • Supports reprint requests without reprocessing
  • Facilitates regulatory compliance (invoice provision upon request)
  • Enables future customer portal integration

Priority: HIGH

Acceptance Criteria:

CriterionMeasurement MethodTest CaseTarget
API endpoint to download PDF by invoice IDGET /invoices/{id}/pdfTC-100PDF file returned
API endpoint to download HTML by invoice IDGET /invoices/{id}/htmlTC-101HTML returned
Download links in batch item responseGET /batches/{id}/items/{itemId}TC-102pdfUrl and htmlUrl present
Access control: org users onlyUser from Org A downloads Org B invoiceTC-103403 Forbidden
Download generates audit log entryDownload invoice, check audit_logsTC-104Entry created
Content-Disposition header for PDFCheck HTTP response headersTC-105Filename in header
Rate limiting on downloadsDownload 1000 invoices rapidlyTC-106Rate limit applied

Dependencies:

  • Blob storage SAS token generation
  • Access control middleware
  • Audit logging service

Risks & Mitigation:

RiskLikelihoodImpactMitigation StrategyOwner
Unauthorized invoice accessLOWHIGH- Organization boundary checks
- Audit all downloads
- Rate limiting
- Anomaly detection
Security Officer
Bandwidth costs for large volumesMEDIUMLOW- CDN for frequently accessed invoices
- Compression
- Monitoring
Technical Architect

2.12 BR-012: Batch History & Search

Requirement: Users shall be able to search, filter, and list historical batches for their organization with support for date ranges, status filters, and vendor format filtering.

Business Value:

  • Enables troubleshooting and root cause analysis
  • Supports operational reporting and SLA tracking
  • Facilitates audit compliance
  • Improves user experience with search capabilities

Priority: MEDIUM

Acceptance Criteria:

CriterionMeasurement MethodTest CaseTarget
List batches with paginationGET /batches?page=1&pageSize=50TC-110Paginated list returned
Filter by date rangeGET /batches?from=2025-11-01&to=2025-11-30TC-111Only Nov batches returned
Filter by statusGET /batches?status=completedTC-112Only completed batches
Filter by vendor formatGET /batches?vendorCode=GASELTC-113Only GASEL batches
Search by batch nameGET /batches?search=NovemberTC-114Name contains "November"
Sort by upload/completion dateGET /batches?sortBy=uploadedAt&order=descTC-115Newest first
Returns last 90 days by defaultGET /batches (no filters)TC-116Last 90 days only
Access restricted to organizationUser queries batches from other orgTC-117Empty results

Dependencies:

  • Batch metadata indexing strategy
  • Query performance optimization
  • Blob storage metadata queries or search index

Risks & Mitigation:

RiskLikelihoodImpactMitigation StrategyOwner
Slow queries with large batch historyMEDIUMMEDIUM- Index on organization_id, date
- Pagination required
- Consider Azure Cognitive Search for Phase 2
Technical Architect

2.13 BR-013: Notification System

Requirement: The system shall send email notifications to designated organization contacts when batch processing completes successfully or fails, including summary statistics and error reports.

Business Value:

  • Reduces manual status checking effort by 80%
  • Enables proactive issue resolution
  • Improves customer satisfaction with timely updates
  • Supports SLA monitoring and reporting

Priority: MEDIUM

Acceptance Criteria:

CriterionMeasurement MethodTest CaseTarget
Email sent on batch completionComplete batch, verify email receivedTC-120Email received within 5 min
Email sent on batch failureForce batch failure, verify emailTC-121Email received within 5 min
Email includes summary statisticsReview email contentTC-122Total, success, failed counts present
Email includes error report for failuresReview failure emailTC-123Failed items listed with reasons
Recipients configurable per organizationUpdate org config, verify new recipientTC-124Email to correct recipients
Email template professional and brandedReview email designTC-125EG branding applied
Links to batch status in emailClick link in emailTC-126Opens to batch detail

Dependencies:

  • Email service (SendGrid or Azure Communication Services)
  • Email templates (HTML)
  • Organization configuration for recipients

Risks & Mitigation:

RiskLikelihoodImpactMitigation StrategyOwner
Email deliverability to org adminsMEDIUMMEDIUM- SPF/DKIM/DMARC for notification domain
- Fallback to SMS for critical alerts
Operations Manager
Notification fatigueMEDIUMLOW- Configurable notification thresholds
- Digest emails for multiple batches
- Opt-in for verbose notifications
Product Owner

3. Functional Requirements

3.1 FR-001: Batch File Upload

Requirement: Users shall upload batch invoice files through a RESTful API with automatic vendor format detection, file validation, and storage in organization-specific blob containers with month/day hierarchy.

Priority: HIGH

API Endpoint: POST /organizations/{organizationId}/batches

Request Format:

POST /v1/organizations/123e4567-e89b-12d3-a456-426614174000/batches HTTP/1.1
Host: api.egflow.com
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbG...
Content-Type: multipart/form-data; boundary=----WebKitFormBoundary

------WebKitFormBoundary
Content-Disposition: form-data; name="file"; filename="invoices_nov.xml"
Content-Type: application/xml

<?xml version="1.0" encoding="UTF-8"?>
<InvoiceBatch xmlns="urn:ediel:se:electricity:invoice:1.0">
  ...
</InvoiceBatch>
------WebKitFormBoundary
Content-Disposition: form-data; name="metadata"
Content-Type: application/json

{
  "batchName": "Invoice_November_2025",
  "priority": "normal"
}
------WebKitFormBoundary--

...

Response Format (201 Created):

{
  "success": true,
  "data": {
    "batchId": "550e8400-e29b-41d4-a716-446655440000",
    "organizationId": "123e4567-e89b-12d3-a456-426614174000",
    "status": "uploaded",
    "uploadedAt": "2025-11-21T10:30:00Z",
    "fileInfo": {
      "fileName": "invoices_nov.xml",
      "fileSize": 15728640,
      "checksum": "sha256:a3d5e7f9b2c4d6e8...",
      "detectedFormat": "GASEL"
    },
    "blobPath": "acme-batches-2025/11/21/550e8400.../source.xml"
  },
  "metadata": {
    "timestamp": "2025-11-21T10:30:00Z",
    "requestId": "req-abc-123",
    "apiVersion": "1.0"
  }
}

...

Blob Storage Path Pattern (Updated per Nov 21 decision):

{org-id}-batches-{year}/{month}/{day}/{batch-id}/source.xml

...

Acceptance Criteria:

CriterionValidation MethodTest DataExpected Result
Accepts XML files up to 100MBUpload 100MB XML filegasel_100mb.xml201 Created
Validates file is well-formed XMLUpload malformed XMLinvalid.xml400 INVALID_XML
Stores in org-specific container with month/day pathUpload, verify blob pathvalid.xmlPath: {org}/2025/11/21/{id}/source.xml
Returns unique UUID batch IDUpload, check batchId formatvalid.xmlValid UUID v4
Detects GASEL formatUpload GASEL XMLgasel_sample.xmldetectedFormat: "GASEL"
Detects XELLENT formatUpload XELLENT XMLxellent_sample.xmldetectedFormat: "XELLENT"
Detects ZYNERGY formatUpload ZYNERGY XMLzynergy_sample.xmldetectedFormat: "ZYNERGY"
Calculates SHA-256 checksumUpload, verify checksumvalid.xmlChecksum matches file
Requires Batch Operator roleUpload without rolevalid.xml403 ACCESS_DENIED
Rate limited (10 uploads/hour/org)Upload 11 files in 1 hourvalid.xml x1111th returns 429
File size > 100MB rejectedUpload 101MB filelarge.xml413 FILE_TOO_LARGE
Non-XML files rejectedUpload PDF fileinvoice.pdf415 UNSUPPORTED_FORMAT

Validation Rules:

FieldRuleError CodeError Message
fileRequiredVALIDATION_ERRORFile is required
file.size1KB ≤ size ≤ 100MBFILE_TOO_LARGEFile must be between 1KB and 100MB
file.contentTypeMust be application/xml or text/xmlINVALID_CONTENT_TYPEFile must be XML format
file.contentWell-formed XML (parseable)INVALID_XMLXML file is not well-formed. Line {line}, Column {column}: {error}
metadata.batchName1-255 characters, no path separatorsVALIDATION_ERRORBatch name must be 1-255 characters without / or \
metadata.priorityMust be "normal" or "high"VALIDATION_ERRORPriority must be 'normal' or 'high'

Error Scenarios:

ScenarioHTTPError CodeMessageDetails
File too large413FILE_TOO_LARGEFile exceeds 100MB limit{ "fileSize": 105906176, "limit": 104857600 }
Invalid XML400INVALID_XMLXML file is not well-formed{ "line": 142, "column": 23, "error": "Unexpected end tag" }
Missing token401UNAUTHORIZEDMissing or invalid authentication token{ "suggestion": "Include Authorization: Bearer {token} header" }
Insufficient permissions403ACCESS_DENIEDUser does not have Batch Operator role{ "requiredRole": "Batch Operator", "userRoles": ["Read-Only"] }
Rate limit exceeded429RATE_LIMIT_EXCEEDEDToo many batch uploads. Limit: 10 per hour{ "limit": 10, "window": "1 hour", "retryAfter": "2025-11-21T11:30:00Z" }

3.2 FR-002: Batch Processing Initiation

Requirement: Users shall initiate batch processing through API, which enqueues the batch to batch-upload-queue for asynchronous processing by ParserService.

Priority: HIGH

API Endpoint: POST /organizations/{organizationId}/batches/{batchId}/start

Request Format:

POST /v1/organizations/{orgId}/batches/{batchId}/start HTTP/1.1
Authorization: Bearer {token}
Content-Type: application/json

{
  "validationMode": "strict"
}

...

Response Format (202 Accepted):

{
  "success": true,
  "data": {
    "batchId": "550e8400-e29b-41d4-a716-446655440000",
    "status": "queued",
    "queuedAt": "2025-11-21T10:35:00Z",
    "estimatedProcessingTime": "15-30 minutes",
    "queuePosition": 2,
    "queueName": "batch-upload-queue"
  }
}

...

Processing Flow:

1. Validate batch exists and status is "uploaded"
2. Create message in batch-upload-queue with batch metadata
3. Update batch status to "queued" in blob metadata
4. Return 202 Accepted with queue position
5. ParserService picks up message asynchronously

...

Acceptance Criteria:

CriterionValidation MethodTest DataExpected Result
Batch must be in "uploaded" statusStart already-processing batchprocessing-batch-id409 CONFLICT
Creates message in batch-upload-queueVerify queue message createdvalid-batch-idMessage present
Updates batch status to "queued"Check batch metadata after startvalid-batch-idstatus: "queued"
Returns estimated time based on queueCheck with empty vs full queuevariousTime varies with queue depth
Idempotent (duplicate calls safe)Call /start twicevalid-batch-idBoth return 202, one processes
Returns current queue positionVerify position accuracyvalid-batch-idPosition matches queue
Requires Batch Operator roleCall without rolevalid-batch-id403 ACCESS_DENIED
Supports "strict" and "lenient" validationSet validationMode=lenientvalid-batch-idMode stored in message
Queue full returns 503Start when queue depth >10000valid-batch-id503 SERVICE_UNAVAILABLE

Validation Rules:

CheckRuleError CodeHTTPAction
Batch existsMust exist in blob storageRESOURCE_NOT_FOUND404Return error immediately
Batch ownershipMust belong to organization in pathACCESS_DENIED403Return error immediately
Batch statusMust be "uploaded", not "queued"/"processing"/"completed"PROCESSING_ERROR422Return error with current status
Organization activeOrganization.isActive = trueORGANIZATION_INACTIVE422Return error
Queue capacityQueue depth < 10,000SERVICE_UNAVAILABLE503Return with retry-after
User permissionUser has BatchOperator or higher roleACCESS_DENIED403Return error
Validation modeMust be "strict" or "lenient"VALIDATION_ERROR400Return error with allowed values

Error Scenarios:

ScenarioHTTPError CodeMessageDetailsUser Action
Batch not found404RESOURCE_NOT_FOUNDBatch does not exist{ "batchId": "{id}" }Verify batch ID
Already processing409CONFLICTBatch is already processing{ "currentStatus": "processing", "startedAt": "2025-11-21T10:00:00Z" }Wait for completion or cancel
Invalid status422PROCESSING_ERRORBatch cannot be started from current status{ "currentStatus": "completed", "allowedStatuses": ["uploaded"] }Re-upload batch
Queue at capacity503SERVICE_UNAVAILABLESystem at capacity, retry later{ "queueDepth": 10500, "retryAfter": "2025-11-21T11:00:00Z" }Schedule for off-peak

3.3 FR-003: Parser Service (XML → JSON Transformation)

Requirement: The ParserService shall listen to batch-upload-queue, download batch XML from blob storage, detect vendor format, validate against XSD schema, parse to individual invoices, transform to canonical JSON format, and enqueue to batch-items-queue in groups of 32.

Priority: CRITICAL

Service Specifications:

Trigger: Message in batch-upload-queue
Input: Batch ID from queue message
Output: Individual JSON files in blob + messages in batch-items-queue

Processing Steps:

1. Dequeue message from batch-upload-queue
2. Download batch XML from blob: {org}-batches-{year}/{month}/{day}/{batch-id}/source.xml
3. Detect vendor format (namespace + structure analysis):
   - urn:ediel → GASEL
   - oio.dk → XELLENT
   - Zynergy → ZYNERGY
4. Load vendor-specific schema mapping from: {org}-data/schemas/{vendor}-mapping.json
5. Validate XML against vendor XSD schema
6. Parse XML using XPath expressions from mapping
7. Transform each invoice to canonical JSON format
8. Store individual JSON files: {org}-invoices-{year}/{month}/{day}/{invoice-id}.json
9. Group invoices into batches of 32
10. Enqueue each 32-item batch to batch-items-queue
11. Update batch metadata: totalItems, vendor info, status="processing"
12. Delete message from batch-upload-queue (on success)
13. On error: Retry (3x with backoff) or move to poison queue

...

Canonical JSON Schema (Output Format):

{
  "invoiceId": "uuid",
  "invoiceNumber": "2025-11-001",
  "invoiceDate": "2025-11-06",
  "dueDate": "2025-11-20",
  "currency": "SEK",
  "periodStart": "2025-10-01",
  "periodEnd": "2025-10-31",
  
  "customer": {
    "customerId": "020624-2380",
    "fullName": "Medeni Schröder",
    "firstName": null,
    "lastName": null,
    "email": "muntaser.af@zavann.net",
    "phone": "09193538799",
    "address": {
      "street": "Strandbo 63B",
      "houseNumber": null,
      "apartment": null,
      "city": "Växjö",
      "postalCode": "352 58",
      "country": "SE"
    },
    "taxIdentifier": "020624-2380",
    "customerType": "private"
  },
  
  "invoiceDetails": {
    "subTotal": 599.42,
    "taxAmount": 149.86,
    "totalAmount": 749.28,
    "lineItems": [
      {
        "lineNumber": 1,
        "description": "Elförbrukning - Fast pris",
        "quantity": 420,
        "unit": "KWH",
        "unitPrice": 1.026,
        "lineAmount": 430.92,
        "taxRate": 25.0,
        "taxAmount": 107.73,
        "category": "electricity"
      },
      {
        "lineNumber": 2,
        "description": "Månadsavgift",
        "quantity": 1,
        "unit": "MON",
        "unitPrice": 15.20,
        "lineAmount": 15.20,
        "taxRate": 25.0,
        "taxAmount": 3.80,
        "category": "fee"
      }
    ]
  },
  
  "delivery": {
    "meteringPointId": "735999756427205424",
    "gridArea": "SE4",
    "gridOwner": "Växjö Energi Elnät AB",
    "previousReading": {
      "date": "2025-09-30",
      "value": 10580,
      "type": "Actual"
    },
    "currentReading": {
      "date": "2025-10-31",
      "value": 11000,
      "type": "Actual"
    },
    "consumption": 420,
    "consumptionUnit": "kWh"
  },
  
  "payment": {
    "paymentId": "202511001",
    "paymentMethod": "Bankgiro",
    "bankAccount": "168-6039",
    "dueDate": "2025-11-20"
  },
  
  "sourceMetadata": {
    "vendorCode": "GASEL",
    "vendorVersion": "1.0",
    "originalBatchId": "BATCH2025110600001",
    "originalInvoiceId": "2025-11-001",
    "contractReference": "CON-2024-001",
    "customFields": {
      "productCode": "telinet_fixed",
      "productName": "Fast Pris Vintersäkra",
      "taxClassification": "Normal"
    },
    "parsedAt": "2025-11-21T10:35:45Z"
  }
}

...

Acceptance Criteria:

CriterionValidation MethodTest DataExpected Result
Listens to batch-upload-queueSend message, verify service picks upQueue messageMessage dequeued within 30s
Downloads batch XML from blobVerify file download loggedBatch in blobFile downloaded successfully
Detects GASEL format (100% accuracy)Test with 50 GASEL samplesgasel_*.xmlAll detected as GASEL
Detects XELLENT format (100% accuracy)Test with 50 XELLENT samplesxellent_*.xmlAll detected as XELLENT
Detects ZYNERGY format (100% accuracy)Test with 50 ZYNERGY sampleszynergy_*.xmlAll detected as ZYNERGY
Loads correct schema mappingVerify mapping file loadedgasel_sample.xmlgasel-mapping.json loaded
Validates XML against XSDUpload invalid GASEL XMLinvalid_gasel.xmlValidation errors in batch metadata
Parses GASEL using XPath mappingsParse GASEL, verify JSON fieldsgasel_sample.xmlAll fields extracted
Parses XELLENT with namespace handlingParse XELLENT (com:, main: prefixes)xellent_sample.xmlAll fields extracted
Parses ZYNERGY nested structureParse ZYNERGYzynergy_sample.xmlAll fields extracted
Transforms to canonical JSONVerify JSON schema complianceAll vendorsAll pass schema validation
Stores JSON: {org}-invoices-{year}/{month}/{day}/{id}.jsonCheck blob pathParsed invoiceCorrect path used
Groups into 32-item batchesParse 100 invoices, count queue messages100-invoice batch4 messages (32+32+32+4)
Enqueues to batch-items-queueVerify messages in queueParsed batchMessages present
Updates batch metadataCheck metadata after parsingParsed batchtotalItems, vendorCode set
Deletes from batch-upload-queue on successVerify message removedSuccessful parseMessage gone
Retries 3x on transient errorsForce blob download errorFailing batch3 retries logged
Moves to poison queue after 3 failuresForce permanent errorFailing batchMessage in poison queue
Parsing completes within 2 minutes for 10K batchPerformance test10K-invoice XML≤ 120 seconds

Vendor-Specific Parsing Rules:

GASEL Format:

Required Elements (validation fails if missing):
- BatchHeader/InterchangeID
- BatchHeader/TotalInvoiceCount
- SupplierParty/PartyName
- Invoice/InvoiceHeader/InvoiceNumber
- Invoice/CustomerParty/PartyName
- Invoice/MonetarySummary/PayableAmount

Optional Elements (null if missing):
- Invoice/DeliveryLocation/MeteringPointID
- Invoice/ContractDetails/ContractID
- Invoice/CustomerParty/Contact/ElectronicMail
- Invoice/CustomerParty/Contact/Telephone

Date Format: ISO 8601 (YYYY-MM-DD)
Amount Format: Decimal with 2 places, period as separator
Namespace: urn:ediel:se:electricity:invoice:1.0

...

XELLENT Format:

Required Elements:
- BatchHeader/BatchID
- com:ID (invoice number)
- com:IssueDate
- com:BuyerParty/com:PartyName/com:Name
- com:LegalTotals/com:ToBePaidTotalAmount

Optional Elements:
- com:BuyerParty/com:ContactInformation/@E-Mail
- com:BuyerParty/com:Address

Special Handling:
- Multiple namespaces (com:, main:, fsv:)
- Amount format: "1 245,00" (space separator, comma decimal)
- Must normalize to standard decimal format

Namespace: http://rep.oio.dk/ubl/xml/schemas/0p71/pie/

...

ZYNERGY Format:

Required Elements:
- BatchHeader/BatchId
- InvoiceData/InvoiceNumber
- Customer/ReadOnlyFullName
- InvoiceData/InvoiceAmount

Optional Elements:
- Customer/FirstName and LastName (if ReadOnlyFullName empty)
- InvoiceAddress/EmailAddress
- VAT details

Special Handling:
- Nested structure (Invoice > Customer, InvoiceData, InvoiceAddress, VAT)
- Multiple company references (CompaniesId throughout)
- InvoiceAmount vs InvoiceBalance distinction

Namespace: http://eg.dk/Zynergy/1.0/invoice.xsd

...

Dependencies:

  • Azure Storage Queue: batch-upload-queue
  • Vendor schema mappings in blob storage
  • XSD schema files for validation
  • Canonical JSON schema definition
  • Error handling and retry infrastructure

Risks & Mitigation:

RiskLikelihoodImpactMitigation StrategyOwner
Large XML file memory issues (>50MB)MEDIUMHIGH- Stream-based parsing (XmlReader, not XDocument)
- Process invoices incrementally
- Worker memory limit monitoring
- File size alerts at 75MB
Technical Architect
Parsing performance bottleneckMEDIUMHIGH- Parallel XPath evaluation where possible
- Compiled XPath expressions cached
- POC: parse 10K invoices in <2 minutes
- Horizontal scaling of ParserService
Technical Architect
XSD validation performanceLOWMEDIUM- Cache compiled XSD schemas
- Make validation optional in lenient mode
- Async validation (don't block parsing)
Technical Architect
Vendor-specific edge casesHIGHMEDIUM- Extensive test suite per vendor (50+ samples)
- Error collection from production
- Vendor liaison for unclear cases
- Lenient mode for known variations
Product Owner

3.4 FR-004: Document Generator Service (JSON → HTML → PDF)

Requirement: The DocumentGeneratorService (based on existing zyn-DocumentGenerator) shall listen to batch-items-queue, load Handlebars templates, render HTML with invoice data, generate PDFs using Playwright, and store documents in blob storage with month/day hierarchy.

Priority: CRITICAL

Service Specifications:

Trigger: Message in batch-items-queue (contains 32 invoice references)
Input: Batch of 32 invoice IDs and organization ID
Output: PDF + HTML files in blob storage, messages in distribution routing queues

Processing Steps (per 32-item batch):

1. Dequeue message from batch-items-queue
2. Acquire blob lease: {org}-batches-{year}/{month}/{day}/{batch-id}/locks/{worker-id}.lock
3. For each of 32 invoices:
   a. Download JSON: {org}-invoices-{year}/{month}/{day}/{invoice-id}.json
   b. Load organization config: {org}-data/organization.json
   c. Determine template category from distribution type
   d. Load Handlebars template: {org}-data/templates/{category}/active.html
   e. Load organization branding (logo from blob URL, colors, fonts)
   f. Compile Handlebars template (cache compiled version)
   g. Render HTML with invoice data + branding
   h. Generate PDF from HTML using Playwright (headless Chromium)
   i. Store HTML: {org}-invoices-{year}/{month}/{day}/{invoice-id}.html
   j. Store PDF: {org}-invoices-{year}/{month}/{day}/{invoice-id}.pdf
   k. Update invoice JSON metadata with file paths and render timestamp
   l. Determine distribution method from invoice data
   m. Enqueue to appropriate queue:
      - Mail (postal) → postal-bulk-queue (processed in bulk)
      - Email → email-queue
      - SMS (future) → sms-queue
      - Kivra (future) → kivra-queue
      - E-faktura (future) → efaktura-queue
4. Release blob lease
5. Update batch metadata: processedItems += 32
6. Delete message from batch-items-queue
7. On error: Retry (3x) or poison queue

...

Acceptance Criteria:

CriterionValidation MethodTest DataExpected Result
Listens to batch-items-queueSend message, verify pickupQueue messageDequeued within 30s
Processes 32 invoices per messageSend 32-item batch, count outputs32 invoices32 PDFs generated
Acquires blob lease before processingCheck lease on blobValid batchLease acquired
Downloads invoice JSON from correct pathVerify download loggedInvoice JSONCorrect path: {org}/2025/11/21/{id}.json
Loads organization templateVerify template file accessedOrg with templateTemplate loaded
Determines template category correctlyInvoice → "invoice" template, Letter → "confirmation"Various typesCorrect template used
Compiles Handlebars templateRender with variablesTemplate with {{invoiceNumber}}Number inserted
Caches compiled templates (24h)Render same template twiceSame templateSecond render faster
Renders HTML with Swedish charactersRender with åäöSwedish invoiceCharacters correct
Generates PDF with PlaywrightConvert HTML to PDFRendered HTMLPDF created, A4 format
PDF includes organization brandingCheck PDF for logo, colorsBranded templateBranding visible
Stores HTML in correct blob pathVerify pathGenerated HTML{org}/invoices/2025/11/21/{id}.html
Stores PDF in correct blob pathVerify pathGenerated PDF{org}/invoices/2025/11/21/{id}.pdf
Updates invoice metadata JSONCheck metadata after renderProcessed invoicefileReferences populated
Determines distribution methodCheck routing logicVarious configsCorrect queue selected
Enqueues to postal-bulk-queue for mailInvoice with postal deliveryMail invoiceMessage in postal queue
Enqueues to email-queue for emailInvoice with email deliveryEmail invoiceMessage in email queue
Releases blob lease on completionVerify lease releasedProcessed batchLease gone
Updates batch statisticsCheck batch metadataProcessed batchprocessedItems incremented
Rendering within 2 seconds per invoice (p95)Performance test 1000 invoicesVariousp95 ≤ 2 seconds
PDF generation within 5 seconds per invoice (p95)Performance test 1000 PDFsVariousp95 ≤ 5 seconds
Retries on transient errorsForce blob errorFailing invoice3 retries attempted
Moves to poison queue after 3 failuresForce permanent errorFailing invoicePoison queue message

Handlebars Template Example:

<!DOCTYPE html>
<html lang="sv">
<head>
    <meta charset="UTF-8">
    <title>Faktura {{invoiceNumber}}</title>
    <style>
        body { font-family: {{organization.fontFamily}}; }
        .header { background-color: {{organization.primaryColor}}; color: white; padding: 20px; }
        .logo { max-width: 200px; }
    </style>
</head>
<body>
    <div class="header">
        <img src="{{organization.logoUrl}}" alt="{{organization.displayName}}" class="logo" />
        <h1>Faktura {{invoiceNumber}}</h1>
    </div>
    
    <div class="customer-info">
        <h2>Kund</h2>
        <p><strong>{{customer.fullName}}</strong></p>
        <p>{{customer.address.street}}</p>
        <p>{{customer.address.postalCode}} {{customer.address.city}}</p>
    </div>
    
    <div class="invoice-details">
        <p><strong>Fakturadatum:</strong> {{invoiceDate}}</p>
        <p><strong>Förfallodatum:</strong> {{dueDate}}</p>
        <p><strong>Period:</strong> {{periodStart}} - {{periodEnd}}</p>
        {{#if delivery.meteringPointId}}
        <p><strong>Mätpunkt:</strong> {{delivery.meteringPointId}}</p>
        <p><strong>Elområde:</strong> {{delivery.gridArea}}</p>
        <p><strong>Förbrukning:</strong> {{delivery.consumption}} {{delivery.consumptionUnit}}</p>
        {{/if}}
    </div>
    
    <table>
        <thead>
            <tr>
                <th>Beskrivning</th>
                <th>Antal</th>
                <th>Enhet</th>
                <th>Pris</th>
                <th>Belopp</th>
            </tr>
        </thead>
        <tbody>
            {{#each invoiceDetails.lineItems}}
            <tr>
                <td>{{this.description}}</td>
                <td>{{formatNumber this.quantity decimals=2}}</td>
                <td>{{this.unit}}</td>
                <td>{{formatCurrency this.unitPrice}}</td>
                <td>{{formatCurrency this.lineAmount}}</td>
            </tr>
            {{/each}}
            <tr class="total-row">
                <td colspan="4"><strong>Delsumma:</strong></td>
                <td><strong>{{formatCurrency invoiceDetails.subTotal}}</strong></td>
            </tr>
            <tr>
                <td colspan="4"><strong>Moms (25%):</strong></td>
                <td><strong>{{formatCurrency invoiceDetails.taxAmount}}</strong></td>
            </tr>
            <tr class="total-row">
                <td colspan="4"><strong>Att betala:</strong></td>
                <td><strong>{{formatCurrency invoiceDetails.totalAmount}} {{currency}}</strong></td>
            </tr>
        </tbody>
    </table>
    
    {{#if payment.paymentId}}
    <div class="payment-info">
        <h3>Betalningsinformation</h3>
        <p><strong>OCR-nummer:</strong> {{payment.paymentId}}</p>
        <p><strong>Bankgiro:</strong> {{payment.bankAccount}}</p>
        <p><strong>Förfallodatum:</strong> {{payment.dueDate}}</p>
    </div>
    {{/if}}
</body>
</html>

...

Dependencies:

  • Handlebars.Net library for template rendering
  • Playwright library for PDF generation
  • Azure Storage Queue: batch-items-queue
  • Blob storage for templates, JSON, PDF, HTML
  • Organization configuration in blob
  • Custom Handlebars helpers (formatCurrency, formatNumber, formatDate)

Risks & Mitigation:

RiskLikelihoodImpactMitigation StrategyOwner
Handlebars rendering performanceHIGHHIGH- Pre-compile templates on first use
- Cache compiled templates (24h TTL)
- Parallel rendering for 32 items
- POC: 1000 renders in <30 seconds
Technical Architect
Playwright memory consumptionHIGHHIGH- Semaphore limit: max 10 concurrent PDFs
- Worker instance memory monitoring
- Graceful degradation if memory high
- Browser instance pooling
Technical Architect
Swedish character encoding (åäö)MEDIUMMEDIUM- UTF-8 throughout entire pipeline
- Font embedding in PDF
- Visual testing with Swedish content
- Sample invoices with all Swedish special chars
QA Team
Template injection securityLOWCRITICAL- Handlebars safe mode (no eval)
- Template sanitization on upload
- No dynamic helper registration
- Security code review
Security Officer
Missing template categoryLOWMEDIUM- Fall back to default "invoice" template
- Log warning for missing category
- Template category validation
Product Owner

3.5 FR-005: Email Delivery Service

Requirement: The EmailDeliveryService shall listen to email-queue, download PDF from blob storage, send via SendGrid with Swedish-localized email template, track delivery status, and retry on transient failures with fallback to postal queue.

Priority: HIGH

Service Specifications:

Trigger: Message in email-queue
Input: Invoice ID, recipient email, organization ID
Output: Email sent via SendGrid, delivery status updated

Processing Steps:

1. Dequeue message from email-queue
2. Download PDF: {org}-invoices-{year}/{month}/{day}/{invoice-id}.pdf
3. Load organization email configuration
4. Load email template (Swedish)
5. Create SendGrid message:
   - From: noreply@{org-domain}.com
   - To: {customer-email}
   - Subject: "Faktura {invoiceNumber} från {orgName}"
   - Body: HTML template with invoice summary
   - Attachment: invoice-{invoiceNumber}.pdf
6. Send via SendGrid API
7. Handle response:
   - Success (2xx): Update invoice status="delivered", log messageId
   - Rate limit (429): Re-queue with Retry-After delay
   - Transient error (5xx): Retry with exponential backoff (3x)
   - Permanent error (4xx): Move to postal-bulk-queue (fallback)
8. Update invoice metadata with delivery attempt
9. Delete from email-queue (on success or permanent failure)

...

Email Template (Swedish):

<!DOCTYPE html>
<html lang="sv">
<head>
    <meta charset="UTF-8">
    <title>Faktura</title>
</head>
<body style="font-family: Arial, sans-serif; max-width: 600px; margin: 0 auto;">
    <div style="background: #0066CC; color: white; padding: 20px; text-align: center;">
        <h1>Faktura från {{organizationName}}</h1>
    </div>
    
    <div style="padding: 20px;">
        <p>Hej {{customerName}},</p>
        
        <p>Din faktura för perioden {{periodStart}} till {{periodEnd}} är nu tillgänglig.</p>
        
        <table style="width: 100%; margin: 20px 0; border-collapse: collapse;">
            <tr style="background: #f5f5f5;">
                <td style="padding: 10px; border: 1px solid #ddd;"><strong>Fakturanummer:</strong></td>
                <td style="padding: 10px; border: 1px solid #ddd;">{{invoiceNumber}}</td>
            </tr>
            <tr>
                <td style="padding: 10px; border: 1px solid #ddd;"><strong>Förfallodatum:</strong></td>
                <td style="padding: 10px; border: 1px solid #ddd;">{{dueDate}}</td>
            </tr>
            <tr style="background: #f5f5f5;">
                <td style="padding: 10px; border: 1px solid #ddd;"><strong>Att betala:</strong></td>
                <td style="padding: 10px; border: 1px solid #ddd;"><strong>{{totalAmount}} SEK</strong></td>
            </tr>
        </table>
        
        <p><strong>Betalningsinformation:</strong></p>
        <p>Bankgiro: {{bankAccount}}<br>
        OCR-nummer: {{ocrNumber}}</p>
        
        <p>Din faktura finns bifogad som PDF.</p>
        
        <p>Vid frågor, kontakta oss på {{supportEmail}} eller {{supportPhone}}.</p>
        
        <p>Med vänlig hälsning,<br>
        {{organizationName}}</p>
    </div>
    
    <div style="background: #f5f5f5; padding: 15px; text-align: center; font-size: 12px; color: #666;">
        <p>Detta är ett automatiskt meddelande. Svara inte på detta e-postmeddelande.</p>
    </div>
</body>
</html>

...

Acceptance Criteria:

CriterionValidation MethodTest DataExpected Result
Listens to email-queueSend message, verify processingQueue messageMessage dequeued
Downloads PDF from correct blob pathVerify blob access loggedInvoice with PDFPDF downloaded
Sends via SendGrid APIMock SendGrid, verify API callEmail invoiceSendGrid API called
PDF attached to emailReceive test email, check attachmentEmail invoicePDF attached
Subject includes invoice number (Swedish)Check email subjectInvoice 123"Faktura 123 från..."
From address uses org domainCheck email headersOrg configFrom: noreply@acme.com
Reply-to set to org supportCheck email headersOrg configReply-To: support@acme.com
Swedish email template usedCheck email bodyEmail invoiceSwedish text
Retry 2x on transient failure (1min, 5min)Force 500 error from SendGridFailing email2 retries logged
Fallback to postal on permanent failureForce 400 error (invalid email)Bad emailPostal queue message
Delivery status tracked in invoice metadataCheck metadata after sendDelivered invoicedeliveryAttempts array updated
SendGrid messageId loggedCheck invoice metadataDelivered invoiceproviderMessageId present
Rate limit handling (429)Simulate rate limitMany emailsRe-queued with delay
Email size validation (<25MB)Large PDF attachment30MB PDFError or compression

SendGrid Configuration:

{
  "sendgrid": {
    "apiKey": "{{from-azure-keyvault}}",
    "fromEmail": "noreply@{org-domain}.com",
    "fromName": "{organizationName}",
    "replyTo": "support@{org-domain}.com",
    "tracking": {
      "clickTracking": false,
      "openTracking": true,
      "subscriptionTracking": false
    },
    "mailSettings": {
      "sandboxMode": false,
      "spamCheck": {
        "enable": true,
        "threshold": 5
      }
    }
  }
}

...

Dependencies:

  • SendGrid account with Nordic IP reputation
  • Azure Storage Queue: email-queue
  • Email templates in Swedish (+ future: Norwegian, Danish, Finnish)
  • Organization email configuration in blob
  • Fallback queue routing to postal

Risks & Mitigation (Nordic Email Deliverability):

RiskLikelihoodImpactMitigation StrategyOwner
Swedish ISP spam filtering (Telia, Tele2, Telenor)MEDIUMHIGH- Dedicated IP warmup (2-week ramp)
- SPF: include:sendgrid.net
- DKIM signing enabled
- DMARC p=quarantine policy
- Monitor bounce rates by ISP
- Request whitelisting from major ISPs
Operations Manager
SendGrid rate limits (enterprise plan needed)MEDIUMMEDIUM- Enterprise plan: 2M emails/month
- Queue-based pacing
- Monitor daily send volume
- Distribute sends over 24 hours
- Priority queue for SLA customers
Product Owner
PDF attachment size (>25MB)LOWLOW- Compress PDFs with Ghostscript
- Target: <5MB per invoice
- Alert if PDF >20MB
- Fallback: send download link
Technical Architect
Email template rendering errorsLOWMEDIUM- Template validation on deployment
- Fallback to plain text if HTML fails
- Error monitoring
- Sample sends for all templates
QA Team
Customer email address invalidMEDIUMLOW- Email validation before send
- Skip email, go directly to postal
- Log invalid addresses for org to correct
Product Owner

3.6 FR-006: Postal Delivery Service (21G Bulk Integration)

Requirement: The PostalDeliveryService shall listen to postal-bulk-queue, collect invoices for bulk processing, create ZIP archive with PDFs and XML metadata in 21G format, upload to 21G SFTP server at scheduled times (12:00 and 20:00 Swedish time), and track delivery confirmations.

Priority: HIGH

Service Specifications:

Trigger: Scheduled execution (12:00 and 20:00 CET/CEST)
Input: All messages in postal-bulk-queue accumulated since last run
Output: ZIP file uploaded to 21G SFTP, delivery confirmations tracked

Processing Steps:

1. Scheduled trigger (12:00 and 20:00 Swedish time)
2. Fetch all messages from postal-bulk-queue (batch retrieval)
3. For each invoice in queue:
   a. Download PDF: {org}-invoices-{year}/{month}/{day}/{invoice-id}.pdf
   b. Download invoice JSON for metadata
   c. Validate recipient address is complete
   d. Add to 21G batch collection
4. Group by organization (21G requires org-specific batches)
5. For each organization batch:
   a. Create 21G XML metadata file with all invoice details
   b. Create ZIP archive: {org-code}_{date}_{sequence}.zip
      - Contains: invoice1.pdf, invoice2.pdf, ..., metadata.xml
   c. Upload ZIP to 21G SFTP: /incoming/{org-code}/
   d. Verify upload success
   e. Update all invoice statuses: status="postal_sent"
   f. Delete messages from postal-bulk-queue
6. Log bulk send statistics to Application Insights
7. Send notification email to organization (bulk send report)

...

21G ZIP Structure:

ACME_20251121_001.zip
├── metadata.xml (21G format)
├── invoice_001.pdf
├── invoice_002.pdf
├── invoice_003.pdf
└── ...

...

21G Metadata XML Format:

<?xml version="1.0" encoding="UTF-8"?>
<PrintBatch xmlns="urn:21g:print:batch:1.0">
  <BatchHeader>
    <BatchId>ACME_20251121_001</BatchId>
    <OrganizationCode>ACME</OrganizationCode>
    <CreationDate>2025-11-21T12:00:00</CreationDate>
    <TotalDocuments>150</TotalDocuments>
    <ServiceLevel>Economy</ServiceLevel>
  </BatchHeader>
  <Documents>
    <Document>
      <DocumentId>invoice_001.pdf</DocumentId>
      <DocumentType>Invoice</DocumentType>
      <Recipient>
        <Name>Medeni Schröder</Name>
        <Street>Strandbo 63B</Street>
        <PostalCode>352 58</PostalCode>
        <City>Växjö</City>
        <Country>SE</Country>
      </Recipient>
      <PrintOptions>
        <Format>A4</Format>
        <Color>false</Color>
        <Duplex>false</Duplex>
      </PrintOptions>
    </Document>
    <!-- ... more documents -->
  </Documents>
</PrintBatch>

...

Acceptance Criteria:

CriterionValidation MethodTest DataExpected Result
Scheduled execution at 12:00 Swedish timeCheck execution logsScheduled timeRuns at 12:00 CET/CEST
Scheduled execution at 20:00 Swedish timeCheck execution logsScheduled timeRuns at 20:00 CET/CEST
Fetches all messages from postal-bulk-queueQueue 100 messages, verify all fetched100 postal invoicesAll 100 fetched
Downloads PDFs from blob storageVerify blob accessPostal invoicesAll PDFs downloaded
Validates recipient address completeInvoice with missing cityIncomplete addressSkipped with error log
Groups by organizationMix of Org A and Org B invoicesMulti-org batchSeparate ZIPs per org
Creates 21G XML metadataVerify XML structurePostal batchValid 21G XML
Creates ZIP archiveVerify ZIP contentsPostal batchPDFs + metadata.xml
Uploads to 21G SFTPMock SFTP, verify uploadZIP fileFile uploaded
Verifies upload successCheck SFTP confirmationUploaded ZIPConfirmation received
Updates invoice status to "postal_sent"Check invoice metadataSent invoicesStatus updated
Deletes messages from queueCheck queue after processingProcessed batchQueue empty
Logs bulk statisticsCheck Application InsightsProcessed batchStatistics logged
Sends org notification emailCheck email receivedProcessed batchEmail with counts
Handles SFTP connection errorsSimulate SFTP downPostal batchRetry logged, alert sent
Respects 21G batch size limitsCreate large batch10,000 invoicesSplit into multiple ZIPs

21G Integration Specifications:

SFTP Connection:

  • Host: sftp.21g.se (or provider-specific)
  • Port: 22
  • Authentication: SSH key (stored in Azure Key Vault)
  • Directory structure: /incoming/{org-code}/
  • File naming: {org-code}{YYYYMMDD}{sequence}.zip

21G SLA:

  • Processing time: 24-48 hours
  • Confirmation: Email notification when processed
  • Tracking: Available via 21G portal

Dependencies:

  • 21G SFTP account and credentials
  • Azure Storage Queue: postal-bulk-queue
  • Scheduled worker (Azure Container Apps with CRON)
  • ZIP file creation library
  • 21G XML schema compliance
  • Email notification service

Risks & Mitigation (Nordic Postal Context):

RiskLikelihoodImpactMitigation StrategyOwner
21G SFTP connectivity issuesLOWHIGH- Retry logic (3 attempts with 5min delay)
- Secondary SFTP credentials
- Alert on connection failure
- Manual upload procedure documented
- 21G support contact documented
Operations Manager
Swedish postal delays (holidays, strikes)MEDIUMMEDIUM- Set customer expectations (5-7 days)
- Monitor 21G processing SLA
- Track delivery confirmations
- Escalation for >10 days
- Alternative print partner identified
Product Owner
Incomplete recipient addressesMEDIUMLOW- Address validation before queueing
- Skip invalid addresses
- Alert organization of invalid addresses
- Provide address correction interface
Product Owner
21G format specification changesLOWMEDIUM- Version 21G XML schema
- Monitor 21G API announcements
- Test uploads to 21G staging
- 21G account manager liaison
Technical Architect
ZIP file corruptionLOWHIGH- SHA-256 checksum in metadata
- Verify ZIP integrity before upload
- Keep ZIP in blob for 30 days
- 21G confirms successful unzip
Technical Architect

3.7 FR-007: Distribution Routing Logic

Requirement: The system shall determine the appropriate distribution queue for each invoice based on organization configuration, customer preferences, and distribution type (invoice vs document) following Swedish regulatory requirements.

Priority: HIGH

Routing Decision Tree:

Invoice Distribution Routing
│
├─ Check customer preference (if available)
│  ├─ Preference = "digital" → email-queue (or kivra-queue in future)
│  └─ Preference = "postal" → postal-bulk-queue
│
├─ Check organization default channels (from config)
│  ├─ Priority 1: email
│  │  └─ Has valid email? → email-queue
│  ├─ Priority 2: kivra (Phase 2)
│  │  └─ Kivra user? → kivra-queue
│  └─ Priority 3: postal
│     └─ postal-bulk-queue
│
├─ Document type consideration
│  ├─ Invoice (faktura) → All channels available
│  └─ Confirmation letter → Email/postal only
│
└─ Swedish regulatory compliance
   └─ Customer always has right to postal ("rätt till pappersfaktura")

...

Queue Selection Logic:

public async Task<string> DetermineDistributionQueueAsync(
    InvoiceDistribution distribution,
    OrganizationConfig config)
{
    // Customer preference overrides (if explicitly set)
    if (distribution.CustomerPreference == "postal")
        return "postal-bulk-queue";
    
    // Try channels in priority order
    var priorities = config.DeliveryChannels.ChannelPriority
        .OrderBy(p => p.Priority);
    
    foreach (var channel in priorities)
    {
        switch (channel.Channel)
        {
            case "email":
                if (!string.IsNullOrEmpty(distribution.CustomerEmail) &&
                    IsValidEmail(distribution.CustomerEmail))
                {
                    return "email-queue";
                }
                break;
                
            case "kivra": // Phase 2
                if (await IsKivraUserAsync(distribution.CustomerPersonnummer))
                {
                    return "kivra-queue";
                }
                break;
                
            case "efaktura": // Phase 2 (B2B only)
                if (distribution.CustomerType == "business" &&
                    !string.IsNullOrEmpty(distribution.OrganizationNumber))
                {
                    return "efaktura-queue";
                }
                break;
                
            case "postal":
                if (distribution.IsCompleteAddress())
                {
                    return "postal-bulk-queue";
                }
                break;
        }
    }
    
    // Ultimate fallback: postal (Swedish law requires paper option)
    return "postal-bulk-queue";
}

...

Acceptance Criteria:

CriterionValidation MethodTest DataExpected Result
Customer preference honoredSet preference="postal"Email-enabled invoiceRoutes to postal queue
Organization priority followedPriority: [email, postal]Valid emailRoutes to email queue
Email validated before routingInvalid email addressbad-email@invalidRoutes to postal queue
Complete address required for postalMissing postal codeIncomplete addressError logged, skipped
Document type consideredConfirmation letterNon-invoice docOnly email/postal
Swedish postal fallbackAll digital channels failFailed digitalpostal-bulk-queue
Business invoices support e-faktura (future)Organization number presentB2B invoiceefaktura-queue (Phase 2)
Routing decision loggedCheck logsAny invoiceDecision reason logged

Dependencies:

  • Organization delivery configuration
  • Customer preference storage (future)
  • Email validation library
  • Address validation library (Swedish postal codes)
  • Kivra user lookup API (Phase 2)

Risks & Mitigation:

RiskLikelihoodImpactMitigation StrategyOwner
Invalid email addresses (>5% in Nordic utilities)HIGHLOW- Email validation regex
- Automatic postal fallback
- Report invalid emails to organization
- Customer data quality improvement program
Product Owner
Incomplete postal addressesMEDIUMMEDIUM- Address validation against Swedish postal database
- Skip invalid addresses with alert
- Organization notification of incomplete addresses
Product Owner
Swedish "rätt till pappersfaktura" complianceLOWCRITICAL- Always enable postal as fallback
- Never force digital-only
- Document compliance in privacy policy
- Legal review of routing logic
Legal/Compliance

3.8 FR-008: Blob Concurrency Control (Note: Read-Only, No Concurrent Updates)

Requirement: The system shall use blob leases to ensure exclusive access during batch processing, preventing concurrent worker instances from processing the same 32-item batch.

Priority: HIGH

Clarification: Based on your feedback, there are no concurrent updates to files - workers only read invoice JSON files. The blob lease is used to ensure only one worker processes a given 32-item batch from the queue.

Lease Implementation:

public async Task ProcessBatchItemsAsync(BatchItemsMessage message)
{
    var lockBlobPath = $"{message.OrganizationId}-batches-{year}/{month}/{day}/{message.BatchId}/locks/{message.MessageId}.lock";
    BlobLease lease = null;
    
    try
    {
        // Acquire lease (5-minute duration)
        lease = await _blobLockService.AcquireLockAsync(
            containerName: $"{message.OrganizationId}-batches-{year}",
            blobName: lockBlobPath,
            leaseDuration: TimeSpan.FromMinutes(5));
        
        // Process 32 invoices (read-only operations)
        foreach (var invoiceId in message.InvoiceIds)
        {
            // Read invoice JSON from blob (no updates to JSON)
            var invoiceJson = await _blobStorage.DownloadJsonAsync(invoiceId);
            
            // Render and generate (creates new HTML/PDF blobs)
            await RenderAndGenerateAsync(invoiceJson);
            
            // No concurrent update risk - creating new blobs only
        }
        
        // Update batch metadata (ETag-based optimistic concurrency)
        await UpdateBatchMetadataAsync(message.BatchId, meta =>
        {
            meta.Statistics.ProcessedItems += message.InvoiceIds.Count;
        });
    }
    finally
    {
        if (lease != null)
        {
            await _blobLockService.ReleaseLockAsync(lease);
        }
    }
}

...

Acceptance Criteria:

CriterionValidation MethodTest DataExpected Result
Acquires blob lease before processingStart processing, check lease32-item batchLease acquired
Lease duration is 5 minutesCheck lease propertiesAny batchDuration = 5 min
Only one worker processes batchSend same message to 2 workersDuplicate messageOne succeeds, one waits
Lease renewed for long processingProcess 32 items slowlySlow batchLease renewed
Lease released on completionCheck lease after processingCompleted batchLease released
Lease released on errorForce error during processingFailing batchLease released
Different batches process in parallelQueue 10 batches10 x 32 itemsAll process concurrently
Batch metadata updates use ETagsConcurrent metadata updates2 workers update statsNo lost updates

Dependencies:

  • Azure Blob Storage lease API
  • ETag-based optimistic concurrency for metadata updates
  • Retry logic for lease acquisition conflicts

3.9 FR-009: Queue Message Handling

Requirement: The system shall process queue messages with proper visibility timeouts, automatic retry on failure, poison queue handling, and message deduplication.

Priority: HIGH

Queue Configuration:

Queue NamePurposeVisibility TimeoutMax Delivery CountDead Letter Queue
batch-upload-queueTriggers ParserService10 minutes3poison-queue
batch-items-queueTriggers DocumentGenerator (32 items)5 minutes3poison-queue
email-queueTriggers EmailService2 minutes3poison-queue
postal-bulk-queueCollected for 21G bulk sendN/A (batch retrieval)1poison-queue
poison-queueFailed messages for manual reviewN/A0None

Message Format Standard:

{
  "messageId": "uuid",
  "messageType": "batch.upload|batch.items|email.delivery|postal.delivery",
  "version": "1.0",
  "timestamp": "2025-11-21T10:30:00Z",
  "data": {
    // Message-specific payload
  },
  "metadata": {
    "correlationId": "uuid",
    "organizationId": "uuid",
    "retryCount": 0,
    "enqueuedAt": "2025-11-21T10:30:00Z"
  }
}

...

Retry Policy (Exponential Backoff):

AttemptDelayTotal Elapsed
1Immediate0
260 seconds1 minute
3300 seconds6 minutes
4900 seconds21 minutes
FailedPoison queue-

Acceptance Criteria:

CriterionValidation MethodTest DataExpected Result
Messages have proper visibility timeoutCheck queue propertiesAny messageCorrect timeout set
Failed messages retry automaticallyForce error, verify retryFailing message3 retries attempted
Retry count incrementedCheck message metadataRetried messageretryCount incremented
Exponential backoff appliedMeasure retry delaysFailing message1min, 5min, 15min
After 3 retries, moved to poison queueForce permanent failureFailing messageIn poison queue
Poison queue triggers alertMessage in poison queueFailed messageAlert email sent
Support team notifiedCheck alert recipientsPoison messageSupport receives email
No duplicate processing (idempotent)Send duplicate messageSame invoice IDProcessed once
Correlation ID traces through systemFollow message across queuesAny messageSame correlationId

Poison Queue Handling:

{
  "messageId": "uuid",
  "originalMessageType": "batch.items",
  "failedAt": "2025-11-21T10:50:00Z",
  "retryCount": 3,
  "lastError": {
    "code": "TEMPLATE_RENDERING_FAILED",
    "message": "Required variable 'customer.address.street' not found in template",
    "stackTrace": "...",
    "attemptTimestamps": [
      "2025-11-21T10:35:00Z",
      "2025-11-21T10:36:00Z",
      "2025-11-21T10:41:00Z",
      "2025-11-21T10:56:00Z"
    ]
  },
  "originalMessage": {
    // Full original message for debugging
  },
  "metadata": {
    "correlationId": "uuid",
    "alertSent": true,
    "alertRecipients": ["support@egflow.com"],
    "manualReviewRequired": true
  }
}

...

Dependencies:

  • Azure Storage Queues with dead-letter queue support
  • Alert service for poison queue notifications
  • Monitoring dashboard for poison queue depth

3.10 FR-010: Health Check Endpoints

Requirement: All services shall expose health check endpoints that verify connectivity to dependencies (database, blob storage, queues) and return health status for Azure Traffic Manager and monitoring.

Priority: HIGH

API Endpoint: GET /health

Response Format (Healthy):

{
  "status": "Healthy",
  "timestamp": "2025-11-21T10:30:00Z",
  "version": "1.0.0",
  "checks": {
    "blobStorage": {
      "status": "Healthy",
      "responseTime": "45ms",
      "lastChecked": "2025-11-21T10:30:00Z"
    },
    "storageQueue": {
      "status": "Healthy",
      "responseTime": "32ms",
      "queueDepth": 150
    },
    "postgresql": {
      "status": "Healthy",
      "responseTime": "12ms",
      "activeConnections": 8
    },
    "keyVault": {
      "status": "Healthy",
      "responseTime": "67ms"
    }
  },
  "environment": "production",
  "region": "westeurope"
}

...

Response Format (Unhealthy):

{
  "status": "Unhealthy",
  "timestamp": "2025-11-21T10:30:00Z",
  "checks": {
    "blobStorage": {
      "status": "Unhealthy",
      "error": "Connection timeout after 5000ms",
      "lastChecked": "2025-11-21T10:30:00Z"
    },
    "postgresql": {
      "status": "Healthy",
      "responseTime": "15ms"
    }
  }
}

...

Acceptance Criteria:

CriterionValidation MethodTest DataExpected Result
Returns 200 when all checks healthyAll dependencies upN/A200 OK, status="Healthy"
Returns 503 when any check unhealthyStop databaseN/A503 Service Unavailable
Checks blob storage connectivityDisconnect blob storageN/AblobStorage.status="Unhealthy"
Checks queue connectivityDisable queue accessN/AstorageQueue.status="Unhealthy"
Checks PostgreSQL connectivityStop databaseN/Apostgresql.status="Unhealthy"
Checks Key Vault accessRevoke Key Vault permissionsN/AkeyVault.status="Unhealthy"
Response time < 1 secondPerformance testN/AHealth check completes quickly
Traffic Manager uses for routingSimulate region failureN/ATraffic routes to healthy region
Includes environment and regionCheck response bodyN/AEnvironment and region present

Dependencies:

  • Health check library (ASP.NET Core HealthChecks)
  • Azure Traffic Manager configuration
  • Monitoring integration

3.11 FR-011: Template Category Management

Requirement: The system shall support template categories (e.g., "Invoice", "Confirmation Letter", "Reminder") to group related templates and enable dynamic template selection based on document type.

Priority: MEDIUM

API Endpoint: GET /organizations/{organizationId}/template-categories

Response Format:

{
  "success": true,
  "data": {
    "categories": [
      {
        "categoryId": "uuid",
        "categoryName": "invoice",
        "displayName": "Faktura",
        "description": "Standard invoice template",
        "activeTemplateId": "uuid",
        "activeTemplateVersion": "2.1.0",
        "templateCount": 3
      },
      {
        "categoryId": "uuid",
        "categoryName": "confirmation",
        "displayName": "Bekräftelsebrev",
        "description": "Contract confirmation letter",
        "activeTemplateId": "uuid",
        "activeTemplateVersion": "1.0.0",
        "templateCount": 1
      },
      {
        "categoryId": "uuid",
        "categoryName": "reminder",
        "displayName": "Påminnelse",
        "description": "Payment reminder",
        "activeTemplateId": null,
        "templateCount": 0
      }
    ]
  }
}

...

Template Category Determination Logic:

Document Type → Template Category Mapping:

Invoice (faktura) → "invoice" template
Confirmation letter (bekräftelsebrev) → "confirmation" template
Payment reminder (påminnelse) → "reminder" template
Termination notice (uppsägning) → "termination" template
Contract change (avtalsändring) → "contract_change" template

Default: If category not found → use "invoice" template

...

Acceptance Criteria:

CriterionValidation MethodTest DataExpected Result
Lists all categories for organizationGET /template-categoriesOrg with 3 categories3 categories returned
Shows active template per categoryCheck activeTemplateIdCategory with active templateTemplate ID present
Returns null for unused categoriesCheck reminder categoryNo reminder templateactiveTemplateId: null
Includes template countVerify countCategory with 3 versionstemplateCount: 3
Category names localized (Swedish)Check displayNameAll categoriesSwedish names

3.12 FR-012: Git Branching Strategy (Development Workflow)

Requirement: The development team shall follow a structured Git branching strategy with development, staging, and main branches, following industry best practices for continuous integration and deployment.

Priority: HIGH

Branch Structure (Updated per Nov 20 decision):

main (production)
  ↑
  └── staging (acceptance testing)
       ↑
       └── development (integration testing)
            ↑
            └── feature/* (short-lived branches)

...

Branch Policies:

BranchPurposeMerge FromDeploy ToProtection
feature/Individual featuresN/AN/ANone (local only)
developmentIntegration testingfeature/*Dev environmentPR required, 1 approval
stagingAcceptance testing (internal)developmentStaging environmentPR required, 2 approvals, all tests pass
mainProductionstagingProduction (West + North Europe)PR required, 3 approvals, security scan, all tests pass

Deployment Flow:

Developer → feature/GAS-12345-batch-upload
   ↓
   PR to development
   ↓
CI/CD: Build, Test, Deploy to Dev
   ↓
Integration testing in Dev
   ↓
   PR to staging (approved by Product Owner)
   ↓
CI/CD: Build, Test, Deploy to Staging
   ↓
UAT testing in Staging (European team only)
   ↓
   PR to main (approved by Product Owner + Architect + Ops)
   ↓
CI/CD: Build, Security Scan, Deploy to Prod (West Europe)
   ↓
Deploy to Prod (North Europe) - manual approval

...

Acceptance Criteria:

CriterionValidation MethodExpected Result
Feature branches merge to development onlyAttempt direct merge to stagingPR blocked
Development branch auto-deploys to dev envMerge to developmentDev environment updated
Staging branch requires 2 approvalsCreate PR to stagingCannot merge with 1 approval
Main branch requires 3 approvalsCreate PR to mainCannot merge with 2 approvals
All tests must pass before staging mergeFailing test in PRMerge blocked
Security scan required for mainPR to mainSAST scan runs
Feature branches deleted after mergeMerge feature branchBranch auto-deleted

Dependencies:

  • Azure DevOps or GitHub repository
  • CI/CD pipeline configuration
  • Branch protection policies
  • Code review requirements

3.13 FR-013: Synthetic Test Data Generation (GDPR Compliance)

Requirement: The system shall provide a synthetic data generation tool that creates realistic but fake Swedish invoice data (personnummer, addresses, names) for use in staging and development environments, with zero production data copying.

Priority: CRITICAL

Synthetic Data Requirements:

Swedish Personnummer Generation:

Format: YYMMDD-XXXX
- YY: Year (00-99)
- MM: Month (01-12)
- DD: Day (01-31)
- XXXX: Last 4 digits with Luhn checksum

Generation Rules:
- Must pass Luhn algorithm validation
- Must NOT match any real personnummer
- Use test ranges: 19000101-19991231 (obviously fake dates)
- Flag as test: personnummer starts with "19"

...

Swedish Address Generation:

Street names: Random from Swedish street database
- Storgatan, Kungsgatan, Vasagatan, Drottninggatan...
- House numbers: 1-150
- Apartment: A-Z (optional)

Cities: Top 50 Swedish cities
- Stockholm, Göteborg, Malmö, Uppsala, Västerås...

Postal codes: Valid format (XXX XX) but non-existent ranges
- Use: 00X XX range (invalid but correct format)

Examples:
- Storgatan 45, 001 23 Stockholm
- Vasagatan 12 A, 002 45 Göteborg

...

Synthetic Invoice Data:

Invoice numbers: Test prefix "TEST-" + sequential
Amounts: Random between 100-5000 SEK
Consumption: Random 100-2000 kWh (residential realistic)
Metering points: Test range 735999999999999XXX
Email addresses: {firstname}.{lastname}@example-test.se
Phone numbers: +46701234XXX (test range)

...

Acceptance Criteria:

CriterionValidation MethodExpected Result
Generates valid personnummer (Luhn check)Validate 1000 generated numbersAll pass Luhn validation
Personnummer obviously fakeReview generated numbersAll start with "19" (invalid birth years)
Addresses realistic but invalidCheck against real postal databaseNo matches found
Email addresses use test domainCheck generated emailsAll @example-test.se
Phone numbers in test rangeCheck generated phonesAll +467012340XX
Can generate 10K invoice batchGenerate full batch10K valid invoices
Zero real data in outputScan for real personnummer patternsNo real data found
Reproducible (seed-based)Generate twice with same seedIdentical output

Dependencies:

  • Swedish personnummer validation library
  • Swedish postal code database (for validation, not generation)
  • Random data generation library

Risks & Mitigation:

RiskLikelihoodImpactMitigation StrategyOwner
Accidental real data generationLOWCRITICAL- Validation against known real ranges
- Visual "TEST DATA" watermark on PDFs
- Automated scanning for real personnummer
- Code review of generation logic
Security Officer
Unrealistic test scenariosMEDIUMMEDIUM- Generate edge cases library
- Long names, special characters
- Missing optional fields
- Various consumption patterns
QA Team
Offshore team needs production debuggingMEDIUMMEDIUM- European team creates synthetic scenarios
- Screen sharing for production issues
- Never copy production data
- Comprehensive logs without PII
Operations Manager

4. Non-Functional Requirements

4.1 NFR-001: Performance Targets

Requirement: The system shall meet specified performance targets for processing throughput, API latency, and rendering speed to support high-volume Nordic utilities operations.

Priority: HIGH

Performance Targets:

MetricTargetMeasurement MethodAcceptance ThresholdTest Scenario
Batch Processing Throughput10M invoices/monthMonthly invoice count in Application Insights≥ 10M in peak monthProduction monitoring
100K Batch Processing Time< 2 hoursTimestamp diff (queuedAt to completedAt)≤ 120 minutesLoad test TC-200
API Response Time (p50)< 200msApplication Insights percentiles≤ 200msLoad test TC-201
API Response Time (p95)< 500msApplication Insights percentiles≤ 500msLoad test TC-202
API Response Time (p99)< 1000msApplication Insights percentiles≤ 1000msLoad test TC-203
PDF Generation Time (p95)< 5 seconds/invoiceCustom metric tracking≤ 5 secondsRender test TC-204
Handlebars Rendering (p95)< 2 seconds/invoiceCustom metric tracking≤ 2 secondsRender test TC-205
Queue Processing Lag< 5 minutesQueue depth / throughput calculation≤ 5 minutesQueue monitoring
Database Query Time (p95)< 100msPostgreSQL slow query log≤ 100msQuery analysis
ParserService: 10K batch< 2 minutesParse duration measurement≤ 120 secondsParser test TC-206

Load Testing Scenarios:

Scenario 1: Steady State (Normal Month)

  • Duration: 24 hours
  • Load: 333K invoices distributed evenly
  • Concurrent batches: 5-10
  • Expected: All targets met, no errors

Scenario 2: Peak Load (Heating Season)

  • Duration: 8 hours
  • Load: 314K invoices (concentrated first week)
  • Concurrent batches: 20+
  • Expected: 2-hour SLA met, >99% success rate

Scenario 3: Spike Test

  • Duration: 30 minutes
  • Load: Sudden 10 batch uploads (50K invoices)
  • Expected: System auto-scales, processes without degradation

Acceptance Criteria:

CriterionValidation MethodTarget
Load test with 100K batch completesEnd-to-end test≤ 2 hours
API maintains p95 < 500ms under loadConcurrent API requests (1000 RPS)≤ 500ms
System processes 10M in peak monthProduction monitoring (Oct-Mar)≥ 10M
No performance degradation with 50 orgs50 orgs upload simultaneouslyAll SLAs met
Worker auto-scaling maintains lag < 5 minMonitor queue depth during peaksLag ≤ 5 min
PDF generation stays within targetRender 1000 PDFs, measure p95≤ 5 seconds

Dependencies:

  • Azure Container Apps auto-scaling configuration
  • Application Insights performance monitoring
  • Load testing tool (NBomber, k6, or JMeter)
  • Blob storage premium tier for high IOPS

Risks & Mitigation (Nordic Peak Season):

RiskLikelihoodImpactMitigation StrategyOwner
Heating season peaks exceed capacity (Oct-Mar)MEDIUMHIGH- Historical data analysis for peak prediction
- Pre-warm workers 1st/last week of month
- Priority queue for SLA customers
- Customer communication: off-peak scheduling
- Capacity planning review quarterly
Operations Manager
Template complexity slows renderingMEDIUMHIGH- Template performance guidelines
- POC testing with customer templates
- Recommend simple templates
- Compiled template caching
- Parallel rendering for 32 items
Technical Architect
Playwright memory issues at scaleHIGHHIGH- Semaphore: max 10 concurrent PDFs
- Worker memory limit: 2GB
- Browser instance pooling
- Monitor memory usage
- Scale horizontally (more workers)
Technical Architect
PostgreSQL connection exhaustionMEDIUMMEDIUM- Connection pooling (max 50 per service)
- Monitor active connections
- Timeout settings (30 seconds)
- Consider read replicas for heavy queries
Technical Architect

4.2 NFR-002: Scalability & Auto-Scaling

Requirement: The system shall scale horizontally without manual intervention to handle peak loads during Nordic heating season (October-March) and monthly invoice cycles.

Priority: HIGH

Scaling Configuration:

ComponentMin InstancesMax InstancesTrigger MetricThresholdScale Up TimeScale Down Time
CoreApiService520CPU Utilization OR Request Rate70% OR 1000 RPS2 minutes10 minutes
ParserService210Queue Length (batch-upload-queue)Length > 01 minute5 minutes
DocumentGenerator2100Queue Length (batch-items-queue)Length > 321 minute5 minutes
EmailService550Queue Length (email-queue)Length > 501 minute5 minutes
PostalService13Scheduled (not queue-based)12:00, 20:00 CETN/AAfter completion

Peak Load Capacity:

Normal Load (non-heating season, mid-month):

  • 333K invoices/day average
  • 5-10 concurrent batches
  • Worker instances: 10-20 total

Peak Load (heating season, first/last week):

  • 2.2M invoices/week (95% of monthly volume)
  • 314K invoices/day
  • 20+ concurrent batches
  • Worker instances: 80-100 total

Scaling Calculation:

Peak Day: 314,000 invoices
Processing time per invoice: 10 seconds (parse + render + PDF + deliver)
Total processing time: 314,000 × 10s = 872 hours
Target completion: 8 hours
Required workers: 872 / 8 = 109 workers
With 32 items/worker: 109 × 32 = 3,488 items processing simultaneously

...

Acceptance Criteria:

CriterionValidation MethodTarget
Auto-scaling triggered on queue depthMonitor scaling eventsScales within 2 min
Scaling up completes within 2 minutesMeasure from trigger to ready≤ 2 minutes
Scaling down after 10 min low loadMonitor scale-down timing≥ 10 minutes
Performance maintained during scalingMonitor API latency during scale eventsNo degradation
No message loss during scalingCount messages before/after100% preserved
Pre-warming for known peaksSchedule scale-up 1st/last weekWorkers ready
Max 100 DocumentGenerator instancesVerify max instance count≤ 100

Pre-Warming Strategy (Heating Season):

Monthly Schedule:
- Day 1-7: Pre-warm to 50 instances at 00:00
- Day 8-23: Scale based on queue (2-20 instances)
- Day 24-31: Pre-warm to 50 instances at 00:00

Heating Season (Oct-Mar): Double pre-warm levels
- Day 1-7: Pre-warm to 80 instances
- Day 24-31: Pre-warm to 80 instances

...

Dependencies:

  • Azure Container Apps with KEDA (Kubernetes Event-Driven Autoscaling)
  • Queue depth monitoring
  • Historical load data for pre-warming schedule

Risks & Mitigation:

RiskLikelihoodImpactMitigation StrategyOwner
Scale-up too slow for sudden spikeMEDIUMHIGH- Pre-warm during known peaks
- Keep min instances higher during peak season
- Queue backpressure (503) if overloaded
- Customer scheduling guidance
Operations Manager
Azure Container Apps 100-instance limitLOWHIGH- Priority queue for SLA customers
- Queue backpressure to throttle intake
- Consider split by organization
- Plan for Phase 2: dedicated worker pools
Technical Architect
Cost escalation during sustained peaksMEDIUMMEDIUM- Cost alerts at thresholds
- Auto-scale down aggressively
- Reserved instances for base load
- Monitor cost per invoice
Finance Controller

4.3 NFR-003: Availability & Reliability (Nordic 24/7 Operations)

Requirement: The system shall maintain 99.9% uptime with automatic failover, multi-region deployment, and recovery procedures to support Nordic utilities' 24/7 invoice delivery operations.

Priority: HIGH

Availability Targets:

MetricTargetAllowed DowntimeMeasurementConsequences of Breach
System Uptime99.9%43 min/monthAzure MonitorSLA credit to customers
Batch Success Rate> 99.5%50 failures per 10KProcessing logsInvestigation required
Delivery Success Rate> 98%200 failures per 10KDelivery trackingAlert to organization
API Availability99.9%43 min/monthHealth check monitoringIncident escalation
MTTR (Mean Time To Recovery)< 30 minutesN/AIncident timestampsProcess improvement
MTBF (Mean Time Between Failures)> 720 hours (30 days)N/AIncident trackingRoot cause analysis

Multi-Region Deployment:

Primary Region: West Europe (Azure westeurope)
- Sweden: Primary processing
- Denmark: Primary processing

Secondary Region: North Europe (Azure northeurope)
- Norway: Primary processing
- Finland: Primary processing
- Failover for Sweden/Denmark

Traffic Routing:
- Azure Traffic Manager with Performance routing
- Health check: /health endpoint every 30 seconds
- Auto-failover on 3 consecutive failed health checks
- Failover time: < 2 minutes

...

Recovery Time Objectives:

ScenarioRTO (Recovery Time)RPO (Data Loss)Recovery MethodResponsible Team
Worker Instance Crash< 5 minutes0 (idempotent)Automatic queue retryAutomatic
Database Failure< 15 minutes< 5 minutesAuto-failover to read replicaAutomatic + Ops verification
Primary Region Failure< 30 minutes< 15 minutesTraffic Manager failover to secondary regionOps Manager
Blob Storage Corruption< 1 hour< 1 hourRestore from blob version/snapshotOps Team
Queue Service Outage< 15 minutes0 (messages preserved)Wait for Azure recovery, messages retainedOps Manager
SendGrid Complete Outage< 2 hours0 (fallback to postal)Route all email invoices to postal queueOps Team
21G SFTP Unavailable< 4 hours0 (retry scheduled)Retry at next scheduled time (12:00/20:00)Ops Team

Backup & Recovery Strategy:

Blob Storage:

Replication: Geo-Redundant Storage (GRS)
  - Primary: West Europe
  - Secondary: North Europe
  - Automatic replication

Soft Delete: 7 days retention
  - Recover accidentally deleted blobs within 7 days

Blob Versioning: 30 days retention
  - Previous versions accessible
  - Rollback capability

Point-in-Time Restore: Not needed (blob versioning sufficient)

...

PostgreSQL:

Backup Schedule: Daily automated backups
Retention: 35 days
Backup Window: 02:00-04:00 CET (low traffic period)
Point-in-Time Restore: 7 days
Geo-Redundant: Enabled
Read Replica: North Europe (for failover)

...

Acceptance Criteria:

CriterionValidation MethodTarget
Multi-region deployment operationalVerify services in both regionsBoth regions active
Traffic Manager routes to healthy regionSimulate West Europe failureRoutes to North Europe
Database auto-failover testedSimulate primary DB failureFailover < 15 min
Blob geo-replication verifiedWrite to primary, read from secondaryData replicated
Health checks on all servicesGET /health on all endpointsAll return 200
Automated incident alerts configuredSimulate service failureAlert received within 5 min
Worker auto-restart on crashKill worker processNew instance starts
Queue message retry testedSimulate worker crash mid-processingMessage reprocessed
Disaster recovery drill quarterlySimulate complete region lossRecovery within RTO
Backup restoration tested monthlyRestore database from backupSuccessful restore

Dependencies:

  • Azure Traffic Manager configuration
  • Multi-region resource deployment
  • Database replication setup
  • Automated failover testing procedures
  • Incident response runbook

Risks & Mitigation (Nordic Context):

RiskLikelihoodImpactMitigation StrategyOwner
Both Azure regions fail simultaneouslyVERY LOWCRITICAL- Extremely rare (Azure multi-region SLA 99.99%)
- Accept risk (probability vs cost of 3rd region)
- Communication plan for extended outage
- Manual failover to Azure Germany (emergency)
Executive Sponsor
Network partition between regionsLOWHIGH- Each region operates independently
- Eventual consistency acceptable
- Manual reconciliation if partition >1 hour
- Traffic Manager handles routing
Technical Architect
Database failover causes brief downtimeLOWMEDIUM- Accept 1-2 minutes downtime during failover
- API returns 503 with Retry-After
- Queue-based processing unaffected
- Monitor failover duration
Operations Manager
Swedish winter storms affect connectivityLOWMEDIUM- Azure datacenter redundancy within region
- Monitor Azure status dashboard
- Communication plan for customers
- No physical office connectivity required
Operations Manager

4.4 NFR-004: Security Requirements

Requirement: The system shall implement comprehensive security controls including OAuth 2.0 authentication, role-based access control, encryption, audit logging, and protection against OWASP Top 10 vulnerabilities.

Priority: CRITICAL

4.4.1 Authentication & Authorization

OAuth 2.0 Implementation:

Grant Type: Client Credentials Flow (machine-to-machine)
Token Provider: Microsoft Entra ID
Token Lifetime: 1 hour
Refresh Token: 90 days
Token Format: JWT (JSON Web Token)
Algorithm: RS256 (RSA signature with SHA-256)

...

Required Claims in JWT:

{
  "aud": "api://eg-flow-api",
  "iss": "https://login.microsoftonline.com/{tenant}/v2.0",
  "sub": "user-object-id",
  "roles": ["Batch.Operator"],
  "organization_id": "123e4567-e89b-12d3-a456-426614174000",
  "exp": 1700226000,
  "nbf": 1700222400
}

...

Role Definitions & Permissions:

RoleScopePermissionsUse Case
Super AdminGlobal (all organizations)Full CRUD on all resources, cross-org visibilityEG internal support team
Organization AdminSingle organizationManage org users, configure settings, view all batchesUtility IT manager
Template AdminSingle organizationCreate/edit templates, manage template versionsUtility design team
Batch OperatorSingle organizationUpload batches, start processing, view statusUtility billing team
Read-Only UserSingle organizationView batches, download invoices, view reportsUtility customer service
API ClientSingle organizationProgrammatic batch upload and status queriesBilling system integration

Acceptance Criteria:

CriterionValidation MethodTarget
OAuth 2.0 token required for all endpoints (except /health)Call API without token401 Unauthorized
JWT token validated (signature, expiration, audience)Tampered token, expired token401 Unauthorized
Refresh tokens work for 90 daysUse refresh token after 30 daysNew access token issued
All 6 roles implemented in PostgreSQLQuery roles table6 roles present
Users can only access their organizationUser A calls Org B endpoint403 Forbidden
All actions logged to audit_logs tablePerform action, query audit_logsEntry created
API authentication middleware on all routesAttempt bypassAll protected
MFA enforced for Super AdminLogin as Super AdminMFA challenge
MFA enforced for Org AdminLogin as Org AdminMFA challenge
Failed logins logged3 failed login attempts3 entries in audit_logs
Account lockout after 5 failed attempts6 failed login attempts15-minute lockout
API key rotation every 90 daysCheck Key Vault secret ageAlert at 80 days

4.4.2 Data Protection

Encryption Standards:

In Transit:
- TLS 1.3 minimum (TLS 1.2 acceptable)
- Cipher suites: AES-256-GCM, ChaCha20-Poly1305
- Certificate: Wildcard cert for *.egflow.com
- HSTS: max-age=31536000; includeSubDomains

At Rest:
- Azure Blob Storage: AES-256 (Microsoft-managed keys)
- PostgreSQL: AES-256 (Microsoft-managed keys)
- Backups: AES-256 encryption
- Customer-managed keys (CMK): Phase 2 option

Sensitive Data Fields (extra protection):
- Personnummer: Encrypted column in database (if stored)
- API keys: Azure Key Vault only
- Email passwords: Never stored
- Customer addresses: Standard blob encryption sufficient

...

Acceptance Criteria:

CriterionValidation MethodTarget
All API traffic over HTTPSAttempt HTTP requestRedirect to HTTPS or reject
TLS 1.3 or 1.2 enforcedCheck TLS version in trafficTLS ≥ 1.2
Data encrypted at rest (blob)Verify Azure encryption settingsEnabled
Data encrypted at rest (PostgreSQL)Verify DB encryptionEnabled
Secrets in Azure Key Vault onlyCode scan for hardcoded secretsZero secrets in code
No credentials in source controlGit history scanZero credentials
Database connections use managed identityCheck connection stringsNo passwords
Personnummer not in URLsURL pattern analysisNo personnummer patterns
Personnummer not in logsLog analysisNo personnummer found

4.4.3 Application Security (OWASP Top 10)

Security Measures:

OWASP RiskMitigationValidation
A01: Broken Access ControlOrganization middleware, RBAC enforcementPenetration testing
A02: Cryptographic FailuresTLS 1.3, AES-256, Key VaultSecurity scan
A03: InjectionParameterized queries, input validationSQL injection testing
A04: Insecure DesignThreat modeling, security reviewArchitecture review
A05: Security MisconfigurationAzure security baseline, CIS benchmarksConfiguration audit
A06: Vulnerable ComponentsDependabot, automated scanningWeekly scan
A07: Authentication FailuresOAuth 2.0, MFA, rate limitingPenetration testing
A08: Software/Data IntegrityCode signing, SRI, checksumsBuild verification
A09: Logging FailuresComprehensive audit loggingLog completeness review
A10: SSRFURL validation, allowlistSecurity testing

Input Validation:

// Example: Batch upload validation with FluentValidation
public class BatchUploadValidator : AbstractValidator<BatchUploadRequest>
{
    public BatchUploadValidator()
    {
        RuleFor(x => x.File)
            .NotNull().WithMessage("File is required")
            .Must(BeValidXml).WithMessage("File must be valid XML")
            .Must(BeLessThan100MB).WithMessage("File must be less than 100MB");
        
        RuleFor(x => x.Metadata.BatchName)
            .NotEmpty().WithMessage("Batch name is required")
            .Length(1, 255).WithMessage("Batch name must be 1-255 characters")
            .Must(NotContainPathSeparators).WithMessage("Batch name cannot contain / or \\")
            .Must(NoSQLInjectionPatterns).WithMessage("Invalid characters in batch name");
        
        RuleFor(x => x.Metadata.Priority)
            .Must(x => x == "normal" || x == "high")
            .WithMessage("Priority must be 'normal' or 'high'");
    }
    
    private bool NoSQLInjectionPatterns(string input)
    {
        var sqlPatterns = new[] { "--", "/*", "*/", "xp_", "sp_", "';", "\";" };
        return !sqlPatterns.Any(p => input.Contains(p, StringComparison.OrdinalIgnoreCase));
    }
}

...

Acceptance Criteria:

CriterionValidation MethodTarget
Input validation on all API endpointsSend malicious inputRejected with error
SQL injection preventedAttempt SQL injection in batch nameSanitized/rejected
XSS prevented in templatesInject script tags in templateSanitized on render
XML external entity (XXE) attack preventedUpload XXE payloadParsing rejects
Billion laughs attack preventedUpload billion laughs XMLParsing rejects/times out safely
File upload size enforcedUpload 101MB fileRejected at API gateway
Rate limiting prevents abuse1000 rapid API calls429 after limit
CSRF protection (future web UI)Attempt CSRF attackBlocked by token
Dependency vulnerabilities scanned weeklyRun DependabotAlerts for high/critical
Security headers presentCheck HTTP responseX-Frame-Options, CSP, etc.

4.4.4 Network Security

Acceptance Criteria:

CriterionStatusPhase
DDoS protection enabled (Azure basic)✅ IncludedPhase 1
IP whitelisting support for API clients✅ Optional featurePhase 1
VNet integration for Container Apps⚠️ Phase 2Phase 2
Private endpoints for Blob Storage⚠️ Phase 2Phase 2
Network Security Groups (NSGs)⚠️ Phase 2Phase 2
Azure Firewall for egress filtering⚠️ Phase 2Phase 2

Dependencies:

  • FluentValidation library
  • OWASP dependency check tools
  • Penetration testing (external vendor)
  • Security code review process

Risks & Mitigation (Nordic/EU Security Context):

RiskLikelihoodImpactMitigation StrategyOwner
NIS2 Directive compliance (EU critical infrastructure)MEDIUMCRITICAL- Energy sector falls under NIS2
- Incident reporting procedures (24h to authorities)
- Security measures documentation
- Annual security audit
- CISO designated
Legal/Compliance
Swedish Säkerhetspolisen (SÄPO) requirementsLOWHIGH- Enhanced security for critical infrastructure
- Incident reporting to MSB (Swedish Civil Contingencies)
- Employee background checks for production access
- Security clearance for key personnel
Security Officer
API key theft/leakageMEDIUMHIGH- Rotate keys every 90 days
- Monitor for leaked keys (GitHub scanning)
- Revoke compromised keys immediately
- API key hashing in database
- Never log full API keys
Security Officer
Insider threat (privileged access abuse)LOWCRITICAL- Least privilege principle
- All actions audited
- Regular access reviews
- Separation of duties
- Anomaly detection in audit logs
Security Officer
Third-party vendor breach (SendGrid, 21G)LOWHIGH- Data Processing Agreements (DPAs) signed
- Regular vendor security assessments
- Minimal data sharing
- Encryption in transit to vendors
- Vendor breach response plan
Legal/Compliance

4.5 NFR-005: Data Retention & Lifecycle Management

Requirement: The system shall manage data retention according to Swedish accounting law (7-year invoice retention) with automated lifecycle policies for cost optimization.

Priority: HIGH

Retention Policies:

Data TypeLegal RequirementRetention PeriodStorage Tier TransitionDisposal Method
Invoices (PDF/HTML/JSON)Bokföringslagen (Swedish Accounting Act)7 years from fiscal year endDay 0-365: Hot
Day 366-2555: Cool
Day 2556+: Archive
Permanent deletion after 7 years
Batch Source Files (XML)None (internal processing)90 daysDay 0-30: Hot
Day 31-90: Cool
Day 91+: Delete
Automatic deletion
Batch Metadata JSONAudit trail90 daysDay 0-90: Hot
Day 91+: Delete
Automatic deletion
Audit Logs (PostgreSQL)GDPR, Swedish law7 yearsYear 0-1: PostgreSQL
Year 1-7: Blob (compressed)
Deletion after 7 years
Application LogsOperational90 daysApplication InsightsAutomatic deletion
TemplatesBusiness continuityIndefinite (archived versions)Hot (active)
Cool (archived)
Never deleted
Organization ConfigBusiness continuityIndefiniteHotNever deleted (updated in place)

Azure Blob Lifecycle Policy:

{
  "rules": [
    {
      "enabled": true,
      "name": "invoice-lifecycle",
      "type": "Lifecycle",
      "definition": {
        "filters": {
          "blobTypes": ["blockBlob"],
          "prefixMatch": ["invoices-"]
        },
        "actions": {
          "baseBlob": {
            "tierToCool": {
              "daysAfterModificationGreaterThan": 365
            },
            "tierToArchive": {
              "daysAfterModificationGreaterThan": 2555
            },
            "delete": {
              "daysAfterModificationGreaterThan": 2920
            }
          }
        }
      }
    },
    {
      "enabled": true,
      "name": "batch-source-cleanup",
      "type": "Lifecycle",
      "definition": {
        "filters": {
          "blobTypes": ["blockBlob"],
          "prefixMatch": ["batches-"]
        },
        "actions": {
          "baseBlob": {
            "tierToCool": {
              "daysAfterModificationGreaterThan": 30
            },
            "delete": {
              "daysAfterModificationGreaterThan": 90
            }
          }
        }
      }
    }
  ]
}

...

Storage Growth Projection:

Assumptions:

  • 10M invoices/month
  • 85KB per invoice (50KB PDF + 30KB HTML + 5KB JSON)
  • 7-year retention

Growth Over Time:

YearNew Data/MonthCumulative TotalPrimary Storage TierSecondary Tier
Year 1850 GB10.2 TBHot (10.2 TB)-
Year 2850 GB20.4 TBHot (10.2 TB)Cool (10.2 TB)
Year 3850 GB30.6 TBHot (10.2 TB)Cool (20.4 TB)
Year 7850 GB71.4 TBHot (10.2 TB)Cool (10.2 TB), Archive (51 TB)

Storage Tier Pricing Impact:

With lifecycle policies (Hot → Cool → Archive):

  • Year 1-2: Manageable with hot storage
  • Year 3-7: Significant savings with tiering (estimated 85% reduction vs all-hot)

Acceptance Criteria:

CriterionValidation MethodTarget
Automated lifecycle policies configuredCheck Azure policyPolicies active
Data transitions to Cool after 1 yearVerify tier of 13-month-old invoiceCool tier
Data transitions to Archive after 7 yearsVerify tier of 7-year-old invoiceArchive tier
7-year invoice retention enforcedAttempt to access 8-year-old invoiceDeleted (404)
Old batch files deleted after 90 daysCheck for 91-day-old batch fileDeleted (404)
Retention policy exceptions supportedTag invoice with legal holdNot deleted despite age
Legal hold prevents deletionSet legal hold, verify no deletionInvoice retained
Data restoration from Archive within 24hRequest archived invoiceRetrieved within 24h
Templates never automatically deletedCheck template ageOld templates present (archived)

Legal Hold Functionality:

{
  "invoiceId": "uuid",
  "legalHold": {
    "enabled": true,
    "reason": "Customer dispute - case #12345",
    "appliedBy": "user-uuid",
    "appliedAt": "2025-11-21T10:00:00Z",
    "expiresAt": null
  }
}

...

Dependencies:

  • Azure Blob lifecycle management
  • Legal hold tagging mechanism
  • Retention compliance monitoring
  • Archive tier data retrieval procedures

Risks & Mitigation (Swedish Legal Context):

RiskLikelihoodImpactMitigation StrategyOwner
Bokföringslagen (Accounting Act) non-complianceLOWCRITICAL- 7-year retention strictly enforced
- Legal opinion obtained
- Retention policy reviewed by auditor
- Automated compliance reporting
- Skatteverket (Tax Agency) audit trail
Legal/Compliance
Premature invoice deletionLOWHIGH- Lifecycle policy testing in staging
- Deletion logging and alerts
- Soft delete (7-day recovery)
- Annual retention audit
Operations Manager
Storage costs exceed budgetMEDIUMMEDIUM- Lifecycle policies reduce costs 85%
- Cost monitoring and alerts
- Quarterly cost review
- Consider compression for PDFs
Finance Controller
Archive retrieval SLA breachLOWMEDIUM- Document 24-hour SLA for archive
- Test archive retrieval monthly
- Maintain critical invoices in Cool (not Archive)
Operations Manager

4.6 NFR-006: Monitoring, Logging & Observability

Requirement: The system shall provide comprehensive monitoring through Application Insights, structured logging with Serilog, real-time dashboards with 5-minute refresh, and automated alerting with escalation procedures.

Priority: HIGH

4.6.1 Structured Logging Standards

Serilog Configuration:

Log.Logger = new LoggerConfiguration()
    .MinimumLevel.Information()
    .MinimumLevel.Override("Microsoft", LogEventLevel.Warning)
    .MinimumLevel.Override("Microsoft.AspNetCore", LogEventLevel.Warning)
    .Enrich.FromLogContext()
    .Enrich.WithProperty("Application", "EGU.Flow")
    .Enrich.WithProperty("Environment", environment)
    .Enrich.WithProperty("Region", azureRegion)
    .Enrich.WithMachineName()
    .Enrich.WithThreadId()
    .WriteTo.ApplicationInsights(
        connectionString,
        TelemetryConverter.Traces,
        LogEventLevel.Information)
    .WriteTo.Console(new CompactJsonFormatter())
    .CreateLogger();

...

Log Entry Structure:

{
  "timestamp": "2025-11-21T10:30:00.123Z",
  "level": "Information",
  "messageTemplate": "Batch {BatchId} processing started for organization {OrganizationId}. Items: {ItemCount}",
  "message": "Batch 550e8400... processing started for organization 123e4567.... Items: 5000",
  "properties": {
    "BatchId": "550e8400-e29b-41d4-a716-446655440000",
    "OrganizationId": "123e4567-e89b-12d3-a456-426614174000",
    "ItemCount": 5000,
    "CorrelationId": "corr-abc-123",
    "Application": "EGU.Flow",
    "Environment": "Production",
    "Region": "westeurope",
    "MachineName": "worker-001"
  }
}

...

PII Masking Rules:

NEVER log:
- Personnummer (Swedish social security numbers)
- Full customer names (log customer ID only)
- Email addresses (log domain only: user@***@example.se)
- Phone numbers (log prefix only: +4670****)
- Street addresses (log city only)
- Bank account numbers
- API keys or tokens

SAFE to log:
- Organization IDs (UUIDs)
- Batch IDs (UUIDs)
- Invoice IDs (UUIDs)
- Invoice numbers (business references)
- Processing statistics
- Error codes and messages
- Performance metrics

...

4.6.2 Application Insights Dashboards

Dashboard 1: Operations (Real-Time)

Refresh: Every 5 minutes

Metrics:

  • Active batches count (gauge)
  • Queue depths (4 queues, time series chart)
  • Worker instance counts per service (bar chart)
  • Processing throughput (items/minute, time series)
  • Error rate (percentage, last hour vs last 24h)
  • System health status (green/yellow/red indicators)
  • Current API request rate (RPS)

Dashboard 2: Performance

Refresh: Every 5 minutes

Metrics:

  • API response times (p50, p95, p99 - line chart)
  • Batch processing duration (histogram)
  • PDF generation times (p50, p95, p99)
  • Handlebars rendering times (p50, p95, p99)
  • Delivery latency by channel (email vs postal)
  • Worker CPU/memory utilization
  • Database query performance (slow query tracking)

Dashboard 3: Business

Refresh: Hourly

Metrics:

  • Invoices processed (today, this week, this month - counters)
  • Delivery channel breakdown (pie chart: email/postal)
  • Failed deliveries by reason (bar chart)
  • Top 10 organizations by volume (bar chart)
  • Processing trends (daily invoice count, 30-day chart)
  • Monthly invoice volumes (seasonal view)

Dashboard 4: Vendor Formats

Refresh: Hourly

Metrics:

  • Batches by vendor format (pie chart: GASEL/XELLENT/ZYNERGY)
  • Parsing success rate by vendor (percentage gauges)
  • Average batch size by vendor
  • Parsing duration by vendor
  • Validation errors by vendor format

4.6.3 Alert Rules & Escalation

Critical Alerts (5-minute evaluation window):

Alert NameCondition (Kusto Query)SeverityRecipientsEscalation (after 15 min)
High Error Ratetraces | where severityLevel >= 3 | count > 50HighOps teamDev team + On-call
Queue Depth CriticalcustomMetrics | where name == 'Queue.Depth' and value > 10000HighOps teamProduct Owner
Worker Crash Spiketraces | where message contains 'Worker crashed' | count > 3CriticalOps + Dev teamsCTO
Delivery Failure RatecustomMetrics | where name startswith 'Delivery' and value < 0.9MediumOps teamCustomer success
API Response Degradedrequests | summarize p95=percentile(duration, 95) | where p95 > 1000MediumOps teamTechnical Architect
Batch Processing TimeoutcustomMetrics | where name == 'Batch.Duration' and value > 120HighOps teamProduct Owner
Database Connection Errorsexceptions | where type contains 'Npgsql' | count > 10CriticalOps + DBACTO
Blob Storage Throttlingexceptions | where message contains '503 Server Busy' | count > 20HighOps teamTechnical Architect
SendGrid Deliverability DropSendGrid webhook: bounceRate > 10%HighOps teamEmail deliverability specialist
21G SFTP Connection FailureSFTP connection exceptionsHighOps team21G account manager

Alert Delivery:

  • Primary: Email to ops team distribution list
  • Secondary: SMS to on-call engineer
  • Escalation: PagerDuty incident creation
  • Integration: Create Jira ticket automatically

On-Call Rotation:

  • European team: 24/7 coverage (Swedish, Danish time zones)
  • Shift schedule: Week-long rotations
  • Handoff procedure: Thursday 09:00 CET

Acceptance Criteria:

CriterionValidation MethodTarget
Dashboards refresh every 5 minutesCheck dashboard timestamp≤ 5 min old
Data retained for 90 daysCheck oldest data in dashboard90 days accessible
Dashboards accessible to authorized usersLogin as different rolesAppropriate access
Critical alerts trigger within 5 minSimulate high error rateAlert within 5 min
Alert escalation after 15 minDon't acknowledge alertEscalation triggered
PII masked in logsSearch logs for personnummer regexZero matches
Correlation IDs trace requestsFollow request across servicesSame ID throughout
Log retention 90 daysCheck Application Insights retention90 days
Structured logging in JSON formatParse log entriesValid JSON

Custom Metrics Tracked:

public class MetricsService
{
    private readonly TelemetryClient _telemetry;
    
    public void TrackBatchProcessing(BatchMetadata batch)
    {
        _telemetry.TrackMetric("Batch.TotalItems", 
            batch.Statistics.TotalItems,
            new Dictionary<string, string> {
                ["OrganizationId"] = batch.OrganizationId,
                ["VendorCode"] = batch.VendorInfo.VendorCode
            });
        
        _telemetry.TrackMetric("Batch.Duration", 
            (batch.Timestamps.CompletedAt - batch.Timestamps.StartedAt)?.TotalMinutes ?? 0);
        
        _telemetry.TrackMetric("Batch.SuccessRate", 
            (double)batch.Statistics.SuccessfulItems / batch.Statistics.TotalItems * 100);
    }
    
    public void TrackDelivery(string channel, bool success, string organizationId)
    {
        _telemetry.TrackMetric($"Delivery.{channel}.Success",
            success ? 1 : 0,
            new Dictionary<string, string> {
                ["OrganizationId"] = organizationId,
                ["Channel"] = channel
            });
    }
    
    public void TrackQueueDepth(string queueName, int depth)
    {
        _telemetry.TrackMetric("Queue.Depth", depth,
            new Dictionary<string, string> {
                ["QueueName"] = queueName
            });
    }
    
    public void TrackVendorParsing(string vendorCode, bool success, long durationMs)
    {
        _telemetry.TrackMetric("Parser.Duration", durationMs,
            new Dictionary<string, string> {
                ["VendorCode"] = vendorCode,
                ["Success"] = success.ToString()
            });
    }
}

...

Kusto Queries for Common Operations:

Query 1: Failed Batches (Last 24 Hours)

traces
| where timestamp > ago(24h)
| where customDimensions.BatchId != ""
| where severityLevel >= 3
| summarize 
    ErrorCount = count(),
    FirstError = min(timestamp),
    LastError = max(timestamp)
    by 
    BatchId = tostring(customDimensions.BatchId),
    OrganizationId = tostring(customDimensions.OrganizationId),
    ErrorMessage = message
| order by ErrorCount desc
| take 50

...

Query 2: Queue Depth Trending

customMetrics
| where name == "Queue.Depth"
| where timestamp > ago(24h)
| extend QueueName = tostring(customDimensions.QueueName)
| summarize 
    AvgDepth = avg(value),
    MaxDepth = max(value),
    MinDepth = min(value)
    by QueueName, bin(timestamp, 5m)
| render timechart

...

Query 3: Vendor Format Performance

customMetrics
| where name == "Parser.Duration"
| where timestamp > ago(7d)
| extend VendorCode = tostring(customDimensions.VendorCode)
| summarize 
    p50 = percentile(value, 50),
    p95 = percentile(value, 95),
    p99 = percentile(value, 99),
    BatchCount = count()
    by VendorCode
| order by p95 desc

...

Dependencies:

  • Application Insights workspace (90-day retention)
  • Serilog sinks for Application Insights and Console
  • Alert action groups configured
  • Dashboard permissions configured
  • PagerDuty or similar on-call system

4.7 NFR-007: Disaster Recovery & Business Continuity

Requirement: The system shall have documented disaster recovery procedures with defined RTO/RPO targets, tested quarterly, to ensure business continuity for Nordic utility customers.

Priority: HIGH

Disaster Recovery Scenarios:

Scenario 1: Worker Instance Crash

Trigger: DocumentGenerator worker crashes during 32-item batch processing

Detection: Worker stops sending heartbeats, Azure Container Apps detects unhealthy instance

Recovery Procedure:

1. Azure Container Apps automatically starts new instance (2 min)
2. Queue message visibility timeout expires (5 min)
3. Message becomes visible in batch-items-queue again
4. New worker instance picks up message
5. Worker checks for already-processed invoices (idempotency)
6. Skips completed invoices, processes remaining items
7. Blob lease ensures no concurrent processing

...

RTO: < 5 minutes (automatic)
RPO: 0 (no data loss, idempotent operations)
Responsible: Automatic + Operations Manager (monitoring)

Scenario 2: PostgreSQL Database Failure

Trigger: Primary PostgreSQL instance becomes unresponsive

Detection: Health check failures, connection timeout errors in logs

Recovery Procedure:

1. Azure detects primary failure via health probes (30 seconds)
2. Auto-failover to read replica in secondary region (5 min)
3. DNS updated to point to new primary
4. Application reconnects automatically (connection retry logic)
5. Verify data integrity post-failover
6. Notify stakeholders of failover event
7. Investigate root cause

...

RTO: < 15 minutes
RPO: < 5 minutes (replication lag)
Responsible: Operations Manager (monitoring), DBA (verification)

Scenario 3: Azure Region Failure (West Europe)

Trigger: Complete West Europe region outage

Detection: Traffic Manager health checks fail for all West Europe endpoints

Recovery Procedure:

1. Traffic Manager detects 3 consecutive health check failures (90 seconds)
2. Traffic Manager routes all traffic to North Europe (2 min)
3. North Europe region activates read replica database as primary
4. North Europe workers process queues (messages replicated via GRS)
5. Verify system operational in North Europe
6. Communicate to customers about region failover
7. Monitor for West Europe recovery
8. Plan failback when West Europe restored

...

RTO: < 30 minutes
RPO: < 15 minutes (blob replication lag)
Responsible: Operations Manager (execution), CTO (decision), Communications (customer notification)

Scenario 4: Blob Storage Corruption

Trigger: Critical blob (organization config, template) becomes corrupted or accidentally deleted

Detection: Blob read errors, validation failures, user reports

Recovery Procedure:

1. Identify corrupted blob path and organization
2. Check soft delete (7-day retention):
   - If within 7 days: Undelete blob immediately
3. If soft delete expired, check blob versions:
   - Restore from previous version
4. If no versions, restore from geo-redundant copy:
   - Access secondary region blob storage
   - Copy to primary region
5. Verify restored blob integrity
6. Test with sample batch
7. Root cause analysis

...

RTO: < 1 hour
RPO: < 1 hour (version interval)
Responsible: Operations Manager (execution), Technical Architect (verification)

Scenario 5: Complete Data Loss (Catastrophic)

Trigger: Theoretical scenario - both regions and all backups lost

Detection: N/A (highly unlikely with Azure GRS)

Recovery Procedure:

1. Declare disaster
2. Restore PostgreSQL from geo-redundant backup (35-day retention)
3. Organizations re-upload batch files (source systems have copies)
4. Customers re-notified about invoices
5. Incident post-mortem and Azure investigation

...

RTO: < 4 hours
RPO: < 24 hours (last backup)
Responsible: CTO (declaration), All teams (execution)

Note: This scenario has probability < 0.001% given Azure GRS + geo-redundant backups + multi-region deployment.

Disaster Recovery Testing:

Test TypeFrequencyScopePass Criteria
Worker Crash TestMonthlyKill random worker mid-processingRecovery < 5 min, no data loss
Database Failover TestQuarterlyForce failover to replicaRecovery < 15 min, queries work
Region Failover DrillAnnuallySimulate West Europe outageRecovery < 30 min, all services operational
Backup Restoration TestMonthlyRestore PostgreSQL from backupSuccessful restore, data integrity verified
Blob Undelete TestQuarterlyDelete critical blob, restoreSuccessful recovery within 1 hour

Acceptance Criteria:

CriterionValidation MethodTarget
Multi-region deployment activeVerify services in both regionsBoth regions operational
Traffic Manager failover testedSimulate region failureFailover < 2 min
Database auto-failover testedForce primary DB failureFailover < 15 min
Blob geo-replication verifiedWrite to primary, read from secondaryData present
Disaster recovery procedures documentedReview runbook completeness100% complete
DR drill conducted quarterlyCheck last drill dateWithin 90 days
Backup restoration tested monthlyCheck last restore testWithin 30 days
Recovery procedures automated where possibleReview manual steps< 5 manual steps

4.8 NFR-008: Data Consistency (Eventual Consistency Model)

Requirement: The system shall maintain data consistency using blob leases for exclusive access, ETag-based optimistic concurrency for metadata updates, and idempotent operations for safe retries.

Priority: HIGH

Consistency Guarantees:

Strong Consistency (Within Single Operation):

  • Blob lease acquisition: One worker per 32-item batch
  • PostgreSQL transactions: User/role operations ACID
  • Queue message delivery: At-least-once delivery

Eventual Consistency (Across System):

  • Batch statistics: Updated within 30 seconds
  • Invoice status: Updated within 1 minute
  • Dashboard metrics: Updated within 5 minutes
  • Audit logs: Written asynchronously

Consistency Mechanisms:

1. Blob Lease for Exclusive Access:

// Ensures only one worker processes 32-item batch
var lease = await AcquireBlobLeaseAsync(
    container: "{org}-batches-{year}",
    blob: "locks/{batch-id}/{message-id}.lock",
    duration: TimeSpan.FromMinutes(5));

try {
    // Process 32 invoices exclusively
    await ProcessBatchItemsAsync(message);
}
finally {
    await ReleaseBlobLeaseAsync(lease);
}

...

2. ETag-Based Optimistic Concurrency:

// Prevents lost updates to batch metadata
var download = await blobClient.DownloadContentAsync();
var etag = download.Value.Details.ETag;
var metadata = JsonSerializer.Deserialize<BatchMetadata>(download.Value.Content);

// Update metadata
metadata.Statistics.ProcessedItems += 32;
metadata.Metadata.UpdatedAt = DateTime.UtcNow;
metadata.Metadata.Version++;

// Upload with ETag condition
await blobClient.UploadAsync(content, new BlobUploadOptions {
    Conditions = new BlobRequestConditions { IfMatch = etag }
});
// Throws if ETag doesn't match (another worker updated)

...

3. Idempotent Operations:

// Safe to retry without duplicates
public async Task ProcessInvoiceAsync(string invoiceId)
{
    // Check if already processed
    var metadata = await TryGetInvoiceMetadataAsync(invoiceId);
    if (metadata?.Status == "delivered")
    {
        _logger.LogInformation("Invoice {InvoiceId} already processed, skipping", invoiceId);
        return; // Idempotent: safe to skip
    }
    
    // Process invoice (creates new blobs, doesn't update existing)
    await RenderInvoiceAsync(invoiceId);
    await GeneratePdfAsync(invoiceId);
    await EnqueueForDeliveryAsync(invoiceId);
}

...

Acceptance Criteria:

CriterionValidation MethodTarget
Blob leases prevent concurrent processingSend same message to 2 workersOnly 1 processes
ETags prevent lost updates2 workers update same metadataNo lost updates
Retry operations are idempotentRetry invoice processing 3xProcessed once only
No duplicate invoices generatedCrash during processing, retryOne PDF created
Concurrent batch updates handled10 workers update statisticsAll updates applied
Race conditions preventedConcurrent access testingNo race conditions
Data integrity after crashKill worker, verify data stateConsistent state

Dependencies:

  • Azure Blob Storage lease API
  • ETag support in blob operations
  • Retry logic with idempotency checks
  • Worker coordination mechanisms

4.9 NFR-009: Multi-Region Data Residency (Nordic Compliance)

Requirement: The system shall enforce data residency requirements for Nordic countries, ensuring Swedish/Danish data stays in West Europe and Norwegian/Finnish data stays in North Europe, with no cross-border transfers except encrypted backups.

Priority: HIGH

Data Residency Rules:

CountryCustomer BasePrimary RegionData ResidencyBackup RegionRationale
Sweden (SE)~10M population, largest marketWest EuropeEnforcedNorth Europe (encrypted)GDPR, Swedish Data Protection Law
Denmark (DK)~6M populationWest EuropeEnforcedNorth Europe (encrypted)GDPR, Danish data laws
Norway (NO)~5M populationNorth EuropeEnforcedWest Europe (encrypted)GDPR, Norwegian data laws, EEA regulations
Finland (FI)~5M populationNorth EuropeEnforcedWest Europe (encrypted)GDPR, Finnish data laws

Traffic Routing Logic:

public class RegionRoutingService
{
    public string GetProcessingRegion(string organizationId)
    {
        var org = await _orgService.GetOrganizationAsync(organizationId);
        
        // Route based on organization's country
        return org.CountryCode switch
        {
            "SE" => "westeurope",    // Sweden → West Europe
            "DK" => "westeurope",    // Denmark → West Europe
            "NO" => "northeurope",   // Norway → North Europe
            "FI" => "northeurope",   // Finland → North Europe
            _ => "westeurope"        // Default: West Europe
        };
    }
    
    public async Task<bool> ValidateDataResidencyAsync(string organizationId, string requestRegion)
    {
        var requiredRegion = GetProcessingRegion(organizationId);
        
        if (requestRegion != requiredRegion)
        {
            _logger.LogWarning(
                "Data residency violation attempted. Org: {OrgId}, Required: {Required}, Attempted: {Attempted}",
                organizationId, requiredRegion, requestRegion);
            
            return false;
        }
        
        return true;
    }
}

...

Organization Configuration:

{
  "organizationId": "uuid",
  "organizationCode": "VATTENFALL-SE",
  "countryCode": "SE",
  "dataResidency": {
    "primaryRegion": "westeurope",
    "allowedRegions": ["westeurope"],
    "backupRegions": ["northeurope"],
    "crossRegionProcessing": false,
    "crossRegionBackup": true
  }
}

...

Acceptance Criteria:

CriterionValidation MethodTarget
Swedish orgs process in West EuropeVerify blob container regionwesteurope
Norwegian orgs process in North EuropeVerify blob container regionnortheurope
Cross-region processing blockedAttempt to process Swedish org in North EuropeRejected
Cross-region backup allowedVerify geo-redundant replicationEnabled
Latency < 100ms within regionAPI latency from Nordic countries< 100ms
Automatic failover to secondary regionSimulate primary region failureFailover works
Data residency config per organizationUpdate org configSetting honored
Audit trail for cross-region accessAttempt cross-region, check logsAttempt logged

Dependencies:

  • Azure Traffic Manager with geographic routing
  • Blob storage geo-redundant replication
  • Organization configuration enforcement
  • Multi-region deployment automation

Risks & Mitigation (Nordic Legal Context):

RiskLikelihoodImpactMitigation StrategyOwner
Schrems II implications (EU-US data transfer)LOWHIGH- No US region usage
- All data in EU (West/North Europe only)
- Azure EU Data Boundary compliance
- Standard Contractual Clauses with vendors
Legal/Compliance
Norwegian data sovereignty concernsLOWMEDIUM- North Europe primary for Norwegian orgs
- Option for Norway-only processing
- No data transfer to other Nordics without consent
- Compliance with Norwegian regulations
Legal/Compliance
Data residency audit by Datatilsynet (NO) or Datatilsynet (DK)LOWHIGH- Documented data flows
- Data residency configuration auditable
- Logs prove compliance
- Annual self-assessment
Legal/Compliance

4.10 NFR-010: Maintainability & Code Quality

Requirement: The system shall be designed for long-term maintainability with clear code standards, comprehensive documentation, high test coverage, and adherence to .NET best practices.

Priority: MEDIUM

Code Quality Standards:

Project Structure (Updated per Oct 27 decision):

EGU.Flow/
├── EGU.Flow.AppHost/                    # .NET Aspire orchestration
│   └── Program.cs
│
├── EGU.Flow.Core/                       # Shared contracts, DTOs, interfaces
│   ├── DTOs/
│   │   ├── BatchUploadRequest.cs
│   │   ├── CanonicalInvoice.cs
│   │   └── DeliveryRequest.cs
│   ├── Enums/
│   │   ├── BatchStatus.cs
│   │   ├── DeliveryChannel.cs
│   │   └── VendorCode.cs
│   ├── Interfaces/
│   │   ├── IXmlParserService.cs
│   │   ├── ITemplateRenderingService.cs
│   │   └── IDeliveryService.cs
│   └── Common/
│       ├── Constants.cs
│       └── ErrorCodes.cs
│
├── EGU.Flow.Domain/                     # Domain models and business logic
│   ├── Models/
│   │   ├── Organization.cs
│   │   ├── Batch.cs
│   │   ├── BatchItem.cs
│   │   ├── Template.cs
│   │   └── TemplateCategory.cs
│   └── DomainServices/
│       ├── VendorDetectionService.cs
│       └── DistributionRoutingService.cs
│
├── EGU.Flow.BusinessLogic/              # Business services
│   ├── Services/
│   │   ├── BatchService.cs
│   │   ├── OrganizationService.cs
│   │   ├── TemplateService.cs
│   │   └── SchemaRegistryService.cs
│   └── Mappers/
│       ├── GaselMapper.cs
│       ├── XellentMapper.cs
│       └── ZynergyMapper.cs
│
├── EGU.Flow.CoreApiService/             # ASP.NET Core REST API
│   ├── Controllers/
│   │   ├── BatchController.cs
│   │   ├── OrganizationController.cs
│   │   ├── TemplateController.cs
│   │   └── SchemaController.cs
│   ├── Middleware/
│   │   ├── OrganizationContextMiddleware.cs
│   │   ├── ErrorHandlingMiddleware.cs
│   │   └── RequestLoggingMiddleware.cs
│   └── Program.cs
│
├── EGU.Flow.ParserService/              # Console app: XML → JSON
│   ├── Workers/
│   │   └── BatchParserWorker.cs
│   ├── Parsers/
│   │   ├── GaselParser.cs
│   │   ├── XellentParser.cs
│   │   └── ZynergyParser.cs
│   └── Program.cs
│
├── EGU.Flow.DocumentGenerator/          # Console app: JSON → PDF
│   ├── Workers/
│   │   └── DocumentGeneratorWorker.cs
│   ├── Services/
│   │   ├── HandlebarsRenderingService.cs
│   │   └── PlaywrightPdfService.cs
│   └── Program.cs
│
├── EGU.Flow.EmailService/               # Console app: Email delivery
│   ├── Workers/
│   │   └── EmailDeliveryWorker.cs
│   └── Program.cs
│
├── EGU.Flow.PostalService/              # Console app: 21G integration
│   ├── Workers/
│   │   └── PostalBulkProcessor.cs
│   ├── Services/
│   │   ├── SftpService.cs
│   │   └── ZipArchiveService.cs
│   └── Program.cs
│
├── EGU.Flow.Web/                        # Future: Blazor UI
│   └── (Phase 2)
│
└── EGU.Flow.Tests/                      # Test projects
    ├── EGU.Flow.UnitTests/
    ├── EGU.Flow.IntegrationTests/
    └── EGU.Flow.LoadTests/

...

Coding Standards:

  • Style: Follow Microsoft C# coding conventions
  • Naming: PascalCase for public members, camelCase for private
  • Comments: XML documentation on all public APIs
  • Async: Always use async/await, never .Result or .Wait()
  • Logging: Structured logging with Serilog, correlation IDs
  • Exceptions: Custom exception types, never swallow exceptions
  • Null safety: Use nullable reference types (C# 12)

Acceptance Criteria:

CriterionValidation MethodTarget
Code follows .NET conventionsEditorConfig + Roslyn analyzersZero warnings
XML documentation on public APIsDocumentation coverage report100% of public members
Unit test coverageCode coverage report> 70%
Integration test coverageTest execution report> 25% of critical paths
Swagger/OpenAPI for all endpointsVerify Swagger UIAll endpoints documented
Correlation IDs in all logsTrace request through systemSame ID across services
Health check endpoints presentGET /health on all servicesAll respond
Feature flags for gradual rolloutVerify feature flag configurationFlags configurable
Database migration scripts versionedCheck migrations folderSequential numbering
Infrastructure as Code (Bicep)All resources in Bicep templates100% infrastructure
No hardcoded valuesCode scanAll config in appsettings/KeyVault

Documentation Requirements:

Document TypeLocationUpdate FrequencyOwner
API DocumentationSwagger UI at /swaggerEvery API changeDev Team
Architecture DecisionsConfluence ADR pagePer decisionTechnical Architect
Deployment ProceduresConfluence + Git (docs/)Per changeOps Team
Operations RunbookConfluenceMonthly reviewOps Manager
Disaster Recovery PlanConfluence (restricted)QuarterlyOps Manager
GDPR DocumentationConfluence (restricted)Annual reviewLegal/Compliance
Test Data Generation GuideGit (docs/)Per updateQA Team

Dependencies:

  • EditorConfig file in repository
  • Roslyn analyzer NuGet packages
  • SonarQube or similar code quality tool
  • Documentation review in PR process

4.11 NFR-011: Usability & Developer Experience

Requirement: The API shall be intuitive, well-documented, easy to integrate, with clear error messages, code samples, and Postman collections.

Priority: MEDIUM

API Design Principles:

RESTful Standards:

  • Resource-based URLs: /organizations/{id}/batches
  • HTTP verbs: GET (read), POST (create), PUT (update), DELETE (not used Phase 1)
  • Status codes: 200 OK, 201 Created, 202 Accepted, 400 Bad Request, 401 Unauthorized, 403 Forbidden, 404 Not Found, 409 Conflict, 422 Unprocessable Entity, 429 Too Many Requests, 500 Internal Error, 503 Service Unavailable
  • Consistent JSON response envelope
  • Pagination for list endpoints
  • Filtering and sorting support

Error Message Quality:

Bad Example:

{
  "error": "Invalid input"
}

...

Good Example:

{
  "success": false,
  "errors": [
    {
      "code": "INVALID_XML",
      "message": "XML file is not well-formed",
      "field": "file",
      "details": {
        "line": 142,
        "column": 23,
        "error": "Unexpected end tag: </Invoice>. Expected: </InvoiceHeader>",
        "suggestion": "Check that all opening tags have matching closing tags in the correct order",
        "documentationUrl": "https://docs.egflow.com/errors/INVALID_XML"
      }
    }
  ]
}

...

Acceptance Criteria:

CriterionValidation MethodTarget
RESTful design principles followedAPI design reviewAll principles followed
Consistent request/response structureReview all endpointsSame envelope
Error messages include suggestionsTest error scenariosActionable guidance
Error messages include documentation linksCheck error responseURLs present
Line/column numbers for XML errorsUpload invalid XMLPosition info present
Comprehensive Swagger documentationReview Swagger UIAll endpoints, examples
Code samples for common operationsCheck documentationC#, curl examples
Postman collection availableImport collection, run requestsAll requests work
API versioning clearCheck URL structure/v1/ in all paths
Deprecation warnings 6 months advanceDeprecate endpointWarning in response

Postman Collection Contents:

EG Flow API Collection/
├── Authentication/
│   └── Get Access Token
├── Batch Management/
│   ├── Upload Batch
│   ├── Start Batch Processing
│   ├── Get Batch Status
│   ├── List Batch Items
│   └── Get Batch Item Details
├── Organization/
│   ├── Get Organization
│   └── Update Organization Config
├── Templates/
│   ├── List Templates
│   ├── Get Template
│   └── List Template Categories
└── Schema Management/
    ├── List Supported Formats
    └── Validate XML

...

Dependencies:

  • OpenAPI/Swagger generator
  • Postman collection export
  • Documentation website/portal
  • Code sample generation

4.12 NFR-012: Internationalization (Nordic Languages)

Requirement: The system shall support Swedish language for invoices, email notifications, error messages, and UI elements, with foundation for future Norwegian, Danish, and Finnish support.

Priority: MEDIUM

Localization Requirements:

Phase 1: Swedish Only

  • Invoice templates: Swedish text
  • Email notifications: Swedish
  • Error messages: Swedish (with English fallback in details)
  • Date formats: YYYY-MM-DD (ISO 8601, universal)
  • Number formats: Space as thousand separator, comma as decimal (Swedish standard)
    • Example: 1 234 567,89 kr
  • Currency: SEK (Swedish Krona)
  • Time zone: CET/CEST (Europe/Stockholm)

Phase 2: Multi-Language (Future)

  • Norwegian (Bokmål): For Norwegian utilities
  • Danish: For Danish utilities
  • Finnish: For Finnish utilities
  • Language detection based on organization country

Swedish-Specific Formatting:

Dates:
- Invoice date: "2025-11-21" (ISO format, universal)
- Display: "21 november 2025" (Swedish long format)

Numbers:
- Amount: 1 234,56 (space separator, comma decimal)
- Percentage: 25,0% (comma decimal)
- Quantity: 1 234 (integer, space separator)

Currency:
- Symbol: "kr" or "SEK"
- Position: After amount "1 234,56 kr"

Swedish Terms:
- Faktura (Invoice)
- Förfallodatum (Due date)
- Mätpunkt (Metering point)
- Elförbrukning (Electricity consumption)
- Att betala (Amount to pay)
- Moms (VAT)

...

Handlebars Helpers for Swedish Formatting:

{{!-- Swedish number formatting --}}
{{formatNumber amount decimals=2}}
→ "1 234,56"

{{!-- Swedish currency --}}
{{formatCurrency amount}}
→ "1 234,56 kr"

{{!-- Swedish date --}}
{{formatDate date format="long"}}
→ "21 november 2025"

{{!-- Swedish percentage --}}
{{formatPercent rate}}
→ "25,0%"

...

Acceptance Criteria:

CriterionValidation MethodTarget
Invoice templates in SwedishReview rendered PDFSwedish text
Email notifications in SwedishReceive test emailSwedish subject/body
Error messages in SwedishTrigger various errorsSwedish messages
Numbers formatted Swedish styleCheck invoice amounts"1 234,56" format
Dates in ISO 8601Check invoice JSON"2025-11-21" format
Currency symbol positioned correctlyCheck rendered invoice"kr" after amount
Swedish characters (åäö) render correctlyPDF with åäöCharacters correct
Time zone CET/CEST usedCheck timestampsEurope/Stockholm

Dependencies:

  • Swedish localization resources
  • Handlebars custom helpers
  • PDF font with Swedish character support
  • Future: i18n library for multi-language

4.13 NFR-013: API Backward Compatibility

Requirement: API versions shall maintain backward compatibility for minimum 12 months after new version release, with clear deprecation warnings and migration guides.

Priority: HIGH

Versioning Strategy:

URL Path Versioning:

Current: https://api.egflow.com/v1/organizations/{id}/batches
Future: https://api.egflow.com/v2/organizations/{id}/batches

Both versions run simultaneously during transition period

...

Version Lifecycle:

v1.0 Released: January 2026
  ↓
v2.0 Released: January 2027
  ↓ (v1 deprecation announced)
v1 Supported: Until January 2028 (12 months)
  ↓
v1 Sunset: January 2028

...

Deprecation Warning (HTTP Header):

HTTP/1.1 200 OK
Deprecation: true
Sunset: Sat, 31 Jan 2028 23:59:59 GMT
Link: <https://docs.egflow.com/api/v2/migration>; rel="successor-version"

...

Acceptance Criteria:

CriterionValidation MethodTarget
v1 supported 12 months after v2 releaseVerify both versions workBoth return 200
Breaking changes only in major versionsReview v1.1, v1.2 changesNo breaking changes
Deprecation warnings 6 months advanceCheck headers 6 months before sunsetDeprecation header present
Migration guide publishedReview documentationComplete guide available
v1 clients continue working during v2 rolloutTest v1 client after v2 deployNo disruption

5. Data Flow Diagrams

5.1 High-Level System Data Flow

┌─────────────────────────────────────────────────────────────────┐
│                     EG Flow Complete Data Flow                   │
└─────────────────────────────────────────────────────────────────┘

External System                    EG Flow System (Azure)
(Utility Billing)                                    
      │                                              
      │  1. Generate monthly                         
      │     invoice batch (XML)                      
      │                                              
      ├──────────────────────────────────────────>  
      │     POST /batches                    ┌──────────────┐
      │     (GASEL/XELLENT/ZYNERGY XML)      │  Core API    │
      │                                      │  Service     │
      │                                      │              │
      │                                      │ • Auth check │
      │                                      │ • Validate   │
      │                                      │ • Store blob │
      │                                      └──────┬───────┘
      │                                             │
      │  2. Return Batch ID                         │
      │<────────────────────────────────────────────┤
      │     {batchId, status: "uploaded"}           │
      │                                             │
      │  3. Start Processing                        │
      ├──────────────────────────────────────────> │
      │     POST /batches/{id}/start                │
      │                                             │
      │                                      ┌──────▼───────┐
      │                                      │  Blob        │
      │                                      │  Storage     │
      │                                      │              │
      │                                      │ {org}-batches│
      │                                      │ /2025/11/21/ │
      │                                      └──────┬───────┘
      │                                             │
      │  4. 202 Accepted (queued)                   │
      │<────────────────────────────────────────────┤
      │                                             │
      │                                      ┌──────▼────────┐
      │                                      │  Storage      │
      │                                      │  Queues       │
      │                                      │               │
      │                                      │ • batch-upload│
      │                                      │ • batch-items │
      │                                      │ • email       │
      │                                      │ • postal-bulk │
      │                                      └───────┬───────┘
      │                                              │
      │                              ┌───────────────┼──────────────┐
      │                              │               │              │
      │                       ┌──────▼─────┐  ┌─────▼──────┐ ┌─────▼──────┐
      │                       │  Parser    │  │ Document   │ │  Email     │
      │                       │  Service   │  │ Generator  │ │  Service   │
      │                       │            │  │            │ │            │
      │                       │ • Detect   │  │ • Render   │ │ • SendGrid │
      │                       │ • Validate │  │ • PDF gen  │ │ • Retry    │
      │                       │ • Parse    │  │ • Store    │ │ • Fallback │
      │                       │ • Transform│  │            │ │            │
      │                       └──────┬─────┘  └─────┬──────┘ └─────┬──────┘
      │                              │              │              │
      │                       Creates 157    Creates PDFs    Sends emails
      │                       JSON files    & queues         │
      │                       (5000/32)     delivery         │
      │                              │              │              │
      │                              ▼              ▼              ▼
      │                       ┌──────────────────────────────────────┐
      │                       │         Blob Storage                 │
      │                       │                                      │
      │                       │ {org}-invoices-2025/11/21/          │
      │                       │ ├── {id}.json (canonical data)      │
      │                       │ ├── {id}.html (rendered)            │
      │                       │ └── {id}.pdf (final invoice)        │
      │                       └──────────────────────────────────────┘
      │                                              │
      │  5. Poll Status                              │
      │     GET /batches/{id}                        │
      ├──────────────────────────────────────────> │
      │                                             │
      │  6. Status Response                         │
      │<────────────────────────────────────────────┤
      │     {status: "processing",                  │
      │      processedItems: 3200/5000}             │
      │                                             │
      │  7. Status: Completed                       │
      │<────────────────────────────────────────────┤
      │     {status: "completed",                   │
      │      successfulItems: 4950,                 │
      │      failedItems: 50}                       │
      │                                             │
      │                                      ┌──────▼─────┐
      │                                      │  Postal    │
      │                                      │  Service   │
      │                                      │            │
      │                                      │ • Collect  │
      │                                      │ • Create ZIP│
      │                                      │ • 21G SFTP │
      │                                      └──────┬─────┘
      │                                             │
End Customer                                        │
      │                                             │
      │  8a. Receive Email                          │
      │<────────────────────────────────────────────┤
      │      (PDF attachment)                       │
      │                                             │
      │  8b. Receive Postal                         │
      │<────────────────────────────────────────────┤
      │      (via 21G print partner)                │

...

5.2 Batch Upload Flow

┌─────────────────────────────────────────────────────────────┐
│                    Batch Upload Data Flow                    │
└─────────────────────────────────────────────────────────────┘

Client                 API Gateway           Core API              Blob Storage
  │                         │                    │                      │
  ├─ POST /batches ────────>│                    │                      │
  │  (XML file)             │                    │                      │
  │                         ├─ Authenticate ────>│                      │
  │                         │  (Entra ID)        │                      │
  │                         │<─ JWT Valid ───────┤                      │
  │                         │                    │                      │
  │                         ├─ Authorize ───────>│                      │
  │                         │  (Check role)      │                      │
  │                         │<─ Access OK ───────┤                      │
  │                         │                    │                      │
  │                         ├─ Upload File ─────>│                      │
  │                         │                    ├─ Generate UUID ─────>│
  │                         │                    │  (Batch ID)          │
  │                         │                    │                      │
  │                         │                    ├─ Store XML ─────────>│
  │                         │                    │  {org}-batches-2025/ │
  │                         │                    │  11/21/{id}/source.xml│
  │                         │                    │<─ Stored ────────────┤
  │                         │                    │                      │
  │                         │                    ├─ Quick Detect ───────>│
  │                         │                    │  (Namespace peek)    │
  │                         │                    │<─ Format: GASEL ─────┤
  │                         │                    │                      │
  │                         │                    ├─ Create Metadata ───>│
  │                         │                    │  metadata.json       │
  │                         │                    │<─ Created ───────────┤
  │                         │                    │                      │
  │                         │<─ 201 Created ─────┤                      │
  │                         │  {batchId}         │                      │
  │<─ 201 Created ──────────┤                    │                      │
  │  {batchId, status}      │                    │                      │

...

5.3 Parser Service Detailed Flow (XML → JSON)

┌──────────────────────────────────────────────────────────────┐
│           Parser Service: XML → Canonical JSON Flow          │
└──────────────────────────────────────────────────────────────┘

Queue Message         Parser Service              Blob Storage
      │                     │                          │
      ├─ {batchId} ────────>│                          │
      │                     │                          │
      │                     ├─ Download XML ──────────>│
      │                     │  GET {org}-batches-2025/ │
      │                     │      11/21/{id}/source.xml│
      │                     │<─ XML Content ───────────┤
      │                     │                          │
      │                     ├─ Parse Root Namespace   │
      │                     │  xmlns="urn:ediel:..."   │
      │                     │  → Detected: GASEL       │
      │                     │                          │
      │                     ├─ Load Schema ───────────>│
      │                     │  GET {org}-data/schemas/ │
      │                     │      gasel-mapping.json  │
      │                     │<─ Mapping Config ────────┤
      │                     │                          │
      │                     ├─ Load XSD ──────────────>│
      │                     │  GET {org}-data/schemas/ │
      │                     │      gasel-v1.0.xsd      │
      │                     │<─ XSD Content ───────────┤
      │                     │                          │
      │                     ├─ Validate XML            │
      │                     │  Against XSD             │
      │                     │  Result: VALID           │
      │                     │                          │
      │                     ├─ Parse XML (XPath)       │
      │                     │  Extract Invoice[1]:     │
      │                     │   InvoiceNumber =        │
      │                     │     "2025-11-001"        │
      │                     │   CustomerName =         │
      │                     │     "Medeni Schröder"    │
      │                     │   TotalAmount = 749.28   │
      │                     │  ... (all fields)        │
      │                     │                          │
      │                     ├─ Transform to Canonical  │
      │                     │  {                       │
      │                     │    invoiceNumber,        │
      │                     │    invoiceDate,          │
      │                     │    customer: {...},      │
      │                     │    invoiceDetails: {...},│
      │                     │    delivery: {...},      │
      │                     │    sourceMetadata: {     │
      │                     │      vendorCode: "GASEL" │
      │                     │    }                     │
      │                     │  }                       │
      │                     │                          │
      │                     ├─ LOOP: All 5000 invoices│
      │                     │                          │
      │                     ├─ Store JSON (5000×) ───>│
      │                     │  PUT {org}-invoices-2025/│
      │                     │      11/21/{inv-id}.json │
      │                     │<─ Stored (5000×) ────────┤
      │                     │                          │
      │                     ├─ Group into 32-batches  │
      │                     │  5000 ÷ 32 = 157 batches │
      │                     │                          │
      │                     ├─ Enqueue 157 Messages   │
      │                     │  TO batch-items-queue    │
      │                     │  Each: 32 invoice IDs    │
      │                     │                          │
      │                     ├─ Update Batch Metadata ─>│
      │                     │  PUT {org}-batches-2025/ │
      │                     │      11/21/{id}/         │
      │                     │      metadata.json       │
      │                     │  {                       │
      │                     │    totalItems: 5000,     │
      │                     │    vendorCode: "GASEL",  │
      │                     │    status: "processing"  │
      │                     │  }                       │
      │                     │<─ Updated ───────────────┤
      │                     │                          │
      │<─ ACK (delete msg)──┤                          │
      │                     │                          │

...

5.4 Document Generator Flow (JSON → HTML → PDF)

┌──────────────────────────────────────────────────────────────┐
│        Document Generator: JSON → HTML → PDF Flow            │
└──────────────────────────────────────────────────────────────┘

Queue Message         Document Generator         Blob Storage
      │                     │                          │
      ├─ 32 Invoice IDs ───>│                          │
      │                     │                          │
      │                     ├─ Acquire Blob Lease ────>│
      │                     │  Lease: {batch}/locks/   │
      │                     │         {worker-id}.lock │
      │                     │<─ Lease Acquired ────────┤
      │                     │  (5 min duration)        │
      │                     │                          │
      │                     ├─ LOOP: 32 invoices      │
      │                     │                          │
      │                     ├─ Download JSON ─────────>│
      │                     │  GET {org}-invoices-2025/│
      │                     │      11/21/{id}.json     │
      │                     │<─ Canonical Invoice ─────┤
      │                     │                          │
      │                     ├─ Load Org Config ───────>│
      │                     │  GET {org}-data/         │
      │                     │      organization.json   │
      │                     │<─ Branding, Settings ────┤
      │                     │                          │
      │                     ├─ Determine Template      │
      │                     │  Category (invoice type) │
      │                     │  → "invoice"             │
      │                     │                          │
      │                     ├─ Load Template ─────────>│
      │                     │  GET {org}-data/         │
      │                     │      templates/invoice/  │
      │                     │      active.html         │
      │                     │<─ Handlebars Template ───┤
      │                     │                          │
      │                     ├─ Compile Template        │
      │                     │  (cache 24h if not cached)│
      │                     │  Handlebars.Compile()    │
      │                     │                          │
      │                     ├─ Render HTML             │
      │                     │  Apply invoice data      │
      │                     │  + organization branding │
      │                     │  Output: HTML string     │
      │                     │  Duration: ~1-2 seconds  │
      │                     │                          │
      │                     ├─ Generate PDF            │
      │                     │  Playwright.NewPage()    │
      │                     │  SetContentAsync(html)   │
      │                     │  PdfAsync(A4, margins)   │
      │                     │  Duration: ~3-5 seconds  │
      │                     │                          │
      │                     ├─ Store HTML ────────────>│
      │                     │  PUT {org}-invoices-2025/│
      │                     │      11/21/{id}.html     │
      │                     │<─ Stored ────────────────┤
      │                     │                          │
      │                     ├─ Store PDF ─────────────>│
      │                     │  PUT {org}-invoices-2025/│
      │                     │      11/21/{id}.pdf      │
      │                     │<─ Stored ────────────────┤
      │                     │                          │
      │                     ├─ Update Invoice JSON ───>│
      │                     │  Add fileReferences,     │
      │                     │  templateInfo, timestamp │
      │                     │<─ Updated ───────────────┤
      │                     │                          │
      │                     ├─ Determine Distribution  │
      │                     │  Has email? → email-queue│
      │                     │  Else → postal-bulk-queue│
      │                     │                          │
      │                     ├─ Enqueue Delivery        │
      │                     │  TO email-queue OR       │
      │                     │  TO postal-bulk-queue    │
      │                     │                          │
      │                     │  (END OF 32 ITEMS LOOP)  │
      │                     │                          │
      │                     ├─ Release Blob Lease ────>│
      │                     │<─ Lease Released ────────┤
      │                     │                          │
      │                     ├─ Update Batch Stats ────>│
      │                     │  (ETag concurrency)      │
      │                     │  processedItems += 32    │
      │                     │<─ Updated ───────────────┤
      │                     │                          │
      │<─ ACK (delete msg)──┤                          │

...

5.5 Email Delivery Flow

┌──────────────────────────────────────────────────────────┐
│              Email Delivery Service Flow                  │
└──────────────────────────────────────────────────────────┘

email-queue      Email Service      SendGrid API      Blob Storage
      │                │                  │                │
      ├─ {invoiceId} ─>│                  │                │
      │                │                  │                │
      │                ├─ Download PDF ──────────────────>│
      │                │  GET {org}-invoices-2025/11/21/  │
      │                │      {invoice-id}.pdf            │
      │                │<─ PDF Bytes ────────────────────┤
      │                │                  │                │
      │                ├─ Load Org Config ───────────────>│
      │                │  GET {org}-data/organization.json│
      │                │<─ Email settings ───────────────┤
      │                │                  │                │
      │                ├─ Create Email    │                │
      │                │  From: noreply@org.se           │
      │                │  To: customer@example.se        │
      │                │  Subject: "Faktura XXX"         │
      │                │  Attach: PDF                    │
      │                │                  │                │
      │                ├─ Send Email ────>│                │
      │                │  POST /v3/mail/send             │
      │                │                  │                │
      │                │                  ├─ Process       │
      │                │                  │  (SendGrid)    │
      │                │                  │                │
      │                │<─ 202 Accepted ──┤                │
      │                │  {messageId}     │                │
      │                │                  │                │
      │                ├─ Update Metadata ───────────────>│
      │                │  deliveryAttempts.push({        │
      │                │    channel: "email",            │
      │                │    status: "delivered",         │
      │                │    messageId,                   │
      │                │    timestamp                    │
      │                │  })                             │
      │                │<─ Updated ──────────────────────┤
      │                │                  │                │
      │<─ ACK (delete)─┤                  │                │
      │                │                  │                │
      │                                                    │
      │  IF SENDGRID FAILS (429 Rate Limit):              │
      │                │                  │                │
      │                │<─ 429 Too Many ──┤                │
      │                │  Retry-After: 60s│                │
      │                │                  │                │
      │                ├─ Re-queue Message                │
      │<─ Re-enqueue ──┤  visibilityTimeout=60s           │
      │  (retry)       │                  │                │
      │                                                    │
      │  IF ALL RETRIES FAIL → FALLBACK TO POSTAL:        │
      │                │                  │                │
      │                ├─ Enqueue to postal-bulk-queue    │
      │                │  (fallback delivery)              │

...

5.6 Postal Bulk Processing Flow (21G Integration)

┌──────────────────────────────────────────────────────────┐
│      Postal Bulk Service: 21G Integration Flow           │
└──────────────────────────────────────────────────────────┘

Scheduled Trigger    Postal Service       21G SFTP         Blob Storage
(12:00, 20:00 CET)         │                  │                │
      │                    │                  │                │
      ├─ CRON Trigger ────>│                  │                │
      │                    │                  │                │
      │                    ├─ Fetch All Messages from         │
      │                    │  postal-bulk-queue               │
      │                    │  (batch retrieval)               │
      │                    │  Result: 150 invoices            │
      │                    │                  │                │
      │                    ├─ Group by Organization          │
      │                    │  Org A: 100 invoices            │
      │                    │  Org B: 50 invoices             │
      │                    │                  │                │
      │                    ├─ FOR Org A:      │                │
      │                    │                  │                │
      │                    ├─ Download 100 PDFs ────────────>│
      │                    │  GET {org-a}-invoices-2025/     │
      │                    │      11/21/{id}.pdf             │
      │                    │<─ PDF Bytes (100×) ─────────────┤
      │                    │                  │                │
      │                    ├─ Create 21G XML Metadata       │
      │                    │  <PrintBatch>                   │
      │                    │    <TotalDocuments>100          │
      │                    │    <Documents>                  │
      │                    │      <Document>                 │
      │                    │        <DocumentId>001.pdf      │
      │                    │        <Recipient>              │
      │                    │          <Name>...</Name>       │
      │                    │          <Address>...</Address>  │
      │                    │                  │                │
      │                    ├─ Create ZIP Archive            │
      │                    │  ORGA_20251121_001.zip          │
      │                    │  ├── metadata.xml               │
      │                    │  ├── invoice_001.pdf            │
      │                    │  ├── invoice_002.pdf            │
      │                    │  └── ... (100 PDFs)             │
      │                    │                  │                │
      │                    ├─ Upload to 21G ─>│                │
      │                    │  SFTP PUT        │                │
      │                    │  /incoming/ORGA/ │                │
      │                    │  ORGA_20251121_  │                │
      │                    │  001.zip         │                │
      │                    │<─ Upload Success ┤                │
      │                    │                  │                │
      │                    ├─ Update Invoice Statuses ──────>│
      │                    │  (100 invoices)                 │
      │                    │  status="postal_sent"           │
      │                    │<─ Updated (100×) ───────────────┤
      │                    │                  │                │
      │                    ├─ Delete Queue Messages          │
      │                    │  (100 messages from queue)      │
      │                    │                  │                │
      │                    ├─ Send Notification Email       │
      │                    │  To: ops@org-a.com              │
      │                    │  Subject: "21G Batch Sent"      │
      │                    │  Body: "100 invoices"           │
      │                    │                  │                │
      │                    ├─ Log to App Insights            │
      │                    │  Metric: Postal.BatchSize=100   │

...

5.7 Error Handling & Retry Flow

┌─────────────────────────────────────────────────────────┐
│          Error Handling & Retry Flow Diagram            │
└─────────────────────────────────────────────────────────┘

Processing      Error Occurs       Retry Logic        Final State
    │                │                  │                  │
    ├─ Render HTML   │                  │                  │
    │                X                  │                  │
    │           Template error          │                  │
    │           (missing variable)      │                  │
    │                │                  │                  │
    │                ├─ Log Error ─────>│                  │
    │                │  Application     │                  │
    │                │  Insights        │                  │
    │                │  Level: Error    │                  │
    │                │  CorrelationId   │                  │
    │                │                  │                  │
    │                ├─ Increment ─────>│                  │
    │                │  retryCount=1    │                  │
    │                │                  │                  │
    │                ├─ Retry Attempt 1 │                  │
    │                │  Wait: 60 seconds│                  │
    │                X Still fails      │                  │
    │                │                  │                  │
    │                ├─ Increment ─────>│                  │
    │                │  retryCount=2    │                  │
    │                │                  │                  │
    │                ├─ Retry Attempt 2 │                  │
    │                │  Wait: 5 minutes │                  │
    │                X Still fails      │                  │
    │                │                  │                  │
    │                ├─ Increment ─────>│                  │
    │                │  retryCount=3    │                  │
    │                │                  │                  │
    │                ├─ Retry Attempt 3 │                  │
    │                │  Wait: 15 minutes│                  │
    │                X Still fails      │                  │
    │                │                  │                  │
    │                ├─ Max Retries ───────────────────────>│
    │                │  Exceeded        │           poison-queue
    │                │                  │                  │
    │                ├─ Create Poison ─────────────────────>│
    │                │  Message with:   │           {      │
    │                │  • Original msg  │    retryCount: 3 │
    │                │  • All errors    │    lastError     │
    │                │  • Timestamps    │    alertSent     │
    │                │                  │   }              │
    │                │                  │                  │
    │                ├─ Send Alert ────────────────────────>│
    │                │  Email to:       │    support@egflow.com
    │                │  support team    │    Subject: "Poison Queue"
    │                │                  │                  │
    │                ├─ Update Batch ──────────────────────>│
    │                │  failedItems++   │    Batch metadata
    │                │  errors.push()   │    updated
      │                │                  │                  │
      │<─ Delete from ─┤                  │                  │
      │   main queue   │                  │                  │

...

5.8 Multi-Vendor Transformation Flow

┌──────────────────────────────────────────────────────────┐
│     Multi-Vendor XML → Canonical JSON Transformation     │
└──────────────────────────────────────────────────────────┘

GASEL XML                     Parser                  Canonical JSON
    │                            │                          │
<InvoiceBatch                     │                          │
 xmlns="urn:ediel:...">           │                          │
  <Invoice>                       │                          │
    <InvoiceHeader>               │                          │
      <InvoiceNumber>             │                          │
        2025-11-001               │                          │
      </InvoiceNumber>            │                          │
    <CustomerParty>               │                          │
      <PartyName>                 │                          │
        Medeni Schröder           │                          │
      </PartyName>                │                          │
    <MonetarySummary>             │                          │
      <PayableAmount>             │                          │
        749.28                    │                          │
      </PayableAmount>            │                          │
    ├────────────────────────────>│                          │
    │                            ├─ Detect: GASEL           │
    │                            │  (namespace match)       │
    │                            │                          │
    │                            ├─ Load Mapping ──────────>│
    │                            │  gasel-mapping.json      │
    │                            │                          │
    │                            ├─ Extract via XPath:     │
    │                            │  invoiceNumber =         │
    │                            │    "InvoiceHeader/       │
    │                            │     InvoiceNumber"       │
    │                            │  customerName =          │
    │                            │    "CustomerParty/       │
    │                            │     PartyName"           │
    │                            │                          │
    │                            ├─ Transform ─────────────>│
    │                            │                  {       │
    │                            │    "invoiceNumber": "2025-11-001",
    │                            │    "customer": {
    │                            │      "fullName": "Medeni Schröder"
    │                            │    },
    │                            │    "invoiceDetails": {
    │                            │      "totalAmount": 749.28
    │                            │    },
    │                            │    "sourceMetadata": {
    │                            │      "vendorCode": "GASEL"
    │                            │    }
    │                            │  }

XELLENT XML                   Parser                  Canonical JSON
    │                            │                          │
<InvoiceBatch                     │                          │
 xmlns="http://oio.dk..."         │                          │
 xmlns:com="...common">           │                          │
  <Invoice>                       │                          │
    <com:ID>                      │                          │
      5002061556                  │                          │
    </com:ID>                    │                          │
    <com:BuyerParty>             │                          │
      <com:PartyName>            │                          │
        <com:Name>               │                          │
          Erik Svensson          │                          │
        </com:Name>              │                          │
    <com:LegalTotals>            │                          │
      <com:ToBePaidTotalAmount>  │                          │
        2 567,00                 │                          │
    ├────────────────────────────>│                          │
    │                            ├─ Detect: XELLENT        │
    │                            │  (oio.dk namespace)      │
    │                            │                          │
    │                            ├─ Load Mapping           │
    │                            │  xellent-mapping.json    │
    │                            │                          │
    │                            ├─ Handle Namespaces      │
    │                            │  (com:, main: prefixes)  │
    │                            │                          │
    │                            ├─ Extract via XPath:     │
    │                            │  invoiceNumber = "com:ID"│
    │                            │  customerName =          │
    │                            │    "com:BuyerParty/      │
    │                            │     com:PartyName/       │
    │                            │     com:Name"            │
    │                            │                          │
    │                            ├─ Normalize Amount:      │
    │                            │  "2 567,00" → 2567.00    │
    │                            │                          │
    │                            ├─ Transform ─────────────>│
    │                            │                  {       │
    │                            │    "invoiceNumber": "5002061556",
    │                            │    "customer": {
    │                            │      "fullName": "Erik Svensson"
    │                            │    },
    │                            │    "invoiceDetails": {
    │                            │      "totalAmount": 2567.00
    │                            │    },
    │                            │    "sourceMetadata": {
    │                            │      "vendorCode": "XELLENT"
    │                            │    }
    │                            │  }

ZYNERGY XML                   Parser                  Canonical JSON
    │                            │                          │
<InvoiceBatch                     │                          │
 xmlns="http://eg.dk/Zynergy">    │                          │
  <Invoice>                       │                          │
    <InvoiceData>                 │                          │
      <InvoiceNumber>             │                          │
        100000                    │                          │
      </InvoiceNumber>            │                          │
    <Customer>                    │                          │
      <ReadOnlyFullName>          │                          │
        Alfred Asplund            │                          │
      </ReadOnlyFullName>         │                          │
    <InvoiceData>                 │                          │
      <InvoiceAmount>             │                          │
        1056.00                   │                          │
    ├────────────────────────────>│                          │
    │                            ├─ Detect: ZYNERGY        │
    │                            │  (Zynergy namespace)     │
    │                            │                          │
    │                            ├─ Load Mapping           │
    │                            │  zynergy-mapping.json    │
    │                            │                          │
    │                            ├─ Extract via XPath:     │
    │                            │  invoiceNumber =         │
    │                            │    "InvoiceData/         │
    │                            │     InvoiceNumber"       │
    │                            │  customerName =          │
    │                            │    "Customer/            │
    │                            │     ReadOnlyFullName"    │
    │                            │                          │
    │                            ├─ Transform ─────────────>│
    │                            │                  {       │
    │                            │    "invoiceNumber": "100000",
    │                            │    "customer": {
    │                            │      "fullName": "Alfred Asplund"
    │                            │    },
    │                            │    "invoiceDetails": {
    │                            │      "totalAmount": 1056.00
    │                            │    },
    │                            │    "sourceMetadata": {
    │                            │      "vendorCode": "ZYNERGY"
    │                            │    }
    │                            │  }
                                 │                          │
                            All 3 formats           Single standard
                            produce same      ───>   canonical JSON
                            structure                for downstream

...

5.9 Distribution Routing Decision Flow

┌─────────────────────────────────────────────────────────┐
│         Distribution Channel Routing Logic              │
└─────────────────────────────────────────────────────────┘

Invoice Data          Routing Logic             Queue Selection
      │                     │                          │
      ├─ Check Customer    │                          │
      │  Preference         │                          │
      │                     │                          │
      │  preference=        │                          │
      │  "postal"? ─────────┤                          │
      │                     ├─ YES ───────────────────>│
      │                     │                   postal-bulk-queue
      │                     │                          │
      │                     ├─ NO                      │
      │                     │  Continue...             │
      │                     │                          │
      ├─ Load Org          │                          │
      │  Channel Priority   │                          │
      │  [email, postal]    │                          │
      │                     │                          │
      │                     ├─ TRY Priority 1: email  │
      │                     │                          │
      ├─ Has email         │                          │
      │  address?           │                          │
      │                     │                          │
      │  email != null? ────┤                          │
      │                     ├─ YES                     │
      │                     │  Validate email format   │
      │                     │                          │
      │  valid email? ──────┤                          │
      │                     ├─ YES ───────────────────>│
      │                     │                   email-queue
      │                     │                          │
      │                     ├─ NO (invalid/missing)    │
      │                     │  Try next channel...     │
      │                     │                          │
      │                     ├─ TRY Priority 2: postal │
      │                     │                          │
      ├─ Complete          │                          │
      │  address?           │                          │
      │                     │                          │
      │  address valid? ────┤                          │
      │                     ├─ YES ───────────────────>│
      │                     │                   postal-bulk-queue
      │                     │                          │
      │                     ├─ NO (incomplete)         │
      │                     │  Log error               │
      │                     │  Skip invoice            │
      │                     │  Alert organization      │
      │                     │                          │
      │  Swedish Law:       │                          │
      │  "Rätt till         │                          │
      │  pappersfaktura"    │                          │
      │  Always ensure      │                          │
      │  postal available ──┤                          │

...

6. API Specifications

6.1 API Design Principles

Standards:

  • RESTful design with resource-based URLs
  • JSON request/response bodies
  • HTTP status codes per RFC 7231
  • OAuth 2.0 Bearer token authentication
  • URL path versioning (/v1, /v2)
  • Consistent error response envelope
  • HATEOAS links for related resources (optional)

Base URL: https://api.egflow.com/v1

Response Envelope (All Responses):

{
  "success": true|false,
  "data": { },
  "metadata": {
    "timestamp": "ISO8601",
    "requestId": "uuid",
    "apiVersion": "1.0"
  },
  "errors": null|[]
}

...

6.2 Complete API Endpoint Catalog

6.2.1 Batch Management APIs

MethodEndpointDescriptionAuth RoleRate Limit
POST/v1/organizations/{orgId}/batchesUpload batch XMLBatch Operator10/hour
POST/v1/organizations/{orgId}/batches/{batchId}/startStart processingBatch Operator30/hour
GET/v1/organizations/{orgId}/batches/{batchId}Get batch statusRead-Only100/min
GET/v1/organizations/{orgId}/batchesList batches (with filters)Read-Only100/min
GET/v1/organizations/{orgId}/batches/{batchId}/itemsList invoice itemsRead-Only100/min
GET/v1/organizations/{orgId}/batches/{batchId}/items/{itemId}Get item detailsRead-Only100/min
PUT/v1/organizations/{orgId}/batches/{batchId}Update batch metadataBatch Operator30/hour

6.2.2 Organization APIs

MethodEndpointDescriptionAuth RoleRate Limit
GET/v1/organizations/{orgId}Get organization detailsRead-Only100/min
PUT/v1/organizations/{orgId}Update organization configOrg Admin10/hour

6.2.3 Template APIs

MethodEndpointDescriptionAuth RoleRate Limit
GET/v1/organizations/{orgId}/templatesList templates (filter: category, status)Read-Only100/min
GET/v1/organizations/{orgId}/templates/{templateId}Get template detailsRead-Only100/min
GET/v1/organizations/{orgId}/template-categoriesList template categoriesRead-Only100/min

6.2.4 Schema Management APIs

MethodEndpointDescriptionAuth RoleRate Limit
GET/v1/organizations/{orgId}/schemasList supported vendor formatsRead-Only100/min
POST/v1/organizations/{orgId}/schemas/validatePre-validate XMLBatch Operator20/hour

6.2.5 Invoice APIs

MethodEndpointDescriptionAuth RoleRate Limit
GET/v1/organizations/{orgId}/invoices/{invoiceId}/pdfDownload PDFRead-Only1000/hour
GET/v1/organizations/{orgId}/invoices/{invoiceId}/htmlDownload HTMLRead-Only1000/hour

6.2.6 System APIs

MethodEndpointDescriptionAuth RoleRate Limit
GET/v1/healthHealth checkNoneUnlimited
GET/v1/versionAPI version infoNoneUnlimited

6.3 Detailed API Specifications

6.3.1 POST /organizations/{orgId}/batches (Batch Upload)

Purpose: Upload batch invoice XML file for processing

Request Headers:

Authorization: Bearer {jwt-token}
Content-Type: multipart/form-data

...

Request Body:

file: [XML file binary]
metadata: {
  "batchName": "Invoice_November_2025",
  "priority": "normal"
}

...

Success Response (201 Created):

{
  "success": true,
  "data": {
    "batchId": "550e8400-e29b-41d4-a716-446655440000",
    "organizationId": "123e4567-e89b-12d3-a456-426614174000",
    "status": "uploaded",
    "uploadedAt": "2025-11-21T10:30:00Z",
    "fileInfo": {
      "fileName": "invoices_nov.xml",
      "fileSize": 15728640,
      "checksum": "sha256:a3d5e7f9...",
      "detectedFormat": "GASEL"
    },
    "blobPath": "acme-batches-2025/11/21/550e8400.../source.xml"
  }
}

...

Error Responses:

// 400 Bad Request - Invalid XML
{
  "success": false,
  "errors": [{
    "code": "INVALID_XML",
    "message": "XML file is not well-formed",
    "field": "file",
    "details": {
      "line": 142,
      "column": 23,
      "error": "Unexpected end tag: </Invoice>",
      "suggestion": "Check that all opening tags have matching closing tags",
      "documentationUrl": "https://docs.egflow.com/errors/INVALID_XML"
    }
  }]
}

// 413 Payload Too Large
{
  "success": false,
  "errors": [{
    "code": "FILE_TOO_LARGE",
    "message": "File exceeds 100MB limit",
    "details": {
      "fileSize": 105906176,
      "limit": 104857600,
      "suggestion": "Split large batches into multiple files"
    }
  }]
}

// 415 Unsupported Media Type
{
  "success": false,
  "errors": [{
    "code": "UNSUPPORTED_FORMAT",
    "message": "Cannot detect vendor format",
    "details": {
      "detectedNamespace": "http://unknown.com/schema",
      "supportedFormats": ["GASEL", "XELLENT", "ZYNERGY"],
      "suggestion": "Ensure XML uses one of the supported vendor formats",
      "documentationUrl": "https://docs.egflow.com/vendor-formats"
    }
  }]
}

// 429 Too Many Requests
{
  "success": false,
  "errors": [{
    "code": "RATE_LIMIT_EXCEEDED",
    "message": "Too many batch uploads",
    "details": {
      "limit": 10,
      "window": "1 hour",
      "retryAfter": "2025-11-21T11:30:00Z"
    }
  }]
}

...

Response Headers:

X-RateLimit-Limit: 10
X-RateLimit-Remaining: 5
X-RateLimit-Reset: 1700226000
Location: /v1/organizations/{orgId}/batches/{batchId}

...

6.3.2 POST /organizations/{orgId}/batches/{batchId}/start

Purpose: Start asynchronous processing of uploaded batch

Request:

{
  "validationMode": "strict"
}

...

Success Response (202 Accepted):

{
  "success": true,
  "data": {
    "batchId": "550e8400-e29b-41d4-a716-446655440000",
    "status": "queued",
    "queuedAt": "2025-11-21T10:35:00Z",
    "estimatedProcessingTime": "15-30 minutes",
    "queuePosition": 2
  }
}

...

Error Responses:

// 409 Conflict - Already Processing
{
  "success": false,
  "errors": [{
    "code": "CONFLICT",
    "message": "Batch is already processing",
    "details": {
      "currentStatus": "processing",
      "startedAt": "2025-11-21T10:00:00Z",
      "estimatedCompletionAt": "2025-11-21T11:30:00Z"
    }
  }]
}

// 503 Service Unavailable - Queue Full
{
  "success": false,
  "errors": [{
    "code": "SERVICE_UNAVAILABLE",
    "message": "System at capacity, retry later",
    "details": {
      "queueDepth": 10500,
      "estimatedWaitTime": "30-60 minutes",
      "retryAfter": "2025-11-21T11:00:00Z",
      "suggestion": "Consider scheduling batch for off-peak hours (22:00-06:00 CET)"
    }
  }]
}

...

6.3.3 GET /organizations/{orgId}/batches/{batchId}

Purpose: Get current batch status and statistics

Success Response (200 OK):

{
  "success": true,
  "data": {
    "batchId": "550e8400-e29b-41d4-a716-446655440000",
    "organizationId": "123e4567-e89b-12d3-a456-426614174000",
    "batchName": "Invoice_November_2025",
    "status": "processing",
    "priority": "normal",
    "vendorInfo": {
      "vendorCode": "GASEL",
      "vendorName": "Telinet Energi / EDIEL",
      "version": "1.0",
      "detectedNamespace": "urn:ediel:se:electricity:invoice:1.0"
    },
    "statistics": {
      "totalItems": 5000,
      "parsedItems": 5000,
      "queuedItems": 1800,
      "processingItems": 200,
      "completedItems": 3000,
      "failedItems": 0,
      "successRate": 100.0,
      "itemsByStatus": {
        "queued": 1800,
        "rendering": 50,
        "rendered": 100,
        "delivering": 50,
        "delivered": 2900,
        "failed": 0
      },
      "deliveryChannelBreakdown": {
        "email": 2500,
        "postal": 500
      }
    },
    "timestamps": {
      "uploadedAt": "2025-11-21T10:30:00Z",
      "queuedAt": "2025-11-21T10:35:00Z",
      "startedAt": "2025-11-21T10:35:10Z",
      "estimatedCompletionAt": "2025-11-21T11:05:00Z",
      "completedAt": null
    },
    "fileInfo": {
      "fileName": "invoices_nov.xml",
      "fileSize": 15728640,
      "format": "xml"
    }
  }
}

...

6.3.4 GET /organizations/{orgId}/batches

Purpose: List and search batches with filtering

Query Parameters:

  • from: ISO 8601 date (default: 90 days ago)
  • to: ISO 8601 date (default: today)
  • status: uploaded|queued|processing|completed|failed
  • vendorCode: GASEL|XELLENT|ZYNERGY
  • search: Batch name search
  • sortBy: uploadedAt|completedAt|batchName
  • order: asc|desc (default: desc)
  • page: integer (default: 1)
  • pageSize: integer (default: 50, max: 500)

Success Response (200 OK):

{
  "success": true,
  "data": {
    "batches": [
      {
        "batchId": "uuid",
        "batchName": "Invoice_November_2025",
        "status": "completed",
        "vendorCode": "GASEL",
        "statistics": {
          "totalItems": 5000,
          "successfulItems": 4950,
          "failedItems": 50
        },
        "timestamps": {
          "uploadedAt": "2025-11-21T10:30:00Z",
          "completedAt": "2025-11-21T11:45:00Z"
        }
      }
    ],
    "pagination": {
      "currentPage": 1,
      "pageSize": 50,
      "totalBatches": 127,
      "totalPages": 3,
      "hasNextPage": true,
      "hasPreviousPage": false
    }
  }
}

...

6.3.5 GET /organizations/{orgId}/batches/{batchId}/items

Purpose: List individual invoice items in batch

Query Parameters:

  • status: queued|processing|completed|failed
  • page: integer (default: 1)
  • pageSize: integer (default: 50, max: 500)

Success Response (200 OK):

{
  "success": true,
  "data": {
    "items": [
      {
        "itemId": "uuid",
        "batchId": "uuid",
        "invoiceNumber": "2025-11-001",
        "customerReference": "020624-2380",
        "customerName": "Medeni Schröder",
        "totalAmount": 749.28,
        "currency": "SEK",
        "status": "delivered",
        "deliveryChannel": "email",
        "processedAt": "2025-11-21T10:45:00Z",
        "deliveredAt": "2025-11-21T10:46:15Z"
      }
    ],
    "pagination": {
      "currentPage": 1,
      "pageSize": 50,
      "totalItems": 5000,
      "totalPages": 100
    }
  }
}

...

6.3.6 GET /organizations/{orgId}/batches/{batchId}/items/{itemId}

Purpose: Get detailed invoice item information

Success Response (200 OK):

{
  "success": true,
  "data": {
    "itemId": "uuid",
    "batchId": "uuid",
    "organizationId": "uuid",
    "invoiceNumber": "2025-11-001",
    "invoiceDate": "2025-11-06",
    "dueDate": "2025-11-20",
    "currency": "SEK",
    "customerInfo": {
      "customerId": "020624-2380",
      "fullName": "Medeni Schröder",
      "email": "muntaser.af@zavann.net",
      "phone": "09193538799"
    },
    "invoiceDetails": {
      "subTotal": 599.42,
      "taxAmount": 149.86,
      "totalAmount": 749.28
    },
    "status": "delivered",
    "deliveryChannel": "email",
    "deliveryStatus": {
      "attemptedAt": "2025-11-21T10:46:00Z",
      "deliveredAt": "2025-11-21T10:46:15Z",
      "providerMessageId": "sendgrid-msg-12345"
    },
    "processingTimeline": [
      {"status": "queued", "timestamp": "2025-11-21T10:35:00Z"},
      {"status": "rendering", "timestamp": "2025-11-21T10:44:00Z"},
      {"status": "rendered", "timestamp": "2025-11-21T10:45:00Z"},
      {"status": "delivering", "timestamp": "2025-11-21T10:46:00Z"},
      {"status": "delivered", "timestamp": "2025-11-21T10:46:15Z"}
    ],
    "sourceInfo": {
      "vendorCode": "GASEL",
      "originalInvoiceId": "2025-11-001"
    },
    "fileReferences": {
      "pdfUrl": "/v1/organizations/{orgId}/invoices/{itemId}/pdf",
      "htmlUrl": "/v1/organizations/{orgId}/invoices/{itemId}/html"
    }
  }
}

...

6.3.7 GET /organizations/{orgId}/schemas/validate

Purpose: Pre-validate XML before upload

Request:

{
  "xmlContent": "<?xml version=\"1.0\"?>\n<InvoiceBatch>...</InvoiceBatch>",
  "vendorCode": "GASEL"
}

...

Success Response (200 OK):

{
  "success": true,
  "data": {
    "valid": true,
    "vendorCode": "GASEL",
    "version": "1.0",
    "invoiceCount": 10,
    "batchId": "BATCH2025110600001",
    "validationDetails": {
      "schemaValid": true,
      "structureValid": true,
      "requiredFieldsPresent": true
    },
    "warnings": [
      {
        "field": "CustomerParty[0]/Contact/ElectronicMail",
        "message": "Email format should be validated",
        "severity": "warning",
        "line": 45
      }
    ],
    "errors": []
  }
}

...

7. Error Handling & Validation

7.1 Comprehensive Field Validation Matrix

7.1.1 Customer Information Validation

FieldTypeMinMaxFormatRequiredValidation
customerIdString150Alphanumeric, dash, underscoreYes`^[A-Za-z0-9_-]+
personnummerString1013YYMMDD-XXXX or YYYYMMDD-XXXXConditionalLuhn algorithm
fullNameString1255Unicode printableYesNot empty, trim
emailString5255RFC 5322NoRegex + optional DNS MX
phoneString820E.164 recommendedNo`^+?[0-9\s-]+
streetString1255Any printableYes (postal)Not empty
postalCodeString510Country-specificYes (postal)Swedish: `^\d{3}\s?\d{2}
cityString1100Any printableYes (postal)Not empty
countryString22ISO 3166-1 alpha-2YesEnum: SE, NO, DK, FI

7.1.2 Financial Data Validation

FieldTypeMinMaxDecimalsRequiredValidation
subTotalDecimal0.00999999999.992Yes≥ 0
taxAmountDecimal0.00999999999.992Yes≥ 0
totalAmountDecimal0.01999999999.992Yes> 0
unitPriceDecimal0.00999999.992-6Yes≥ 0
quantityDecimal0.01999999.992Yes> 0
taxRateDecimal01001YesSwedish: 0, 6, 12, 25

7.1.3 Business Logic Validation Rules

RuleLogicError CodeMessage
Total ConsistencytotalAmount == subTotal + taxAmountAMOUNT_MISMATCHTotal must equal subtotal plus tax
Line Items Sumsum(lineItems.lineAmount) == subTotalLINE_ITEMS_MISMATCHLine items must sum to subtotal
Date LogicdueDate >= invoiceDateINVALID_DATE_RANGEDue date must be on or after invoice date
Tax Rate ValidtaxRate in [0, 6, 12, 25]INVALID_TAX_RATESwedish VAT: 0%, 6%, 12%, or 25%
Currency MatchAll amounts same currencyCURRENCY_MISMATCHAll amounts must use same currency
Personnummer LuhnLuhn checksumINVALID_PERSONNUMMERInvalid Swedish personnummer
Swedish Postal CodeFormat XXX XXINVALID_POSTAL_CODEFormat must be: XXX XX

7.2 Error Handling Scenarios

7.2.1 Scenario: Malformed XML Upload

Trigger: User uploads non-well-formed XML

System Action:

  1. XML parser throws exception during parse attempt
  2. Catch exception, extract line/column from error
  3. Do NOT store file in blob storage
  4. Log error to Application Insights with file details (no content)
  5. Return 400 Bad Request with detailed error

Response:

{
  "success": false,
  "errors": [{
    "code": "INVALID_XML",
    "message": "XML file is not well-formed",
    "field": "file",
    "details": {
      "line": 142,
      "column": 23,
      "error": "Unexpected end tag: </Invoice>. Expected: </InvoiceHeader>",
      "suggestion": "Verify all XML tags are properly closed and nested",
      "documentationUrl": "https://docs.egflow.com/xml-format"
    }
  }]
}

...

7.2.2 Scenario: Unsupported Vendor Format

Trigger: XML namespace doesn't match GASEL, XELLENT, or ZYNERGY

System Action:

  1. Store file in blob for manual review
  2. Update batch status to "failed"
  3. Log unsupported format to Application Insights
  4. Send alert to support team
  5. Return 415 Unsupported Media Type

Response:

{
  "success": false,
  "errors": [{
    "code": "UNSUPPORTED_FORMAT",
    "message": "Cannot detect vendor format",
    "details": {
      "detectedNamespace": "http://custom-vendor.com/invoices",
      "rootElement": "InvoiceBatch",
      "supportedFormats": [
        {
          "vendorCode": "GASEL",
          "namespace": "urn:ediel:se:electricity:invoice:1.0",
          "description": "Telinet Energi / EDIEL format"
        },
        {
          "vendorCode": "XELLENT",
          "namespace": "http://rep.oio.dk/ubl/xml/schemas/0p71/pie/",
          "description": "Karlskoga Energi / OIOXML format"
        },
        {
          "vendorCode": "ZYNERGY",
          "namespace": "http://eg.dk/Zynergy/1.0/invoice.xsd",
          "description": "EG Software Zynergy format"
        }
      ],
      "suggestion": "Contact EG Support to add support for your vendor format",
      "supportEmail": "support@egflow.com"
    }
  }]
}

...

7.2.3 Scenario: Template Rendering Failure

Trigger: Handlebars template references undefined variable

System Action:

1. DocumentGenerator attempts to render template
2. Handlebars throws exception: Variable 'customer.address.street' not found
3. Log error with full context (template24h to authorities)<br>- Security measures documentation<br>- Annual security audit<br>- CISO designated | Legal/Compliance |
| Swedish Säkerhetspolisen (SÄPO) requirements | LOW | HIGH | - Enhanced security for critical infrastructure<br>- Incident reporting to MSB (Swedish Civil Contingencies)<br>- Employee background checks for production access<br>- Security clearance for key personnel | Security Officer |
| API key theft/leakage | MEDIUM | HIGH | - Rotate keys every 90 days<br>- Monitor for leaked keys (GitHub scanning)<br>- Revoke compromised keys immediately<br>- API key hashing in database<br>- Never log full API keys | Security Officer |
| Insider threat (privileged access abuse) | LOW | CRITICAL | - Least privilege principle<br>- All actions audited<br>- Regular access reviews<br>- Separation of duties<br>- Anomaly detection in audit logs | Security Officer |
| Third-party vendor breach (SendGrid, 21G) | LOW | HIGH | - Data Processing Agreements (DPAs) signed<br>- Regular vendor security assessments<br>- Minimal data sharing<br>- Encryption in transit to vendors<br>- Vendor breach response plan | Legal/Compliance |

---

## 4.5 NFR-005: Data Retention & Lifecycle Management

**Requirement:** The system shall manage data retention according to Swedish Bokföringslagen (7-year invoice retention) with automated lifecycle policies for cost optimization through storage tier transitions.

**Priority:** **HIGH**

**Retention Policies:**

| Data Type | Legal Requirement | Retention Period | Storage Tier Transition | Disposal Method |
|-----------|------------------|-----------------|------------------------|-----------------|
| **Invoices (PDF/HTML/JSON)** | Bokföringslagen (Swedish Accounting Act) | 7 years from fiscal year end | Day 0-365: Hot<br>Day 366-2555: Cool<br>Day 2556+: Archive | Permanent deletion after 7 years |
| **Batch Source Files (XML)** | None (internal processing) | 90 days | Day 0-30: Hot<br>Day 31-90: Cool<br>Day 91+: Delete | Automatic deletion |
| **Batch Metadata JSON** | Audit trail | 90 days | Day 0-90: Hot<br>Day 91+: Delete | Automatic deletion |
| **Audit Logs (PostgreSQL)** | GDPR, Swedish law | 7 years | Year 0-1: PostgreSQL<br>Year 1-7: Blob (compressed) | Deletion after 7 years |
| **Application Logs** | Operational | 90 days | Application Insights | Automatic deletion |
| **Templates** | Business continuity | Indefinite (archived versions) | Hot (active)<br>Cool (archived) | Never deleted |
| **Organization Config** | Business continuity | Indefinite | Hot | Never deleted (updated in place) |

**Azure Blob Lifecycle Policy:**

```json
{
  "rules": [
    {
      "enable---

## 4.3 NFR-003: Availability & Reliability (Nordic 24/7 Operations)

**Requirement:** The system shall maintain 99.9% uptime with automatic failover, multi-region deployment, and recovery procedures to support Nordic utilities' 24/7 invoice delivery operations.

**Priority:** **HIGH**

**Availability Targets:**

| Metric | Target | Allowed Downtime | Measurement | Consequences of Breach |
|--------|--------|-----------------|-------------|----------------------|
| **System Uptime** | 99.9% | 43 min/month | Azure Monitor | SLA credit to customers |
| **Batch Success Rate** | > 99.5% | 50 failures per 10K | Processing logs | Investigation required |
| **Delivery Success Rate** | > 98% | 200 failures per 10K | Delivery tracking | Alert to organization |
| **API Availability** | 99.9% | 43 min/month | Health check monitoring | Incident escalation |
| **MTTR (Mean Time To Recovery)** | < 30 minutes | N/A | Incident timestamps | Process improvement |
| **MTBF (Mean Time Between Failures)** | > 720 hours (30 days) | N/A | Incident tracking | Root cause analysis |

**Multi-Region Deployment:**

...

Primary Region: West Europe (Azure westeurope)

  • Sweden: Primary processing
  • Denmark: Primary processing

Secondary Region: North Europe (Azure northeurope)

  • Norway: Primary processing
  • Finland: Primary processing
  • Failover for Sweden/Denmark

Traffic Routing:

  • Azure Traffic Manager with Performance routing
  • Health check: /health endpoint every 30 seconds
  • Auto-failover on 3 consecutive failed health checks
  • Failover time: < 2 minutes

**Recovery Time Objectives:**

| Scenario | RTO (Recovery Time) | RPO (Data Loss) | Recovery Method | Responsible Team |
|----------|---------------------|-----------------|-----------------|------------------|
| **Worker Instance Crash** | < 5 minutes | 0 (idempotent) | Automatic queue retry | Automatic |
| **Database Failure** | < 15 minutes | < 5 minutes | Auto-failover to read replica | Automatic + Ops verification |
| **Primary Region Failure** | < 30 minutes | < 15 minutes | Traffic Manager failover to secondary region | Ops Manager |
| **Blob Storage Corruption** | < 1 hour | < 1 hour | Restore from blob version/snapshot | Ops Team |
| **Queue Service Outage** | < 15 minutes | 0 (messages preserved) | Wait for Azure recovery | Ops Manager |
| **SendGrid Complete Outage** | < 2 hours | 0 (fallback to postal) | Route all to postal queue | Ops Team |
| **21G SFTP Unavailable** | < 4 hours | 0 (retry at next scheduled run) | Retry at 12:00 or 20:00 | Ops Team |

**Backup & Recovery Strategy:**

**Blob Storage:**
```yaml
Replication: Geo-Redundant Storage (GRS)
  - Primary: West Europe
  - Secondary: North Europe
  - Automatic replication

Soft Delete: 7 days retention
  - Recover accidentally deleted blobs within 7 days

Blob Versioning: 30 days retention
  - Previous versions accessible
  - Rollback capability

Point-in-Time Restore: Not needed (versioning sufficient)

...

PostgreSQL:

Backup Schedule: Daily automated backups
Retention: 35 days
Backup Window: 02:00-04:00 CET (low traffic period)
Point-in-Time Restore: 7 days
Geo-Redundant: Enabled
Read Replica: North Europe (for failover)

...

Acceptance Criteria:

CriterionValidation MethodTarget
Multi-region deployment operationalVerify services in both regionsBoth regions active
Traffic Manager routes to healthy regionSimulate West Europe failureRoutes to North Europe
Database auto-failover testedSimulate primary DB failureFailover < 15 min
Blob geo-replication verifiedWrite to primary, read from secondaryData replicated
Health checks on all servicesGET /health on all endpointsAll return 200
Automated incident alerts configuredSimulate service failureAlert received within 5 min
Worker auto-restart on crashKill worker processNew instance starts
Queue message retry testedSimulate worker crash mid-processingMessage reprocessed
Disaster recovery drill quarterlySimulate complete region lossRecovery within RTO
Backup restoration tested monthlyRestore database from backupSuccessful restore

Dependencies:

  • Azure Traffic Manager configuration
  • Multi-region resource deployment
  • Database replication setup
  • Automated failover testing procedures
  • Incident response runbook

Risks & Mitigation (Nordic Context):

RiskLikelihoodImpactMitigation StrategyOwner
Both Azure regions fail simultaneouslyVERY LOWCRITICAL- Extremely rare (Azure multi-region SLA 99.99%)
- Accept risk (probability vs cost of 3rd region)
- Communication plan for extended outage
- Manual failover to Azure Germany (emergency)
Executive Sponsor
Network partition between regionsLOWHIGH- Each region operates independently
- Eventual consistency acceptable
- Manual reconciliation if partition >1 hour
- Traffic Manager handles routing
Technical Architect
Database failover causes brief downtimeLOWMEDIUM- Accept 1-2 minutes downtime during failover
- API returns 503 with Retry-After
- Queue-based processing unaffected
- Monitor failover duration
Operations Manager
Swedish winter storms affect connectivityLOWMEDIUM- Azure datacenter redundancy within region
- Monitor Azure status dashboard
- Communication plan for customers
- No physical office connectivity required
Operations Manager

4.4 NFR-004: Security Requirements

Requirement: The system shall implement comprehensive security controls including OAuth 2.0 authentication, role-based access control, encryption, audit logging, and protection against OWASP Top 10 vulnerabilities.

Priority: CRITICAL

4.4.1 Authentication & Authorization

OAuth 2.0 Implementation:

Grant Type: Client Credentials Flow (machine-to-machine)
Token Provider: Microsoft Entra ID
Token Lifetime: 1 hour
Refresh Token: 90 days
Token Format: JWT (JSON Web Token)
Algorithm: RS256 (RSA signature with SHA-256)

...

Required Claims in JWT:

{
  "aud": "api://eg-flow-api",
  "iss": "https://login.microsoftonline.com/{tenant}/v2.0",
  "sub": "user-object-id",
  "roles": ["Batch.Operator"],
  "organization_id": "123e4567-e89b-12d3-a456-426614174000",
  "exp": 1700226000,
  "nbf": 1700222400
}

...

Role Definitions & Permissions:

RoleScopePermissionsUse Case
Super AdminGlobal (all organizations)Full CRUD on all resources, cross-org visibilityEG internal support team
Organization AdminSingle organizationManage org users, configure settings, view all batchesUtility IT manager
Template AdminSingle organizationCreate/edit templates, manage template versionsUtility design team
Batch OperatorSingle organizationUpload batches, start processing, view statusUtility billing team
Read-Only UserSingle organizationView batches, download invoices, view reportsUtility customer service
API ClientSingle organizationProgrammatic batch upload and status queriesBilling system integration

Acceptance Criteria:

CriterionValidation MethodTarget
OAuth 2.0 token required for all endpoints (except /health)Call API without token401 Unauthorized
JWT token validated (signature, expiration, audience)Tampered token, expired token401 Unauthorized
Refresh tokens work for 90 daysUse refresh token after 30 daysNew access token issued
All 6 roles implemented in PostgreSQLQuery roles table6 roles present
Users can only access their organizationUser A calls Org B endpoint403 Forbidden
All actions logged to audit_logs tablePerform action, query audit_logsEntry created
API authentication middleware on all routesAttempt bypassAll protected
MFA enforced for Super AdminLogin as Super AdminMFA challenge
MFA enforced for Org AdminLogin as Org AdminMFA challenge
Failed logins logged3 failed login attempts3 entries in audit_logs
Account lockout after 5 failed attempts6 failed login attempts15-minute lockout
API key rotation every 90 daysCheck Key Vault secret ageAlert at 80 days

4.4.2 Data Protection

Encryption Standards:

In Transit:
- TLS 1.3 minimum (TLS 1.2 acceptable)
- Cipher suites: AES-256-GCM, ChaCha20-Poly1305
- Certificate: Wildcard cert for *.egflow.com
- HSTS: max-age=31536000; includeSubDomains

At Rest:
- Azure Blob Storage: AES-256 (Microsoft-managed keys)
- PostgreSQL: AES-256 (Microsoft-managed keys)
- Backups: AES-256 encryption
- Customer-managed keys (CMK): Phase 2 option

Sensitive Data Fields (extra protection):
- Personnummer: Encrypted column in database (if stored)
- API keys: Azure Key Vault only
- Email passwords: Never stored
- Customer addresses: Standard blob encryption sufficient

...

Acceptance Criteria:

CriterionValidation MethodTarget
All API traffic over HTTPSAttempt HTTP requestRedirect to HTTPS or reject
TLS 1.3 or 1.2 enforcedCheck TLS version in trafficTLS ≥ 1.2
Data encrypted at rest (blob)Verify Azure encryption settingsEnabled
Data encrypted at rest (PostgreSQL)Verify DB encryptionEnabled
Secrets in Azure Key Vault onlyCode scan for hardcoded secretsZero secrets in code
No credentials in source controlGit history scanZero credentials
Database connections use managed identityCheck connection stringsNo passwords
Personnummer not in URLsURL pattern analysisNo personnummer patterns
Personnummer not in logsLog analysisNo personnummer found

4.4.3 Application Security (OWASP Top 10)

Security Measures:

OWASP RiskMitigationValidation
A01: Broken Access ControlOrganization middleware, RBAC enforcementPenetration testing
A02: Cryptographic FailuresTLS 1.3, AES-256, Key VaultSecurity scan
A03: InjectionParameterized queries, input validationSQL injection testing
A04: Insecure DesignThreat modeling, security reviewArchitecture review
A05: Security MisconfigurationAzure security baseline, CIS benchmarksConfiguration audit
A06: Vulnerable ComponentsDependabot, automated scanningWeekly scan
A07: Authentication FailuresOAuth 2.0, MFA, rate limitingPenetration testing
A08: Software/Data IntegrityCode signing, SRI, checksumsBuild verification
A09: Logging FailuresComprehensive audit loggingLog completeness review
A10: SSRFURL validation, allowlistSecurity testing

Input Validation:

// Example: Batch upload validation with FluentValidation
public class BatchUploadValidator : AbstractValidator<BatchUploadRequest>
{
    public BatchUploadValidator()
    {
        RuleFor(x => x.File)
            .NotNull().WithMessage("File is required")
            .Must(BeValidXml).WithMessage("File must be valid XML")
            .Must(BeLessThan100MB).WithMessage("File must be less than 100MB");
        
        RuleFor(x => x.Metadata.BatchName)
            .NotEmpty().WithMessage("Batch name is required")
            .Length(1, 255).WithMessage("Batch name must be 1-255 characters")
            .Must(NotContainPathSeparators).WithMessage("Batch name cannot contain / or \\")
            .Must(NoSQLInjectionPatterns).WithMessage("Invalid characters in batch name");
        
        RuleFor(x => x.Metadata.Priority)
            .Must(x => x == "normal" || x == "high")
            .WithMessage("Priority must be 'normal' or 'high'");
    }
    
    private bool NoSQLInjectionPatterns(string input)
    {
        var sqlPatterns = new[] { "--", "/*", "*/", "xp_", "sp_", "';", "\";" };
        return !sqlPatterns.Any(p => input.Contains(p, StringComparison.OrdinalIgnoreCase));
    }
}

...

Acceptance Criteria:

CriterionValidation MethodTarget
Input validation on all API endpointsSend malicious inputRejected with error
SQL injection preventedAttempt SQL injection in batch nameSanitized/rejected
XSS prevented in templatesInject script tags in templateSanitized on render
XML external entity (XXE) attack preventedUpload XXE payloadParsing rejects
Billion laughs attack preventedUpload billion laughs XMLParsing rejects/times out safely
File upload size enforcedUpload 101MB fileRejected at API gateway
Rate limiting prevents abuse1000 rapid API calls429 after limit
CSRF protection (future web UI)Attempt CSRF attackBlocked by token
Dependency vulnerabilities scanned weeklyRun DependabotAlerts for high/critical
Security headers presentCheck HTTP responseX-Frame-Options, CSP, etc.

4.4.4 Network Security

Acceptance Criteria:

CriterionStatusPhase
DDoS protection enabled (Azure basic)✅ IncludedPhase 1
IP whitelisting support for API clients✅ Optional featurePhase 1
VNet integration for Container Apps⚠️ Phase 2Phase 2
Private endpoints for Blob Storage⚠️ Phase 2Phase 2
Network Security Groups (NSGs)⚠️ Phase 2Phase 2
Azure Firewall for egress filtering⚠️ Phase 2Phase 2

Dependencies:

  • FluentValidation library
  • OWASP dependency check tools
  • Penetration testing (external vendor)
  • Security code review process

Risks & Mitigation (Nordic/EU Security Context):

...


...


...