Overview
A global enterprise organization was replacing a legacy, Windows-based ERP system (delivered via Citrix) with a modern cloud ERP platform. The new platform needed to integrate with more than 60 enterprise services while supporting over 200 independent organizations and approximately 15,000 users.
The legacy integration mechanism relied on a Win32 client application and FTP-based file transfers — a pattern that was incompatible with the new SaaS ERP architecture and its security, reliability, and scalability requirements.
I was responsible for designing and leading the implementation of a new integration architecture that would meet modern SLAs, security expectations, and operational efficiency goals.
The Problem
Legacy Constraints
- Integration built on FTP + Win32 client
- Tight coupling between ERP and integration mechanism
- No cloud-native reliability guarantees
- Limited observability
- Security controls primarily perimeter-based
- Not designed for API-first integration
Business & Technical Requirements
- Support file ingestion and processing for 200+ organizations
- Integrate with 60+ upstream enterprise services
- Achieve 99.95% uptime SLA
- Minimize operational cost
- Support CI/CD automation
- Secure API-based integration
- Scalable and modular for future growth
Architectural Strategy
1. Move to Event-Driven, Serverless Architecture
Instead of rebuilding another centralized integration service, I designed a serverless, fan-out architecture using managed cloud services.
Key design principles:
- Event-driven processing to decouple producers and consumers
- Stateless compute for horizontal scalability
- Secure API-based file retrieval
- Private network isolation via VPC segmentation
- JWT-based authentication with token caching
- Clear separation of ingestion, processing, and storage layers
This removed tight coupling and allowed components to evolve independently.
2. Replace FTP with Secure API Integration
The legacy FTP mechanism was replaced with:
- API-based file retrieval
- OAuth/JWT authentication via identity provider
- Token caching layer for performance optimization
- Explicit state tracking for file processing lifecycle
Security was designed in from the start rather than bolted on.
3. Fan-Out Processing Model
The architecture used:
- A file list retrieval service
- A download service
- Event-triggered file processing functions
- Independent processing flows per file
This allowed:
- Parallel processing
- Fault isolation
- Clear retry strategies
- Observability at each stage
4. CI/CD and Operational Discipline
The solution included:
- CI/CD pipeline automation via Azure DevOps
- Infrastructure-as-Code provisioning
- Modular deployment units
- Clear ownership boundaries
- Automated monitoring and alerting
The system was designed for maintainability from day one.
Security Model
Security decisions included:
- Private subnets for processing components
- Public subnet only where necessary for ingress
- Role-based IAM policies
- JWT validation at ingestion boundary
- No implicit trust between services
- Explicit state transitions recorded in storage
This reduced the attack surface while enabling integration flexibility.
Results
- Achieved 99.95% availability target
- Supported critical enterprise integrations
- Enabled scalable onboarding of 200+ organizations
- Reduced operational cost to under $6,000/year
- Modular architecture enabled independent updates
- Improved observability and troubleshooting
Most importantly, the architecture established a reusable integration pattern that could be replicated across additional enterprise initiatives.
Architecture Diagram
Lessons Learned
- Keep the design simple — complexity creeps quickly in integration systems.
- Event-driven patterns dramatically improve decoupling and resilience.
- Token caching is critical when interacting with high-frequency authenticated APIs.
- Clear separation of ingestion, state management, and processing reduces cognitive load.
- Serverless is powerful when paired with strong governance and observability.
