The solution

Automated population of a new Data Mart was implemented, consolidating a unique data structure for all countries, so that KPIs and all information used for support and decision making is standardized. 

The Data Mart was developed with a Serverless Architecture on the AWS Cloud to facilitate maintenance, scalability and data availability. Below, we list some of the services used in the solution:

Storage & Data:

• Amazon S3: For scalable object storage, including data files to be queried by Amazon Athena.
• Amazon Redshift Serverless: For Historical Data
• Amazon Athena: For querying Databases & Tables
• AWS Lake Formation: For centralized permission management and Data sharing
• AWS Direct Connect: For a dedicated network connection to AWS.

Compute & Code:

• AWS Lambda: Serverless Python Functions, used to clean, process and ingest data, generate mappings, and monitor the solution.
• AWS CodeCommit: Used for Source Control Management
• AWS CodeBuild: Used for the deployments needed for the solution

Security:

• AWS Identity and Access Management (IAM): Manages Identities and their permissions, configured to use least-privilege principles
• Amazon GuardDuty: Continuously monitors the AWS Account for anomalous behaviors and events.

Monitoring:

• AWS CloudTrail: Registers Administrative Events in the account.
• Amazon CloudWatch & Amazon Simple Notification Service (Amazon SNS): Monitors Metrics and Admin events logged by CloudTrail, and uses an Amazon SNS topic to notify the interested parties if needed.
• AWS Cost Explorer: Keeps track of costs and is used for reporting.