Working with Strata and AWS tools for Customer Retention and ML process standardization.

C&W Communications is one of the leading telecommunications and entertainment providers in the Caribbean and Latin America. As many other services providers, one of their main challenges is retaining customers. They were facing challenges with customer churn and needed to develop a customized machine learning solution to predict and prevent involuntary churn, they also wanted to standardize their ML development lifecycle and automate their data science processes. Strata partnered with the customer to develop a scalable MLOps platform using AWS services, resulting in a 10% increase in retention and collection campaigns, 90% reduction in operational tasks and errors, and 70% less effort for future releases. The solution allowed the customer’s analytical team to focus on performance development and improvement while reducing deployment time by 95%.

Common Challenges require Innovative Solutions

C&W Communications faced a pressing business challenge when they observed a surge in the number of subscribers who were involuntarily leaving their services. This problem could be addressed using machine learning (ML) techniques. The company needed to predict the propensity to involuntary churn on a daily basis so that they could design targeted collection campaigns and take timely actions to prevent churn. By doing so, they not only aimed to reduce the number of subscribers leaving their services but also minimize the outstanding debt.

Likewise, they experienced common challenges that are often encountered when putting machine learning models into production. These challenges include difficulties in monitoring, the need for manual interventions, and scalability issues. Similar to other organizations, more than half of their machine learning projects were unable to be successfully deployed to production due to inconsistent and labor-intensive workflows. Model improvements were often initiated manually, in response to deteriorating performance, and scalability in the development process was limited. Additionally, the success of these projects heavily relied on key personnel, making staff turnover a major concern.

The main obstacles that C&W Communications faced were two. The first was the ability to predict the propensity to involuntary churn for each customer daily to optimize Retention & Collection campaigns. The second challenge involved the complete management of the machine learning (ML) solution’s lifecycle, including standardizing operational procedures to reduce dependency on personnel and ensure scalability. Additionally, there was a need to enable or disable features remotely without code deployment, develop an automatic retraining process that helps to overcome ML model drifts, and prevent the misuse of resources since staff members were devoting significant amounts of time to manual operational duties rather than impactful business tasks.

AWS – Reliable Partner

C&W Communications has been utilizing various AWS services for their business needs, including the deployment of their data lakes. This prior experience with AWS has allowed for a seamless integration of the model serving with the campaign management platform, utilizing the data lakes already in place. This integration has greatly facilitated the collection of customer churn propensity scores for the Retention & Collection campaigns, streamlining the process and increasing efficiency. Furthermore, AWS’s reputation for reliability, scalability, and constant improvements has provided C&W with the confidence to trust their services and rely on them for their business needs.

Strata and C&W Communications Joint Work

Strata had been working for C&W Communications for many years, providing Advanced Analytics and Data Science solutions, Marketing Automation for its main campaigns and Data Driven Business Consulting. Strata has extensive experience in descriptive and predictive modeling techniques providing deeper insights, using multiple leading-edge algorithms and agile models’ development, providing data key relationships and new pattern discovery, using statistical techniques for modeling and testing, delivering data visualization Dashboards to help businesses understand their situation or results of models, employing MLOps framework as a best practice.

A MLOps Approach for Greater Scalability

Strata worked closely with C&W to develop a comprehensive solution that addressed both their churn prediction and ML management challenges. The MLOps solution developed by Strata automates common data science processes such as data preparation, model training, and testing. Strata’s solution uses high- performance ML algorithms for data processing, prediction, monitoring, and automatic retraining, and is deployed in the customer’s cloud account. The system is serverless and event-driven, developed under agile methodologies to enable granular releases into production. Throughout the project, Strata collaborated with various C&W departments, including business, finance, analytics, and ML, to ensure that the solution met the specific needs and requirements of the organization.

The solution includes a machine learning model that predicts involuntary churn using a high-level, tree-based algorithm that was chosen after extensive testing and experimentation. The architecture is fully orchestrated for AWS Step Functions, using AWS Glue for the Data Pipeline and CWC’s DataLake as the data source. Amazon SageMaker Job is used for the Model Pipelines, with Amazon CloudWatch for logging and Amazon DynamoDB for storing model metadata and process tracking. Overall, the MLOps framework provides high scalability and availability, reduces time to market, and allows for standardized, easy control, operation, and maintenance of the ML solution.

The goal of the system is to perform daily inference, monitoring, metrics, and weekly retraining, as shown in the architecture diagram below. The daily process begins when new data for the partitioned day becomes available. Each Pipeline was designed to be atomic, meaning that incomplete or erroneous pipelines will not produce corrupted data. Instead, they will only generate metadata for analysis. This allows each Pipeline to be triggered again and rewrite its output while creating new metadata.

High Level Architecture

Due to the large amount of data that needed to be processed, a distributed data compute solution was required. After careful consideration, Glue Spark was determined to be the best suited option for the customer. The DataPipelines run on Spark Glue Jobs, which are connected to AWS Glue Data Catalog and the processed data is saved partitioned in the project data bucket.

After the DataPipelines are completed, the ML model workflow proceeds to several SageMaker Processing Jobs containers, which perform various tasks such as computing model metrics, generating data preprocessing for the model, generating inference (scores), retraining the model and running the best model selection process on a weekly basis. The processing jobs run in a serverless manner, taking advantage of the scalability and flexibility of AWS SageMaker. This enables the team to easily manage the inference and monitoring stages of the ML model lifecycle in a streamlined and automated manner.

Better Model and Retention Campaign Performance – Less manual tasks

The implemented solution resulted in a significant positive impact, including a 10% increase in Retention & Collection campaigns, a 90% reduction in operational manual tasks and errors thanks to pipeline automation and standardized operations, and a 70% reduction in effort required for later releases into new markets due to the replicable and scalable architecture. Following the successful development of the churn model for the first market (Jamaica), Strata was approached to develop additional models for two other markets (Bahamas and Trinidad and Tobago). Additionally, the ML framework solution allows the analytical team to focus 100% on developing and improving model performance, making more effective use of their talent, and reducing deployment time by 95%.

The solution grants consistency due to the presence of own developed containers, flexibility and reproducibility for the development of new markets, reusability of components across projects and scalability thanks to MLOps framework, monitoring and explainability through the metadata tracking.

The joint work continues

There is an ongoing contract of C&W Communications with Strata, that consists of delivering Machine Learning solutions and Marketing Campaigns, which include the daily release of retention, stimulation and collection campaigns for different audiences and purposes. There are several model solutions provided for different business issues and markets (churn predictions, no recharge prediction, top up segmentation, behavioral segmentation, among others) and executed on daily, weekly or monthly basis as inputs for the aforementioned campaigns. This service also includes the accompaniment of the customer in the analysis and monitoring of main KPIs, helping to understand possible issues or data-driven insights. Strata is in the Advanced Tier of AWS and is validated in its Services Path.