Value creation with Posit Team: improving model accuracy by more than 50 percent

As a loyalty company, Air Miles serves our partners with insights, campaigns, and predictive models to support their businesses. Over the last few years, we realized our tools and workflow started to become a bottleneck in achieving our goal of becoming a personalized rewarding system by processing individual customer preferences. 

Our data team was using a drag-and-drop modeling tool, in which several standardized models were developed and put into production. These propensity-to-buy models predicted users’ next visit to each of our partners. At the time, those models were sufficient for our business. However, with our commitment to becoming a more data-driven company, we had to evaluate the tooling and model performance.

At the time, we were facing model runtimes of eight hours, inefficient load times, and a lack of transparency and documentation—there was a clear need for a more professional approach for our business. 

Until then, we were utilizing a local version of R and Posit was only being used for ad hoc analysis in a separate environment.

In 2020, we started the process by installing Posit Workbench as our new standard for model development, with GitHub for version control. As we transitioned from drag-and-drop tools to using Posit, our models were rebuilt with the use of more advanced machine learning models.

Workbench has helped us improve the quality, reproducibility, and flexibility of our workflow. We use Posit Connect to publish our insights, which range from reports to dashboards and scheduled predictive models. This dramatically increases the transparency and adaptability of our production models. 

With Posit Team, we are able to select the relevant models for our business, tune them with our preferred performance metrics and most importantly track their performance along the way. We then continually test and implement improvements on our models and processes with the combination of GitHub and Posit.

Throughout this process, Posit offered useful advice and suggestions for next steps in our development. Their support helped in growing our use of Posit Team across the business through the installation of Python IDEs, such as Jupyter Lab and VSCode, within the Posit Workbench environment.

The biggest advantage we have experienced is the decrease in runtime of our models. Some of our models have gone from running over eight hours to only one. Additionally, our processes are now scheduled and automated so model training and scoring are no longer a manual task. 

Further professionalization includes proper documentation, version control, and standardization of workflow. The ability to work together and collaborate has also boosted the learning curve within our team and decreased errors in our coding.

With these fundamentals in place, we have seen an overall improvement in process. The accuracy and recall of one of our main models has improved over 50 percent.

Loyalty Management Netherlands (LMN), the organization behind Air Miles, was founded in 1994 and has a large and loyal customer base. We are focused on growing toward a shared intelligence center in collaboration with A-label companies (like Albert Heijn, Shell and Praxis). Our mission is to create a world where people are appreciated and connected to relevant brands they trust with a personalized rewarding system that processes individual customer preferences. We aim to provide a 360 degree customer view where customers are in charge of their data and are rewarded for sharing it.