Real-time dashboards to evaluate a churn prediction model

A cyber security software provider had already solved the harder problem: it had a churn and downsell prediction model live in production. What it could not do was tell the board, with any confidence, whether that model actually worked. QuantSpark closed that gap with a suite of real-time dashboards, replacing slow, ad-hoc analysis with continuous visibility into the model's performance, and turning every governance question into a self-serve answer rather than a fresh data-science task.

The problem: a model in production, but no reliable way to see it working

The company's churn and downsell model was generating risk scores in the live environment, but the reporting around it had not kept pace with the deployment. Whenever the board or the C-suite wanted to know how well the model was performing, whether performance held up across business units and customer segments, or whether the customer-success team was actually engaging the accounts the model flagged as high risk, the only route to an answer was a bespoke analysis, built from scratch each time.

That pattern is a familiar failure mode for models that reach production without persistent monitoring built in from the start. A model's accuracy in testing says little about how it behaves once it is live across different regions, segments and shifting customer behaviour, and without instrumentation designed for the purpose, every governance question becomes billable analyst time rather than a repeatable answer. For a board trying to hold a model accountable, that is a slow and expensive way to get comfortable.

The approach: build the reporting the business actually needed, not the reporting the data allowed

QuantSpark started with the questions rather than the data. Working with the company's data science team and senior management, the team mapped the business questions and user journeys the dashboards actually needed to answer, then agreed the metrics that mattered: precision and recall over time, performance broken down by region and customer segment, and the underlying signals driving individual risk scores.

With those requirements fixed, QuantSpark used DBT and Snowflake to engineer a set of well-structured data tables from the raw model outputs and business data, creating a dependable foundation to build on rather than reporting built directly on messy source tables. On top of that data layer, the team designed and built the dashboard suite itself: real-time views of model performance and its drivers, sliced by the dimensions that mattered to the business, plus a dedicated view tracking whether customer-success engagement was actually concentrated on the highest-risk accounts, connecting the model's output to the process meant to act on it.

The final design choice was the one that made the dashboards a governance tool rather than a reporting tool: every view was paired with a recommended call to action, so a question about model performance resolved not just in a number but in a next step, whether that meant retraining a segment of the model or redirecting customer-success attention.

The result: real-time answers, not ad-hoc analysis

The dashboard suite gave the board and senior management real-time answers to questions that had previously required repeated, time-consuming ad-hoc work, freeing analyst time for improving the model rather than re-proving it every time someone asked a question about it. It surfaced where the model was succeeding and where it needed improvement, by region and by customer segment, and made visible whether the accounts the model flagged as highest risk were the accounts customer success was actually working.

The source does not report a quantified return, so no percentage or pound figure can responsibly be attached to this project. The value is better understood as a shift in kind: a recurring cost, the analyst time spent answering the same category of board question over and over, replaced by a one-off build that pays that cost back on every future question. For a business considering the same move, the honest way to size the return is to total the analyst hours spent on ad-hoc model-performance requests over a typical reporting cycle, and weigh that recurring cost against the cost of building the equivalent dashboard suite once.

The build sits on Snowflake as the data warehouse and DBT as the transformation layer, with a real-time dashboarding layer presenting the views; the source does not name the specific BI tool used for that presentation layer.