Stop Losing Days to Model Debugging
What happens when your model goes wrong?
If you deal with customer transactions and your fraud detection model goes wrong, you’re hemorrhaging money as customers take their business elsewhere or you’re turning it off and losing money from refunding fraudulent transactions and paying fines for your negligence.
If you’re in manufacturing and your model goes wrong, production needs to be turned off, possibly to the tune of millions of pounds a day, or the products are faulty and need to be destroyed, likely costing even more.
And while you can put processes and guardrails in place to prevent these problems as much as possible, it’s a fact of life that at some point something is going to go wrong. So today’s post is about how to recover from modelling disasters before one small mistake costs millions of pounds.
An Expensive Mistake
Company X is a payment company processing millions of dollars in transactions every day. Like any financial services company, they’re regulated, therefore they need to have a Fraud and Anti Money Laundering process.
In particular, they handle payments, so they need a fraud detection model to flag potentially suspicious transactions for further investigation. Think of it like the texts your bank sends you when you’ve bought something expensive from a foreign website and you’re made to confirm you recognise the recent transactions they show you.
It’s a normal part of business as usual for the company… until one day it suddenly goes wrong.
The first Company X hears about it is on social media platform X, formerly known as twitter. Someone has tagged them, complaining they’re trying to make a payment and continually getting blocked for suspected fraud. It’s something that happens of course and isn’t usually an issue, but the problem this time is the volume.
The phones start ringing constantly, unhappy customer after unhappy customer unable to make purchases because they’re being flagged for investigation. And those are the customers they know about. Thousands more could be giving up without saying a word, the money going unspent.
This is of course a huge issue for the company. If people aren’t able to buy then the company isn’t making any money and neither are their clients, who rely on them for their payment processing. The cost of not fixing this problem isn’t just a huge monetary bill, it’s reputational as well with the real risk of clients abandoning them for more “reliable” payment services.
They have an alternative of course. They could turn the fraud detection model off. That would stop the customers getting blocked at the very least. But that doesn’t fix the problem either. In that case, potentially fraudulent transactions then would get through, meaning customers would need reimbursing and even worse, the company is at real risk of fines due to not meeting fraud regulations. It’s a lose lose situation until the problem gets fixed.
Table showing the risks if payments where kept blocked and if the fraud detection was switched off
Fix it ASAP
The real issue is that fixing the problem takes too long. When they look at the algorithm they see a 3000% increase in transactions being flagged as fraudulent which explains all the complaints, but that doesn’t answer the important question: why? This is where the data team comes in to investigate the issue.
The first thing someone will do, be it a data engineer, ML engineer or whatever their job title is, they’ll check the logs to see if there’s any information that could tell them why the model was producing strange numbers. In this particular case, the logs look fine. There’s nothing immediately obvious from looking that explains what the problem might be. Which means someone needs to investigate what’s going on.
It’ll then be passed to the team that’s responsible for the model, usually a team of data scientists or a similarly named role. It will be someone’s job on that team to find out why the number the model is producing is strange.
So the first thing they do is try and reproduce the issue. If they run the model locally themselves, do they get the same result? This might take a few attempts to get the right conditions to reproduce and depending on how quickly the model runs could involve a lot of sitting around waiting for the code to run. But eventually it does run and they’re able to reproduce with an example transaction that shouldn’t have been flagged.
Next they try to work out why it happened. They might have an inkling of where the problem occurred that gives them a head start but in reality they’re working backwards, examining the model in small sections of code to try and trace down exactly where the issue occurred.
One issue is, of course, that you can’t run code backwards, you can only run forwards, so every time the data scientist’s hunch doesn’t pan out, they have to start the code running again, searching earlier and earlier sections of code to try and trace down the issue.
Eventually, after hours, if not days, of questioning their code, their maths, redoing calculations manually to cross verify and waiting time and time again for code to run, they hit the jackpot: The input data from the risk score supplier has changed, it’s coming back with numbers on a completely different scale to what the model expects and this is likely the culprit. So they identify the problem and send it back to the data engineer, or equivalent to fix.
The problem is that the data the data scientist sees is probably not the raw output of the supplier’s API. It’s likely been massaged and reformatted by the data engineer first. Depending on how confident the data engineer is, they might go straight to the supplier and raise the issue or they might spend a bit of time verifying before they do. In the latter case, even more time is spent verifying the real source of the issue.
But eventually it’s confirmed! The supplier DID change the data and forgot to pass on that information. They provide the relevant logic that needs to be implemented to adapt to the new change and either the data engineer or data scientist implements it, thus resolving the problem and getting the model working again…
Only after days of investigation that have cost the company millions of pounds.
The real problem was that the company didn’t have incremental computing.
Fix it faster
Now let’s take the same problem for a company using Evil Work’s puff platform. Their model can be viewed as a graph, a bit like the below image, where the top nodes are the inputs into the model, internal data and data from suppliers in this instance, and the bottom one here is the fraud detection model output.
Each node in between is an operation. Whether filtering data or doing a calculation, we can represent any model by drawing the sequence of what happened in this way.
So in the case of Company X, the fraud detection node here was what failed, and the cause was this supplier data in the top layer here. But Company X had no idea about that.
With PUFF, however, we track computations at the lowest level of code which means that for any model in PUFF, we generate this graph as a representation for one hugely advantageous reason: We can use it to work out what we need to recompute or not.
PUFF has the basic fundamental feature of incremental computing which is: If the code ran successfully once and the code hasn’t changed, the inputs haven’t changed and the data hasn’t changed, then don’t rerun it, which means that if any node in PUFF has rerun, it means that something has changed.
This means that the model graph you see from PUFF actually looks like this.
The supplier data, the fraud model and all nodes upstream, as well as any other nodes that use the supplier data, have all turned orange because they recomputed. That means that the inputs, or the code, for each one changed, and we differentiate between the two by the way and in this case, only the inputs changed.
So when all of these nodes have changed, it means that we can trace the graph all the way to the top and see immediately that the data from the risk score supplier has changed. No need to debug, no need to even pass it to the data scientist. The engineer looks at the data, sees the risk score data has changed and immediately raises it to the supplier without needing to touch a single line of code.
That means the days of investigation where the company is hemorrhaging millions of pounds just don’t exist. The problem can be solved within hours, preventing a mistake that the company has no control of by the way, it was their supplier’s fault, from costing them millions of pounds while they work out what was going on.
This also works with code by the way. If the issue had been a bug introduced by one of the team, PUFF identifies exactly which line has been changed so no one needs to do any investigation.
And that’s really the key benefit of PUFF. There are many companies out there already using DAGs to track calculations which allow dependency tracing and in theory are more efficient by saving you from rerunning nodes that have already completed. But they’re usually set up on a task level, with configuration needed to make the code compatible and debugging still needed to trace down the exact issue.
PUFF does this instead at every single operation of the code so you can isolate the exact issue without needing to debug anything.
If, like Company X, you’re losing time and money trying to work out exactly what blew up your model, we can help you solve it. Get in touch here
Our closed beta is now Live. Sign up via the link below and keep being Evil