Rules based vs machine learning in fraud prevention: Understanding the differences and benefits

When it comes to rules based vs machine learning, which is more effective? Your business should be familiar with the benefits and challenges of each so you can decide which approach is suitable to address your fraud risk.

What’s an effective enterprise fraud management approach to reduce payment processing losses?

The accelerating cost of online fraud remains a major threat for merchants, with online payment fraud estimated to cost merchants $3 for every $1 lost to fraud.¹ The challenge now is to manage fraud more effectively, while still working to prevent more traditional tactics such as stolen credit cards and identities.

To do so, merchants have two obvious enterprise fraud management options:

1. Traditional rule-based fraud detection and prevention
2. Adaptive machine learning fraud prevention

Yet many struggle to decide between the two. Is one better than the other? Or can both be used in concert?

Rule-based vs machine learning fraud prevention modeling

Machine learning modeling may be effective for minimizing payment processing losses compared to traditional rules. But that may not be the whole story. Basic analysis has demonstrated that while machine learning is a solid approach, it may lack validation against real-world transactional traffic.²

Settling the rule-based vs machine learning debate would require analysis of historical data that has run in both scenarios, in order to generate actual performance comparisons for each. However, this would require a massive amount of transactional data with the same schema and the known outcome represented in chargeback and decline rates.

Why choosing an effective fraud prevention system matters

Fraudsters have been attacking merchants since the dawn of e-commerce – and they’re using new technology to defeat fraud prevention techniques.³

This is a major issue for fraud management tools, in that they become less and less effective as fraudsters continue to test their limits, and design attacks to circumvent filters. As a result, fraud prevention teams simply can’t keep up with continually changing attack techniques—exposing merchants to significant financial and reputational damage.

The pros and cons of a rule-based approach

There are, of course, both advantages and disadvantages of a rule-based fraud prevention approach.

Advantages of a rule-based approach to fraud prevention

Effective rule-based fraud prevention approaches require human interaction. It can sometimes be relatively easy for analysts monitoring events to spot errors and make a swift correction. This can save the day for merchants, especially where machine learning is deployed but hasn’t yet learned to identify these issues. Smaller and growing merchants may benefit from a rule-based fraud detection approach. For example, putting in place rules that don’t allow an order to ship out where the IP address is in a different country than the billing address may help reduce fraud.

Rules also make it easy for analysts to understand why a transaction was declined, affording them the opportunity to create new rules as needed. And if they see new fraud patterns in transactional data, additional rules can be created to tackle them. Again, that has value in scenarios where machine learning models are still getting up to speed.

Disadvantages of a rule-based approach to fraud prevention

While rule-based fraud prevention may be a good choice for smaller organizations, these systems can become overwhelmingly complex and cumbersome to maintain as they grow. Once a rule is created, it can’t be changed. That means rule-based systems grow and grow as fraud evolves and the bad guys invent new ways to circumvent them. It can be challenging knowing which rules to discard and which to keep. The need for more staff to manually observe trends and create new rules can become prohibitive.

Advantages of machine learning for fraud prevention

By creating algorithms to process large datasets with multiple variables, organizations can leverage the advantages of machine learning to rapidly discover correlations that might indicate more sophisticated fraud attempts. Not only do they identify trends and patterns that the human eye might miss, they’re also able to continuously adapt to keep pace with the fraudsters. This can result in a higher degree of precision in identifying fraud. Effective fraud prevention using machine learning powered by clean, robust, and proprietary data helps enable a model that is more sophisticated than those available to purchase ready-made. Better models powered by better data may help provide an advantage over fraudsters.

Whereas rules are created in response to a particular fraud pattern recognition, machine learning considers various signals before deciding a transaction is fraudulent.⁴ These systems typically also look at multiple factors based on administrator input when they are in the initial learning phase.

This stops fraud prevention with machine learning models from getting stuck on a particular trend – enabling them to capture more fraudulent attempts with fewer false positives. Their adaptive abilities can outshine rules by fighting fraud protection in real time, detecting patterns as they start to form, and using models based on thousands of transactions to identify future fraudulent activity. Plus, these models can be retrained using the freshest data, making fraud prevention using machine learning more precise.

Machine Learning – used the right way

As with any fraud management tool, the way it is used makes a difference. Machine learning models are not a silver bullet for fraud prevention if they are not properly designed and trained or fail to leverage the necessary features.

There are ways to address this, including SHAP (Shapley Additive Explanations) or LIME (local interpretable model-agnostic explanations) algorithms which help explain what machine learning models are doing behind the scenes. By visualizing the output, SHAP can be used to easily explain a model’s prediction through computation of each feature’s contribution. Meanwhile, LIME focuses on interpreting local models by modifying a sample and observing the results. Both have their advantages, but they don’t necessarily provide the straight line needed to provide a clear picture of transactional behaviors.

Machine learning fraud management tools require time to absorb data and analyze information to produce results and learn user behavior. And it needs a sufficient volume of data. Once it’s functional, machine learning in fraud prevention can be automated. This ramp up/training period typically requires 1-2 months as rejections/chargebacks must be realized (identified) to begin training the model. Fully realized historical data allows for this training to be shortened, assuming the features required are included in the data set.

Combining great aspects of both rule and machine learning-based fraud prevention systems

The good news is that this doesn’t have to be an “either/or” decision for merchants. In fact, organizations can benefit from combining great aspects of both rule and machine learning-based systems to drive down fraud rates without increasing false positives.

While machine learning models are “powering up,” rules can step in to do some of the heavy lifting for enterprise fraud protection. Because rules are based on particular trends, administrators can write them on the fly while fraud prevention using machine learning is being trained to see the same patterns. This approach can help merchants protect themselves and their customers from fraud with the immediacy of rules and the sophistication of machine learning. In other cases, anti-fraud platforms might use machine learning algorithms to suggest new rules for analysts to create based on evolving fraud patterns.

Organizations which have an open mind to using both fraud management resources will ultimately be well placed to manage eCommerce fraud risk effectively than those that do not.

If you’re ready to learn more about the many ways PayPal's 20+ years of building risk models can benefit your business, manage risk here.