Self-Learning & Feedback Loop

How the mule detection model improves from analyst decisions, validated by continuous backtesting

A static detection model decays — mule networks adapt their patterns specifically to evade whatever’s currently catching them. The AML model is designed to improve continuously from real analyst outcomes, rather than only being retrained on a fixed schedule.

The feedback loop

1. Model flags a cluster

   Graph and behavioral signals produce a candidate case, as described in Mule Graph Detection.

2. Analyst disposition

   Every case an analyst closes — confirmed mule ring, false positive, or inconclusive — is captured as a labeled outcome, not just an audit-log entry.

3. Outcomes accumulate as training signal

   Confirmed cases reinforce the patterns that correctly flagged them; false positives are just as valuable — they tell the model which patterns it’s over-weighting.

4. Backtesting before any model update ships

   A candidate model update is run against historical labeled cases before it’s promoted — confirming it improves precision/recall on known outcomes, not just performing well on paper. See Backtesting below.

5. Promoted model informs future flags

   Only after backtesting validates an improvement does the updated model start generating new candidate cases — closing the loop.

Backtesting

Before any model change is promoted to production, it’s evaluated against a held-out set of historically confirmed cases — both true positives (confirmed mule rings) and true negatives (accounts investigated and cleared). This catches two specific failure modes a naive “retrain and ship” process wouldn’t:

Regression on known patterns

A model update that gets better at catching a new pattern but worse at catching previously well-detected ones — backtesting against the full historical case set catches this before it reaches production.

False-positive drift

A model that becomes more aggressive (catches more true positives) at the cost of a rising false-positive rate — analysts bear that cost directly in caseload, so this is tracked explicitly, not just accuracy in aggregate.

Why this matters for compliance

A model that only improves via infrequent manual retraining lags behind evolving mule tactics by definition — by the time a retraining cycle catches up, the pattern has often already shifted again. A continuous feedback loop, gated by backtesting rather than shipped blindly, keeps detection current without sacrificing the precision regulators and analysts both depend on — an ever-more-aggressive model that drowns analysts in false positives is not actually a better model, even if its raw catch rate looks higher.

What analysts see

The feedback loop is designed to be invisible to day-to-day case work — analysts disposition cases the same way regardless of whether that disposition happens to be valuable training data. Model performance trends (precision, recall, and false-positive rate over time) are visible to compliance officers in aggregate, not surfaced as a burden on individual case review.