Passive Analysis of Leaky Data

By Editor Last - April 23rd, 2019 Posted in Product Announcements

Last week we rolled out a set of new features to our Trillion platform, one of which was our Proof Of Life feature.  The purpose of this particular feature is to try and help further improve the accuracy of the “potential risk” that leaked data could have on an organisation, and we thought that it would be useful to share some interesting results that came out of our deployment tests.

One of the reasons we introduced Proof Of Life is because there are a number of variables that need to be considered when evaluating leaked information.  Data breaches are occurring so frequently now that its like being told its going to rain tomorrow (we’re in the UK and it rains a lot!).  Almost every morning, news of several new breaches make the wire, security managers across the world tut in disappointment and then move on with the rest of their day.  This is pretty standard behaviour when the exceptions become the norm and if you’re monitoring for this data yourself then you are probably seeing information relating to old accounts repeatedly turning up which no longer matter to you.  False positives are a classic IT Security distraction and quickly causes false positive fatigue.

On the flip side the challenge with “tuning” breach data is that it’s very difficult for anyone outside of an organisation to know the value or risk of individual data records such as leaked credentials.  Organisations are unique, Individuals change roles, staff leave the business, change names, email address formats and even domain names might be changed and all of this can make interpreting discovered data difficult.

Proof of Life is a component in Trillion which has been designed to walk over data which appears to belong to our subscribers and tries to passively validate if the accounts found still exists in the organisation using a number of mailbox response tests.


We added Proof Of Life into our risk engine to improve data quality, reduce the amount of client analysis required and just bubble up what matters.  During testing we ran the new feature back over 7,000 previously detected alerts from various size organisations and business verticals and observed some interesting results.  Firstly of the original 7000 samples 26% of the data just couldn’t be verified as some organisations will just accept anything as a valid email by implementing catch-all addresses, so where Trillion detected a sinkhole in place it just stopped.  Of the remaining, 62% looked like the accounts failed to respond to queries and 37.7% were still in use.

Verify Stats 2

This is interesting for a couple of reasons.  The first is that it highlights a significant amount of data being turned up in breaches may no longer present a current business risk, therefore under normal situations its understandable why organisations stop using basic alerting services or manual processes – 62% is a high false positive rate if you’re trying to work with this data without context or validation.

However 37% of the data we tested did confirm valid accounts still existed for the records that had been leaked.  This is an extremely high number of valid accounts that shouldn’t be ignored.  This doesn’t in anyway validate that the credentials found were correct as the tests are non-intrusive, but it shows that of the data sets Trillion tested there were a lot of records that had the potential to provide a foothold to an attacker.

Its easy to see why this high mix of false positive / true positive data could be problematic for organisations trying to manage these risks on their own, with a constant stream of new data being added every day.

These numbers have been summarised over a group test so at an organisational level they may look very different.  We’ll be monitoring that over the next couple of months and will publish updated stats as soon as they become available.