Posted by Douglas Wood, Editor.
As regular readers of this blog know, I spend a great deal of time writing about the use of technology in the fight against crime – financial and otherwise. In Part One of this series, I overviewed the concept of Major Investigation Analytics and Investigative Case Management.
I also overviewed the major providers of this software technology – Palantir Technologies, Case Closed Software, and Visallo. The latter two recently became strategic partners, in fact.
The major case for major case management (pun intended) was driven home at a recent crime and investigation conference in New York. Full Disclosure: I attended the conference for educational purposes as part of my role at Crime Tech Weekly. Throughout the three day conference, speaker after speaker talked about making sense of data. I think if I’d have heard the term ‘big data’ one more time I’d have gone insane.  Nevertheless, that was the topic du jour as you can imagine, and the 3 V’s of big data – volume, variety, and velocity – remain a front and center topic for the vendor community serving the investigation market.
According to one report, 96% of everything we do in life – personal or at work – generates data. That statement probably best sums up how big ‘big data’ is.  Unfortunately,  there was very little discussion about how big data can help investigate major crimes. There was a lot of talk about analytics, for sure, but there was a noticeable lack of ‘meat on the bone’ when it came to major investigation analytics.
Nobody has ever yelled out “Help, I’ve been attacked. Someone call the big data!”. That’s because big data doesn’t, in and by itself, do anything.  Once you can move ‘big data’ into ‘smart data’, however, you have an opportunity to investigate and adjudicate crime. To me, smart data (in the context of investigations) is a subset of an investigator’s ability to:

1. Quickly triage a threat (or case) using only those bits of data that are most immediately relevant
2. Understand the larger scope of the crime through experience and crime analytics, and
3. Manage that case through intelligence-led analytics and investigative case management, data sharing, link exploration, text analytics, and so on.

Connecting the dots, as they say. From an investigation perspective, however, connecting dots can be daunting. In the children’s game, there is a defined starting point and a set of rules.  We simply need to follow the instructions and the puzzle is solved. Not so in the world of the investigator. The ‘dots’ are not as easy to find. It can be like looking for a needle in a haystack, but the needle is actually broken into pieces and spread across ten haystacks.
Big data brings those haystacks together, but only smart data finds the needles… and therein lies the true value of major investigation analytics.