Barriers to leveraging open source intelligence

Open Source Intelligence (OSINT) can be a powerful intelligence and investigative tool, but is often overlooked. In many organizations, there are significant barriers to the adoption of effective open source intelligence and a failure to adapt quickly enough to emerging technologies. A cultural shift is needed to raise the profile of open source intelligence and ensure it reaches its full potential.

Open source intelligence is a critical component of modern intelligence and investigative tools. The growing volume of data available online provides investigators with a rich source of information. The insights that open source intelligence can provide are unlikely to be found in in-house datasets, select databases, or sanctioned lists.

Failure to leverage open source data can lead to embarrassment and intelligence failures. There are many strong examples of open source intelligence playing an important role in solving cases: Bellingcat's insights into the downed MH17 flight in 2014 relied entirely on open source intelligence.

Open source intelligence should also be considered an essential element of counter-terrorism and counter-misinformation programs. Mapping terrorist networks on social media, especially the more grassroots right-wing extremist groups now appearing on platforms such as Parler, is a highly effective means of identifying the individuals behind these crimes.

Investigators have also had great success in identifying networks that spread misinformation/disinformation and the real identities behind them. Imran Khawaja was sentenced to 12 years in prison for preparing acts of terrorism, attending training camps and possessing firearms. This was all because open source intelligence provided the bulk of the evidence.

It is difficult not to conclude that open source investigations are of growing strategic importance. In addition, they can save money as a quick and economical way to learn about criminals early in an investigation before deploying more expensive and intrusive tactics. So why are so many organizations still failing to leverage open source intelligence?

Barriers to Leveraging Open Source Intelligence

1. Misconceptions

The lack of investment in open source intelligence is often based on a misunderstanding of what it really means and what it is worth. "Open source intelligence" may conjure up a negative image with connotations of hacking and privacy violations. However, the type of adoption of open source intelligence we advocate can be better described as online open source investigation: the use of freely available online information in a targeted and non-intrusive manner.

2. Cultural and Technical Barriers

Cultural and technological deficits also factor into this attitude toward open source intelligence. Many are wrestling with outdated technical architectures and spending most of their energy on how to better manage their internal data. However, this is driven by the culturally outdated assumption that the greatest insights can always be found in the vast amounts of data that large organizations spend decades accumulating. Open source intelligence will increasingly provide greater insights for individuals and companies than those found internally.

As organizations realize the importance of open source data, they typically use it only in the form of curated datasets that do not capture all of the rich, valuable information available on the Internet. For example, the well-known curated dataset LexisNexis offers 6 petabytes of data.

The entire Internet is thought to hold over 1200 PB (as of 2020). By relying on this database alone, investigators could miss 99% of the available data, meaning they would almost certainly miss valuable insights.