How to break through the barriers to leveraging open source intelligence?

Barriers to leveraging open source intelligence

Open Source Intelligence (OSINT) can be a powerful intelligence and investigative tool, but is often overlooked. In many organizations, there are significant barriers to the adoption of effective open source intelligence and a failure to adapt quickly enough to emerging technologies. A cultural shift is needed to raise the profile of open source intelligence and ensure it reaches its full potential.

Open source intelligence is a critical component of modern intelligence and investigative tools. The growing volume of data available online provides investigators with a rich source of information. The insights that open source intelligence can provide are unlikely to be found in in-house datasets, select databases, or sanctioned lists.

Failure to leverage open source data can lead to embarrassment and intelligence failures. There are many strong examples of open source intelligence playing an important role in solving cases: Bellingcat's insights into the downed MH17 flight in 2014 relied entirely on open source intelligence.

Open source intelligence should also be considered an essential element of counter-terrorism and counter-misinformation programs. Mapping terrorist networks on social media, especially the more grassroots right-wing extremist groups now appearing on platforms such as Parler, is a highly effective means of identifying the individuals behind these crimes.

Investigators have also had great success in identifying networks that spread misinformation/disinformation and the real identities behind them. Imran Khawaja was sentenced to 12 years in prison for preparing acts of terrorism, attending training camps and possessing firearms. This was all because open source intelligence provided the bulk of the evidence.

It is difficult not to conclude that open source investigations are of growing strategic importance. In addition, they can save money as a quick and economical way to learn about criminals early in an investigation before deploying more expensive and intrusive tactics. So why are so many organizations still failing to leverage open source intelligence?

1. Misconceptions

The lack of investment in open source intelligence is often based on a misunderstanding of what it really means and what it is worth. "Open source intelligence" may conjure up a negative image with connotations of hacking and privacy violations. However, the type of adoption of open source intelligence we advocate can be better described as online open source investigation: the use of freely available online information in a targeted and non-intrusive manner.

2. Cultural and Technical Barriers

Cultural and technological deficits also factor into this attitude toward open source intelligence. Many are wrestling with outdated technical architectures and spending most of their energy on how to better manage their internal data. However, this is driven by the culturally outdated assumption that the greatest insights can always be found in the vast amounts of data that large organizations spend decades accumulating. Open source intelligence will increasingly provide greater insights for individuals and companies than those found internally.

As organizations realize the importance of open source data, they typically use it only in the form of curated datasets that do not capture all of the rich, valuable information available on the Internet. For example, the well-known curated dataset LexisNexis offers 6 petabytes of data.

The entire Internet is thought to hold over 1200 PB (as of 2020). By relying on this database alone, investigators could miss 99% of the available data, meaning they would almost certainly miss valuable insights.

How to break through the barriers to leveraging open source intelligence?

Increasing flexibility and the role of technology

To facilitate increased investment in open source intelligence, systemic, strategic and technical changes are needed.

1. Organizations need to move to a more flexible approach to business and procurement that reflects the reality that many high-quality open source tools can be found in early-stage companies. These companies often find themselves accidentally designed to be outside the complex procurement processes of governments and other large organizations.

2. A new strategic approach to the survey process is needed. Organizations need to recognize the changing environment and make a conscious decision to dedicate a percentage of their technology investment and training budget to equipping investigators with cutting-edge open source tools.

3. Technology that provides a complex combination of capabilities designed to professionalize open source intelligence investigations should be supported and invested in. Technology plays a critical role in reducing the operational difficulties of using open source intelligence by increasing：

a. Security: Collecting online data can reveal the identity of investigators and disrupt operations

b. Speed: Without technology that helps you access relevant information quickly, data can be overwhelming

c. Insight: Finding connections and presenting data from different sources

d. Connectivity to other data: open source intelligence will always be part of a broader strategy that combines various data chains to help investigators get the full picture. The ability to merge data from different sources (structured and unstructured) is critical.

Knowlesys provides the best system, Knowlesys Intelligence System, to assist open source investigators and help organizations better advance their investigations through the use of social media. Knowlesys Intelligence System is developed based the world's leading open source intelligence extraction technology, with the advantages of quick identification and full coverage. It enables users to monitor the whole Internet, including social media(Twitter, Facebook, YouTube, Instagram...), traditional websites (Forum, Chat Room, News...) and dark web in a timely manner. It will help clients mine massive information from social media and turn them into actionable intelligence by automatic analysis and manual processing.

There are many advantages to incorporating open source intelligence into an investigative strategy. Knowlesys Intelligence System helps investigators use open source information quickly, securely and effectively.