What Is Open Source Intelligence

Key Takeaways

Open source intelligence is derived from data and information that is available to the general public. It’s not limited to what can be found using Google, although the so-called “surface web” is an important component.

As valuable as open source intelligence can be, information overload is a real concern. Most of the tools and techniques used to conduct open source intelligence initiatives are designed to help security professionals (or threat actors) focus their efforts on specific areas of interest.

There is a dark side to open source intelligence: anything that can be found by security professionals can also be found (and used) by threat actors.

Having a clear strategy and framework in place for open source intelligence gathering is essential — simply looking for anything that could be interesting or useful will inevitably lead to burnout.

What Is Open Source Intelligence?

Before we look at common sources and applications of open source intelligence, it’s important to understand what it actually is.

According to U.S. public law, open source intelligence:

Is produced from publicly available information

Is collected, analyzed, and disseminated in a timely manner to an appropriate audience

Addresses a specific intelligence requirement

The important phrase to focus on here is “publicly available.”The term “open source” refers specifically to information that is available for public consumption. If any specialist skills, tools, or techniques are required to access a piece of information, it can’t reasonably be considered open source.

Crucially, open source information is not limited to what you can find using the major search engines. Web pages and other resources that can be found using Google certainly constitute massive sources of open source information, but they are far from the only sources.

For starters, a huge proportion of the internet (over 99 percent, according to former Google CEO Eric Schmidt) cannot be found using the major search engines. This so-called “deep web” is a mass of websites, databases, files, and more that (for a variety of reasons, including the presence of login pages or paywalls) cannot be indexed by Google, Bing, Yahoo, or any other search engine you care to think of. Despite this, much of the content of the deep web can be considered open source because it’s readily available to the public.

In addition, there’s plenty of freely accessible information online that can be found using online tools other than traditional search engines. We’ll look at this more later on, but as a simple example, tools like Shodan and Censys can be used to find IP addresses, networks, open ports, webcams, printers, and pretty much anything else that’s connected to the internet.