The Deep Web and Open Source Intelligence (OSINT): Two Peas in a Pod

At BrightPlanet we often throw around the acronym OSINT and talk about open source intelligence but what is it, what/if anything does it have to do with the Deep Web and how is it being used? We answer those questions in this post.

What is OSINT?

For the purposes of this post, we’ll keep the definition of OSINT at a high level. If you want to dig deeper into OSINT, check out a past post of ours focused on OSINT.

The term OSINT was first, and still is, employed by government agencies to refer to any unclassified, publicly-available information. This definition includes anything freely available on the web including content on the Deep Web.

The Deep Web and OSINT

If you are looking to get your hands on open source intelligence, look no further than the Deep Web. All the data found on the Deep Web is open source intelligence. The data found on the Surface Web is also open source intelligence but as the name suggests, you aren’t getting very much of the data available. Search engines like Google pull data from the Surface Web.

If you want to go beyond the Surface Web information found by Google search, you need to utilize a Deep Web harvest to get more open source intelligence. This includes any webpages found only through a specific site’s search.

We often use the example of grants.gov. A Deep Web harvest can pull any of the pages found when something is inserted in the search box on that page; Google cannot. Think of all the OSINT that is missed by a regular search engine?!

Using OSINT

Organizations often have a handle on the data inside their organization but what about outside of it? The open source intelligence available on the Deep Web is a great place to start mining external data for business insight.

Here are some examples of how industries are currently using open source intelligence from the Deep Web harvested and enriched by BrightPlanet:

Insurance -Insurance companies are looking to open source intelligence to begin augmenting creditworthiness for underwriting. They are tasked with the challenge of identifying risk for their clients and rely on open source intelligence on the Deep Web to make informed decisions about policies and claims.

Mining -Mining companies want to keep their employees safe and understand what is happening in the regions where they have sites. They use open source intelligence on the Deep Web to monitor data on disease outbreaks, local government issues and any other news in the regions relevant to the functioning of their business.

Criminal Intelligence -More and more suspects are “accidentally" admitting to and/or sharing information about crimes through Deep Web sources including social media. Using open source intelligence harvested from the Deep Web, law enforcement is able to have situational awareness and collect information about potential crimes happening in their area.

Online Fraud -Counterfeited and fraudulent goods cost companies billions of dollar each year. By harvesting open source intelligence on the Deep Web, companies selling expensive goods online are tracking down sites that are selling their products illegally.

Finding the Right OSINT For You Are you looking to start collecting open source intelligence or improve the intelligence you are currently gathering? Sign up to talk to one of our Data Acquisition Engineers about what open source intelligence is out there and ready to be harvested for your organization.