Searching The Dark Web For Business Intelligence

For many, Google Search defines the known boundaries of the internet. To some degree that's true, with a majority of the commercial web activity occurring on easily searchable common domains. But there are other levels to the internet.

Beyond the roughly five percent of the sites that are commercially searchable today, there's also what's called the "deep web." This comprises the majority of the remaining internet, a vast region containing unstructured data from sensors and devices, temporary pages, or pages hidden behind password protection. Think of your public library. Or your workplace.

There's yet another layer. Although the dark web comprises a small portion of the overall internet, it is accessible only through specialized browsers such as the Onion Router (TOR) and I2P. The pages here are not searchable although indexes do exist. These are the dark web sites, with no common names only sixteen digit strings of alphanumeric characters, which are traded personally among users, and therefore these sites are increasingly being used for sex, drugs, and other criminal activity such as discussing, trading, and selling computer vulnerabilities.

At the RSA Conference in San Francisco in April, Silobreaker, an intelligence platform, shared their partnership with Flashpoint, a company that delivers business risk intelligence, in a luncheon presentation. The two companies spoke jointly about the need for businesses to navigate the closed communities of the Dark Web, understand how adversaries operate, and separate rumor from reality.

The two companies use open-source intelligence (OSINT) from public sources as opposed to covert or hidden activities. On the searchable internet, OSINT would consist of posts on twitter and other social media for intelligence. OSINT on the Dark Web consists of memberships in various forums.