How real-time social data is transforming national security

As online platforms grow, critical information can easily go unnoticed if security teams and intelligence agencies don't look beyond standard sources. Real-time data, especially from edge and international sources, can play an important role in a variety of use cases. In the face of national security threats, organizations need to be prepared and make informed decisions quickly to protect assets and save lives.

Here I recommend the white paper "How Real-time Social Data is Transforming National Security" by Flashpoint, an open source intelligence company, and make a brief summary.

Open source intelligence (OSINT) has become valuable in driving decision making. A comprehensive all-source intelligence toolkit, including network and online discussion monitoring software, can save organizations millions of dollars, preserve national security, and maintain public trust. As online platforms evolve, critical information can easily go unnoticed if security teams and intelligence agencies aren't looking beyond standard sources for information. Real-time data, especially from edge and international sources, can play an important role in a variety of use cases:

1. Counter-terrorism and extremism

Jihadist groups such as ISIS and Al Qaeda are no longer the only culprits of the terrorist and extremist threat. Domestic extremist movements based on conspiracy theories, right-wing ideologies, and discriminatory world views now also pose serious threats to national security. Public cyberspace is also exploited by both types of extremists, playing a huge role in spreading propaganda, recruiting, financing, and sometimes even plotting. open source intelligence data can help governments understand how extremist groups operate to predict public safety risks and protect citizens and assets from domestic and global terrorism.

Fringe social networks and the deep web have historically been used by violent criminals to post announcements and discuss plans. After these events, like-minded users turned to sites like 8kun, 4Chan, Telegram, and Gab to express their support in groups and forums in these communities.

2. False information monitoring

National security threats have expanded to include online influence campaigns, which can compromise national security and lead to real-world security risks. Disinformation (intentional deception) and misinformation (false information not necessarily spread with malicious intent) are widespread online. Monitoring cyberspace is critical to tracking disinformation campaigns so governments can mitigate their impact and keep the public safer and more informed.

Disinformation can take the following forms:

• Impersonate company or personal social media accounts

• Spread of false information or “fake news” about a brand, person or event

• Create photos or videos that do not represent reality

• Reposting false content on legitimate sources

• Rapid emergence of popular phrases or hashtags

Misinformation can spread quickly through social media, especially in emergencies. But in some cases, not all rumors are false or untrue, sometimes they are facts that have not been confirmed by official sources. This can lead the public to take action before the information has been verified. Due to the viral nature of social media, misinformation can spread quickly and widely, leading to unwise actions or decisions based solely on the lack of official information.

3. Crisis response

When a national crisis strikes, governments must make timely and informed decisions to protect their data, assets, and citizens. Whether it's a natural disaster, a public health crisis, or a terrorist attack, intelligence teams need to know how and where a crisis is occurring and how to allocate resources to respond. Online spaces are often the earliest sources of information to provide this context—for example, social media users often post public updates and images of crisis sites. Matching this data with other feeds can help provide faster, more informed responses.

Real-time social media data combined with information gathered from more traditional sources can help emergency responders gain and maintain situational awareness and assist in decision-making, planning and resource allocation. In post-natural disaster response efforts, government officials are now using social media to share information and connect with citizens at all stages of the crisis.

For example, in a disaster response situation, an organization can use keyword searches and general monitoring to identify community needs. Social media tools can also help rescue groups, agencies and organizations advise the public on available resources during emergencies.

4. Sentiment Analysis

Through constant engagement with the public through social media, organizations can “listen” for specific information or monitor overall situational awareness. Analysts can use social media monitoring and analysis to gauge sentiment or public support within a geographic area.

Research conducted by the U.S. Naval Postgraduate School has shown that social media data combined with traditional polling methods has a positive impact on analysis, especially when negative sentiment is involved.

Some proponents of behavior-based metrics argue that they can assess operational efficiency by analyzing variables such as security, economic indicators, judicial indicators, and governance indicators. To do this, they gather and assess public attitudes, beliefs, climate and opinions through mass media analysis, public profile analysis and opinion polls. The study argues that intelligence analysts should conduct social media analysis and combine the results with polling analysis to support operational assessments.

5. Geopolitical risk assessment

Geopolitical risks can here be considered risks related to tensions between or within countries, which may affect the course of international relations. Geopolitical risk includes both the risk of event realization and new risks associated with the escalation of existing events.

Climate change will lead to continued conflict between governments, citizens and companies, as the public is torn between government commitments to reduce emissions and waste and corporate profits. Climate change will make natural disasters more likely, more frequent, and more severe, and has the potential to render parts of the world uninhabitable, displacing large numbers of people from their homes, and increasing political, social, and economic tensions from unintended influxes of people to other areas unstable.

In politics, as bad actors continue to work to influence elections around the world, widespread disinformation, or "fake news," can play a role in destabilizing entire countries, possibly even altering election outcomes at the national level. Monitoring the online space is critical to tracking disinformation campaigns so governments can mitigate their impact and keep the public safer and more informed.

6. Application programming interface (API)

According to the U.S. National Strategy for Intelligence (2019), the intelligence community faces the challenge of increasing online data collection, processing, analysis, and classification. The Western world also faces a shortage of data analysts and a growing need for military AI. As a result, data scientists in the public sector tend to tackle more complex tasks, developing tools and datasets to support lower-level analysts on intuitive platforms.

Intelligence teams also face the challenge of not being able to access some emerging cyber resources. For example, fringe networks (such as alt-tech platforms, deep and dark web image boards and paste sites, etc.) do not offer their own APIs, or are not available through commercial API providers. To gather data from these sources, analysts often need to create dummy accounts, make group requests, and manually browse the web. This requires significant human intelligence resources that could be allocated to other areas of the intelligence cycle.

Intelligence professionals need specialized software to gather information and generate actionable intelligence. Commercial open source intelligence tools can help intelligence teams collect open source data more efficiently and align with the team's unique needs. Because intelligence teams often use their own interfaces and tools, they often need direct access to raw data that can be plugged into their existing systems.

APIs are becoming an integral part of any organization's digital transformation investments, and intelligence and enterprise security entities are no exception.

APIs help connect data with applications, saving users the resources needed to manually integrate data entry. In the context of gathering threat intelligence, the quality of API data and delivery is a high priority:

• Defense and intelligence teams requiring access to online data sources

• An enterprise security operations center that uses online data sources for security alerts

• Data companies seeking valuable online data inputs to provide to their own customers

As the broader online space becomes relevant to security initiatives—whether in private or public sector environments—addressing the data needs of the coming years will depend heavily on the breadth of resources available through commercial API solutions. Security and intelligence professionals may prioritize expanding data coverage in their tools. This can be achieved by leveraging API vendors who offer a greater variety of standard and alternative threat feeds than is typically provided through commercial solutions. This looks like a combination of standard intelligence feeds with emerging sites.

This has many advantages. First, more data is covered, i.e. less information is overlooked. Access to direct APIs allows analysts to spend less time manually collecting data. Crawling more obscure sources also means that any posts that are later removed from the original site are retained for analysis—a benefit that manual collection cannot provide.

Additionally, combining various inputs makes it easier to cross-reference and pivot between data sources. This is extremely valuable as trivial intelligence is becoming more intricate as the online risk environment diversifies and expands. As a result, analysts can glean insights that might not be apparent or available when standard and alternative data sources are not integrated.

A solution with more data can also better support the development of machine learning. Many online resources, such as content on obscure social networking sites and chat applications, cannot be properly cataloged and stored for data science applications without access through APIs.