Intelligence gathering: comprehensive video extraction with open source tools

Open source investigations often require in-depth research into specific events, including photos, videos and witness accounts that may be useful for analysis. This is especially true when unexpected events occur, and many users are quick to upload photos and videos, but this is also a key time for marketers to confuse them by posting fake news.

This article is intended to provide advice and some tips on how to collect as much video as possible at a given event from sources including, but not limited to, eyewitnesses of natural disasters, mass events, man-made disasters, and terrorist attacks. Most of the examples presented below are based on the assumption that the event is a large protest or mass event, but the same applies to other events.

Video extraction via major live streaming platforms

The first video to show an immediate event is a live stream. For large group events, there will be a large number of participants or members of the media streaming the event as it happens.

1.Facebook Live

URL: https://www.facebook.com/live

As one of the most popular platforms, Facebook Live is rapidly gaining popularity due to its large user base and ease of use. Finding live videos is as simple as opening the URL and clicking on the blue dot in the location of personal interest. However, these videos are only displayed on the map when streaming.

To find previously played videos from Facebook Live, you can visit the person or home page of someone you know is playing the stream and search for their video. For example, past streamed Facebook Live videos on the popular American protest stream Rebelutionary_Z can be found at facebook.com/RebZtv/videos.

The second way to find most results is to search extensively for live videos through the Facebook search feature, i.e., by clicking on the filter for " Live" videos only, but this way is not very efficient.

2. YouTube Live

Although YouTube Live is a bigger and more professional news media platform, it is not as popular as Fabebook Live.

Like Facebook, it can filter search results, but only shows "live" videos.

3. Periscope

One of the most popular live streaming platforms is Periscope, which was acquired by Twitter in 2015, but it is much easier to search for videos using Google or Twitter than using the interface on Periscope's website.

4.Livestream.com

URL: http://livestream.com/

Livestream.com is one of the oldest video sites and has been one of the live streaming options for many groups and individuals since the social media platform offered live streaming options, despite its declining popularity.

Thankfully, its search function is more robust than Periscope and some other platforms, but it still has flaws.

Livestream.com has difficulty searching for videos posted on specific dates, and the search filters are more akin to YouTube's "past 24 hours," "past week" and other filters than to a specific date range. Nevertheless, there are far fewer videos on this platform than on

YouTube and Periscope, making it less of a hassle to sort through useful and irrelevant results.

As with other platforms, previous broadcasts are archived for future viewing. As shown below, Unicorn Riot, the popular Unicorn Riot channel, which regularly broadcasts live protests, has nearly 300 previously broadcast videos archived for viewing.

Video extraction via date specific filters on various platforms

1. Twitter

The case study where we used this tool was the Mexican earthquake. Using Twitter's advanced search interface, you can find a large number of live messages delivered by tweets from ordinary people, and you can easily find these videos by searching for operators at twitter.com/search-advanced and selecting the "Videos" tab to see only videos uploaded to Twitter.

The purpose of tracking these videos is to know what keywords to look for when searching next. When looking for videos related to an event, don't just search for the obvious subject tags and keywords that will bring up most results. For example, the August 12 protests with the hashtags #UnitetheRight and Charlottesville; also use specific phrases, words and subject tags that locals are likely to use to find results from accounts that may not have tried to broadcast to a large audience.

Imagine what we would write to describe the event if we were locals observing it - perhaps using the name of the city, or more likely, if only intending to broadcast to a smaller audience, the name of the place most familiar to locals.

Another obvious strategy is to use the local language rather than just English. In the case of the Mexico City earthquake, you'll find more successful results searching for earthquakes in Spanish, "sismo" or a common Mexico City term such as CDMX (short for Ciudad deMéxico). Otherwise, you will only get results for English-speaking tourists and journalists who post.

2. YouTube and Google Video

On YouTube it is possible to use filters to find only videos posted in a specific time span. However, these filters are limited to not allowing us to search for a specific date or date range, only the relative time span relevant to when you performed the search. If you are searching for videos of an event within a week of the event, then click on "this week" and you will be fine.

However, if we are searching for videos after a longer period of time has elapsed since the event, it may be more useful to search for YouTube videos in Google, which offers far more powerful search options than the YouTube user interface.

In the search below, we asked Google to return results for each YouTube video posted on August 12, 2017 (date parameter under "Tools") for the "Charlottesville" keyword.

By using the same search parameters in video.google.com, minus the YouTube format, we will primarily find videos posted to news organizations that may not be able to share their content on YouTube.

No single platform can collect all the content from professional news organizations in one place. So our best choice is a combination of YouTube search (for news organizations that share their content on official YouTube channels) and Google Video Search, which will detect video content on each site.

3. Facebook

Searching for videos shared on a specific date on Facebook is much more difficult than using YouTube or Twitter, because there is no search operator on Facebook that can specify a date range, only the year.

Similar to YouTube, we still need to outsource the operation to Google due to the limitations of the Facebook user interface. A video search on video.google.com for the keyword "Charlottesville" for August 12, 2017 will yield many useful results, similar to our date-specific search for YouTube videos.

4. Vkontakte and Odnoklassniki

Depending on the event of interest, many regional social networks are also worth using. Two of the most popular regional social networks in the former Soviet Union countries are Vkontakte (VK) and Odnoklassniki (OK), and video search is easy using the site interfaces of both platforms.

On VK, the video search is limited to sorting by upload date, but on other platforms there are far fewer videos available, which can greatly reduce the search time. After entering keywords, if you cannot change the language setting, please select "Video" (or "Видео" in Russian) to limit the search results.

Odnoklassniki has roughly the same options as Facebook for video search, as it includes the option to include rarely filmed live videos from Odnoklassniki. As with VK, search on the site (you may need to create an account) and then select "Videos" to filter the results. Unless the event of interest occurs in a location with multiple VK and OK users, searching for new video clips on these platforms may be half the battle, as a large number of clips on these sites are shared from other sources.

Video extraction via Snapchat and Instagram content

Video capture via Snapchat and Instagram content

1. Instagram

Searching for video clips on the Instagram website is nearly useless unless one can get lucky or search by geotagging to a specific location. Searching only for keywords or hashtags on a topic makes it impossible to sort video results only, or to search within a specific date range or video length criteria.

We can sort the latest videos using specific keywords or hashtags, but for popular words such as city names, this certainly increases the burden of needing to exclude. Searching by geotags can be useful, but not all videos have tags, or even use tags other than the ones we are searching for. For example, a hashtag for Charlottesville, Virginia will give you some results, while a hashtag for a particular park or street name will give you different results.

In contrast, Google is usually better to use. By using search operators, we can find video results for specific keywords and date ranges, but the results are very limited.

There is no doubt that more than five videos with the word "Charlottesville" were posted on Instagram on the day of the protest, but these were the only results Google gave. Unfortunately, searching for videos on Instagram is a daunting task, and we're more likely to find Instagram videos on other platforms like Twitter than on the Instagram platform itself.

2. Snapchat

It is currently not possible to search for Snapchat videos on laptops or desktops unless you use the Android emulator, and you can only use the Snapchat app on a tablet or phone. In the Snapchat app, Snapchat stories can be found by simply searching for a keyword or location. However, as with Instagram and Facebook, this content may be restricted by user privacy settings.

Conclusion

For journalists and researchers, the ability to find and verify videos is critical. While the most explosive and interesting videos often take the spotlight due to retweets and shares on a variety of social media platforms, we can still uncover the truth by finding and verifying overlooked details in related videos on more little-known platforms.