Research studies that analyze information regarding various communities on the deep web and dark web have unfolded novel angles in the domain of security informatics. The existence of dark web communities that operate with high levels of anonymity opens the door for data driven approaches that target different means of adversarial reasoning. Such approaches would require certain data that is more or less classified when adopted outside of the dark web.
Hsinchun Chen and colleagues conducted a pioneering research study on dark web mining in 2016, which established the foundations for the relationship between cyber threat intelligence and dark web data. A recently published paper builds on the results of Hsinchun Chen and his colleagues’ work to analyze how dark web data could be geared towards influencing cyber threat intelligence in a myriad of ways, which include analysis of hacker communities, describing adversarial models, risk assessment, and prediction of data driven cyber attacks. Authors of this paper were astonished by the response to their work in this field, which coincided with innovative scientific research studies, government grants, and commercial efforts served to study this area.
Why is it hard for cyber threat intelligence analysts to obtain information from the dark web?
There is an evolution going on in the field of utilizing the dark web to improve cyber threat intelligence strategies. All data should be obtained, arranged perfectly, analyzed, and potentially utilized in various prediction objectives, which all represent enormous challenges to deal with.
First of all, obtaining information from various forms of dark web communities, such as forums, marketplaces, etc, represents a unique set of significant challenges. Deploying web crawlers to obtain such information is an extremely complex task. Moreover, the inherent adversarial characteristics of dark web communities represents a challenging hurdle to researchers. Also, the techniques used by researchers might cease to be viable as they are exposed to malicious adversaries. These discussions have taken place at specialized conferences including IEEE Intelligence & Security Informatics and ASONAM. For instance, Richard Frank’s research on dark web mining, which was awarded the best paper at ASONAM in 2015, delves into the great challenges he had to face while carrying out his research. In the issue, published in the conference’s proceeding, titled “A Framework for More Effective Dark Web Marketplace Investigations,” a thorough analysis of the techniques of scraping the dark web’s hidden services is provided, offering a very comprehensive case study that researchers could previously only obtain via offline conversations.
What are the most recent cyber threat intelligence strategies?
Even though collection of information is pivotal, data by itself cannot mitigate cyber security challenges in a real world setting. Currently available threat intelligence entities at major corporations all over the world analyze this data regularly. They run searches relevant to their corporations, map out various threat actors, and organize information obtained from multiple sources. Nowadays, cyber security operations are utilizing law enforcement and anti-terrorism techniques. Techniques such as link analysis are currently commonly utilized by cyber threat intelligence entities. Research that utilizes novel data mining techniques on data gathered from the dark web will enable threat intelligence analysts to formulate an accurate picture of potential threats more swiftly. A pivotal challenge is the potential reconciliation of malicious adversaries across multiple platforms.
Currently, utilization of data obtained from the dark web to support cybersecurity operations in a real world setting has been centered on augmentation of intelligence practices. Nevertheless, with the recent advancements in the technologies of security information and event management (SIEM), recent research studies have proven that dark web indicators can be combined with event data to make predicting cyber attacks easier. Recent research studies have added sentiment mining as a valuable prediction element, even though it was initially presented as means for identification of relevant hacker conversations.
Utilizing information obtained from hacker communities over the dark web has proven to yield a more threat focused cybersecurity strategy. The cornerstone of a meaningful approach to further progress in this field is continuous automation and evolution so that effective cyber threat intelligence strategies can be accessible to a wide array of entities and corporations in order to formulate better security decisions and protect their infrastructures via more effective plans.