Cybersecurity Risk Initiative

2022 Cybersecurity Data Science Student Summer Program

During the summer of 2022, in collaboration with the Northeast Big Data Innovation Hub (NEBDHub), Rochester Institute of Technology (RIT) invited students to curate cross-organizational cyber intrusion data and use the data to produce benchmarking information about ongoing cyberattacks. This information can be used to enhance current risk and threat assessments, which are typically done through hypothesized threat reports (not necessarily relevant) or via penetration testing (costly). This effort was designed to benefit the student cohort to learn and produce relevant cybersecurity data science results with real-world data.

For this project, Dr. Yang recruited and mentored five undergraduate students and one graduate student to explore data science approaches for cross-organizational cyber threat and risk assessment.

Student Participants:

• Chanel Cheng (Undergraduate, Computer Science, RIT)
• Serena Yang (Undergraduate, Information Science, Cornell University; remote participant)
• Wei-Ting Chen (Undergraduate, National Chi-Nan University, Taiwan; remote participant)
• Vazgen Tadevosyan (MS, Data Science, RIT)
• Matthew Heller (Undergraduate, Computer Engineering, RIT)
• Pradumna Gautam (Undergraduate, Anna University, Chennai, India; visiting student to RIT)

Specifically, two sets of efforts were put forth with the award:

1. Cross-organization attack pattern analysis for data collected through STINGAR, supported by Serena Yang, Wei-Ting Chen, and Vazgen Tadevosyan.

Common honey-nets are used to assess potential risks and threats to the networked systems being modeled after. STINGAR is a community honey-net project run by Duke University’s cybersecurity team. PI Jay Yang has engaged the STINGAR team to obtain honey-net data across many higher-ed institutions that use STINGAR. Two students, Serena Yang and Wei-Ting Chen, were recruited in the summer of 2022 to explore and assess statistical and data-driven means to identify threat patterns across victim organizations and attacker origins. In September of 2022, they presented their findings to the STINGAR team. Serena Yang further presented their findings to NSF Big Data Hubs’ Data Sharing and  Cyberinfrastructure Working Group in October 2022, and in the 2023 NEBDHub Inaugural Student Research Symposium. Vazgen Tadevosyan was later recruited to continue working on additional STINGAR data with clustering analysis of engineered attack features, and presented to the STINGAR team in the March of 2023. Serena Yang and Vazgen Tadevosyan are now working on preparing a paper based on their findings.

Key Findings and Impacts:

• Statistical analysis reveals attack frequencies originated from different countries (as indicated by IP addresses) to the community honey-nets. This analysis is certainly limited by the data being collected. STINGAR team has improved its system since summer of 2022 and further looked into generating more contextually rich attack features and data science techniques.
• Data reveals a large variety of attack attributes across victim organizations and attacking origins. Intelligent engineering of features that can be used for data science and machine learning techniques is critical. We have since derived several features that transform the large variety of attack attribution into more behavioral summary and shown their effectiveness for clustering analysis. The details of the results will be forthcoming in a paper being prepared.
• Regular exchanges between community-based honey-net teams and data scientists can be helpful for both sides, where the system designed to collect data and the threat analysis using such data will be more meaningful.

2. Continual learning across organizational cyberattack data for efficient threat recognition, supported by Chanel Cheng, Matthew Heller, and Pradumna Gautam

Chanel Cheng and Matthew Heller were recruited in the summer of 2023 to explore the use of continual learning to differentiate cyberattacks by analyzing network traffic across organizations. Typical machine learning approaches that differentiate cyberattacks focus on traffic one network at a time. This effort develops network agnostic features and an efficient continual learning approach to process data across datasets generated by different data providers (which uses different networks). Matthew Heller assisted in identifying the open-domain network flow datasets that share similar features and normalized the features. Chanel Cheng investigated and implemented a continual learning approach to demonstrate the effectiveness of our approach when treating DoS attack data across datasets. Chanel Cheng continued after summer and further investigated the limitations and enhancements of the continual learning approach when treating a broader variety of cyberattack behaviors. Meanwhile, Pradumna Gautam performed a preliminary literature review into the use of Natural Language Processing to interpret various cyberattack tactics.

Additional Findings:

• Continual learning with memory replies that replace old data points with uncertain predictions is effective for learning new attack behaviors even for traffic across organizations.
• The above findings have been shown to be effective for a variety of DoS attacks [Cheng ’22], and further investigation to broaden the attack types is ongoing and expected to be disseminated in 2023.
• There are significant limitations on open-domain network traffic datasets used for cyber intrusion and threat analysis. PI Jay Yang is in the process of sharing the findings and requesting quality datasets from the community to conduct responsible research using these datasets.
• During the process of exploring cyber risk and threat assessment, PI Yang’s group also sees the need of advancing NLP techniques. Follow-up research efforts in this area are ongoing in the community and by PI Jay Yang’s research group.

Summary of Outputs/Dissemination:
Papers:

• Chanel Cheng and S. Jay Yang, “Cross-Organizational Continual Learning of Cyber Threat Models,” Poster Paper, ACSAC 2022, December 7-9, 2022, Austin TX, USA.
• Reza Fayyazi, Steve Wufeng, Pradumna Gautam, and S. Jay Yang, “Translating Cybersecurity Descriptions into Interpretable MITRE Tactics using Transfer Learning,” Poster Paper, ACSAC 2022, December 7-9, 2022, Austin TX, USA.
• Pradumna assisted in curating data and conducting this analysis. He is not the main author of this poster paper.
• Vazgen Tadevosyan and Serena Yang, “Cross-organization attack pattern analysis through clustering of engineered features,” in preparation.
• Chanel Cheng and S. Jay Yang, “Continual Learning of Large Variety of Cyberattacks across Networks,” in preparation.

Presentations:

• Chanel Cheng and Serena Yang, “Data-driven Pattern Analysis and Continual Learning of Cyber Attacks across Organizations,” webinar presentation to Data Sharing and Cyberinfrastructure Working Group, NSF Big Data Hubs Data Sharing and Cyberinfrastructure Working Group, October 7, 2022. 
• Chanel Cheng, “Cross-Organizational Continual Learning of Cyber Threat Models,” presentation in NEBDHub Inaugural Student Research Symposium, January 27, 2023. Video linked below.
• Serena Yang and Wei-Ting Chen, “A Data Driven Approach to Cyber Risk and Threat Assessment,” presentation to Duke University STINGAR team, September 23, 2022. 
• Serena Yang, “A Data-Driven Approach for Cross-organization Cyber Risk and Threat Assessment,” presentation in NEBDHub Inaugural Student Research Symposium, January 27, 2023. Video linked below.
• Vazgen Tadevosyan, “Clustering of Malicious Attacks,” presentation to Duke University STINGAR team, March 3, 2023.