Applying Machine Learning Techniques to Understand User Behaviors When Phishing Attacks Occur

Emails have been widely used in our daily life. It is important to understand user behaviors regarding email security situation assessments. However, there are very challenging and limited studies on email user behaviors. To study user security-related behaviors, we design and investigate an email test platform to understand how users behave differently when they read emails, some of which are phishing. Specifically, we conduct two experimental studies, where participants take part in our experiments on site in a lab contained environment and online through Amazon Mechanical Turk that are referred to on-site study and online study, respectively. In the two experimental studies, we design questionnaires for the two studies and use a set of emails including phishing emails from the real world with some necessary modifications for personal information protection. Furthermore, we develop necessary software tools to collect experimental data include participants’ basic background information, time measurement, mouse movement, and their answers to survey questions. Based on the collected data, we investigate what factors, such as intervention, phishing types, and an incentive mechanism, play a key role in user behaviors when phishing attacks occur. The difficulty of such investigation is due to the qualitative analysis of user behaviors and the limited number of data in the on-site study. For these reasons, we develop an approach to quantify user behavior metrics and reduce the number of user attributes by evaluating the significance of each attribute and analyzing the correlation of attributes. Moreover, we propose a machine learning framework, which contains attribute reduction, to find a critical point that classifies the performance of a participant into either ‘good’ or ‘bad’ through 10-fold cross-validation with randomly selected attributes cross-validation models. The proposed machine learning model can be used to predict the performance of a user based on the user profile. Our data analysis shows that intervention and an incentive mechanism play a significant role while phishing type I is more harmful to users compared to the other two types. The findings of this research can be used to help a user identify a phishing attack and prevent the user from being a victim of such an attack. Received on 21 November 2019; accepted on 13 January 2020; published on 29 January 2020

电子邮件在我们的日常生活中已被广泛使用。了解用户在电子邮件安全状况评估方面的行为至关重要。然而，关于电子邮件用户行为的研究极具挑战性且数量有限。为了研究用户与安全相关的行为，我们设计并研究了一个电子邮件测试平台，以了解用户在阅读电子邮件（其中一些是钓鱼邮件）时的不同行为方式。具体而言，我们进行了两项实验研究，参与者分别在实验室受限环境中现场参与我们的实验以及通过亚马逊 Mechanical Turk 在线参与，分别称为现场研究和在线研究。在这两项实验研究中，我们为两项研究设计了调查问卷，并使用了一组电子邮件，包括来自现实世界的钓鱼邮件，并对个人信息进行了一些必要的修改以保护隐私。此外，我们开发了必要的软件工具来收集实验数据，包括参与者的基本背景信息、时间测量、鼠标移动以及他们对调查问卷的回答。基于所收集的数据，我们研究了在钓鱼攻击发生时，诸如干预、钓鱼类型和激励机制等因素在用户行为中起关键作用。这种研究的困难在于对用户行为的定性分析以及现场研究中数据数量有限。出于这些原因，我们开发了一种方法来量化用户行为指标，并通过评估每个属性的重要性和分析属性的相关性来减少用户属性的数量。此外，我们提出了一个包含属性约简的机器学习框架，通过对随机选择的属性进行10折交叉验证模型来找到一个将参与者的表现分类为“好”或“坏”的临界点。所提出的机器学习模型可用于根据用户资料预测用户的表现。我们的数据分析表明，干预和激励机制起着重要作用，而与其他两种类型相比，钓鱼类型I对用户的危害更大。这项研究的结果可用于帮助用户识别钓鱼攻击并防止用户成为此类攻击的受害者。2019年11月21日收到；2020年1月13日接受；2020年1月29日发表