SaTC: CORE: Small: Collaborative: Understanding and Mitigating Adversarial Manipulation of Content Curation Algorithms

SaTC：核心：小型：协作：理解和减轻内容管理算法的对抗性操纵

• 批准号: 1814816
• 负责人: Damon McCoy
• 金额: $ 24.16万
• 依托单位: New York University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-07-15 至 2022-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1814816&HistoricalAwards=false
• 关键词: SaTC; CORE; Small; Collaborative; Understanding

Online social networks (OSNs) have fundamentally transformed how billions of people use the Internet. These users are increasingly discovering books, music bands, TV shows, movies, news articles, products, and other content through posts from trusted users that they follow. All major OSNs have deployed content curation algorithms that are designed to increase interaction and act as the "gatekeepers" of what users see. While this curation and filtering is useful and necessary given the amount of content available, it has also exposed people and platforms to manipulation attacks whereby bad actors attempt to promote content people would otherwise prefer not to see. This has driven the creation of an underground ecosystem that provides services and techniques tailored towards subverting OSNs' content curation algorithms for economic and ideological gains. This project will conduct open research to improve our understanding of current algorithmic curation attackers. The team will devise content curation algorithms and defenses which are hardened against manipulation and that can be adopted by these OSN platforms, providing a systematic approach to improving design and practice in an area of critical national importance. Technology transfer from this project will protect the integrity of social media discourse from adversarial manipulation. This project will train students with expertise in security and machine learning, areas of broad national need, and produce educational materials to engage both high school students and the public in these critical questions. The team will holistically explore the economic, social, and technical perspectives of machine learning-based content curation algorithms' weaknesses. The research comprises three main activities: 1) understand how OSNs are currently being successfully manipulated at large scales, 2) investigate the defenses OSNs have in place, and 3) design more resilient defenses. The team will build the first-ever taxonomy of manipulation services and techniques that are actively used to manipulate curation algorithms. Another thrust of the project is to create a framework for the external evaluation of deployed manipulation defenses based on the collection of both public data from the OSN's platform and external data to compare it against. The team will then develop robust and scalable algorithms to detect OSN manipulation within the collected data. Finally, the team will use the insights from the taxonomy of effective manipulation techniques and the exploration of the limitation of current defenses to design fundamentally resilient content curation algorithms. The project will explore both new curation algorithms and more effective mitigation techniques for existing algorithms. The project's findings will deepen our understanding of social network manipulation and adversarial learning and produce reliable approaches to algorithmic content curation.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在线社交网络 (OSN) 从根本上改变了数十亿人使用互联网的方式。这些用户越来越多地通过他们关注的可信用户的帖子发现书籍、乐队、电视节目、电影、新闻文章、产品和其他内容。所有主要 OSN 都部署了内容管理算法，旨在增加交互并充当用户所见内容的“看门人”。虽然考虑到可用内容的数量，这种管理和过滤是有用且必要的，但它也使人们和平台面临操纵攻击，不良行为者试图推广人们不希望看到的内容。这推动了地下生态系统的创建，该生态系统提供量身定制的服务和技术，旨在颠覆 OSN 的内容管理算法，以获取经济和意识形态收益。该项目将进行开放研究，以提高我们对当前算法管理攻击者的理解。该团队将设计内容管理算法和防御措施，这些算法和防御措施经过强化以防止操纵，并且可以被这些 OSN 平台采用，从而提供一种系统方法来改进对国家至关重要的领域的设计和实践。该项目的技术转让将保护社交媒体话语的完整性免受对抗性操纵。该项目将培训具有安全和机器学习（国家广泛需求领域）专业知识的学生，并制作教育材料，让高中生和公众参与这些关键问题。该团队将从经济、社会和技术角度全面探讨基于机器学习的内容管理算法的弱点。该研究包括三项主要活动：1）了解目前如何成功地大规模操纵 OSN；2）研究 OSN 现有的防御措施；3）设计更具弹性的防御措施。该团队将建立第一个操纵服务和技术的分类法，这些服务和技术被积极用于操纵管理算法。该项目的另一个目标是创建一个框架，用于基于从 OSN 平台收集的公共数据和外部数据进行比较，对已部署的操纵防御进行外部评估。然后，该团队将开发强大且可扩展的算法来检测收集的数据中的 OSN 操纵。最后，该团队将利用有效操纵技术分类的见解以及对当前防御局限性的探索来设计根本上有弹性的内容管理算法。该项目将探索新的管理算法和针对现有算法的更有效的缓解技术。该项目的研究结果将加深我们对社交网络操纵和对抗性学习的理解，并产生可靠的算法内容管理方法。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Conspiracy Brokers: Understanding the Monetization of YouTube Conspiracy Theories

• DOI: 10.1145/3485447.3512142
• 发表时间: 2022-01-01
• 期刊: PROCEEDINGS OF THE ACM WEB CONFERENCE 2022 (WWW'22)
• 作者: Ballard, Cameron;Goldstein, Ian;McCoy, Damon
• 通讯作者: McCoy, Damon

An Audit of Facebook’s Political Ad Policy Enforcement

对 Facebook 政治广告政策执行情况的审计

• 发表时间: 2022
• 期刊: USENIX Security Symposium
• 作者: Le Pochat, Victor;Edelson, Laura;Van Goethem, Tom;Joosen, Wouter;McCoy, Damon;Lauinger, Tobias
• 通讯作者: Lauinger, Tobias

A Security Analysis of the Facebook Ad Library

Facebook 广告库的安全分析

• DOI: 10.1109/sp40000.2020.00084
• 发表时间: 2020
• 期刊: 2020 IEEE Symposium on Security and Privacy (SP
• 作者: Edelson, Laura;Lauinger, Tobias;McCoy, Damon
• 通讯作者: McCoy, Damon

The Pod People: Understanding Manipulation of Social Media Popularity via Reciprocity Abuse

• DOI: 10.1145/3366423.3380256
• 发表时间: 2020-04
• 期刊: Proceedings of The Web Conference 2020
• 作者: Janith Weerasinghe;Bailey Flanigan;Aviel J. Stein;Damon McCoy;R. Greenstadt

Understanding Incentivized Mobile App Installs on Google Play Store

• DOI: 10.1145/3419394.3423662
• 发表时间: 2020-10
• 期刊: Proceedings of the ACM Internet Measurement Conference
• 作者: Shehroze Farooqi;Álvaro Feal;Tobias Lauinger;Damon McCoy;Zubair Shafiq;Narseo Vallina-Rodriguez