FW-HTF-RL: Collaborative Research: Shared Autonomy for the Dull, Dirty, and Dangerous: Exploring Division of Labor for Humans and Robots to Transform the Recycling Sorting Industry

FW-HTF-RL：协作研究：沉闷、肮脏和危险的共享自治：探索人类和机器人的分工以改变回收分类行业

• 批准号: 1928506
• 负责人: Berk Calli
• 金额: $ 60.43万
• 依托单位: Worcester Polytechnic Institute
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-09-01 至 2024-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1928506&HistoricalAwards=false
• 关键词: FW; HTF; RL; Collaborative; Research

This Future of Work at the Human-Technology Frontier (FW-HTF) project investigates a novel human-robot collaboration architecture to improve efficiency and profitability in the recycling industry, while re-creating recycling jobs to be safer, cleaner, and more meaningful. The specific goal is to improve the waste sorting process, that is, the separation of mixed waste into plastics, paper, metal, glass, and non-recyclables. The US scrap recycling industry -- which represents $117 billion in annual economic activity and more than 530,000 US jobs -- is struggling to meet increasingly challenging standards in domestic and international markets. A major problem for the industry is poor sorting of waste, resulting in materials impurity and a significant decrease in the quality and value of the recycled product. Human perception and judgement are essential to handle the object variety, clutter level and changing characteristics of the waste stream. Yet waste-sorting workers currently face health risks and discomfort arising from sharp and heavy objects, toxic materials, noise, vibration, dust, noisome odors, and poor heating, ventilation, and air conditioning. The innovative robotics component of this project, especially in object detection, manipulation, and human-robot interaction, will allow new sorting facility architectures, creating new, safer roles for human workers. The project complements these technological advances with economic analyses to determine the facility configurations that best remove processing bottlenecks, target materials of high value, and boost the end-to-end efficiency of the recycling process. Division of labor between humans and robots will be investigated to improve job desirability and worker motivation, incorporating consideration of the workers' well-being. In particular, the project will explore ways to utilize robots to amplify worker expertise and value. A holistic and interconnected research approach will be taken for all these aspects, i.e. developing robotics technology, designing the human-machine interfaces, investigating workers' workers' role in the new sorting plant architectures, and understanding and incorporating workers' needs and well-being into the design process.This project will develop the appropriate robotics technology for recycling industry deployment, which will require advancing the state of the art in waste classification and manipulation to handle the conditions associated with recycling facilities. Deep Neural Networks-based object detection and semantic segmentation frameworks will be designed for rich, multi-modal sensor data in order to solve challenges regarding a high-level of clutter, occlusion and object variety. Novel robotic manipulation algorithms based on dynamic and soft manipulation strategies will be utilized to separate and pick classified items from the cluttered waste stream. Robust and dexterous robot hardware will be developed, including the robotic arms and end effectors. Human-machine interfaces will be designed and implemented to achieve these tasks in an intuitive, efficient and practical workflow that optimizes the contributions of both human workers and automated technologies. The robotics technology will also allow expanding the facilities from simply sorting the incoming materials into a whole recycling ecosystem; additional process lines for onsite materials processing units will enable conveying partially-finished products to next stage manufacturers. This expansion will require a novel systems approach, and will help achieve more efficient recycling plants and a much more comprehensive employment ladder for current and new workers. These technological and structural changes in the interactional system of work will shift both the task and relational landscape of the work. The effect of these shifts on worker satisfaction and motivation will be investigated via worker interviews with simulated systems. The new technological landscape will be formed accordingly for improved work experience.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

人类技术边界（FW-HTF）项目的这一工作未来研究了一种新型的人类机器人协作体系结构，以提高回收行业的效率和盈利能力，同时重新创造回收工作以使其更安全，清洁和更有意义。具体目标是改善废物排序过程，即将混合废物分离为塑料，纸张，金属，玻璃和非回收物。美国废品回收行业（代表1,170亿美元的年度经济活动，超过530,000个美国就业机会）正在努力达到国内和国际市场中日益挑战的标准。该行业的一个主要问题是浪费不足，导致材料杂质，并显着降低可回收产品的质量和价值。人类的看法和判断对于处理物体的多样性，混乱水平以及废物流的特征不断变化至关重要。然而，浪费的工人目前面临着锋利和重物，有毒物质，噪音，振动，灰尘，异常气味以及较差的加热，通风和空调，面临健康风险和不适。该项目的创新机器人技术组成部分，尤其是在对象检测，操纵和人类机器人互动中，将允许新的分类设施架构，为人类工人创造新的，更安全的角色。该项目通过经济分析来补充这些技术进步，以确定最能消除加工瓶颈的设施配置，具有高价值的目标材料，并提高回收过程的端到端效率。将调查人类和机器人之间的劳动分工，以提高工作期望和工人动力，并考虑工人的福祉。特别是，该项目将探索利用机器人扩大工人专业知识和价值的方式。所有这些方面都将采取一种整体和相互联系的研究方法，即开发机器人技术，设计人机界面，调查工人在新的分类植物体系结构中的作用，并理解和了解工人的需求，并将工人的需求融入设计过程中，以开发出适当的机器人技术，并将其用于浪费的机器人的范围，以供应范围，以促进浪费范围，以供应范围，以供应范围，以供应范围，以供应范围的范围，以供应范围的范围，以供应范围的范围，以适应浪费的范围，以适应范围的范围。与回收设施相关的条件。基于深度神经网络的对象检测和语义分割框架将设计用于丰富的多模式传感器数据，以解决有关高级混乱，遮挡和对象多样性的挑战。基于动态和软操作策略的新型机器人操纵算法将用于将分类物品与混乱的废物流分开并挑选分类物品。将开发出强大而灵巧的机器人硬件，包括机器人臂和终点效应器。人机接口将被设计和实施，以在直观，高效和实用的工作流程中实现这些任务，从而优化人工和自动化技术的贡献。机器人技术还将允许将设施扩展到简单地将传入材料分类为整个回收生态系统。现场材料加工单元的其他过程线将使下一阶段制造商将部分生产的产品传达。这种扩展将需要一种新型的系统方法，并将有助于实现更有效的回收厂，并为当前和新工人提供更全面的就业阶梯。工作的互动系统中的这些技术和结构变化将改变工作的任务和关系格局。这些转变对工人满意度和动机的影响将通过工人对模拟系统的访谈进行研究。新的技术格局将相应地为改善工作经验而形成。该奖项反映了NSF的法定任务，并被认为是使用基金会的知识分子优点和更广泛影响的审查标准的评估值得支持的。

项目成果

How to Give Imperfect Automated Guidance to Learners: A Case-Study in Workplace Learning

• DOI: 10.1007/978-3-031-11644-5_1
• 发表时间: 2022
• 作者: J. Whitehill;Amitai Erfanian

Compositional clustering: Applications to multi-label object recognition and speaker identification

• DOI: 10.1016/j.patcog.2023.109829
• 发表时间: 2021-09
• 期刊: Pattern Recognit.
• 作者: Zeqian Li;Xinlu He;J. Whitehill

Learning to Work in a Materials Recovery Facility: Can Humans and Machines Learn from Each Other?

学习在材料回收设施中工作：人类和机器可以互相学习吗？

• DOI: 10.1145/3448139.3448183
• 发表时间: 2021
• 期刊: Learning Analytics and Knowledge
• 作者: Kyriacou, Harrison;Ramakrishnan, Anand;Whitehill, Jacob
• 通讯作者: Whitehill, Jacob

ZeroWaste Dataset: Towards Deformable Object Segmentation in Cluttered Scenes

ZeroWaste 数据集：杂乱场景中的可变形对象分割

• DOI: 10.1109/cvpr52688.2022.02047
• 发表时间: 2022
• 期刊: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR
• 作者: Bashkirova, Dina and

VisDA 2022 Challenge: Domain Adaptation for Industrial Waste Sorting

VisDA 2022 挑战赛：工业废物分类的领域适应

• 发表时间: 2022
• 期刊: Proceedings of Machine Learning Research
• 作者: Dina Bashkirova, Samarth Mishra