Collaborative Research: LightningBug, An Integrated Pipeline to Overcome The Biodiversity Digitization Gap

合作研究：LightningBug，克服生物多样性数字化差距的综合管道

• 批准号: 2104152
• 负责人: Robert Guralnick
• 金额: $ 8.42万
• 依托单位: University of Florida
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-08-01 至 2024-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2104152&HistoricalAwards=false
• 关键词: Collaborative; Research; LightningBug; Integrated; Pipeline; Collaborative; Research; LightningBug; Integrated; Pipeline

Insects are the largest and most diverse class of animals on our planet where they play essential roles in ecosystems and the services those provide to society. Entomologists have long been engaged in collecting, preserving and depositing nearly one billion insect specimens at natural history museums around the globe. These collections form the basis for much of our knowledge about insects and provide critical information about the past from which scientists can assess current and future global change impacts. To fully realize the value of these collections, data from insect specimens must first be digitized. However, their small size, delicate structures, and traditional storage and labeling methods creates enormous challenges for large-scale digitization. Consequently, at present, only 5% of specimens have transcribed labels and less than 1% of specimens are imaged. The LightningBug project will break through this digitization bottleneck by establishing a semi-automated workflow involving advancements in robotic multi-view imaging, information extraction and 3D reconstruction. Results from this work will provide researchers with the unprecedented capability to capture specimen metadata representing time, place and taxonomic identity along with accurate three-dimensional surface morphology representing color and shape. We expect LightningBug and related technologies will promote ecomorphological studies at a scale that has not been possible to date.The LightningBug project seeks to create an end-to-end pipeline for high-throughput data acquisition from pinned insects in entomological collections. To accomplish this goal, we will: (1) further develop an existing hardware and software platform to capture multi-view imagery of both labels and specimens; (2) build robust algorithms to automatically process fragmentary views of multiple labels into separate integrated “virtual labels;" (3) connect virtual labels to structured text extraction services; and (4) apply photogrammetric analysis to assemble the 3D shape and structure of specimens. Guided by real-world science use cases that highlight the use of specimen-based multi-view imaging in studies of global change and functional morphology, the entomological collections of the Yale Peabody Museum and the Harvard Museum of Comparative Zoology will be used in rigorous test-case implementations. Results will include robust sets of annotated multi-view images, 3D models of specimens (point clouds, textured meshes), 2D reconstructed “virtual labels” and digitized specimen metadata generated from those labels. These digital specimens will present new challenges for data preservation and access, but they will also catalyze new solutions for large-scale storage and delivery of research imagery. This challenge will be addressed via a partnership with MorphoSource to develop a linked institutional repository model for data access to large digital assets such as those produced by multi-view imaging. Ultimately, the ability to capture multi-view image suites and generate virtual specimens at scale will permit new avenues for remote access to research resources, and enable the application of computer vision and machine learning to trait identification and evolution, species recognition and new species discovery. Label data from pinned insects will give researchers access to critical temporal and geospatial information necessary for relating changes in biodiversity to other biotic and environmental variables. It will also provide collections staff with a complete digital portrait of their holdings, which can enable historical research, streamline collections use and tracking, and improve data quality control. Results from this project will also have applications beyond the natural history collections and research communities, such as computer graphics, product imaging, motion pictures, 3D animation, virtual and augmented realities, and education. More information and results from this project can be found at http://lightningbug.techThis award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

昆虫是我们地球上最大，最多样化的动物，它们在生态系统以及为社会提供的服务中起着重要作用。昆虫学家长期以来一直从事在全球自然历史博物馆的收集，保存和存放近十亿个绝缘标本。这些收藏构成了我们关于绝缘材料的大部分知识的基础，并提供了有关过去和未来全球变化影响的过去的关键信息。为了充分实现这些收藏的价值，必须首先将来自昆虫标本的数据数字化。但是，它们的尺寸小，精致的结构以及传统的存储和标签方法为大规模数字化带来了挑战。因此，目前，只有5％的标本已转录标签，不到1％的标本成像。 LightningBug项目将通过建立一个半自动化的工作流程来突破这种数字化瓶颈，涉及机器人多视图成像，信息提取和3D重建方面的进步。这项工作的结果将为研究人员提供前所未有的能力，可捕获代表时间，地点和分类学身份的标本元数据，以及代表颜色和形状的准确的三维表面形态。我们预计，闪电症和相关技术将以迄今为止不可能促进生态形态学研究。闪电般的项目旨在从昆虫学收集中固定的隔热材料中创建一条端到端的端到端管道。为了实现这一目标，我们将：（1）进一步开发现有的硬件和软件平台，以捕获标签和标本的多视图图像； （2）构建强大的算法，以自动将多个标签的碎片视图处理为单独的集成“虚拟标签”； （3）将虚拟标签连接到结构化的文本提取服务； （4）应用摄影分析来组装样品的3D形状和结构。在现实世界的科学用例的指导下，强调了基于标本的多视图成像在全球变化和功能形态的研究中，耶鲁·皮博迪博物馆的昆虫学收集和哈佛对比较动物学博物馆的昆虫学收集将用于严格的测试案例实施。结果将包括可靠的带注释的多视图图像，样品的3D模型（点云，纹理网格），2D重建的“虚拟标签”，并从这些标签中生成的数字化标本元数据。这些数字标本将为数据准备和访问带来新的挑战，但它们还将催化新的解决方案，以进行大规模存储和提供研究图像。该挑战将通过与Morphosourece的合作伙伴关系来解决，以开发链接的机构存储库模型，以访问数据访问大型数字资产，例如由多视图成像产生的资产。最终，捕获多视图图像套件并大规模生成虚拟规范的能力将允许远程访问研究资源的新途径，并能够将计算机视觉和机器学习应用于特质识别和进化，物种识别和新物种发现。固定昆虫的标签数据将使研究人员获得将生物多样性变化与其他生物和环境变量联系起来所需的关键临时和地理空间信息。它还将为收藏人员提供其持股的完整数字肖像，这可以使历史研究，简化收集和跟踪以及改善数据质量控制。该项目的结果还将在自然历史集合和研究社区之外提供应用程序，例如计算机图形，产品成像，电影，3D动画，虚拟和增强现实以及教育。该项目的更多信息和结果可以在http：//lightningbug.techis Award上找到反映NSF的法定任务，并使用基金会的知识分子优点和更广泛的影响审查标准，认为通过评估被认为是珍贵的支持。