I-Corps: Using neural radiance fields (NeRF) and photogrammetry algorithms for creating 3D models

I-Corps：使用神经辐射场 (NeRF) 和摄影测量算法创建 3D 模型

• 批准号: 2412147
• 负责人: George Stiny
• 金额: $ 5万
• 依托单位: Massachusetts Institute of Technology
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2024
• 资助国家: 美国
• 起止时间: 2024-02-01 至 2025-01-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2412147&HistoricalAwards=false
• 关键词: Corps; Using; neural; radiance; fields

The broader impact/commercial potential of this I-Corps project is the development of a three-dimensional (3D) scanning technology to visualize a space immersively using a smartphone. Currently, millions of people around the globe operate as content creators and viewers, sellers, and buyers. The proposed technology provides the ability to view large intricate objects and spaces. The expected first entry point for this application is e-commerce in real estate. The goal is to streamline the rental process for both landlords and tenants by leveraging artificial intelligence (AI)-driven matching, virtual AR tours, and digitizing the leasing experience. Additional applications are anticipated.This I-Corps project is based on the development of neural radiance fields (NeRF) and photogrammetry algorithms to create three-dimensional (3D) models with more detail and information than is available with images and videos alone. Currently, immersive technologies are cumbersome and require expertise and special equipment to build a remote experience. In addition, 3D reconstruction algorithms based on purely photogrammetry processing methods require large amounts of data (e.g., a room needs 1,000 images, while a medium scale historic building requires 100,000 images). In contrast, the proposed approach through NeRF enables 3D reconstruction with smaller data sets due to the ability of NeRF to use inference (e.g., a room needs 100 images and dramatically less time to 3D scan). From a user experience point of view, whereas images and videos present predetermined content, 3D models may allow users to simply explore by “walking through” complex spaces.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该I-Corps项目的更广泛的影响/商业潜力是开发三维（3D）扫描技术，以使用智能手机将空间可视化。目前，全球成千上万的人是内容创建者和观众，卖方和买家。提出的技术提供了查看大型复杂物体和空间的能力。该应用程序的预期第一个入口点是房地产中的电子商务。目的是通过利用人工智能（AI）驱动的匹配，虚拟AR旅行以及数字化租赁体验来简化房东和租户的租赁过程。预计会有其他申请。该I-Corps项目基于神经元辐射场（NERF）和摄影算法的开发，以创建与单独图像和视频相比，具有更多详细信息和信息的三维（3D）模型。当前，沉浸式技术很麻烦，需要专业知识和特殊设备来建立远程体验。此外，基于纯摄影处理方法的3D重建算法需要大量数据（例如，房间需要1,000张图像，而中等规模的历史建筑需要100,000张图像）。相比之下，由于NERF使用推断的能力，通过NERF提出的方法启用了具有较小数据集的3D重建（例如，房间需要100张图像，并且在3D扫描的时间上大幅减少了3D扫描的时间）。从用户体验的角度来看，尽管图像和视频呈现了预测的内容，但3D模型可以简单地通过“浏览”复杂空间来探索。该奖项反映了NSF的法定任务，并被认为是通过基金会的智力优点和更广泛影响的评估评估来获得的支持。