I-Corps: Using neural radiance fields (NeRF) and photogrammetry algorithms for creating 3D models

I-Corps：使用神经辐射场 (NeRF) 和摄影测量算法创建 3D 模型

• 批准号: 2412147
• 负责人: George Stiny
• 金额: $ 5万
• 依托单位: Massachusetts Institute of Technology
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2024
• 资助国家: 美国
• 起止时间: 2024-02-01 至 2025-01-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2412147&HistoricalAwards=false
• 关键词: Corps; Using; neural; radiance; fields

The broader impact/commercial potential of this I-Corps project is the development of a three-dimensional (3D) scanning technology to visualize a space immersively using a smartphone. Currently, millions of people around the globe operate as content creators and viewers, sellers, and buyers. The proposed technology provides the ability to view large intricate objects and spaces. The expected first entry point for this application is e-commerce in real estate. The goal is to streamline the rental process for both landlords and tenants by leveraging artificial intelligence (AI)-driven matching, virtual AR tours, and digitizing the leasing experience. Additional applications are anticipated.This I-Corps project is based on the development of neural radiance fields (NeRF) and photogrammetry algorithms to create three-dimensional (3D) models with more detail and information than is available with images and videos alone. Currently, immersive technologies are cumbersome and require expertise and special equipment to build a remote experience. In addition, 3D reconstruction algorithms based on purely photogrammetry processing methods require large amounts of data (e.g., a room needs 1,000 images, while a medium scale historic building requires 100,000 images). In contrast, the proposed approach through NeRF enables 3D reconstruction with smaller data sets due to the ability of NeRF to use inference (e.g., a room needs 100 images and dramatically less time to 3D scan). From a user experience point of view, whereas images and videos present predetermined content, 3D models may allow users to simply explore by “walking through” complex spaces.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该 I-Corps 项目更广泛的影响/商业潜力是开发三维 (3D) 扫描技术，可使用智能手机以身临其境的方式可视化空间。目前，全球有数百万人作为内容创作者、观看者和销售者。所提出的技术提供了查看大型复杂物体和空间的能力，该应用程序的预期切入点是房地产电子商务，其目标是通过利用人工来简化房东和租户的租赁流程。智能（AI）驱动预计还会有更多应用。该 I-Corps 项目基于神经辐射场 (NeRF) 和摄影测量算法的开发，旨在创建更详细、更详细的三维 (3D) 模型。目前，沉浸式技术非常繁琐，需要专业知识和特殊设备来构建远程体验，此外，基于纯粹摄影测量处理方法的 3D 重建算法需要大量数据。 （例如，一个房间需要 1,000 个图像，而一个中等规模的历史建筑需要 100,000 个图像）相比之下，由于 NeRF 能够使用推理（例如，一个房间需要），通过 NeRF 提出的方法可以使用较小的数据集进行 3D 重建。 100 张图像，并且 3D 扫描时间显着减少）从用户体验的角度来看，虽然图像和视频呈现预定内容，但 3D 模型可以让用户简单地进行操作。通过“走过”复杂的空间进行探索。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。