The Terabase Search Engine

Terabase 搜索引擎

• 批准号: 8882493
• 负责人: Steven L. Salzberg
• 金额: $ 34.61万
• 依托单位: JOHNS HOPKINS UNIVERSITY
• 依托单位国家: 美国
• 财政年份: 2014
• 资助国家: 美国
• 起止时间: 2014-07-01 至 2017-04-30
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/8882493
• 关键词: Acceleration; Affect; Algorithms; Archives; Chromosome Structures; Code; Communities; Complex; Computational algorithm; Computer software; Computers; Coupled; DNA Sequence; DNA Sequence Databases; Data; Data Compression; Data Set; Databases; Deposition; Disease; Galaxy; Genes; Genome; Goals; Health; Human; Human Genetics; Human Genome; Infectious Diseases Research; Investigation; Modeling; Molecular; Mutation; Positioning Attribute; Process; Reaction Time; Reading; Real-Time Systems; Research; Research Personnel; Resources; Retrieval; Running; Scientist; Sequence Analysis; Services; Site; Solutions; Sorting - Cell Movement; Speed; System; Time; Validation; Variant; Writing; base; cloud based; design; human DNA sequencing; human disease; human genome sequencing; indexing; instrument; microbial; next generation sequencing; novel strategies; open source; programs; tool; trait; user-friendly; web interface

DESCRIPTION (provided by applicant): We propose to create a new system, the Terabase Search Engine that will make it possible for biomedical researchers to search all human DNA sequences that have been sequenced and deposited in public archives. The vast and growing resource of human DNA sequences provides a wealth of opportunities for scientific discovery and for validation of results, but the size of the data sets has already far exceeded the ability o most researchers to use them. For more than two decades, geneticists and geneticists have relied on DNA sequence databases for a wide range of scientific endeavors, including the discovery of new genes and new mutations, the investigation of evolutionary changes within and between species, the forces affecting chromosomal structure and change, and many other molecular and evolutionary processes. The ability to search all known genes and genomes using BLAST and similar programs has long been assumed, and sequence search engines throughout the world provide this ability. However, the raw data pouring out of next-generation sequencing (NGS) projects has exceeded our ability to provide rapid access to it. A single NGS instrument can generate six billion reads encompassing 600 billion bases in a single run, and this capacity is still growing. Traditional alignment programs like BLAST cannot sort through this data in a reasonable amount of time. Newer, faster programs such as Bowtie (developed by our group) allow far faster alignment of NGS reads to the genome, but today the size of the data sets, now in excess of 1 trillion reads, far exceeds the ability of most computers to store it. And even the fastest alignment programs today could not search all this data in a reasonable amount of time. A new approach is required in order to serve up these huge and hugely valuable DNA sequences to the research community. The Terabase Search Engine will be a new, highly efficient system for searching trillions of bases in real time. Using a hierarchical search strategy with extensive pre-processing to speed up response time, the TSE will allow a scientist to align any sequence, human or non-human, to all publicly-available human sequence reads. Reads that match the human genome will be indexed and stored on very high-speed disks for rapid retrieval. Reads that match microbial sequences will be captured and stored separately for use in micro biome and infectious disease research. The system will be made available through a user-friendly web interface, and a local database will store each user's results for further analysis on the TSE site or for download to a local site. This system will make it possible, for the first time ever, for any scientist to align a sequence to the complete set of human DNA sequences and to retrieve everything that matches, without the need to write special-purpose programs or to use complex cloud-based software interfaces. All of the software for this project will be developed under an open-source model that will permit others to use, modify, share, and re-distribute the code without restriction.

描述（由申请人提供）：我们建议创建一个新系统，即terabase搜索引擎，这将使生物医学研究人员可以搜索已将并存放在公共档案中的所有人类DNA序列。人类DNA序列的庞大而不断增长的资源为科学发现和验证结果提供了丰富的机会，但是数据集的规模已经超出了大多数研究人员使用它们的能力。二十多年来，遗传学家和遗传学家一直依赖DNA序列数据库，用于广泛的科学努力，包括发现新基因和新突变，物种内部和物种之间进化变化的研究，影响染色体结构和变化的力以及许多其他分子和进化过程。长期以来，已经假定了使用BLAST和类似程序搜索所有已知基因和基因组的能力，并且全世界的序列搜索引擎都提供了这种能力。但是，从下一代测序（NGS）项目中涌出的原始数据超出了我们提供快速访问的能力。单个NGS仪器可以在一次运行中产生60亿个读取，其中包含6000亿个基础，并且这种能力仍在增长。像BLAST这样的传统对齐程序无法在合理的时间内整理这些数据。诸如Bowtie（由我们的小组开发）之类的更新的程序允许NGS读取的速度更快地对齐基因组，但是如今，数据集的大小，如今已超过1万亿读的读取，远远超过了大多数计算机存储它的能力。和 即使是当今最快的对齐程序，也无法在合理的时间内搜索所有这些数据。为了为研究社区提供这些巨大且非常有价值的DNA序列，需要采用一种新的方法。 Terabase搜索引擎将是一个新的，高效的系统，用于实时搜索数万亿个基础。使用层次搜索策略和广泛的预处理以加快响应时间，TSE将允许科学家将任何序列（人类或非人类）与所有公共可用的人类序列读取。与人类基因组相匹配的读物将被索引并存储在非常高速的磁盘上，以快速检索。读取匹配微生物序列的读物将被捕获并分别存储在微型生物群落和传染病研究中。该系统将通过用户友好的Web界面提供，本地数据库将存储每个用户的结果以在TSE站点上进行进一步分析或下载到本地站点。这个系统将使 有史以来，任何科学家都有可能将序列与完整的人类DNA序列保持一致，并取回与之匹配的所有内容，而无需编写特殊用途程序或使用复杂的基于云的软件接口。该项目的所有软件都将在开源模型下开发，该模型将允许其他人使用，修改，共享和重新分配代码而无需限制。