Accelerated Vonvergence and Structure Determination of the Backpropagation Neural Network

反向传播神经网络的加速收敛和结构确定

• 批准号: 9211691
• 负责人: Luke Achenie
• 金额: $ 10万
• 依托单位: University of Connecticut
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 1992
• 资助国家: 美国
• 起止时间: 1992-08-01 至 1996-01-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=9211691&HistoricalAwards=false
• 关键词: Accelerated; Vonvergence; Structure; Determination; Backpropagation

Neural nets can identify and learn correlative patterns between sets of input data and corresponding target values. The most widely used neural net architecture, the back- propagation net, loosely mimics the human learning process and "learns" to recognize patterns relating input and output variables. Such nets are trained by being repeatedly fed input data together with corresponding target outcomes. After a sufficient number of training iterations, the net learns to recognize patterns in the data and, effectively, creates an internal model of the process governing the data. The net can then use this internal model to make predictions for new input conditions. Supervised training of back-propagation neural networks is usually achieved through the solution of an appropriate optimization problem. Subsequently, training times are affected by the nonlinear programming algorithms used. The training algorithm that is often used is the delta rule, which is a steepest descent derivative and as such exhibits a linear rate of convergence around a local minimum. This results in very long training time, often on the order of hours or days for practical problems. In this project the PI plans to: (1) accelerate training of the back-propagation network using Newton type algorithms, (2) determine network structure through the use of the singular value decomposition of the analytic hessian, (3) use the concept of a Minimal Spanning Network to derive a network of linear elements that will provide a performance lower bound on the neural network, and (4) impose appropriate bounds (or constraints) on design variables to enhance convergence. He hopes that the results will significantly speed up the training of neural networks. The structure of both the analytic gradients and the analytic hessian will be exploited in an implementation of the back-propagation algorithm on parallel computers, resulting in further increases in speed up. With such speed up, it will be possible to tackle difficult industrially relevant problems in a reasonable time frame.

神经网可以识别和学习输入数据集和相应目标值之间的相关模式。 使用最广泛的神经网架构，后传播网，松散地模仿人类学习过程，并“学习”以识别与输入和输出变量有关的模式。 通过反复馈送输入数据以及相应的目标结果来训练此类网。 经过足够数量的训练迭代，NET学会了识别数据中的模式，并有效地创建了控制数据的过程的内部模型。 然后，NET可以使用此内部模型对新输入条件进行预测。 通常通过解决适当的优化问题来实现后传播神经网络的监督培训。 随后，培训时间会受到所使用的非线性编程算法的影响。 经常使用的训练算法是Delta规则，它是最陡峭的下降衍生物，因此在局部最小值附近表现出线性收敛速率。 这会导致训练时间很长，通常是在几个小时或数天的情况下解决实际问题。 In this project the PI plans to: (1) accelerate training of the back-propagation network using Newton type algorithms, (2) determine network structure through the use of the singular value decomposition of the analytic hessian, (3) use the concept of a Minimal Spanning Network to derive a network of linear elements that will provide a performance lower bound on the neural network, and (4) impose appropriate bounds (or constraints) on design variables to增强收敛。 他希望结果将大大加快神经网络的训练。 分析梯度和分析性Hessian的结构将在平行计算机上的后传播算法实现中被利用，从而导致速度进一步增加。 随着这种速度，可以在合理的时间范围内解决困难的工业相关问题。