Neural networks with excellent abilities in learning and recognition problems for complex patterns

神经网络在复杂模式的学习和识别问题方面具有出色的能力

• 批准号: 15500150
• 负责人: FUJIKI Nahomi M.
• 金额: $ 0.64万
• 依托单位: Sendai National College of Technology
• 依托单位国家: 日本
• 项目类别: Grant-in-Aid for Scientific Research (C)
• 财政年份: 2003
• 资助国家: 日本
• 起止时间: 2003 至 2004
• 项目状态: 已结题
• 来源: https://kaken.nii.ac.jp/en/grant/KAKENHI-PROJECT-15500150/
• 关键词: Neural network; Kullback Divergence; Error Back Propagation; Identification; Scalability; Multi-layered network; Learning Algorithm; 汎化能力; 手書き文字; 認識能力; 確率ニューロン; 逆問題

Error back propagation is a well-known and powerful supervised learning algorithm for feed-forward multi-layered neural networks, but it is known to introduce the problem of its learning process being often trapped in a meta-stable state. It has been reported that the error back propagation algorithm with Kullback divergence instead of conventional quadratic error as an error measure demonstrates an excellent learning tendency and is expected to be applicable to various problems, even without the foreknowledge of their optimal network size.We have studied this algorithm carefully based on the identification problem of handwritten numerical characters and discussed its abilities such as scalability, flexibility, damage tolerance, and recognition of unlearned data. The numerical studies have been done on the three-layered feed-forward network ; an input layer, a hidden middle layer with various numbers of neurons up to 8129, and an output layer. Results of numerical studies indicate that this learning algorithm can have high generalization ability and be sufficiently powerful for practical use. Those network abilities are also improved by increasing its size. If we prepare an enough size of network, we could easily obtain the identification rate of higher than 99% after a few learning steps and recognition rate for unlearned characters over 90%.Those research results have been published on several papers in Research Reports on Sendai National College of technology, and also presented the papers at various conferences such as Tohoku-section Joint Convention of Institutes of Electrical And Information Engineers.

误差反向传播是一种众所周知且强大的前馈多层神经网络监督学习算法，但众所周知，它会引入其学习过程经常陷入亚稳态的问题。据报道，用 Kullback 散度代替传统的二次误差作为误差度量的误差反向传播算法表现出极好的学习倾向，并且有望适用于各种问题，即使在不预先知道其最佳网络规模的情况下。我们已经研究了该算法仔细地基于手写数字字符的识别问题，讨论了其可扩展性、灵活性、损伤容限和未学习数据识别等能力。对三层前馈网络进行了数值研究；一个输入层，一个隐藏的中间层，具有不同数量的神经元，最多 8129 个，以及一个输出层。数值研究结果表明，该学习算法具有较高的泛化能力，足以满足实际应用的需要。这些网络能力也通过增加其规模而得到提高。如果我们准备足够规模的网络，经过几个学习步骤，我们就可以轻松获得高于99%的识别率，并且对未学习的字符的识别率超过90%。这些研究成果已发表在《仙台研究报告》的多篇论文中国立工业大学，并在电气和信息工程师学会东北地区联合会议等各种会议上发表了论文。

项目成果

カルバック測度を用いたエラーバックプロパゲーション学習則の学習及び識字能力の評価

使用 Kullback 度量学习误差反向传播学习规则并评估读写能力

• 发表时间: 2003
• 期刊: 仙台電波工業高等専門学校紀要 33
• 作者: Fujiki Nahomi M.;Baba Yoji;Harada Yusuke;藤木なほみ
• 通讯作者: 藤木なほみ

カルバック測度をエラー測度とするニューラルネットワークの認識と汎化能力の検討

使用 Kullback 测度作为误差测度检验神经网络的识别和泛化能力

• 发表时间: 2004
• 期刊: 仙台電波工業高等専門学校紀要 34
• 作者: Fujiki Nahomi M.;Sasaki Ryutarou;Fujiki Sumiyoshi;藤木なほみ
• 通讯作者: 藤木なほみ

Learning and Identification abilities of Error Back Propagation Algorithm with Kullback Divergence

Kullback散度误差反向传播算法的学习与识别能力

• 发表时间: 2003
• 期刊: Research Reports of Sendai National College of Technology No.33
• 作者: Fujiki Nahomi M.;Baba Yoji;Harada Yusuke
• 通讯作者: Harada Yusuke

フィードフォワード型確率的層状ニューラルネットワークの学習能力

前馈随机分层神经网络的学习能力

• 发表时间: 2004
• 期刊: 仙台電波工業高等専門学校研究紀要 34
• 作者: 藤木なほみ;藤木なほみ
• 通讯作者: 藤木なほみ

藤木なほみ: "カルバック測度を用いたエラーバックプロパゲーション学習則の学習及び認識能力の評価"仙台電波工業高等専門学校研究紀要. 33. 65-70 (2003)

Nahomi Fujiki：“使用 Kullback 度量的误差反向传播学习规则的学习和识别能力的评估”仙台国立技术大学研究报告 33. 65-70 (2003)。