[1]王强,达飞鹏,宋文忠.前向神经网络临时极小点动态性能[J].东南大学学报(自然科学版),2005,35(4):641-644.[doi:10.3969/j.issn.1001-0505.2005.04.032]
 Wang Qiang,Da Feipeng,Song Wenzhong.Dynamics of temporary minima in feed-forward neural network[J].Journal of Southeast University (Natural Science Edition),2005,35(4):641-644.[doi:10.3969/j.issn.1001-0505.2005.04.032]
点击复制

前向神经网络临时极小点动态性能()
分享到:

《东南大学学报(自然科学版)》[ISSN:1001-0505/CN:32-1178/N]

卷:
35
期数:
2005年第4期
页码:
641-644
栏目:
自动化
出版日期:
2005-07-20

文章信息/Info

Title:
Dynamics of temporary minima in feed-forward neural network
作者:
王强 达飞鹏 宋文忠
东南大学自动化研究所, 南京 210096
Author(s):
Wang Qiang Da Feipeng Song Wenzhong
Research Institute of Automation, Southeast University, Nanjing 210096, China
关键词:
BP算法 临时极小点 Jacobian矩阵 特征值
Keywords:
back-propagation algorithm temporary minima Jacobian matrix eigenvalue
分类号:
TP183
DOI:
10.3969/j.issn.1001-0505.2005.04.032
摘要:
对于只有一个隐含层的前向神经网络,分析了隐含层不同神经元之间权值数值相近但符号相反时会产生临时极小点的情况.并以临时极小点为平衡点建立了动力学模型,在平衡点附近线性化后得到了系统的Jacobian矩阵,证明了Jacobian矩阵一定是不定矩阵,因此Jacobian矩阵有符号相异的特征值,系统的平衡点即临时极小点为鞍点.并以异或问题为例进行仿真,仿真结果表明所得到的结论是正确的.
Abstract:
For a feed-forward neural network with only one hidden-layer, when initial weights have almost identical numerical value and opposite sign between different hidden-layer neurons, neural network would get stuck on temporary minima during training. A dynamical system was derived, in which the equilibrium point corresponded to temporary minimum. And the Jacobian matrix of this system was established by linearizing the system around equilibrium point. It is proved that Jacobian matrix is indefinite matrix, so Jacobian matrix has opposite sign eigenvalues. The result indicates that equilibrium point that is temporary minimum is saddle point. The exclusive OR(XOR)problem was used to simulate. Simulation result shows the validity of our conclusion.

参考文献/References:

[1] Murray A F.Analog VLSI and multi-layer perceptrons-accuracy,noise and on-chip learning[J].Neurocomputing,1992,4(6):301-310.
[2] Phansalkar V V,Sastry P S.Analysis of the back-propagation algorithm with momentum[J].IEEE Transactions on Neural Networks,1994,5(3):505-506.
[3] RoyChowdhury P,Singh Y P,Chansarkar R A.Hybridization of gradient descent algorithms with dynamic tunneling methods for global optimization[J].IEEE Transactions on Systems,Man and Cybernetics Part A:Systems and Humans,2000,30(3):384-390.
[4] Ampazis N,Perantonis S J,Taylor J G.Dynamics of multilayer networks in the vicinity of temporary minima[J].Neural Networks,1999,12(1):43-58.
[5] Ampazis N,Perantonis S J,Taylor J G.A dynamics model for the analysis and acceleration of learning in feedforward networks[J].Neural Networks,2001,14(8):1075-1088.
[6] Vitela J E,Reifman J.Premature saturation in backpropagation network:mechanism and necessary conditions[J].Neural Networks,1997,10(4):721-735.
[7] Zweiri Y H,Seneviratne L D,Althoefer K.Parameter estimation for excavator arm using generalized Newton method[J].IEEE Transactions on Robotics and Automation,2004,20(4):762-767.
[8] Ampazis N,Perantonis S J.Two highly efficient second-order algorithms for training feedforward networks[J].IEEE Transactions on Neural Networks,2002,13(5):1064-1074.
[9] 王太勇,商同,任成祖,等.一种改进BP算法及其在滚动轴承故障诊断中的应用[J].中国机械工程,2001,12(10):1179-1181.
  Wang Taiyong,Shang Tong,Ren Chengzu,et al.A kind of modified BP algorithm and application in faults diagnosing of roller bearing[J].Chinese Mechanical Engineering,2001,12(10):1179-1181.(in Chinese)
[10] Haykin S.神经网络的综合基础.第2版[M].北京:清华大学出版社,2001.171-172.

相似文献/References:

[1]刘春林,何建敏.神经网络用于模式识别分类的改进算法[J].东南大学学报(自然科学版),1999,29(1):20.[doi:10.3969/j.issn.1001-0505.1999.01.004]
 Liu Chunlin,He jianmin.An Improved ANN Algorithm for Pattern Recognition[J].Journal of Southeast University (Natural Science Edition),1999,29(4):20.[doi:10.3969/j.issn.1001-0505.1999.01.004]

备注/Memo

备注/Memo:
基金项目: 江苏省自然科学基金资助项目(BK2003405).
作者简介: 王强(1977—),男,博士生; 宋文忠(联系人),男,教授,博士生导师,swzjwz@seu.edu.cn.
更新日期/Last Update: 2005-07-20