[1]李建,刘红星,王仲宇.前馈网络构造性设计中基于GP实现神经元激活函数类型优化[J].东南大学学报(自然科学版),2004,34(6):746-750.[doi:10.3969/j.issn.1001-0505.2004.06.007]
 Li Jian,Liu Hongxing,Wang Zhongyu.Optimizing neuronal activation function types based on GP in constructive FNN design[J].Journal of Southeast University (Natural Science Edition),2004,34(6):746-750.[doi:10.3969/j.issn.1001-0505.2004.06.007]
点击复制

前馈网络构造性设计中基于GP实现神经元激活函数类型优化()
分享到:

《东南大学学报(自然科学版)》[ISSN:1001-0505/CN:32-1178/N]

卷:
34
期数:
2004年第6期
页码:
746-750
栏目:
自动化
出版日期:
2004-11-20

文章信息/Info

Title:
Optimizing neuronal activation function types based on GP in constructive FNN design
作者:
李建 刘红星 王仲宇
南京大学电子科学与工程系, 南京 210093
Author(s):
Li Jian Liu Hongxing Wang Zhongyu
Department of Electronic Science and Engineering, Nanjing University, Nanjing 210093, China
关键词:
神经网络 构造性设计 遗传编程 神经元激活函数
Keywords:
neural networks constructive algorithms genetic programming neuronal activation function
分类号:
TP183
DOI:
10.3969/j.issn.1001-0505.2004.06.007
摘要:
讨论了在前馈网络构造性设计中如何基于遗传编程(GP)实现神经元激活函数类型自动优化的问题.首先,提出了典型前馈网络的一种构造性设计方法框架,将整个网络的设计分解为单个神经元的逐个设计; 然后,在此框架下提出了基于GP的单个神经元的设计方法,该方法可实现对激活函数类型的优化.仿真实验显示,本文的前馈网络构造性设计方案是可行的,与其他几种不优化激活函数类型的网络设计方法相比,本方法更有效,能用较小的网络规模获得更满意的泛化特性.
Abstract:
Aiming at typical feedforward neural networks(FNN), a constructive FNN designing algorithm with the auto-optimization of neuronal activation function types based on genetic programming(GP)is investigated. First, a frame of the constructive FNN design is given, in which the design of the whole FNN breaks down to the design of single neurons one by one. Then, based on GP the design algorithm of single neuron, which realizes the auto-optimization of neuronal activation function types, is proposed. Finally, with many function approximation experiments, it is shown that the proposed constructive FNN design scheme is feasible. Compared with some other designing algorithms without activation function type optimization, it is more effective, being able to achieve better FNN generalization with smaller network size.

参考文献/References:

[1] Karnin E D. A simple procedure for pruning backpropagation trained neural networks [J]. IEEE Transactions on Neural Networks, 1990,1(2):239-242.
[2] Reed R.Pruning algorithms—A survey [J].IEEE Transactions on Neural Networks, 1993,4(5):740-747.
[3] Giles C L,Chen Dong,Sun Guozheng,et al.Constructive learning of recurrent neural networks:problems with recurrent cascade correlation and a simple solution [J]. IEEE Transactions on Neural Networks, 1995,6(4):829-836.
[4] Kwok T Y,Yeung D Y.Constructive algorithms for structure learning in feedforward neural networks for regression problems [J]. IEEE Transactions on Neural Networks, 1997,8(3):630-645.
[5] Treadgold N K,Gedeon T D.Exploring constructive cascade networks [J]. IEEE Transactions on Neural Networks, 1999,10(6):1335-1350.
[6] Wu Y S,Zhao M S.A neuron model with trainable activation function(TAF)and its MFNN supervised learning [J]. Science in China(Information Sciences), 2001,44(5):366-375.
[7] Koza J.Genetic programming:on the programming of computers by means of natural selection [M].Cambridge,MA:The MIT Press,1992.1-45.
[8] Cybenco G.Approximations by superpositions of a sigmoid function [J]. Mathematics of Control,Signals and Systems,1989,2(2):303-314.
[9] Hornik Kurt.Approximation capabilities of multilayer feedforward networks [J]. Neural Networks, 1991,4(2):251-257.
[10] MacKay D J C.Bayesian interpolation [J]. Neural Computation, 1992,4(3):415-447.
[11] Moller M F.A scaled conjugate gradient algorithm for fast supervised learning [J]. Neural Networks, 1993,6(5):525-533.

相似文献/References:

[1]胡伍生,沙月进.神经网络BP算法的误差分级迭代法[J].东南大学学报(自然科学版),2003,33(3):376.[doi:10.3969/j.issn.1001-0505.2003.03.032]
 Hu Wusheng,Sha Yuejin.Error grade iterative method of BP neural networks[J].Journal of Southeast University (Natural Science Edition),2003,33(6):376.[doi:10.3969/j.issn.1001-0505.2003.03.032]
[2]朱大奇,于盛林.电子电路故障诊断的神经网络数据融合算法[J].东南大学学报(自然科学版),2001,31(6):87.[doi:10.3969/j.issn.1001-0505.2001.06.021]
 Zhu Daqi,Yu Shenglin.Neural Network Data Fusion Algorithm of Circuit Fault Diagnosis[J].Journal of Southeast University (Natural Science Edition),2001,31(6):87.[doi:10.3969/j.issn.1001-0505.2001.06.021]
[3]倪富健,屠伟新,黄卫.基于神经网络技术的路面性能预估模型[J].东南大学学报(自然科学版),2000,30(5):91.[doi:10.3969/j.issn.1001-0505.2000.05.021]
 Ni Fujian,Tu Weixing,Huang Wei.Pavement Performance Forecasting Model by Using Neural Network[J].Journal of Southeast University (Natural Science Edition),2000,30(6):91.[doi:10.3969/j.issn.1001-0505.2000.05.021]
[4]窦东阳,杨建国,李丽娟,等.基于规则的神经网络在模式分类中的应用[J].东南大学学报(自然科学版),2011,41(3):482.[doi:10.3969/j.issn.1001-0505.2011.03.010]
 Dou Dongyang,Yang Jianguo,Li Lijuan,et al.Application of rule-based neural network in pattern classification[J].Journal of Southeast University (Natural Science Edition),2011,41(6):482.[doi:10.3969/j.issn.1001-0505.2011.03.010]
[5]黄清.自适应宽带稳健波束形成及神经网络算法[J].东南大学学报(自然科学版),2002,32(2):172.[doi:10.3969/j.issn.1001-0505.2002.02.005]
 Huang Qing.Robust adaptive broadband beamforming and neural network[J].Journal of Southeast University (Natural Science Edition),2002,32(6):172.[doi:10.3969/j.issn.1001-0505.2002.02.005]
[6]朱明程,徐健,李昆华.采用MCU多层神经网络模型设计探讨[J].东南大学学报(自然科学版),2000,30(2):21.[doi:10.3969/j.issn.1001-0505.2000.02.005]
 Zhu Mingcheng,Xu Jian,Li Kunhua.A Novel Model Design Adopting MCU for Multilayer Neural Network[J].Journal of Southeast University (Natural Science Edition),2000,30(6):21.[doi:10.3969/j.issn.1001-0505.2000.02.005]
[7]吴新根,柏毅,罗立民.一种基于概率神经网络模型的胃病诊断专家系统[J].东南大学学报(自然科学版),1999,29(4):57.[doi:10.3969/j.issn.1001-0505.1999.04.013]
 Wu Xingen,Bai Yi,Luo Limin.A Neural Network Model Based on Bayesian Posterior Probability[J].Journal of Southeast University (Natural Science Edition),1999,29(6):57.[doi:10.3969/j.issn.1001-0505.1999.04.013]
[8]程乃毅,赵显曾.点模式识别ANN的区域分划特征[J].东南大学学报(自然科学版),1995,25(1):69.[doi:10.3969/j.issn.1001-0505.1995.01.013]
 Cheng Naiyi,Zhao Xianzeng,Zhao Xianzeng.Domain Partition Features of ANN for Point Pattem Recognition[J].Journal of Southeast University (Natural Science Edition),1995,25(6):69.[doi:10.3969/j.issn.1001-0505.1995.01.013]
[9]夏又生,黄俊良.一类线性互补问题的神经网络求解[J].东南大学学报(自然科学版),1995,25(6):25.[doi:10.3969/j.issn.1001-0505.1995.06.005]
 Xia Youshen,Huang Junliang.Neural Network for Solving a Class of Linear Complementary Problems[J].Journal of Southeast University (Natural Science Edition),1995,25(6):25.[doi:10.3969/j.issn.1001-0505.1995.06.005]
[10]汤新梁.自由手写体数字识别的神经网方法[J].东南大学学报(自然科学版),1991,21(3):8.[doi:10.3969/j.issn.1001-0505.1991.03.002]
 Tang Xinliang.Handwritten Numeral Recognition Using Neural Networks[J].Journal of Southeast University (Natural Science Edition),1991,21(6):8.[doi:10.3969/j.issn.1001-0505.1991.03.002]

备注/Memo

备注/Memo:
基金项目: 国家自然科学基金资助项目(60275041,59905011).
作者简介: 李建(1979—),男,硕士生; 刘红星(联系人),男,副教授,njhxliu@nju.edu.cn.
更新日期/Last Update: 2004-11-20