[1]张毅锋,蒋程,程旭,等.基于基完备化理论和嵌入多层感知机的深度网络结构设计[J].东南大学学报(自然科学版),2018,48(5):933-938.[doi:10.3969/j.issn.1001-0505.2018.05.022]
　Zhang Yifeng,Jiang Cheng,Cheng Xu,et al.Deepnetwork structure design based on base completion and embedded multi-layer perceptron[J].Journal of Southeast University (Natural Science Edition),2018,48(5):933-938.[doi:10.3969/j.issn.1001-0505.2018.05.022]
点击复制

基于基完备化理论和嵌入多层感知机的深度网络结构设计()
分享到：

《东南大学学报(自然科学版)》[ISSN:1001-0505/CN:32-1178/N]

卷:
48
期数:
2018年第5期
页码:
933-938
栏目:
计算机科学与工程
出版日期:
2018-09-20

文章信息/Info

Title:
Deepnetwork structure design based on base completion and embedded multi-layer perceptron
作者:
张毅锋1; 2; 3; 蒋程1; 程旭4; 刘袁1
1东南大学信息科学与工程学院, 南京 210096; 2东南大学南京通信技术研究院, 南京 211100; 3南京大学计算机软件新技术国家重点实验室, 南京 210093; 4南京信息工程大学计算机与软件学院, 南京 210044
Author(s):
Zhang Yifeng 1; 2; 3;  Jiang Cheng 1;  Cheng Xu4;  Liu Yuan1
1School of Information Science and Engineering, Southeast University, Nanjing 210096, China
2Nanjing Institute of Communications Technologies, Southeast University, Nanjing 211100, China
3State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210093, China
4School of Computer and Software, Nanjing University of Information Science and Technology, Nanjing 210044, China
关键词:
基完备化;  期望网络;  嵌入多层感知机;  期望图
Keywords:
base completion;  expectation network;  embedded multi-layer perceptron;  expectation graph
分类号:
TP391
DOI:
10.3969/j.issn.1001-0505.2018.05.022
摘要:
为了进一步改善经典卷积神经网络的识别性能,首先证明了跨层传输实质上是基的完备化过程,然后基于多类型特征提取结构、嵌入多层感知机以及跨层传输结构提出一种新型深度卷积网络——期望网络. 经分析发现,期望网络中的多类型特征提取结构可提取不同类型的特征,嵌入多层感知机可生成期望图并标定不同类型特征的权重,跨层传输结构可缓解网络性能退化的问题.仿真实验结果表明,在数据集CIFAR-10、数据集CIFAR-100和数据集SVHN 上,相比于ResNet网络、深度监督网络和Highway网络等经典深度卷积网络, 期望网络的误识别率均有不同程度的下降.
Abstract:
In order to improve the recognition performance of classical convolutional neural networks, the cross-layer transmission was proved to be essentially a base completion process, and then a new type of deep convolutional network named expectation network(ExpNet)was proposed based on the multi-type feature extraction structure, the embedded multi-layer perceptron(MLP), and the cross-layer transmission structure. The analysis results show that the multi-type feature extraction structure in the expectation network can extract different types of features. The embedding multi-layer perceptron can generate the expectation graph and calibrate the weights of different types of features. The cross-layer transmission structure can alleviate the problem of network performance degradation. The simulation results show that on the CIFAR-10, the CIFAR-100 and the SVHN datasets, the misrecognition rate in the expectation network is lower than those of the ResNet, the deep supervision, and the Highway networks.

参考文献/References:

[1] Lecun Y, Bottou L, Bengio Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324.
[2] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition [EB/OL].(2015-04-10)[2018-01-23]. https://arxiv.org/abs/1409.1556.
[3] He K, Zhang X, Ren S, et al. Deep residual learning for image recognition [C]// Computer Vision and Pattern Recognition. Las Vegas, Nevada, USA, 2016: 770-778.
[4] Huang G, Liu Z, Weinberger K Q, et al. Densely connected convolutional networks[C]//Computer Vision and Pattern Recognition. Ha-waii, USA, 2017:2261-2269.
[5] Srivastava R K, Greff K, Schmidhuber J,et al.Training very deep networks [C]// Neural Information Processing Systems. Montreal, Canada, 2015: 2377-2385.
[6] Springenberg J T, Dosovitskiy A, Brox T, et al. Striving for simplicity: The all convolutional net[EB/OL].(2015-04-13)[2018-01-23]. https://arxiv.org/abs/1412.6806.
[7] Fei W, Mengqing J, Chen Q, et al. Residual attention network for image classification [EB/OL].(2017)[2018-01-10]. https://arxiv.org/abs/1704.06904.
[8] Hariharan B, Arbeláez P, Girshick R, et al. Hypercolumns for object segmentation and fine-grained localization [C]// Computer Vision and Pattern Recognition. Boston, USA, 2015: 447-456.
[9] Sermanet P, Kavukcuoglu K, Chintala S, et al. Pedestrian Detection with Unsupervised Multi-stage Feature Learning [C]// Computer Vision and Pattern Recognition. Oregon, USA, 2013: 3626-3633.
[10] Krizhevsky, A. Learning multiple layers of features from tiny images [EB/OL].(2009)[2017-12-09]. https://www.researchgate.net/publication/265748773_Learning_Multiple_Layers_of_Features_from_Tiny_Images.
[11] Huang G, Sun Y, Liu Z, et al. Deep networks with stochastic depth [C]// European Conference on Computer Vision. Amsterdam, Netherlands, 2016: 646-661.
[12] Lin M, Chen Q, Yan S, et al. Network in network [EB/OL].(2014-03-04)[2017-11-30]. https://arxiv.org/abs/1312.4400.
[13] Romero A, Ballas N, Kahou S E, et al. FitNets: Hints for thin deep nets [EB/OL].(2015)[2018-01-23]. https://arxiv.org/abs/ 1412.6550.
[14] Lee C, Xie S, Gallagher P W, et al. Deeply-supervised nets [C]// International Conference on Artificial Intelligence and Statistics. Reykjavik, Iceland, 2014: 562-570.
[15] Netzer Y, Wang T, Coates A, et al. Reading digits in natural images with unsupervised feature learning[EB/OL].(2011)[2017-12-13].http://ai.sta-nford.edu/~twangcat/papers/nips2011_housenum-bers.pdf.
[16] Sutskever I, Martens J, Dahl G, et al. On the importance of initialization and momentum in deep learning [C]// International Conference on Machine Learning. Atlanta, USA, 2013: 1139-1147.

备注/Memo

备注/Memo:
收稿日期: 2018-01-28.
作者简介: 张毅锋(1963—),男,博士,副教授,yfz@seu.edu.cn.
基金项目: 国家自然科学基金资助项目(61673108)、江苏省自然科学基金资助项目(BK20151102)、北京大学机器感知与智能教育部重点实验室开放课题资助项目(K-2016-03)、东南大学水声信号处理教育部重点实验室开放课题资助项目(UASP1502).
引用本文: 张毅锋,蒋程,程旭,等.基于基完备化理论和嵌入多层感知机的深度网络结构设计[J].东南大学学报(自然科学版),2018,48(5):933-938. DOI:10.3969/j.issn.1001-0505.2018.05.022.
更新日期/Last Update: 2018-09-20