Image generation based on conditional-Wassertein generative adversarial network

doi:10.11772/j.issn.1001-9081.2020071138

Journal of Computer Applications ›› 2021, Vol. 41 ›› Issue (5): 1432-1437.DOI: 10.11772/j.issn.1001-9081.2020071138

Special Issue: 多媒体计算与计算机仿真

• Virtual reality and multimedia computing • Previous Articles Next Articles

Image generation based on conditional-Wassertein generative adversarial network

GUO Maozu^1,2, YANG Qiannan^1,2, ZHAO Lingling³

1. School of Electrical and Information Engineering, Beijing University of Civil Engineering and Architecture, Beijing 100044, China;
2. Beijing Key Laboratory of Intelligent Processing for Building Big Data(Beijing University of Civil Engineering and Architecture), Beijing 100044, China;
3. School of Computer Science and Technology, Harbin Institute of Technology, Harbin Heilongjiang 150001, China

Received:2020-07-31 Revised:2020-10-05 Online:2021-05-10 Published:2020-12-23
Supported by:
This work is partially supported by Surface Program of National Natural Science Foundation of China (61871020), the Key Project of Science and Technology Plan of Beijing Municipal Education Commission (KZ201810016019), High-level Innovation Team Building Program in Beijing Municipal Colleges and Universities (IDHT20190506), the Graduate Innovation Project of Beijing University of Civil Engineering and Architecture in 2020 (PG202005).

基于条件Wassertein生成对抗网络的图像生成

郭茂祖^1,2, 杨倩楠^1,2, 赵玲玲³

1. 北京建筑大学电气与信息工程学院, 北京 100044;
2. 建筑大数据智能处理方法研究北京市重点实验室(北京建筑大学), 北京 100044;
3. 哈尔滨工业大学计算机科学与技术学院, 哈尔滨 150001

通讯作者: 赵玲玲
作者简介:郭茂祖(1966-),男,山东德州人,教授,博士生导师,博士,主要研究方向:机器学习、智慧城市、生物信息学;杨倩楠(1995-),女,甘肃兰州人,硕士研究生,主要研究方向:机器学习、智慧城市;赵玲玲(1980-),女,黑龙江齐齐哈尔人,讲师,博士,主要研究方向:机器学习、智慧城市、生物信息学。
基金资助:
国家自然科学基金面上项目（61871020）；北京市教委科技计划重点项目（KZ201810016019）；北京市属高校高水平创新团队建设计划项目（IDHT20190506）；北京建筑大学2020年度研究生创新项目（PG202005）。

Abstract

Abstract: Generative Adversarial Network (GAN) can automatically generate target images, and is of great significance to the generation of building arrangement of similar blocks. However, there are problems in the existing process of model training such as the low accuracy of generated images, the mode collapse, and the too low efficiency of model training. To solve these problems, a Conditional-Wassertein Generative Adversarial Network (C-WGAN) model for image generation was proposed. First, the feature correspondence between the real sample and the target sample was needed to be identified by this model, and then the target sample was generated according to the identified feature correspondence. The Wassertein distance was used to measure the distance between the distributions of two image features in the model, the GAN training environment was stablized, and mode collapse was avoided during model training, so as to improve the accuracy of the generated images and the training efficiency. Experimental results show that compared with the original Conditional Generative Adversarial Network (CGAN) and the pix2pix models, the proposed model has the Peak Signal-to-Noise Ratio (PSNR) increased by 6.82% and 2.19% at most respectively; in the case of the same number of training rounds, the proposed model reaches the convergence state faster. It can be seen that the proposed model can not only effectively improve the accuracy of image generation, but also increase the convergence speed of the network.

Key words: image generation, Generative Adversarial Network (GAN), Conditional Generative Adversarial Network (CGAN), Wassertein distance

摘要： 生成对抗网络（GAN）能够自动生成目标图像，对相似地块的建筑物排布生成具有重要意义。而目前训练模型的过程中存在生成图像精度不高、模式崩溃、模型训练效率太低的问题。针对这些问题，提出了一种面向图像生成的条件Wassertein生成对抗网络（C-WGAN）模型。首先，该模型需要识别真实样本和目标样本之间特征对应关系，然后，根据所识别出的特征对应关系进行目标样本的生成。模型采用Wassertein距离来度量两个图像特征之间分布的距离，稳定GAN训练环境，规避模型训练过程中的模式崩溃，从而提升生成图像的精度和训练效率。实验结果表明，与原始条件生成对抗网络（CGAN）和pix2pix模型相比，所提模型的峰值信噪比（PSNR）分别最大提升了6.82%和2.19%；在训练轮数相同的情况下，该模型更快达到收敛状态。由此可见，所提模型不仅能够有效地提升图像生成的精度，而且能够提高网络的收敛速度。

关键词: 图像生成, 生成对抗网络, 条件生成对抗网络, Wassertein距离

CLC Number:

TP183

GUO Maozu, YANG Qiannan, ZHAO Lingling. Image generation based on conditional-Wassertein generative adversarial network[J]. Journal of Computer Applications, 2021, 41(5): 1432-1437.

郭茂祖, 杨倩楠, 赵玲玲. 基于条件Wassertein生成对抗网络的图像生成[J]. 计算机应用, 2021, 41(5): 1432-1437.

References

[1] YI X,WALIA E,BABYN P. Generative adversarial network in medical imaging:a review[J]. Medical Image Analysis,2019,58:No. 101552.
[2] CHANG H,LU J,YU F,et al. PairedCycleGAN:asymmetric style transfer for applying and removing makeup[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2018:40-48.
[3] HAN X,WU Z,WU Z,et al. VITON:an image-based virtual tryon network[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2018:7543-7552.
[4] ZHU J Y,PARK T,ISOLA P,et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway:IEEE,2017:2242-2251.
[5] 金连文, 钟卓耀, 杨钊, 等. 深度学习在手写汉字识别中的应用综述[J]. 自动化学报,2016,42(8):1125-1141.(JIN L W, ZHONG Z Y,YANG Z,et al. Applications of deep learning for handwritten Chinese character recognition:a review[J]. Acta Automatica Sinica,2016,42(8):1125-1141.)
[6] GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems. Cambridge:MIT Press,2014:2672-2680.
[7] YU L,ZHANG W,WANG J,et al. SeqGAN:sequence generative adversarial nets with policy gradient[C]//Proceedings of the 31st AAAI Conference on Artificial Intelligence. Palo Alto,CA:AAAI Press,2017:2852-2858.
[8] 王坤峰, 苟超, 段艳杰, 等. 生成式对抗网络GAN的研究进展与展望[J]. 自动化学报,2017,43(3):321-332.(WANG K F, GOU C,DUAN Y J,et al. Generative adversarial networks:the state of the art and beyond[J]. Acta Automatica Sinica,2017,43(3):321-332.)
[9] CRESWELL A,WHITE T,DUMOULIN V,et al. Generative adversarial networks:an overview[J]. IEEE Signal Processing Magazine,2018,35(1):53-65.
[10] GAUTHIER J. Conditional generative adversarial nets for convolutional face generation[EB/OL].[2020-03-02]. https://www.foldl.me/uploads/2015/conditional-gans-face-generation/paper.pdf.
[11] GULRAJANI I,AHMED F,ARJOVSKY M,et al. Improved training of Wasserstein GANs[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook,NY:Curran Associates Inc.,2017:5768-5779.
[12] ISOLA P,ZHU J,ZHOU T,et al. Image-to-image translation with conditional adversarial networks[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2017:5967-5976.
[13] RATLIFF L J,BURDEN S A,SASTRY S S. Characterization and computation of local Nash equilibria in continuous games[C]//Proceedings of the 51st Annual Allerton Conference on Communication,Control,and Computing. Piscataway:IEEE, 2013:917-924.
[14] SANTURKAR S,TSIPRAS D,ILYAS A,et al. How does batch normalization help optimization?[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems. Red Hook,NY:Curran Associates Inc.,2018:2488-2498.
[15] WU Y,HE K. Group normalization[J]. International Journal of Computer Vision,2020,128(3):742-755.

Image generation based on conditional-Wassertein generative adversarial network

基于条件Wassertein生成对抗网络的图像生成

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

[1]	GUAN Qijie, ZHANG Ting, LI Deya, ZHOU Shaojing, DU Yi. Indefinite reconstruction method of spatial data based on multi-resolution generative adversarial network [J]. Journal of Computer Applications, 2021, 41(8): 2306-2311.
[2]	SUN Xiao, XU Jindong. Remote sensing image dehazing method based on cascaded generative adversarial network [J]. Journal of Computer Applications, 2021, 41(8): 2440-2444.
[3]	TANG Guihua, SUN Lei, MAO Xiuqing, DAI Leyu, HU Yongjin. Generative adversarial network synthesized face detection based on deep alignment network [J]. Journal of Computer Applications, 2021, 41(7): 1922-1927.
[4]	WANG Xianwu, ZHANG Ting, JI Xin, DU Yi. 3D shale digital core reconstruction method based on deep convolutional generative adversarial network with gradient penalty [J]. Journal of Computer Applications, 2021, 41(6): 1805-1811.
[5]	LI Yanzhi, FAN Yong, GAO Lin. Anomaly detection of oil drilling water flow based on shape flow [J]. Journal of Computer Applications, 2021, 41(6): 1842-1848.
[6]	SUN Heli, SUN Yuzhu, ZHANG Xiaoyun. Event description generation based on generative adversarial network [J]. Journal of Computer Applications, 2021, 41(5): 1256-1261.
[7]	DUAN Youxiang, ZHANG Hanxiao, SUN Qifeng, SUN Youkai. Image super-resolution reconstruction algorithm based on Laplacian pyramid generative adversarial network [J]. Journal of Computer Applications, 2021, 41(4): 1020-1026.
[8]	OU Lili, SHAO Fengjing, SUN Rencheng, SUI Yi. Cerebral infarction image recognition based on semi-supervised method [J]. Journal of Computer Applications, 2021, 41(4): 1221-1226.
[9]	LI Hongxia, QIN Pinle, YAN Hanmei, ZENG Jianchao, BAO Qianyue, CHAI Rui. Face frontalization generative adversarial network algorithm based on face feature map symmetry [J]. Journal of Computer Applications, 2021, 41(3): 714-720.
[10]	ZHANG Ya, JIN Xin, JIANG Qian, LEE Shin-jye, DONG Yunyun, YAO Shaowen. Deepfake image detection method based on autoencoder [J]. Journal of Computer Applications, 2021, 41(10): 2985-2990.
[11]	QIU Yaoru, SUN Weijun, HUANG Yonghui, TANG Yuqi, ZHANG Haochuan, WU Junpeng. Person re-identification method based on GAN uniting with spatial-temporal pattern [J]. Journal of Computer Applications, 2020, 40(9): 2493-2498.
[12]	ZHANG Shuping, WU Wen, WAN Yi. Single image shadow removal method based on multistage generative adversarial network [J]. Journal of Computer Applications, 2020, 40(8): 2378-2385.
[13]	LIN Jing, HUANG Yuqing, LI Leimin. Image super-resolution reconstruction based on spherical moment matching and feature discrimination [J]. Journal of Computer Applications, 2020, 40(8): 2345-2350.
[14]	ZHANG Hao, ZHANG Qiang, SHAO Siyu, DING Haibin. Application of deep learning to 3D model reconstruction of single image [J]. Journal of Computer Applications, 2020, 40(8): 2351-2357.
[15]	CHEN Foji, ZHU Feng, WU Qingxiao, HAO Yingming, WANG Ende. Infrared image data augmentation based on generative adversarial network [J]. Journal of Computer Applications, 2020, 40(7): 2084-2088.