| 摘要: |
| 大数据时代的到来使得网络上图像数据信息越来越丰富, 篡改图像也越来越多, 为了区分篡改图和原图, 图像篡改检测方法层出不穷。虽然篡改检测可以定位图像的篡改区域, 但是仍无法了解原图的图像以及原图所表达的语义信息。为了解决上述问题, 本文提出了一种基于生成对抗网络(Generative adversarial network, GAN)图像语义指导的取证修复图像模型。首次在篡改环境下, 针对于copy-move和拼接篡改这两种篡改手段进行图像修复。该模型利用GAN网络对抗式进行训练, 最终得到训练好的生成器进行图像取证修复工作。模型利用图像语义识别模块, 提供原图像的语义特征, 对篡改环境下的图像修复进行指导和约束, 这样使得修复图像遵循原始图像的语义, 且生成器不会过度对图像进行美化, 从而达到还原真实图像的目的。该模型在GAN网络中, 将生成器修复的图像进行对应位置掩模拼接后再放入判别器, 使得图像真实像素区域较多, 这样可以训练出更加严格的判别器, 最终使得生成器生成的图片更加真实。模型在语义识别模块和生成器特征拼接前设置了遗忘门, 使得模型根据自己的需要选择适当的语义指导强度, 使得修复图像更加真实。我们在三个数据集上的大量实验证明了所提模型在像素级修复图片方面的可行性和有效性。 |
| 关键词: GAN 取证修复 图像语义识别 图像篡改 语义指导 |
| DOI：10.19363/J.cnki.cn10-1380/tn.2025.09.07 |
| 投稿时间：2023-09-18修订日期：2024-03-23 |
| 基金项目:本课题得到国家自然科学基金(No. 62072250, No. 62172435, No. U1804263, No. U20B2065, No. 61872203, No. 71802110, No. 61802212);中原科技创新领军人才项目(No. 214200510019); 江苏自然科学基金(No. BK20200750); 河南省网络空间态势感知重点实验室开放基金(No. HNTS2022002); 江苏省研究生研究与实践创新项目(No. KYCX200974); 广东省信息安全技术重点实验室开放项目(No.2020B1212060078); 山东省计算机网络重点实验室开放课题基金(No. SDKLCN-2-22-05); 人文社会科学教育部项目(No.19YJA630061) 国家重点研发计划项目(No. 2021QY0700)资助。 |
|
| A Forensic Restoration Image Model Based on GAN Network for Image Caption Guidance |
| WANG Jinwei,ZHANG Zihe,LUO Xiangyang,MA Bin |
| Engineering Research Center of Digital Forensics, Ministry of Education, Nanjing University of Information Science and Technology, Nanjing 210044, China;Department of Computer Science, Department of Software, Department of Cyberspace Security, Nanjing University of Information Science and Technology, Nanjing 210044, China;State Key Laboratory of Mathematical Engineering and Advanced Computing, Zhengzhou 450001, China;State Key Laboratory of Mathematical Engineering and Advanced Computing, Zhengzhou 450001, China;PLA Strategic Support Force Information Engineering University, Zhengzhou 450001, China;School of Cyberspace Security, Qilu University of Technology, Jinan 250353, China |
| Abstract: |
| The arrival of the big data era has made image data information on the network increasingly rich, and there are also more and more tampered images. In order to distinguish between tampered images and original images, image tamper detection methods are emerging one after another. Although tamper detection can locate the tampered area of an image, it still cannot understand the original image and the semantic information expressed by the original image. To address the aforementioned issues, this paper proposes a forensic restoration image model based on Generative Adversarial Network (GAN) image semantic guidance. For the first time in a tampering environment, image restoration was carried out using two tampering methods: copy move and concatenation tampering. This model utilizes GAN network adversarial training to obtain a trained generator for image forensics and restoration work. The model utilizes an image semantic recognition module to provide semantic features of the original image, guiding and constraining image restoration in a tampered environment. This ensures that the repaired image follows the semantics of the original image, and the generator does not excessively beautify the image, thus achieving the goal of restoring the real image. In the GAN network, this model concatenates the images repaired by the generator with corresponding position masks and then puts them into a discriminator, making the image have more real pixel regions. This can train a more rigorous discriminator, ultimately making the images generated by the generator more realistic. The model sets a forget gate before the semantic recognition module and generator feature concatenation, allowing the model to choose appropriate semantic guidance intensity according to its own needs, making the repaired image more realistic. Our extensive experiments on three datasets have demonstrated the feasibility and effectiveness of the proposed model in pixel level image restoration. |
| Key words: GAN forensic repair image caption image tampering semantic guidance |
