| 引用本文： |
-
魏建好,李一骁,张锦,赵明,刘利枚,吴鑫,叶松涛.时空相关性驱动的多模态社交推荐隐私保护[J].信息安全学报,已采用 [点击复制]
- Weijianhao,Liyixiao,Zhangjin,Zhaoming,Liulimei,Wuxin,Yesongtao.Spatiotemporal Correlation-driven Privacy-preserving Multimodal Social Recommendation[J].Journal of Cyber Security,Accept [点击复制]
|
|
| 摘要: |
| 多模态社交推荐系统通过收集和分析多模态数据为用户推荐感兴趣的活动。然而，该系统在广泛应用的同时，也面临严峻的社交数据隐私泄露问题。现有方法主要侧重于保护单模态数据隐私，忽视了多模态社交数据之间和深度学习模型参数之间的时空相关性所带来的潜在隐私风险。因此，本文提出一种云-边-端框架下时空相关性驱动的多模态社交推荐隐私保护(SPMR)方案，在保护多模态数据时空相关性和预训练模型梯度双重隐私的同时，实现高精度的多模态社交活动推荐。具体而言，在终端层引入一种基于避免模态松弛的多模态时空对比学习隐私保护(MPMC)算法，降低模态松弛带来的干扰，挖掘多模态社交图的时空相关性特征并进行高效融合，采用拉普拉斯-高斯噪声聚合机制，有效保护多模态融合嵌入的时空特征隐私。其次，提出多层次多模态用户关联性增强(MUAE)算法，采用分层注意力机制构建模态内和模态间的时空关联矩阵，以增强多用户的交互特征。接着，设计基于自适应噪声衰减的梯度时空相关性保护(DGSP)方法，利用卷积循环神经网络挖掘梯度之间的时空关联性并添加自适应高斯噪声，确保预训练模型的安全性和可用性。最后，提出基于图神经网络的时空活动推荐(GSIR)算法，通过表征用户-活动和用户-用户的时空关联关系，实现高精度的多模态社交活动推荐。同时，通过安全性分析和性能分析证明了本文SPMR方案具有较高的隐私保护水平和有效性，在真实数据集上的大量实验表明，本文提出的SPMR方案在保护多模态时空相关性隐私的同时，其推荐精度比现有先进方法DGNN的性能提高5.92%。 |
| 关键词: 多模态时空相关性 社交推荐隐私保护 多模态关联增强 模型梯度相关性 |
| DOI： |
| 投稿时间：2025-02-09修订日期：2025-04-12 |
| 基金项目:国家自然科学基金项目（面上项目，重点项目，重大项目），教育部人文社会科学研究青年项目，湖南省自然科学基金项目，湖南省教育厅科学研究项目，湖南省教育厅教学改革研究重点项目，湖南省研究生科研创新重点项目 |
|
| Spatiotemporal Correlation-driven Privacy-preserving Multimodal Social Recommendation |
|
Weijianhao1, Liyixiao1, Zhangjin2, Zhaoming3, Liulimei1, Wuxin1, Yesongtao4
|
| (1.Hunan University Of Technology and Business;2.Changsha University of Science and Technology;3.Central South University;4.Xiangtan University) |
| Abstract: |
| Multimodal social recommendation systems suggest items of interest to users by collecting and analyzing multimodal data. However, while these systems are widely applied, they face significant challenges regarding social data privacy leakage. Existing methods primarily focus on protecting privacy in single-modal data, overlooking the potential privacy risks arising from the spatiotemporal correlations between multimodal social data and deep learning model parameters. Therefore, this paper proposes a spatiotemporal correlation-driven privacy-preserving multimodal social recommendation (SPMR) scheme within a cloud-edge-end framework, ensuring high-accuracy multimodal social item recommendations while preserving both the spatiotemporal correlation privacy of multimodal data and the gradient privacy of pre-trained models. Specifically, we introduce a modality slackness avoidance-based privacy-preserving multimodal spatiotemporal contrastive learning (MPMC) algorithm at the end layer to reduce interference from modality slackness, efficiently extract and integrate spatiotemporal correlation features of multimodal social graphs, and apply a Laplace-Gaussian noise aggregation mechanism to effectively protect the privacy of spatiotemporal features in multimodal fusion embeddings. Furthermore, we propose a multilevel multimodal user association enhancement (MUAE) algorithm that constructs spatiotemporal correlation matrices both within and across modalities using a hierarchical attention mechanism to enhance multi-user interaction features. Next, we design an adaptive decay-based gradient spatiotemporal correlation protection (DGSP) method that utilizes convolutional recurrent neural networks to mine spatiotemporal correlations between gradients and add adaptive Gaussian noise, ensuring the security and usability of the pre-trained model. Finally, we introduce a GNN-based spatiotemporal item recommendation (GSIR) algorithm that characterizes the spatiotemporal relationships between users-item and users-users, achieving high-accuracy multimodal social item recommendations. Additionally, through security and performance analysis, we demonstrate that the proposed SPMR scheme offers a high level of privacy protection and effectiveness. Extensive experiments on real-world datasets show that the proposed SPMR scheme not only protects the privacy of multimodal spatiotemporal correlations but also improves recommendation accuracy by 5.92% compared to the state-of-the-art method DGNN. |
| Key words: multimodal spatiotemporal correlation privacy-preserving social recommendation multimodal association enhancement model gradient correlation |
