Abstract
Retinal blood vessel segmentation is crucial for early detection of diseases such as diabetic retinopathy, glaucoma, and retinal vein occlusion. Centralized deep learning approaches raise privacy concerns and risk violating regulations like Health Insurance Portability and Accountability Act (HIPAA) and General Data Protection Regulation (GDPR). Federated learning (FL) enables collaborative training without raw data sharing but remains vulnerable to gradient leakage, model poisoning, and communication interception. Moreover, the widely used FedAvg aggregation algorithm often suffers from slow convergence, oscillations, and reduced performance in the presence of heterogeneous client data. Researchers have proposed many U-Net variants for medical image segmentation; however, existing models often fail to balance multi-scale feature extraction, long-range context modeling, and fine vessel continuity, leading to suboptimal results. To address these challenges, we propose FedFusion-U-Net, a privacy-preserving federated U-Net variant integrating: (i) Inception modules for multi-scale feature extraction, (ii) Ghost modules to generate efficient and less redundant feature maps,(iii) Residual connections for stable optimization, and (iv) Squeeze-and-Excitation blocks to recalibrate channel responses and emphasize vessel-specific features and (v) Pyramid Attention to capture long-range dependencies more effectively than conventional attention. These enhancements collectively improve continuity and recovery of thin vessels. Privacy is preserved through CKKS homomorphic encryption (HE), which secures transmission and aggregation of updates. For global aggregation, we employ the FedAvgM algorithm, an extension of FedAvg that incorporates server momentum, leading to faster and more stable convergence compared to vanilla FedAvg in heterogeneous environments. Experimental results demonstrate that FedFusion-U-Net with FedAvgM outperforms baseline U-Net models in accuracy, vessel continuity, and robustness, while ensuring privacy and regulatory compliance. This highlights the effectiveness of combining advanced U-Net modules, CKKS HE, and FedAvgM aggregation in a secure and collaborative framework for medical image segmentation.