IEEE Transactions on Consumer Electronics · Research Project

Secure Federated LoRA Fine-Tuning of Small Language Models for Women’s Health Support

A privacy-preserving framework for menstrual phase classification that compares post-quantum-secure communication with client-level differential privacy.

Malithi Mithsara Wanniarachchi Kankanamge, Ning Yang, Mohd Farhan Israk Soumik, Ahmed Imteaj, Hussein Zangoti, and Abdur R. Shahid

Southern Illinois University · Florida Atlantic University · Jazan University

LOCAL CLASSIFICATION

Predicted phase

Luteal
Hormonal features
Physiological features
Self-reported symptoms
Raw data remains local
Only protected updates are shared
5,659records
18features
15participants
5backbone models
2privacy settings

Research motivation

Privacy-aware menstrual health intelligence for consumer devices

Menstrual health applications may process sensitive information related to bleeding, pain, mood, hormone levels, sleep, fertility, and reproductive health. The proposed system keeps participant records local and shares only protected model updates.

01

Subject-level federation

Each participant is treated as an independent federated client to avoid centralizing raw records.

02

Lightweight adaptation

LoRA is used for decoder models, while DistilBERT is trained as a sequence classifier.

03

Deployment evaluation

Accuracy, F1, latency, throughput, memory, training time, communication, and privacy are measured.

Proposed framework

Two privacy-preserving federated learning settings

Proposed PQC and differential privacy federated learning framework
Participant-level data preparation, local LoRA fine-tuning, protected client updates, and server aggregation.
PQC

Post-quantum-secure communication

Protects global adapters and local client updates during transmission.

  • ML-KEM key establishment
  • HKDF-SHA256 key derivation
  • AES-256-GCM encryption and authentication
  • FedAvg after server-side decryption
DP

Client-level differential privacy

Limits information leakage from each participant’s model update.

  • L2 clipping with C = 0.5
  • Gaussian noise multiplier σ = 0.1
  • δ = 10⁻⁵
  • RDP accounting for accumulated ε

Experimental results

Compare models across utility and deployment cost

Select a privacy setting and model to view the reported results.

ACCURACY

84.32%

DistilBERT-base · PQC

Best overall balance of classification utility and deployment efficiency.

Macro-F184.22%
Weighted-F184.68%
Rounds16
Training0.1 h
Latency1 ms/ex.
Throughput774.9 ex./s
Peak GPU1.0 GB
Ciphertext/round0.26 GB
Efficiency comparison across models
Training time, inference latency, throughput, and peak GPU memory under PQC and DP.
ModelSettingRoundsAccuracyMacro-F1Weighted-F1TrainingLatencyPeak GPU

Privacy–utility trade-off

PQC and differential privacy protect different risks

PQC

Preserves model utility because updates are encrypted rather than perturbed, but communication cost grows with adapter size.

Differential privacy

Limits update-level information leakage through clipping and noise, but may reduce accuracy or increase training and memory cost.

Main finding

DistilBERT maintains 0.8432 accuracy under both settings while requiring much less time, memory, latency, and communication.

Important privacy note

The reported differential privacy ε values are large because a small noise multiplier was selected to preserve utility. These experiments should not be interpreted as providing strong worst-case privacy.