Federated learning / Concept drift

When data changes, how should federated learning adapt?

A visual guide to helping distributed models keep up with changing data.

2-minute overview · Bhargav Ganguly · With Vaneet Aggarwal
Based on our 2023 IEEE/ACM Transactions on Networking paper

The takeaway

Data changes. Models become stale. Our MASTER-FL framework helps distributed learning systems detect change and decide when to restart—without knowing future shifts in advance.

01 / The problem

Devices can learn a shared model without pooling their raw data. But when new classes appear or labels change, yesterday’s model may no longer fit today’s task. Simply continuing the same training run can miss that change.

02 / The idea

Train at different time scales. Look for change. Restart when needed. MASTER-FL coordinates short and long training runs, using their losses to test whether the environment has shifted. It works with established optimizers such as FedAvg and FedOMD.

Training runs of different lengths pause and resume on a shared timeline. — **Keep several time scales available.** Each line is a training run; bold segments show when it is active. Figure 1 from the paper.

A fresh training run reaches a lower loss than the reference estimate, revealing a change in the environment. — **A fresh run can reveal what an older model misses.** The green curve falling below the blue reference provides evidence of change. Figure 2 is a conceptual illustration, not measured results.

03 / Does it help?

We tested 20 clients over 500 training rounds on two classification datasets, introducing new classes and swapping labels. In these experiments, both MASTER-FL variants achieved higher average accuracy than FedNova and FedProx.

How to read the plots

Lower loss is better. Spikes reflect changes in the data. Follow the red and green MASTER-FL curves against the pink and yellow baselines.

Covtype loss curves when new classes appear. — **Covtype · New classes appear.** Figure 3(a).

Covtype loss curves when class labels swap. — **Covtype · Labels swap.** Figure 3(b).

MNIST loss curves when new classes appear. — **MNIST · New classes appear.** Figure 3(c).

MNIST loss curves when class labels swap. — **MNIST · Labels swap.** Figure 3(d).

Select any figure to open its full-size view.

78.0% vs 64.4%

Average accuracy for MASTER-FL + FedOMD versus FedNova on MNIST with label swaps: 13.6 percentage points higher in this experiment. Source: Table II.

Why this work matters

It connects a practical question—when should a model adapt?—to an algorithm and a mathematical performance guarantee. The contribution spans distributed ML, change detection, and online optimization.

Scope: These are controlled experiments, not production results. The guarantees assume convex losses and depend on how much the environment changes; they do not automatically extend to arbitrary deep networks.

Technical context & sources

Joint work by Bhargav Ganguly and Vaneet Aggarwal. The analysis bounds dynamic regret: cumulative loss relative to the best model at each round. Sublinear regret requires sufficiently limited change, alongside the paper’s bounded, Lipschitz, convex-loss and base-optimizer assumptions.

The experiments do not report uncertainty intervals in Table II. Keeping raw data local is not itself a formal privacy guarantee; detailed privacy-preserving loss computation is left to future work.

Figures 1 and 3 were extracted from the supplied arXiv v2 manuscript (6 May 2023), pages 5 and 10. Figure 2 uses the existing illustration. See Sections III–IV, Theorem 3, and Section VI / Table II.