Lethe: Adapter-Augmented Dual-Stream Update for Persistent Knowledge Erasure in Federated Unlearning (arxiv.org)
arXiv:2601.22601v2 Announce Type: replace
Abstract: Federated unlearning (FU) aims to erase designated client-level, class-level, or sample-level knowledge from a global model. Existing studies commonly assume that the collaboration ends with the unlearning operation, overlooking the follow-up situation where federated training continues over the remaining data. We identify a critical failure mode, termed knowledge resurfacing, by revealing that continued training can re-activate unlearned knowledge and cause the removed influence to resurface in the global model. To address this, we propose Lethe, a novel federated unlearning method that de-correlates knowledge to be unlearned from knowledge to be retained, ensuring persistent erasure during continued training. Lethe follows a Reshape--Rectify--Restore pipeline: a temporary adapter is first trained with gradient ascent on the unlearning data to obtain magnified updates, which are then used as corrective signals to guide layer-wise rectification of the remaining updates in two streams. Finally, the adapter is removed, and a short recovery stage is performed on the retained data. Our experiments show that Lethe supports unlearning at all levels in federated systems in a unified manner and maintains superior persistence, with a resurfacing rate below 1% in most cases, even after numerous rounds of follow-up training.
Abstract: Federated unlearning (FU) aims to erase designated client-level, class-level, or sample-level knowledge from a global model. Existing studies commonly assume that the collaboration ends with the unlearning operation, overlooking the follow-up situation where federated training continues over the remaining data. We identify a critical failure mode, termed knowledge resurfacing, by revealing that continued training can re-activate unlearned knowledge and cause the removed influence to resurface in the global model. To address this, we propose Lethe, a novel federated unlearning method that de-correlates knowledge to be unlearned from knowledge to be retained, ensuring persistent erasure during continued training. Lethe follows a Reshape--Rectify--Restore pipeline: a temporary adapter is first trained with gradient ascent on the unlearning data to obtain magnified updates, which are then used as corrective signals to guide layer-wise rectification of the remaining updates in two streams. Finally, the adapter is removed, and a short recovery stage is performed on the retained data. Our experiments show that Lethe supports unlearning at all levels in federated systems in a unified manner and maintains superior persistence, with a resurfacing rate below 1% in most cases, even after numerous rounds of follow-up training.
Comments