Instruction: Explain how to effectively benchmark Federated Learning algorithms, including key metrics and considerations.
Context: This question probes the candidate's knowledge of evaluating and comparing the performance of Federated Learning algorithms, emphasizing the importance of comprehensive benchmarking practices.
Official answer available
Preview the opening of the answer, then unlock the full walkthrough.
I would benchmark federated algorithms across both model and systems dimensions. That means tracking accuracy or loss, client-level performance spread, communication rounds, total bytes transferred, time to convergence, robustness under non-IID data, and failure behavior under dropout...