One of the recurring themes at CMIMI were attempts at implemented federated or distributed learning. The motivation was simple: need lots of data, but HIPAA. Therefore let’s learn without giving up data. Different techniques: Coefficient averaging | Round robin | Gradient contrib Gradient contrib is reminiscent of the computational argument…
But it also raised the possibility of “hyper-federated” learning.
Which raises the question what other architectural problems can be solved like federated learning?