diffdist

github.com ↗

diffdist is a python library for pytorch. It extends the default functionality of torch.autograd and adds support for differentiable communication between processes.