A Python distributed test don't exit properly when one rank fails.

[wujingyue]

This is apparently caused by the use of MPI_barrier. Because each test waits for a barrier before finishing, the failed rank(s) won't enter the barrier, causing other ranks to hang.

[wujingyue]

This appears to be a known limitation of mpi4py: https://mpi4py.readthedocs.io/en/latest/mpi4py.run.html#exceptions-and-deadlocks. The solution proposed there is to python -m mpi4py. But I'm having troubles doing that with pytest.

My best bet so far is to expose Communicator to nvFuser's Python API and then a test fixture. Communicator::barrier doesn't suffer from this AFAIK. In addition, Communicator can host some common features between C++ tests and Python tests, e.g., #3091.