Consistency-Training-with-Supervision

Contains experimentation notebooks for my Keras Example Consistency Training with Supervision. This example also provides a template for performing semi-supervised / weakly supervised learning.

Promising results on CIFAR-10-C with the process shown in the example:

More things one can incorporate:

• Incorporate more data during training the student.
• Filter high confidence predictions from teacher during training the student.
• Use recipes like Stochastic Depth for training the teacher. The current example uses Stochastic Weight Averaging to induce geometric ensembling.

Full-scale experiments are available here.

Acknowledgements

• ML-GDE program for providing GCP credits.