DtSc Course Note
Pi-model & Temporal Ensemble
Nvidia team 2017 propose this temporal ensemble framework. The famous contrastive learning framework, SimCLR, is also one of this kind. {Medium Pi-model Temporal Ensemlbe Distinguish}{ICLR ArXiv Page Pi Model}.
Features I have captured so far:
- Partially labeled dataset
- Augment input with stochastic Gaussian noise.
- Momentum in loss function weighting the supervised (accuracy) and unsupervised (stability)
- Prediction from previous epoch/step involves the unsupervised part of the current training epoch.
pi-model
temporal-ensemble
!
statistics
Besides mean and median:
MAD = median-absolute-deviation
$$MAD=median(|X_{i}-\bar{X}|)$$
sklearn ensemble learning pipeline
Official learning materials that compares different encoders and use gradient boost ensemble learning https://scikit-learn.org/stable/auto_examples/preprocessing/plot_target_encoder.html#sphx-glr-auto-examples-preprocessing-plot-target-encoder-py