});

DtSc Course Note

Pi-model & Temporal Ensemble

Nvidia team 2017 propose this temporal ensemble framework. The famous contrastive learning framework, SimCLR, is also one of this kind. {Medium Pi-model Temporal Ensemlbe Distinguish}{ICLR ArXiv Page Pi Model}.
Features I have captured so far:

  1. Partially labeled dataset
  2. Augment input with stochastic Gaussian noise.
  3. Momentum in loss function weighting the supervised (accuracy) and unsupervised (stability)
  4. Prediction from previous epoch/step involves the unsupervised part of the current training epoch.

pi-model

pi-model-pseudo-code
pi-model-flow-chart

temporal-ensemble

temporal-ensemble-pseudo-code
!temporal-ensemble-flow-chart

statistics

Besides mean and median:
MAD = median-absolute-deviation
$$MAD=median(|X_{i}-\bar{X}|)$$

sklearn ensemble learning pipeline

Official learning materials that compares different encoders and use gradient boost ensemble learning https://scikit-learn.org/stable/auto_examples/preprocessing/plot_target_encoder.html#sphx-glr-auto-examples-preprocessing-plot-target-encoder-py

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.