Overview

Summary

WOODS has 2 model selection methods based on DomainBed (Gulrajani and Lopez-Paz, 2021):

• Train Domain Validation: We split each trainin domain into training (in) and validation (out) subsets. We tran models using the training subset and choose the model maximizing the accuracy on the union of validation subsets. We then report the test accuracy of the chosen model on the test domain.

• Test Domain Validation: We choose the model maximizing the accuracy on a validation set that followes the distribution of the test domain. We allow one query (the last checkpoint) per choice of hyperparameters, disallowing early stopping. This breaks the OOD generalization assumption that models does not have access to the test domain, but this metric is still insightful as to performance of different objectives.

---

Spurious Fourier

---

Temporal Colored MNIST Sequences

---

Temporal Colored MNIST Steps

---

CAP

---

SEDFx

---

PCL

---

HHAR

---

LSA64