Environment and Climate Change Canada

Introduction

The training of a neural network is the most cost-consuming part of scientific machine learning. The optimizers are algorithms used to adjust model parameters to minimize the loss function. The optimizers determine the efficiency of the training approach and the quality of the resulting model. No one single optimizer is fit for all problems.

The PARADIS model is a data-driven model developed by Environment and Climate Change Canada (ECCC) for medium-range global weather forecasting. The goal of this project is to compare various optimization protocols during the training of the PARADIS model.

Data set

Shallow Water Equations

(SWE) Due to the size of the ERA-5 global weather dataset, it is more suitable to use the data generated by SWE for this project.

Optimizers for comparison

• AdamW
• SGD
• Muon
• etc.

Metric of comparison

• Efficiency: the rate of convergence
• Quality: The loss of the optimized point, the spectral of the optimized point.

N.B. Students are required to have access to GPU for training.