Environment and Climate Change Canada

Tue, 24 Mar 2026 00:00:00 +0000

Introduction

The training of a neural network is the most cost-consuming part of scientific machine learning. The optimizers are algorithms used to adjust model parameters to minimize the loss function. The optimizers determine the efficiency of the training approach and the quality of the resulting model. No one single optimizer is fit for all problems.

The PARADIS model is a data-driven model developed by Environment and Climate Change Canada (ECCC) for medium-range global weather forecasting. The goal of this project is to compare various optimization protocols during the training of the PARADIS model.

Data set

Shallow Water Equations

(SWE) Due to the size of the ERA-5 global weather dataset, it is more suitable to use the data generated by SWE for this project.

Optimizers for comparison

AdamW
SGD
Muon
etc.

Metric of comparison

Efficiency: the rate of convergence
Quality: The loss of the optimized point, the spectral of the optimized point.