One more step towards transparency and traceability

The development of Eradiate started two years ago with the idea of building a new radiative transfer framework designed for accuracy and usability. One of our goals is to have Eradiate recognised as a reference in terms of numerical accuracy. For that purpose, benchmarking and comparison are of prime importance: model intercomparison is how we document the strengths and weaknesses of our models and are an excellent opportunity to test and improve simulation code. A significant part of our development work consists in comparing our results with the ones produced by other models.

The RAMI benchmark series #

Initiated in 1999, the Radiative transfer model intercomparison (RAMI) series benchmarks models designed for the simulation of radiative transfer at the Earth’s surface. Test cases feature scenes reproducing vegetated surfaces and soil, and atmospheric effects are not accounted for. Since 1999, the European Commission’s Joint Research Centre has launched and closed four phases of RAMI, each one building on the previous and offering more advanced test cases. Results have been published in peer-reviewed journals, and these publications notably document how the models which participated in each phase performed.

The RAMI test cases aim at testing models against three-dimensional radiative effects due to the morphological properties of vegetated surfaces. Many models contributed to the comparison exercise during its successive phases. Although still in beta version, Eradiate is currently participating in the fifth phase of RAMI, launched in November 2020 and closing in September 2021.

ROMC: RAMI outside of RAMI #

While intercomparison phases are a good opportunity to assess radiative transfer model performance at a certain time, code under development may also require comparison. A “highly available” counterpart of RAMI, the RAMI online checker (ROMC), uses RAMI-3 scenarios and provides an online interface to submit simulation results and compare them with a reference established as an outcome of RAMI-3, outside of any RAMI submission phase. Eradiate’s results are routinely compared to the ROMC reference, and we provide below a few examples using ROMC’s Debug mode.

Beyond vegetation with RAMI4ATM #

Despite being routinely used in different Earth observation workflows (e.g. calibration, lookup table generation, inversion), radiative transfer models are much less compared in situations reflecting real-world usage. How a radiative transfer model performs, in terms of accuracy or resource usage, is not only dependent on the numerical methods and models it implements, but also on how it is used and the input it is given. This can lead to situations where models behaving very well when used to solve “well-posed” (i.e. unambiguous, perfectly specified) problems will yield significantly less good results when used for a practical application.

While this is not surprising, it is problematic: model accuracy is critical in real-world applications, but we have little idea of how well we can trust radiative transfer models in those situations. A special RAMI4ATM phase of RAMI will try to contribute to addressing this currently open issue. RAMI4ATM notably introduces atmosphere-surface coupling, which is one of the key features on the Eradiate roadmap. It is therefore only logical that Eradiate will participate in RAMI4ATM when submission will open.