Eradiate Winter News

The Eradiate team wishes you a merry Christmas and a happy New Year!

In our last post, we summarised the work accomplished so far on Eradiate for the MetEOC-3 project. Work for the project is still going on, with now a focus on the general design of the radiative transfer model. In this post, we will present the current state for the design of the Eradiate.

The Eradiate team needs extra workforce #

Rayference, Brussels-based company, is hiring two staff members to support the development of Eradiate:

• 3D Radiation Transfer Modelling Expert: contribution to design and development of the model, physical property database preparation.
• Software engineer: contribution to design and development, parallel computing, continuous delivery.

Do not hesitate to apply or forward the job descriptions to your contacts!

Ongoing activities #

An ambitious project such as Eradiate requires careful thinking to ensure that the resulting software package possesses the qualities we want it to have. Multiple discussions with users and the gathering of requirements convinced us that completeness, accuracy and performance will not be sufficient to make Eradiate a tool useful to the Earth observation community; Eradiate must to be high-quality open-source software, which has implications often overlooked when writing scientific software.

• High-quality software architecture. The code should be comprehensive, easy to maintain, extend and amend. We are here referring to the way the code is structured and decomposed: the quality of a software package’s architecture can be evaluated without looking at the code. It should even, ideally, be designed prior to writing code. In practice, however, writing some code greatly helps during the architectural design phase. This is the part of the design process we are working on at the moment, and which we will present in a few months when we are happy with the design.
• Intuitive interface. Many users complain about impractical interfaces and obscure input and output file formats. We know for a fact that manipulating the model must be easy, and we put a lot of care in the design of a multi-layer interface with which both users and developers should have a good time using Eradiate. We converged a few months ago to a solution, which has slightly evolved since we first presented it.

In the following, we will further elaborate on the current interface concept foreseen for Eradiate.

Current interface model #

The user requirement elicitation process allowed us to draft a user interface concept taking advantage of coupling possibilities between the C++ and Python programming languages. Further research allowed us to refine our design and propose a solution based on a three software component types [Application, API, Library], two programming languages [Native, Scripting] and two abstraction levels [Low, High].

Component types. We distinguish Library, API and Application components.

• Library components contain the core routines of the model. They contain code which allows building and manipulating scenes (i.e. defining shapes and attaching them radiative properties), defining observations, running radiative transfer equation (RTE) integration algorithms and collecting results. Not all the details of the implementation of these components need to be accessible to application developers: some utility components are purely internal facilities and only maintainers of the model need to use them. This is why access to the library components will be granted to application developers through an application programming interface (API).
• The application programming interface (API) is necessary to provide application developers with a set of programming entities sufficient to write applications while keeping library internals private. The API serves as a bridge between library and application components and will be structured in an intuitive way.
• Application components are intended to be used by end-users and will run the RTM with varied degrees of flexibility. They can be run from a text or graphical shell, and they can range from a preset 1D model for the evaluation of upwelling and downwelling fluxes in a standard atmosphere model, to a flexible, multi-purpose 3D radiative transfer simulator for arbitrary scenes. One of the design goals is to use the same library components to build all types of applications.

Programming languages. Eradiate is structured around two programming languages, both of have specific advantages and disadvantages.

• C++ is the native language. It implements multiple programming paradigms which encourage abstraction and ease code debugging, while still allowing for low-level optmisation.
• Python is the scription language. It does not require compilation, also implement multiple programming paradigms and interfaces very easily with C++. A very strong argument in favour of Python is that it is widespread in scientific communities, which should make Eradiate easier to integrate in the workflow of many users. Finally, the interpreted nature of Python makes it easy to integrate Eradiate into an interactive environment, which is a definite advantage when it comes to teaching software usage and investigating with the model.

Abstraction levels. We define two abstraction levels: Low and High.

• Low-level components use the least abstract concepts. A low-level description of a tree in the scene, for instance, consists in placing its trunk and leaves one by one, providing the shapes in the simplest form allowed by the model. Low-level components are directly used by the model when the RTE integration process starts.
• High-level components are more abstract and are eventually broken down into low-level components for RTE integration. The aforementioned tree, at the high level, is simply designated as “a tree at a certain position,” and the model then takes care by itself of breaking it down into low-level components.

Interface overview. Now that the key concepts are defined, we get a clearer view on the interface concept summarised by the figure below.

The native layer of the model is written in C++. The native library components are the core of the model and implement basic scene and observation setup, as well as radiative transfer equation integration algorithms. Selected library components are exposed through the native API, so that application developers can write native applications with great flexibility. It is worth noticing that native implementation of high-level features should be avoided: we plan to leave that to the scripting layer.

The scripting layer, written in Python, connects to the native layer through a low-level API which is planned to be as close as possible to the native API. It then becomes possible to build scripting library components using this API, which can then be exposed by enriching the scripting API, possibly with high-level components. For instance, the scripting library may include a tree generator, built with basic scripting API elements, themselves calling directly the native API. The tree generator is then exposed through the scripting API. Both low- and high-level components can then be used to build scripting applications, tailored for specialised applications.

The scripting API is a critical component for the usability of the model. It will be designed with interactive usage in mind: Eradiate will be easy to embed in Jupyter Notebooks.

What is next? #

With this interface concept in mind, we are now working on the native architecture, getting inspiration from multiple Monte Carlo ray tracing software packages from both physics and computer graphics communities. We hope to be ready with a concept soon! Stay tuned for more updates, and if you have not yet, be sure to subscribe to our mailing list!