DEEP Project series
Official website: https://www.deep-projects.eu/
The DEEP projects is a series of research projects in which, amongst many individual contributions, the Modular Supercomputing Architecture (MSA) has been developed.
Around 2010, we at JSC and our cluster partners started considering how to close the gap between general purpose clusters and highly-scalable systems. The first heterogeneous node designs were appeing, consisting of CPUs and accelerators, but as far as graphics cards (GPU) were concerned, they could only be used as coprocessors, since CPUs and GPUs are statically assigned to each other. We realized that it would be much more beneficial to take the accelerators apart from the node as an autonomous unit and flexibly assign them to the CPUs. We made these ideas the basis of the proposal for the first DEEP project, which was approved and launched end of 2011.
As part of DEEP, our first cluster-booster prototype was built with an Intel Xeon-based general-purpose cluster running on a Mellanox network, connected through a specific network interface with the EXTOLL network of a 384-node system of Intel Xeon Phi processors (KNC); the latter we referred to as the "booster." In parallel, the full software stack was developed around the ParaStation cluster middleware that eventually led to the ParaStation Modulo system we enjoy today, and quite a few initial application use-cases were adapted and tested. In the follow-up DEEP-ER project, we continued work on the DEEP pilot architecture and its environment, focusing in particular on improving input-output and resilience capabilities.
Building on these successful results, the DEEP-EST project that ran from 2017 to 2021 generalized the Cluster-Booster concept in order to address the requirements of a wider variety of applications. The resulting Modular Supercomputing Architecture (MSA) developed in DEEP-EST is designed to meet the requirements of both large-scale simulations traditionally run on HPC systems and data-intensive workloads from the field of artificial intelligence like deep learning.
The continuos further development of MSA proceeds within the SEA Projects, which run from 2021 to 2024:
JSC contribution
The JSC is leader and coordinator of all the four DEEP projects and leads the Design and Development Group (DDG), in which the main technical decissions of the projects are taken. Furthermore, contributions from JSC are done in the following areas, amongst others: infrastructure preparation, prototype hosting and operation, benchmarking, monitoring, performance analysis, application support, application porting, codesign, dissemination.
Contact Person at JSC
Prof. Dr. Estela Suarez
Co-Lead of JSC-Division "Novel System Architecture Design"
- Institute for Advanced Simulation (IAS)
- Jülich Supercomputing Centre (JSC)
Room 222
Acknowledgements
The DEEP Projects have received funding from the European Commission's FP7, H2020, and EuroHPC JU Programmes, under Grant Agreements n° 287530, 610476, 754304, and 955606. The EuroHPC Joint Undertaking (JU) receives support from the European Union’s Horizon 2020 research and innovation programme and Germany, France, Spain, Greece, Belgium, Sweden, United Kingdom, Switzerland.
