As part of Phase B WP3.10's contribution to DESC, I have been working on porting the GalSim sensor model to GPU over the past several months. GalSim is the low level library that forms the core of the ImSim image simulation code used extensively in DESC, and we have previously contributed various improvements to this library, including parallelising the main photon accumulation loop using OpenMP, and modifying how the sensor pixel geometry is stored.

I have added GPU acceleration to the photon accumulation and pixel boundary update loops using OpenMP Target Offload. This same technology is being used elsewhere in DESC's image simulation pipeline, and it meets DESC's requirement for a portable, platform-neutral GPU solution. It uses a directives-based approach requiring relatively little modification to the original C++ code. However, OpenMP Target Offload is still quite immature, and I encountered several problems (mostly compiler related) that had to be worked around. The porting work was done on NERSC's Perlmutter GPU system, though I am also now testing on Cirrus at EPCC, using the Clang compiler.

The porting is now complete and after debugging I am able to run the entire GalSim sensor test suite on GPU. The performance is better than expected, with the main accumulation loop running on average 3.9x faster on GPU than on CPU across all tests. However, when the photon arrays are already stored in GPU memory, this increases to 18x faster. The eventual aim is to integrate this work with other parts of the simulation workflow so that the photon data can be generated on the GPU and fed straight into the sensor model without ever having to leave GPU memory, and it appears that this may unlock a potentially large speed up.