Introduction
UK participation in Rubin Observatory Commissioning
Rubin Observatory Science Advisory Committee
LSST:UK Science Working Group
The LSST:UK Executive committee have reconstituted the Science Working Group, with a term of office defined as the duration of the Phase B work. The inital SWG was a large body which was charged with raising the awareness of LSST survey data within the UK community an across other projects with major UK interest. We felt that it had achieved its goal, with the UK’s involvement in LSST now widely known and we have become embedded in the LSST Science Collaborations, holding four leadership roles within the LSST Science Collaborations now. The SWG is made up of “Points of Contact” which are related to the Science Collaborations and other UK projects which will jointly exploit LSST data. The roles of the POCs are to actively help review the Phase B work being carried out and commenting if the science requirements are being met or now. They will also provide a link and insight into the ongoing processes within the collaborations. Some have already contributed to early Phase B work reviews and we thank them for their efforts - /wiki/spaces/LUSCSWG/pages/770310145 We encourage contact with these scientists if you are just starting out in LSST related work.
Running the LSST pipeline stack
Over the past few months I have been working to get the LSST software pipeline up and running for processing the images generated by DESC's Data Challenge 2 on UK compute resources. I installed the software on the CSD3 system at Cambridge via the newinstall.sh script and initialised a data repository using calibration data supplied by DESC. I then worked through each stage of the processing pipeline, documenting the process thoroughly in D3.10.5 - Processing DC2 data using the LSST DM Stack on UK Facilities. I was assisted greatly in this by earlier work done by James Mullaney and Darren White.
For ease of installation and use, it is highly desirable to make use of container and workflow technologies. Using container images greatly simplifies the installation of the software and ensures that the correct version of each component is present, while workflow management tools can automate the process of invoking each stage of the pipeline at the right time and on the right data, something which can otherwise be an arduous process when working with large data sets.
A workflow for this purpose is currently under development by Ben Clifford, using the Python-based Parsl workflow management system. This takes care of automatically submitting tasks to a batch system whenever their inputs become available, the software running within a container. Initially this workflow had only been tested on US systems, using Shifter to run container images built by DESC, but I have been working with Ben over the past few weeks to get it running at Cambridge. This necessitated switching to Singularity to run the containers.
The main problem encountered was an MPI error which occurs in many of the pipeline stages on the Cambridge system, believed to be due to an incompatibility between the version of MPI within the container images and the version installed on CSD3. However, the MPI functionality within the LSST stack is not actually required here, since the intention is for all of the parallelism to be handled at a high level by Parsl rather than within the individual jobs. I was able to get the workflow running successfully by replacing the "process pool" module used in the stack with a "fake" version that runs everything serially instead. A better solution, however, is to avoid using the high level "driver scripts" within the stack and instead directly call lower level components that do not use MPI, since this gives a finer granularity of task for Parsl to work with, and this is currently being implemented by Ben Clifford.
2020 Rubin Observatory Project and Community Workshop
The Covid-19 pandemic has necessitated some changes to arrangements for the up-coming Rubin Observatory Project and Community Workshop. This year’s even, which will run during 10th–14th August, will now be run entirely online and this means it is potentially easier for UK-based astronomers to participate in some or all of the sessions.
The Workshop website is now online and includes both the programme for the event and registration instructions. Each day, the programme runs during 5pm--9pm BST. While outside of the usual working day, the timing potentially gives people the option to attend those presentations/ discussions that are particularly interesting to them.
Recent Appointments
We are pleased to announce that Alastair Edge (Durham) has been appointed to the position of Deputy Chair for the LSST:UK Board. Alastair commented “I am looking forward to putting my experience on the Executive Committee to good use”.
Full details of Board Membership and other project appointments is always available on the LSST:UK public website.
The 2020 mid-year call for LSST:UK Junior Associates was open during 23rd June–6th July. Applications are currently being assessed by the LSST:UK Selection Committee and we hope to confirm the outcome to applicants before the end of July.
Recent LSST:UK outputs
LSST:UK has recently produced the following technical reports.
Title | Author | Description |
|---|---|---|