WP1.2 Maintenance of Science Requirements Document

With the LSST:UK in-kind proposal being approved in the first instance by the Rubin Management Board, a new version of deliverable D1.2.3 “Science Requirements Document (Mar '22)” has been generated.

WP1.4: Coordinating of LSST:UK Contributions to Commissioning

The proposals submitted to the call for UK commissioning contributions were reviewed by the LSST:UK Executive Committee and forwarded to the Rubin Commissioning Team (SIT-Com) in November. Onboarding of staff from successful proposals started in April 2022.

WP1.7: International Program Coordinator.

Aprajita Verma, joined the Rubin Operations Executive in October'21 as the In-kind Programme (IPC) team lead. Aprajita has also continued to manage the Rubin IPC team, supporting the international in-kind community and managing the Rubin general pool of software development resource. Other highlights over this reporting period are the development of the in-kind contribution viewer and the in-kind program tracking tools.

WP 2.1 DAC Management

The DAC team agreed a new and streamlined process with the Observatory for periodically updating the LSST:UK list of data rights holders. Discussions have also begun with LIneA (Brazil), who are also going to run an IDAC. The aim of these is to identify areas of collaboration and information sharing, and seed a wider cooperation between IDACs. Finally, the DAC team have proposed a Data Facilities transition plan to the two other operational Data Facilities (SLAC, USA and CC-IN2P3, France).

WP2.2 Data Ingestion and Publication

The DAC team have successfully completed a major upgrade of the DAC catalogue service (Qserv). The DAC team have also, in collaboration with colleagues in CC-IN2P3, deployed a 10-node instance with the latest software versions and enough capacity for Data Preview 0.2 and relevant ancillary surveys. On a related note, the DAC team have successfully run the Qserv Ingest workflow to ingest a third-party survey -- specifically, the CatWISE survey, which is a precursor survey for WP3.11.

WP2.3/3.2 Lasair

For the Lasair team the major highlights in this period have concerned the configuration and testing of new IRIS hardware and the subsequent migration of Lasair onto this new platform.

WP2.4 Provision of the DAC Platform

Members of the DAC team have maintained an operational UK-based Rubin Science Platform instance and the collaboration with Rubin (SQuaRE) team about standardising infrastructure support. The team have also completed an upgrade to Somerville (proto-DAC infrastructure) with a mix of IRIS and LSST:UK hardware funding.

WP2.5 Science Support

The DAC team have begun optimisation work on the Macauff cross-match software (produced by WP3.11) to make it production-ready. In addition, the DAC team now have agreement from Rubin Observatory to create a dedicated Lasair channel in the Community Forum, by way of a pilot for UK DAC science support longer term.

WP3.3: Spectroscopic classification of transients

The TIDES survey strategy project/publication was formally approved by DESC and the publication draft was made available to the working group. Chris Frohmaier presented the project at a DESC telecon. The 4MOST project approval for publication is underway with informal approval for a paper from the Strategy and Simulation working group chairs. We successfully created and submitted a catalogue of real-galaxies hosting mock SNe in the GAMA09/eFEDs fields as required by IWG1 in 4MOST. This real-target catalogue was performed by TiDES on a best effort basis to test infrastructure issues such as shared targets and fibre-collision within 4MOST observations. We coordinated with the other surveys in 4MOST in the construction of this catalogue. This effort contributes towards the 4MOST IWG1 WG3's goal of developing a pipeline to identify common targets between the different surveys - Chris Frohmaier is a key member of this WG. We developed a pipeline to create realistic spectra, as observed by 4MOST, following each strategy simulation. This pipeline gives us a clear understanding of the quality of each object’s spectrum according to the exposure time variations and weather conditions expected over the survey lifetime. The spectra have been shared with other groups within TiDES to enable secondary projects. We submitted the LSST:UK WP.3.3 deliverable relating to the TiDES target selection algorithm and software to analyse candidate targets from the real-time ZTF stream. We had to adapt the workflow as ZTF suffered a prolonged outage due to hardware issues. We paused development of our software to interface with the Lasair Kafka stream and instead changed to an analysis of historical targets within ZTF. Our pipeline now replicates a nightly flow of observations and analyses individual targets against a user defined and customisable selection function. This software is currently under code-review by LSST:UK and will be publicly released.

WP3.5 LSST and near-infrared data fusion

The WP3.5 team completed the transition of all code to the third generation Butler. All processing is now up to date with the latest version of the LSST Science Pipelines. We implemented a new confidence map into the variance plane within the LSST Science Pipeline such that exposure numbers and bad pixels are properly accounted for. We installed the gen3 workflow code developed for DESC allowing large gen 3 runs to be conducted on SLURM HPC queuing systems. We completed the first trial gen 3 run of the SXDS field and have started to test this latest prototype run “2022.1”. We tested the integration of the VISTA-HSC Butler in the UK based RSP allowing access to the full processed imaging and catalogue data sets.

WP3.6 Photometric Redshift Estimation

Work in this WP commenced at the end of January 2022. Thus far a survey of available simulation tools, current data products, and their connectivity and level of realism has been carried out as well as a first study of spatially varying observing conditions and their impact on photometry on existing simulation data. For a simulated 5-year survey, variability was found to be unexpectedly low, the reasons for which are still under investigation.

WP3.7 Low-surface-brightness science using LSST

The LSST Project has made Aaron Watkins a full member of the Rubin Algorithms and Pipeline team which is responsible for delivering the Rubin data products to the worldwide community. Aaron can now access the Algorithms and Pipeline team’s code base and development server and much of the work that we will do going forward will use these facilities. Aaron’s membership demonstrates the importance that the Algorithms and Pipeline team place on the sky subtraction work that is being performed by WP 3.7 which will be critical for LSST to access its discovery space. The Algorithms and Pipeline team now envisages the WP 3.7 sky subtraction package, when fully developed and tested on LSST data (e.g. from commissioning surveys such as DP2) to be the default sky subtraction algorithm for all LSST data releases. Aaron Watkins will continue to lead this effort, now with the significant resources placed at his disposal by the Project via his membership of the Algorithms and Pipeline team. Aaron Watkins and Sugata Kaviraj (WP 3.7 lead) are in the process of setting up a ‘Low-surface-brightness Coordination Group’. This will be the first community-led coordination group within the Rubin eco-system. This coordination group will solicit feedback from all science collaborations (particularly from their UK members) regarding the discovery science that the community wants to perform using LSST. This will ensure that the data products from WP3.7 benefit the largest number of LSST scientists and that our work is guided by what is actually needed from the sky subtraction to enable LSST science. Watkins and Kaviraj continue to give invited talks, both to working groups within science collaborations and also to external communities (e.g. Euclid) to present the sky subtraction work that is being performed within WP3.7 and highlight its significance to the LSST community.

WP3.9 LSST Point Spread Function, sensor characterisation and modelling

We have demonstrated the ability to calibrate our optical interferometer in the OPMD lab to an absolute distance of <200nm between two reflective surfaces. We have begun work on achieving this accuracy in the less reflective surfaces (i.e. CCDs) needed for the final PSF measurements. The measurements needed to calibrate our TCAD models of the CCD pixels were completed, and we can now proceed to begin tuning these models to the experimental measurements.

WP3.10 UK Contributions to DESC Operations

Over this reporting period, the WP3.10 team have: built a draft interface for TXPipe to the RAIL photo-z summarisation tools, which will enable DESC to generate number density distribution of galaxies; created a series of validation tests of TXPipe on CosmoDC2 and Gaussian simulation; built the first TXPipe module to use GPUs at NERSC; kicked off the STAR challenge to test their complete 3x2pt pipeline; completed GalSim pixel boundary code refactoring and optimisation; written a pure C++ GalSim test case for testing and benchmarking GPU port; and, ported the GalSim photon accumulate loop to GPU using OpenMP Target Offload.

WP3.11 Cross matching and astrometry at LSST depths

Over this period, the WP3.11 team have been optimising their code base as well improving and extending their algorithms. The code optimisation improved the efficiency of existing routines and they are working with WP2.5 to enable the code base to exploit multi-node, massive parallelisation. Algorithms have been improved by: the addition of further descriptions of effects on source positions; adding a model for unknown proper motions; and, including external galaxies in the perturbation astrometric model. The algorithms have been extended to include descriptions of transition from photon-dominated to background-dominated AUF algorithms to an all-sky model, allowing for much more robust probabilities of match between sources both in the Galactic plane and outside the galaxy. The team have also been invited to give a seminar talk at the University of Delaware to highlight the work they are doing with LSST:UK in Rubin cross-matching. Finally they have submitted a paper to the RAS Techniques and Instrumentation on their new model for handling the effects of unknown proper motions on the probabilistic cross-matching of photometric catalogues.

WP3.12: Support of EPO software

During this reporting period, the Zooniverse system has been upgraded to allow easy ingest of images and data from observatory systems. In addition there has been development of systems for template projects, allowing observatory scientists to benefit from previous design work and best practice. An agreement has also been reached on data storage for images used in production projects. Finally, the WP3.12 lead, Chris Lintott, has been appointed as LSST:UK EPO Coordinator.