Issued by George Beckett (LSST:UK Science Centre, Data Facility Liaison); Wil O’Mullane (Rubin Observatory, AD Data Management); Mike Watson (LSST:UK Consortium, Board Chair) on behalf of the LSST:UK Consortium
...
| Info |
|---|
The call was updated on Monday 6th November 2023 to include details of the grade profiles for the funding award from STFC. The new information is highlighted in the text below for convenience. |
The LSST:UK Science Centre Phase C funding, awarded by STFC, includes support for a team to work on the UK contribution to LSST Data Release Processing. The LSST:UK Consortium is now ready to allocate that funding for the three-year period from 1st April 2024 to the end of Phase C on 31st March 2027, with a likelihood of continued support for the selected team during Rubin operations.
...
Full details of the LSST:UK in-kind contribution to Data Release Processing can be found in the mature draft of the UK Data Facility Operational Plan.
LSST:UK has secured funding for 5.0 FTE of effort during April 2024—March 2027, with the following grade profile based on standard UK University salary scales: 2.5 FTE at Grade 7; 1.5 FTE at Grade 8; and 1.0 FTE at Grade 9. An indicative assignment of grades to roles is included below, though this is not a prescription. The final recommendation from the select panel may be based on a different distribution of grades to roles, though the selection panel is expected to make a recommendation that is within the effort levels available at each grade.
The team roles are, with reference to the Operational Plan, defined as follows:
Production Scientist (0.5 FTE, indicative Grade 8) – working in Data Production and contributing to the science validation of DRP and helping to ensure that Data Release Products are fit for purpose. Specifically, the Data Release Production Scientist will:
Oversee UK processing activities, to identify processing problems, off-spec data products, and underperforming processing stages.
Undertake a real-time (that is, during production) assessment of the astronomical validity of the data release processing undertaken at the UK Data Facility.
Coordinate with other Production Scientists (that is, at US and French Data Facilities) regarding data issues that are not specific to UK DRP.
Understand the scientific intent of the processing pipeline and be able to troubleshoot issues with elements of the pipeline configuration.
Processing Scientist (0.5 FTE, indicative Grade 7) – as part of Data Facilities, to be responsible for the completion of DRP in the UK and ensure the timely delivery of data products to maintain processing momentum. The Processing Scientist will be responsible for:
Coordinating the curation of datasets via the Data Butler as well as other required database platforms for DRP.
Monitoring day-to-day progress with the UK contribution to DRP and measure real-time DRP performance.
Undertaking basic validation of UK processing activities, to identify and coordinate resolution of day-to-day processing problems, off-spec data products, or under-performing processing stages.
To prepare resource estimates and forecasts for IRIS provisioning.
To support the QA team by providing suitable sample products and data to enable them to validate UK processing contributions.
To liaise with the Data Wrangler, Workflow Manager and Operations Support to prepare for each new Data Release campaign, confirming software and hardware configurations to be used and to collate feedback from UK pre-Data Release testing and validation activities.
Seeing problems through to resolution.
Data Wrangler (Data Curation – Rucio) (1.0 FTE, indicative Grade 8) – as part of Data Facilities, to configure, operate, and maintain the data-distribution, staging and curation, required for DRP—e.g., using Rucio. The Data Wrangler will be responsible for:
Contributing to the overall operation of Rucio across the campaign.
Oversee any bulk-download/ transfer operations and audit location and availability of data assets, in line with campaign requirements.
Configuring UK endpoints and interfaces to Rucio, to allow UK infrastructure to effectively contribute to DRP.
Problem-solving for Rucio workflows and configuration elements.
Liaising with UK (storage and transport) infrastructure providers to enable effective use of Rucio.
Workflow Manager (Workload Management – PanDA) (1.0 FTE, indicative Grade 9) – as part of Data Facilities, to configure, operate, and maintain the (compute) processing workflow, required for DRP. It is expected that the processing workflow will be implemented using PanDA. The Data Wrangler (WM) will be responsible for:
Contributing to the overall operation of PanDA across the campaign.
Configuring UK endpoints and interfaces to Panda, to allow UK infrastructure to effectively contribute to DRP.
Install and maintain Pipeline software and dependencies on UK infrastructure.
Setting up and monitoring UK elements of the Data Butler service.
Problem-solving PanDA workflows and configuration elements.
Liaising with UK (computing) infrastructure providers to monitor and improve effectiveness and efficiency of PanDA configuration in the UK.
DRP Operations Support (1.0 FTE, indicative Grade 7) – as part of Data Facilities, a Research Software Engineer who will have a thorough understanding of the whole UK contribution to DRP and be able to trouble-shoot and contribute to processing tasks as required.
Addressing issues with the configuration and operation of the DRP environment
Liaise with network providers (transit and endpoints) regarding efficient transport of data products to processing sites, etc.
Liaising with the Rubin Infrastructure Team regarding implementation, configuration, and optimisation of the DRP environment.
The DRP team is completed by the inclusion of the UK Data Facility Liaison (George Beckett, University of Edinburgh) and 1.0 FTE of Fabric Administrator effort (indicative Grade 7) that will be divided between the IRIS sites hosting the DRP hardware and, hence, not allocated as part of the current call.
...