LSST:UK Newsletter 32 (April 2023)
- Bob Mann
- Terry Sloan
- George Beckett
- James Mullaney
Introduction
Construction work in Chile continues well, with the Telescope Mount Assembly declared “substantially complete” in early April. This means that it has been handed over by the contractor who built and installed it to the Rubin Observatory, enabling further integration of components into the functioning TMA structure.
The passing of such milestones enables more confident updating of the project schedule, and a recent Construction/Commissioning Schedule workshop has updated the expected date for System First Light (i.e. the completion of on-sky engineering work and the start of the System Optimization and Science Validation phases) to October 2024, with the completion of the construction project (and the start of the LSST survey) now scheduled for February 2025. These dates are still subject to some change, as the project still has some schedule contingency that may be deployed as system integration and testing continues.
The Observatory has released the latest of its annual “Day in the life of Rubin” videos (see right), showing a large number of Rubin staff at work, in the US and Chile - plus, slightly cheekily, some photos from the LSST@Europe4 conference last October.
The Observatory has also released further details of the 2023 Project and Community Workshop, including the ability to suggest sessions and apply to present a talk or poster (up to a deadline of May 19th)
Closer to home, April 1st marked a significant milestone for LSST:UK, with the start of Phase C of the LSST:UK Science Centre (LUSC) programme, which now comprises the STFC-funded work implementing the in-kind contribution to Rubin operations through which LSST data rights for UK astronomers are obtained. The knock-on effects of the Covid-induced delays to the Rubin schedule meant that a number of the Phase B Work Packages could not be completed by March 31st, so several of the Phase B grants have no-cost extensions which will enable them to finish their scheduled work over the coming months. As a result of that, the change from Phase B to Phase C will be an extended process, rather than a discrete transition. The LUSC Phase C programme was outlined in the December/January newsletter, and further details will be presented in future newsletters: for example, @James Mullaney, who has recently started as Production Scientist within the UK’s Data Release Processing team describes below some of his prior work with the LSST pipelines, applied to data from the GOTO telescope.
Those with ideas for future newsletter items should contact the LSST:UK Project Managers (@George Beckett and @Terry Sloanlusc_pm@mlist.is.ed.ac.uk), while everyone is encouraged to subscribe to the Rubin Observatory Digest for more general news from the US observatory team.
@Bob Mann
First Rubin IDACs Workshop
LSST will be a substantial data resource (well in excess of 100 Petabytes), which means traditional approaches to survey astronomy will not be tractable. Instead, astronomers will access survey data through specialised centres called Data Access Centres (DAC), effectively implementing the model of bringing compute to the data. The Observatory will operate two DACs -- one in USA and one in Chile. In addition, through the in-kind contribution programme, international groups can operate Independent Data Access Centres (IDACs) to increase the computing capacity available to the community.
At the time of writing, there are thirteen IDACs planned, spread across the Americas, Europe and Australasia, including the only planned, "Full IDAC" hosted by LSST:UK, which will serve complete Data Releases to the community, alongside uswer-generated products and services reflecting UK science priorities and with comparable services to the official Observatory DACs.
Recognising that running an IDAC is a complex undertaking and that there are likely to be many common problems to be tackled by every IDAC, a team led by Knut Olsen (Rubin) and including George Beckett (LSST:UK), Luiz da Costa (LIneA, Brazil), Jeremy Kubica (Rubin), Sierra Villareal (Rubin) organised a workshop in March, intended to make introductions between the IDAC teams, to establish links between key science users and IDACs, and to begin coordinate activities and identify opportunities for collaboration.
The four-day workshop had two elements. In the first two days, representatives from the various Rubin science communities and domains described their plans for using Observatory data in the early years of Operations. Then, during the third and fourth days, a closed meeting was held involving only the IDAC teams and key figures in the Rubin Observatory, to begin planning towards operational readiness.
Some of the first outputs from the workshop include: the formation of an IDAC Coordination Group on the Rubin Slack service; commitment to create an IDAC service catalogue for the science community; and agreement to begin work on some of the common challenges, including data movement and ingestion, simulation of science cases, and enforcement of data rights. There are also concrete plans to run further workshops to progress the dialogue and planning.
Materials from the workshop are available on the Rubin website [https://project.lsst.org/meetings/rubin-idacs/welcome] and a Google Drive folder, while a technical report is being prepared by the organising committee, detailing findings and likely next steps, and should be available in the summer.
@George Beckett
Rubin’s Science Pipelines: Not just a one-trick pony
As most readers of this newsletter will already know, turning Rubin’s millions of raw exposures into useable science products is a major challenge. To address this challenge, the international LSST consortium is developing a software pipeline to perform all the calibration, coadding, detection, measuring, etc., that the astronomy community needs to turn the raw imaging data into scientific discoveries. What is perhaps less well known is that these science pipelines have been designed and developed with universality in mind, allowing other surveys to utilise the expertise and effort that has gone into developing Rubin’s Science Pipeline for their own gain.
While Rubin’s Science Pipelines have been developed with broader application in-mind, the sheer diversity of astronomical surveys mandates a flexible structure that can be adapted to different datasets taken by different telescopes/instruments with different pixel scales, filters, metadata (i.e., headers) etc. The way this adaptation is achieved is via so-called “obs packages”. As well as providing configuration data, such as detector size, defects, and pixel scales, an obs package dictates how the raw exposures are ingested into the data repository that is referred-to throughout data processing. Further, what steps are carried-out by the science pipelines - whether that be flat-fielding and bias subtraction, PSF-modelling, coaddition of science frames, source detection and measurement, etc. - can also be controlled by the obs package. As such, obs packages are a fundamental component of the Rubin Science Pipeline that allows it to process data from different telescopes and instruments.
Because each instrument (or, indeed, simulated dataset) requires its own obs package, a number of obs packages already exist for the various different datasets that have been processed by the Rubin Science Pipeline. Of course, the LSST camera has its own obs package, which includes adaptations to processed data from the Commissioning Camera as well as the simulated images used to create the DP0 datasets, which some of you may already have accessed. But obs packages have also been produced for other telescope and instruments, including Subaru, DECam, and CFHT. On the UK-side, Raphael Shirley has been leading the development of obs_vista, an obs package that allows the Rubin Science Pipelines to interface with and process near-infrared exposures obtained by the VISTA telescope, which formed a key part of LSST:UK’s Phase B WP3.5 (“LSST and near-infrared data fusion”). My own introduction to obs packages was while developing one for GOTO, which my former PhD student, Lydia Makrygianni, and I used to process and analyse a subset of the data obtained by the GOTO prototype. I’ve also recently been developing obs_necam (“anycam”), which is a stripped-down obs package that comes with instructions and scripts to help people get started processing data and developing their own obs package.
As we move into the era of massive multiwavelength astronomical datasets, much will be gained from being able to combine these datasets as seamlessly as possible. By developing a robust science pipeline that is flexible enough to be applied to very different datasets, the LSST consortium is making great strides to reaching this goal.
@James Mullaney
Outcome of Consortium Board Chair election process
As mentioned in the March Newsletter, Mike Watson’s current term as Consortium Board Chair is coming to a close, so, as Deputy Chair, I solicited nominations from the Board to fill that role for the next three years. The outcome of that process is that Mike will continue as LSST:UK Board Chair for another term, having been elected unopposed, which I take as a clear indication of the confidence the Board has in Mike's stewardship. LSST:UK has benefited greatly from that stewardship and we all look forward to Mike guiding us to the start of operations.
@Alastair Edge
DP0 Virtual Summer School (12-16 June)
The Rubin Data Previews progamme provides LSST data rights holders (including UK Affiliate PIs and Junior Associates) with access to LSST-like data products - simulated data in Data Preview 0 (DP0) and commissioning data in Data Previews 1 and 2 - plus support provided by the Community Science Team (CST) to “DP0 Delegates” in gaining familiarity with using the Rubin Science Platform to access and analyse them. As noted in the March newsletter, access to the current version of Data Preview 0 (DP0.2) has now been opened up, with all data rights holders now able to sign up to be a DP0 Delegate, and the latest announcement from the CST concerns a DP0 Virtual Summer School, which they will run from June 12-16. The summer school will comprise a three-hour session each day, which run twice to accommodate attendees in different timezones - the most convenient for UK attendees being 16.00-19.00 BST - which will include tutorials, plus collaborative hackathon sessions. The summer school is intended to be suitable both for those new to DP0, as well as those who have been participating in the data preview programme already: those in the former category who want to take advantage of this opportunity to kick-start their engagement with the data preview programme should consult the Getting Started with DP0 Checklist for instructions as to how to sign up for a DP0 account and prepare for the workshop.
@Bob Mann
Recent LSST:UK Science Centre outputs
The LSST:UK Science Centre has recently produced the following technical reports.
Title | Author(s) | Description |
|---|---|---|
D2.1.4 LSST:UK DAC Roadmap and Integration Plan for DEV Activities (updated) | Bob Mann and George Beckett (University of Edinburgh) | This document provides an update to the “LSST:UK DAC Roadmap and Integration Plan for DEV Activities” presented in D2.1.1 in the light of current Rubin and LUSC plans. By updating the DAC Roadmap it defines the capabilities of the UK DAC that should be in place at the end of Phase B (nominally 31/MAR/23) and outlines the plans for further DAC development in the early stages of Phase C, although the latter remain somewhat uncertain due to the Rubin Construction Project’s recent decision not to perform on-sky observing with ComCam, which has implications for the timing and content of Data Preview 1 and, hence, for the UKDAC release schedule. This document also presents an updated account of the interaction between LSST:UK and IRIS, its main infrastructure provider, which has changed significantly since the publication of D2.1.1 with the advent of the Somerville cloud system. Furthermore, it presents a revised discussion of the support provided by the DAC to the Phase B DEV WP teams, although the details of these interactions are now captured in separate DAC-DEV interface documents. |
Bob Mann and George Beckett (University of Edinburgh) | In the light of the Covid-induced delay to the start of survey operations, the scope of this Deliverable was changed, from a release of documentation through the UK’s Data Access Centre (DAC) to a discussion of the requirements for such documentation in future. The areas in which UK DAC documentation is likely to be required are identified, and a set of principles – non-duplication, sufficiency, currency and use of similar format, jargon and style to Rubin documentation – for their development is outlined. These principles should inform the creation of a UK DAC documentation plan early in Phase C, which will provide a specification of each planned piece of documentation and which is intended, thereby, to aid their production. | |
D3.11.4 Report on the Preparation for Full-Scale DAC Matches | Tom J Wilson and Tim Naylor (University of Exeter) | The goal of WP3.11, Cross-Matching and Astrometry at LSST Depths, is to create a cross-match tool through which common detections can be identified between LSST objects and sources in numerous other (non-LSST) datasets. Alongside identifying matches, the generated products should be able to report on the probability of such a match (to identify unreliable assignments for users), and provide key derived secondary information such as is necessary or useful for the astronomer. Until now each WP3.11 deliverable has extended this cross-match functionality. This began with a “simple” Bayesian cross-match and we then added to the algorithms to include those needed to overcome the systematic effects such as the effects of hidden, unresolved contaminant objects on the positions of sources, or the inclusion of photometric information in the discernment of “false positive” matches. By contrast this deliverable, D3.11.4, reports the efforts of WP3.11 to accomplish the goal of performing matches, rather than making a match-capable software. This report summarises those efforts, a three-pronged work. First, the incorporation of previous testing which identified the need to increase the scope of parallelisation within the codebase to reduce total runtimes. Second, the inclusion of additional software to ensure that the matches are both precise and accurate in their reported match likelihoods. Finally the beginning of “full scale” matches, running all-sky tests and simulating the wider workflow with the DAC team in which catalogues are generated, matched, and the consolidated results ingested into the UK DAC for access by in the community the UK RSP. |
@Terry Sloan
Forthcoming meetings of interest
There is a busy schedule of potentially interesting meetings in the coming months. Registration is now open for the National Astronomy Meeting, in Cardiff, during 3rd--7th July (http://nam2023.org ) at which LSST:UK will run three sessions (details to follow). The website for the Rubin Project and Community Workshop 2023 is now available, with options to suggest sessions and to contribute a poster or talk abstract. Earlier in the Summer, Rubin Data Delegates might be interested to join the DP0 Virtual Summer School (online, 12th–16th June).
Other meetings of potential interest for the coming months include:
12th-14th June: LSST Galaxies Collaboration meeting, Paris
20th–22nd June: The Transient and Variable Universe, held at University of Urbana Champaign, Illinois, USA
3rd-7th July: The National Astronomy Meeting 2023, held in Cardiff, with a three-part session on “UK involvement in the Vera C. Rubin Observatory: Legacy Survey of Space and Time”
24th–28th July: DESC Collaboration Meeting (SLAC), California, USA
24th–28th July: ZTF Summer School, University of Minnesota/ virtual
Members of the Consortium (not in receipt of travel funding through one of the Science Centre grants) may apply for travel support for meetings of this kind via the the LSST:UK Pool Travel Fund. Details are available at Forthcoming LSST-related Meetings .
Note that the current list of forthcoming meeting is always available on the Relevant Meetings page. You may also wish to check information held on the LSST organisation website LSST-organised events and the LSST Corporation website.
@George Beckett
Announcements
If you have significant announcements that are directly relevant to LSST:UK and would like to share the announcement in a future newsletter, please contact the LSST:UK project managers.
If you require this document in an alternative format, please contact the LSST:UK Project Managers lusc_pm@mlist.is.ed.ac.uk or phone +44 131 651 3577