LSST:UK Newsletter 7 (December 2020/January 2021)
Introduction
Welcome to the first LSST:UK newsletter of 2021, which promises to be a crucial year for UK preparation for participation in the Rubin LSST - and, hopefully, an easier year for us all than was 2020.
In this newsletter we present an update on two important proposals: the UK’s proposed in-kind contributions to Rubin LSST operations, which will secure our data rights; and the community broker proposal that will provide the Lasair system with a copy of the full Rubin LSST alert stream. The outcomes from these two proposals will determine the UK role in the Rubin LSST and mark two important milestones due during 2021. Further items report on the latest challenge run by the Dark Energy Science Collaboration - testing methods for assigning galaxies to redshift bins to maximise the scientific return from future tomographic weak lensing analyses by DESC - and introduce the new logo of the Vera C. Rubin Observatory. We also solicit the community’s input to the planning of a virtual LSST:UK All-Hands Meeting to be held in April or May, and advertise the latest technical report arising for the STFC-funded LSST:UK Science Centre programme.
Those with ideas for future newsletter items should contact the LSST:UK Project Managers (@George Beckett and @Terry Sloan: lusc_pm@mlist.is.ed.ac.uk), while everyone is encouraged to subscribe to the Rubin Observatory Digest for more general news from the US observatory team.
@Bob Mann
The UK in-kind proposal
As mentioned in the September newsletter, the document outlining the UK’s proposed contribution to Rubin LSST operations was submitted on September 25th. The details of three of those contributions - to Commissioning and to the annual Data Release Processing, plus our operation of an Independent Data Access Centre - were not complete at that time, as they required further discussion with Rubin staff, so we took advantage of an extension mechanism provided by the Observatory and submitted our final proposal on November 6th. A copy of that proposal is available from the LSST:UK Science Working Group wiki page.
That proposal describes a broad range of significant contributions, reflecting the breadth of expertise within the UK community and the scale of our ambitions for UK participation in the Rubin LSST. They can be grouped under two headings:
The DAC and DEV streams from the LSST:UK Science Centre programme, as originally outlined in our Phase A proposal to STFC in 2014: the DAC activity covers the development and operation of an LSST Data Access Centre in the UK, while the DEV activity comprises a set of software development work packages leading to the production of user-generated Products, to be published through the UK DAC. The in-kind proposal seeks credit for work done on this programme during Phases A and B, and includes their continuation through Phases C and D (i.e. survey operations).
Several new activities that the Rubin leadership asked us to consider taking on and which, taken together, would help integrate us into the operations consortium, namely: (i) taking a 25% share of the annual Data Release Processing workload; (ii) supporting Education and Public Outreach software developed by the Zooniverse team; (iii) providing a half-time International Contributions Coordinator to the Rubin Director’s office; (iv) contributing to Rubin Commissioning; and (v) supporting a stream of Community Scientists secondments.
This proposal is now being assessed by the Contribution Evaluation Committee and by Observatory staff, which will lead to feedback in February and a recommendation to the US funding agencies of its value in terms of data rights for the UK community. That should, by April, yield an outline of our final agreement, which is then expected to be signed in mid-2021. So, in less than six months' time, we should have confirmation of the scale and scope of UK participation in the Rubin LSST.
@Bob Mann
New logo for the Vera C. Rubin Observatory
In early December 2020, the Rubin Observatory unveiled its new logo.
According to the press release, the new logo “is a visual representation of Rubin Observatory’s central purpose: to collect light from celestial objects and transform it into data for scientific discovery.” Rubin also released an infographic (see below) that explained the various design elements.
@Terry Sloan
The LSST:UK Community Alert Broker
R. D. Williams, K. W. Smith, G. Francis, A. Lawrence, S. Smartt, D. R. Young, M. Schwamb, C. Frohmaier, and T. Sloan
The LSST survey is focused on the dynamic sky: a source is not just brightness, but a history of brightness. One major product is the “alerts”, that report new sources and changes in brightness. Most observing nights are expected to produce millions of alerts, each perhaps 50 kB, so the data volume per night can be of order a terabyte. For a scientist to get what they want from this firehose, there will be “community brokers” that ingest the data and allow scientists to utilise it effectively. Only a limited number of brokers can be supported – because of the high data rates – so there is a competition. Thus in December 2020 the LSST:UK submitted a proposal to become a community broker.
Our broker is Lasair[1], the main partners being the University of Edinburgh and Queen’s University Belfast (Lasair means flame or flash in Scottish and Irish Gaelic). Although the LSST survey has not started, there is already a prototype transient stream – ZTF – on which we have built the first versions of Lasair. The architecture of the LSST stream will be similar to ZTF.
Lasair will provide a flexible and powerful platform that will enable worldwide users -- individual users, other projects, and citizen science -- to achieve their own science. Lasair will provide access to a rich variety of added value information and external data sources alongside the alert data, all of which can be interrogated, using queries, filters, watchlists, streaming queries and a programming interface. Lasair uses scalable technology and runs on the STFC-funded IRIS infrastructure. Although powerful, Lasair has an easy on-ramp for scientists from web pages and simple SQL, then on to Jupyter notebooks on their own machines, and then to high-performance mining co-located with the data. Lasair offers direct access with a staged approach: scientists can start with a simple, immediate mechanism using familiar SQL-like languages. These SQL-like queries can be custom made or users can choose and modify one of our pre-built and tested queries. These queries return an initial selection of objects, based on our rich value-added data content, and users can then run their own local code on the results. Users can build up to running their own code on both the stream and the database with high-throughput resources in the IRIS cloud. The SQL filters and code can be made public, shared with a group of colleagues, copied, modified, and excellent examples and their outputs are featured on the Lasair web page.
A broad overview of the Lasair design is shown in the figure below.
Alerts arrive from the Rubin Observatory at left, and are cached and saved. Several “tagging” systems add value about sky context (Sherlock), external multi-messenger alerts, user-created watchlists of their own sources, classification engines, and featured of the light curves. User queries and filters are processed and results despatched, and the enriched alert stream kept in databases. The alerts can be utilised in several ways: by web, jupyter-style notebook, a programming interface (API), or received as real-time streams.
The Lasair team welcomes astronomers and technologists to have a try at the new (beta) version[1], and report comments and suggestions to lasair-help@lists.roe.ac.uk.
[1] https://lasair-iris.roe.ac.uk/
@Roy Williams
DESC Tomographic Challenge
Last year the Dark Energy Science Collaboration (DESC) ran a semi-public analysis challenge, designed to find new methods for assigning galaxies to tomographic redshift bins as optimally as possible.
The main weak lensing methods split galaxies into separate radial bins before correlating those bins with themselves and each other. This is usually done by using a photometric redshift code to get a very approximate redshift and then comparing that to a set of pre-defined nominal bin edges. One challenge is of course to choose these edges in the first place as optimally as possible, to maximize the signal-to-noise of the resulting correlations, but a further complication arises when we use a galaxy shape measurement method called "metacalibration" to measure galaxy shapes in the first place.
This method allows us to find very accurate corrections for selection biases that arise whenever you split galaxies into groups (it turns out that almost any split, e.g. on magnitude, will correlate in some way with the shear of the galaxy, which is what we are trying to measure). But the cost is that we can only split galaxies using fluxes in bands with sufficiently precise PSF measurements, and the u, y, and possibly g bands will not be suitable. To get the advantages of this method we would have to split galaxies using only the riz and maybe g bands.
The tomography challenge asked entrants to attempt this, to maximise the constraining power of the correlations using only a restricted subset of bands and colours. More than twenty entrants submitted methods that first trained on a representative sample and then analysed a test sample to get a score. Methods ranged from machine learning tools with convolutional neural networks, self-organizing maps, or Gaussian processes, to more classic methods optimizing linear colour cut.
We have now analyzed the results, and found them encouraging - many methods managed impressively well-separated galaxy bins even using only the riz bands. This bodes very well for the use of metacalibration in early LSST data. We are now writing up the results in a paper, and evaluating the methods to see how the best ones succeeded.
@Joe Zuntz
Preliminary plans for a virtual LSST:UK All-Hands Meeting in April/May 2021
The first LSST:UK All-Hands Meeting (AHM) took place in Cardiff in May 2019. It was a great success, and the consensus of opinion afterwards was that the level of LSST-related activity within the community motivated holding such a meeting annually. In the event, the LSST:UK Executive Group decided not to hold an AHM in 2020, preferring to invest in funding a strong UK presence at the LSST@Europe4 conference then planned to take place in Rome in June 2020. LSST@Europe4 may now take place later this year, but the Exec feel that 2021 is too important a year for UK involvement in the Rubin LSST to pass without a UK consortium meeting.
As noted above, we should know the outline of our in-kind package - and, hence, of our data rights position and our relationship to the operations consortium - by April, so the plan is to hold the LSST:UK AHM in late April or early May, once that information is known. The meeting will take place online, and will include plenary sessions, providing information to the community on our in-kind package and starting preparations for our Phase C funding proposal. It is also likely to include parallel science sessions, but its exact format is still TBD, and will be influenced by the outcome of bids to hold LSST-related sessions at NAM2021 and whether LSST@Europe4 will take place during 2021.
These plans will start to take more definite form in the next month or two, but, for the moment, the Exec are keen to get input from the community on what you would like to see at the 2021 AHM. To that end:
We have set up a Doodle poll - https://doodle.com/poll/2dtfzci3zqt4tipn - with dates in the second half of April and the first half of May, on which we invite you to record your availability - and, more importantly, periods of unavailability due to clashes with meetings likely to involve other members of the community. Please use the Comments box to annotate your entries, if appropriate.
We solicit suggestions for themes for parallel science sessions and for topics to be covered in the plenary sessions.
We seek LSST:UK consortium members willing to contribute some time to serving on the organising committee(s) for the meeting.
Please record any availability/unavailability information on the Doodle by February 19th and feel free to email me (rgm@roe.ac.uk) regarding items 2 and 3. Further information on the AHM plans will appear in later newsletters and/or messages to the lusc_announce mailing list.
@Bob Mann
Recent LSST:UK outputs
LSST:UK has recently produced the following technical reports.
Title | Author | Description |
|---|---|---|
Aaron Watkins, Chris Collins, Sugata Kaviraj | To expand LSST’s scientific reach into the low surface brightness (LSB) regime, where nearly all of its extragalactic discovery space lies, an accurate sky subtraction is paramount. The current LSST pipeline sky subtraction routine must therefore be optimized for LSB work. The first step in this optimization is to test the current implementation and determine how much improvement is required for LSB work to proceed. This requires the development of metrics for measuring the over-subtraction currently induced by the sky subtraction. We have devised such a metric using model galaxy injections: the difference in model magnitudes pre- and post-sky subtraction, or Δm. Using this metric, we have tested both the final, local sky subtraction done at the deep coadd level, as well as the full focal plane sky subtraction done to remove night sky emission. While both show systematic over-subtraction below µλ~26 mag/arcsec2 , the final local sky subtraction’s effect is significantly worse, and also shows a trend with model size for high surface brightness models that is absent from the full focal plane sky subtraction. Though these tests only established a baseline, it is already apparent that the final sky subtraction step makes LSB work infeasible with LSST, and even heavily impacts high surface brightness objects with scales larger than 10". In future work, we will expand the parameter space to include more realistic galaxy profiles to determine the full scope of the problem, and then begin devising mitigation strategies. |
@Terry Sloan
If you require this document in an alternative format, please contact the LSST:UK Project Managers lusc_pm@mlist.is.ed.ac.uk or phone +44 131 651 3577