| Table of Contents |
|---|
Introduction
Construction work continues apace on the summit of Cerro Pachon. Recent drone photos (see right) shows that the dome cladding is essentially complete, which is important since, as shown in the accompanying video, there is now snow on the ground, as the Chilean winter approaches.
Closer to home, a significant milestone for LSST:UK was passed with the selection of Lasair as one of the Community Brokers that will receive the full Rubin LSST alert stream. The Lasair team write about this below, and their selection reflects the great work they have done in recent, producing a broker that is being used now for ZTF transients and that will scale to LSST data rates.
The Lasair team also featured prominently in the NAM 2021 session on Case Studies of Transient and Extreme Variability Science, while many other LSST:UK colleagues spoke in the Early science and commissioning opportunities with the Vera C. Rubin Observatory (slides from which are available here), and the Rubin LSST was much in the minds of those attending the session on Satellite megaconstellations: the challenge for astronomy.
(Credit: Rubin Observatory)
Finally, a reminder that registration for the 2021 Project and Community Workshop (PCW) is open, with this year’s event taking place online from August 9-13 and featuring a wealth of programmatic, technical and scientific information related to the Rubin LSST.
Those with ideas for future newsletter items should contact the LSST:UK Project Managers (George Beckett and Terry Sloanlusc_pm@mlist.is.ed.ac.uk), while everyone is encouraged to subscribe to the Rubin Observatory Digest for more general news from the US observatory team.
UK contributions to Rubin commissioning: call for proposals
Colleagues are invited to submit proposals for value added contributions to Rubin commissioning. This opportunity follows acceptance of the commissioning element of the UK’s in-kind proposal, that was in turn based on the Expressions of Interest submitted by UK colleagues in 2020. It is therefore already agreed that the UK will contribute a summit team and a remote team. The focus of the former will include commissioning the wavefront sensor sub-systems, bringing expertise from VISTA. The latter will be a broader and larger team of colleagues contributing a wide range of scientific and operational expertise, much of which will concentrate on analysing commissioning data and giving feedback to the core commissioning team on the performance of the system. This is a great opportunity to get early hands on experience with Rubin data and science platform. Whilst there are strict rules around any scientific output from the commissioning period, experience gained with commissioning data will inevitably help with achieving early science. We are looking to agree the UK team with US colleagues by the end of 2021 with a view to UK team members getting started in early 2022. Travel and subsistence funding to support the UK team is already in place, and modest additional funding of commissioning effort is also available through this call. Full details of this opportunity are described in the call for proposals, which has a submission deadline of September 15, 2021. Please contact Graham Smith if you have any questions or would like to discuss proposal ideas in advance of submission.
Lasair selected as LSST community alert broker
The Rubin Observatory announced on Tuesday 27th July that the UK’s Lasair project has been selected as an official LSST community alert broker and will receive the full Rubin alert stream. Every image that Rubin takes will have a reference sky subtracted and every object that is variable or transient, or moving, will be catalogued and released in a stream of alerts within 60 seconds. The University of Edinburgh and Queen’s University Belfast have been partnering, with STFC Phase B funding, to build a broker that will provide a user friendly and scientifically powerful platform for world-wide users to exploit this information reach data stream. We expect Rubin’s data stream to have a peak production rate of 1 terabyte per night in just the alert data, a data rate that would overwhelm the average science user trying to feed off the complete transient stream. For every visit (i.e. approximately every minute) there will be about 30,000 alerts. Lasair will take the full Rubin alert stream and add scientific value to the data stream, providing computing power, tools and storage for science users to manage selection of objects from the stream. We will manage a large database store of all objets and provide SQL, API and python notebook query functionality. Our added value will be cross-matching to multi-wavelength catalogues, classification, and provision of realtime spectroscopic redshifts and spectral types from the 4MOST and SOXS surveys, photometric redshifts from major catalogues and Rubin itself and a basic lightcurve classifications. We have built the prototype and have had multiple reviews and workshops with the user community in the UK and Europe to shape the design, most recently at the RAS National Astronomy Meeting 2021 hosted by the University of Bath. We welcome user input and feedback at any time. Read more from Roy Williams below on the new concept of streaming data in comparison to database queries, which is core to our vision for science functionality for the LSST data stream. While our Lasair prototype works on ZTF data and is being scientifically exploited by a range of users, we now look forward to developing the Lasair system for LSST data rates.
The Lasair Team : Stephen Smartt Andy Lawrence Roy Williams Ken Smith Gareth Francis Dave Young Dave Morris Stelios Voutsinas
The New World of Streaming Astronomical Data
Many of us are looking for a new home. Even if we are not, we all have a friend or family member who enjoys “property fantasy” – browsing the homes for sale and imagining themselves the buyer. We connect to a property broker, just as astronomers connect to an event broker: not just to find, but to have new results pushed to us, as soon as the property comes on the market (or as soon as the supernova explodes).
The “asking” way is to run the query every so often, and pick through the results to see what is new. There is also the “listening” way to get updates pushed to us: once a query is saved, simply check a box and you will get notified every time a suitable property comes on the market. Your query has been converted to a filter, that takes the stream of properties, and filters out the ones you want. In the screenshot shown here, it says “Create Alert” at top right, which causes an email whenever a property passes the filter.
You can (1) run your query and get all the results, consisting of both new properties and those you have already seen. Or (2) You can get notified whenever a new property comes up. Note that you only get new results, because the server remembers what it gave you before.
The Lasair event broker of LSST:UK has a “query builder” based on SQL syntax, with some extras: users can also query an intelligent crossmatch from catalogues, or upload a list of their favourite sources, or specify a sky area of interest.
In the past, an astronomer would query the database with their morning coffee, and try to find the new ones – method (1) above. Now however, the astronomer can check the “streaming” box on their saved query in Lasair. Alerts will be delivered as in method (2). While email is available for low-frequency streams, the “Kafka” protocol is more suitable for delivering alerts for further processing by machines.
The Kafka server remembers what has been delivered already, through the mechanism of the “groupID”. Each user-made query/filter generates a stream of alerts, that Kafka calls a “topic”. Having yesterday read from a topic with a given groupID, and then read today, you will get only the new alerts that have appeared on that topic since yesterday. You can run a program forever and continuously pull the stream, or you can start the program periodically, and in both cases each alert is received exactly once. Of course, you can get everything in the topic by simply choosing a new groupID, and it will start at the beginning of its cache – generally about a week’s worth of the stream. Details are available at the Lasair website -- see the last notebook at http://lasair-iris.roe.ac.uk/jupyter .
The streaming data has a link to the Lasair page corresponding to that object, showing the lightcurve and sky context, and links that give more information about the object, from the databases of the world. If you have a team, the alerts could be fed into a “marshal” -- so that different people can write comments and vote on whether telescopic follow-up is warranted. Or you could even have automated telescopic follow-up with no human in the loop.
Feedback from the LSST:UK All Hands Meeting
Seventeen attendees of the All Hands Meeting (AHM) completed the post-meeting feedback questionnaire. That represents a little over 10% of those who registered for the meeting, so it is difficult to derive very strong conclusions from the survey, but what follows is a summary of the responses that were provided.
Respondents felt that it is important for LSST:UK to hold UK-focussed meetings in addition the Project and Community Workshop and the various workshops organised by the Science Collaborations.
Even after the end of Covid restrictions, most would prefer future AHMs to be online or “in person, but allowing meaningful remote attendance” although few specific suggestions were made as to how to make remote attendance meaningful, beyond having remote attendees visible to those attending in person, and ensuring that session chairs ensure that remote attendees are properly included in Q&A sessions.
The general feeling was that three days was an appropriate length for an online meeting and that the 2021 AHM had about the right amount of content divided over three days. Most participants used all three online tools - Zoom, Slack and GatherTown - and saw the benefit of all three, although several respondents noted that attendance in GatherTown could have been higher.
The respondents generally preferred plenary sessions, both in principle and in comparison to the parallel sessions and other activities (e.g. posters and flash talks) that took place.
The topics that interested the respondents most were the Rubin Observatory updates and Science Collaboration talks, with a general preference expressed for science over organisational content, although there was interest in the introductory material aimed at Early Career Researchers and those new to the Rubin LSST and LSST:UK amongst those to whom that was targeted.
The general free-text comments were overwhelmingly supportive and appreciative, although one respondent complained vehemently about the timing of the meeting and the notice given of its timing, suggesting that polling for clashing dates and circulating a “Save the date” email on the lusc-announce list six weeks before the AHM had not been completely effective.
N.B. the respondents were split 3/7/7 between PhD students/postdocs/faculty, with most (14/17) currently holding data rights, and there were 6 female and 11 male respondents.
Bob Mann and Roy Williams for the LOC
Placeholder title for Dan
Forthcoming meetings of interest
The schedule of events is relatively quiet over the Summer months, though we expect new workshops and meetings to be announced in the coming weeks. The most significant event in August is the 2021 Rubin Project and Community Workshop, which will take place online from 9th--13th August. Visit the website to register.