Introduction
Finally, a reminder that registration for the 2021 Project and Community Workshop (PCW) is open, with this year’s event taking place online from August 9-13 and featuring a wealth of programmatic, technical and scientific information related to the Rubin LSST.
Those with ideas for future newsletter items should contact the LSST:UK Project Managers (George Beckett and Terry Sloanlusc_pm@mlist.is.ed.ac.uk), while everyone is encouraged to subscribe to the Rubin Observatory Digest for more general news from the US observatory team.
Placeholder title for Graham
The New World of Streaming Astronomical Data
Many of us are looking for a new home. Even if we are not, we all have a friend or family member who enjoys “property fantasy” – browsing the homes for sale and imagining themselves the buyer. We connect to a property broker, just as astronomers connect to an event broker: not just to find, but to have new results pushed to us, as soon as the property comes on the market (or as soon as the supernova explodes).
The “asking” way is to run the query every so often, and pick through the results to see what is new. There is also the “listening” way to get updates pushed to us: once a query is saved, simply check a box and you will get notified every time a suitable property comes on the market. Your query has been converted to a filter, that takes the stream of properties, and filters out the ones you want.
You can (1) run your query and get all the results, consisting of both new properties and those you have already seen. Or (2) You can get notified whenever a new property comes up. Note that you only get new results, because the server remembers what it gave you before.
The Lasair event broker of LSST:UK has a “query builder” based on SQL syntax, with some extras: users can also query an intelligent crossmatch from catalogues, or upload a list of their favourite sources, or specify a sky area of interest.
In the past, an astronomer would query the database with their morning coffee, and try to find the new ones – method (1) above. Now however, the astronomer can check the “streaming” box on their saved query in Lasair. Alerts will be delivered as in method (2). While email is available for low-frequency streams, the “Kafka” protocol is more suitable for delivering alerts for further processing by machines.
The Kafka server remembers what has been delivered already, through the mechanism of the “groupID”. Each user-made query/filter generates a stream of alerts, that Kafka calls a “topic”. Having yesterday read from a topic with a given groupID, and then read today, you will get only the new alerts that have appeared on that topic since yesterday. You can run a program forever and continuously pull the stream, or you can start the program periodically, and in both cases each alert is received exactly once. Of course, you can get everything in the topic by simply choosing a new groupID, and it will start at the beginning of its cache – generally about a week’s worth of the stream. Details are available at the Lasair website -- see the last notebook at http://lasair-iris.roe.ac.uk/jupyter .
The streaming data will have a link to the Lasair page corresponding to that object, showing the lightcurve and sky context, and links that give more information about the object, from the databases of the world. If you have a team, the alerts could be fed into a “marshal” -- so that different people can write comments and vote on whether telescopic follow-up is warranted. Or you could even have automated telescopic follow-up with no human in the loop.
Placeholder title for Catherine
Placeholder title for Dan
Recent LSST:UK outputs
LSST:UK has recently produced the following technical reports.
Title | Author | Description |
|---|---|---|
Tom Wilson and Tim Naylor | WP3.11’s primary objective is to create a cross-match service, to identify sources in common between two photometric catalogues and allow for their detections in each dataset to be combined. The previous deliverable D3.11.2 outlined the main infrastructure of this codebase, allowing for a full “many-to-many” match between two catalogues, for a simple match setup, purely considering the positions and centroid uncertainties of each source. This latest deliverable, D3.11.3, extends the options available in the cross-matches to include two important aspects for the matching of LSST to ancillary datasets, mostly infrared catalogues such as WISE. WP3.11 now have a robust and accurate cross-match tool, and will be able to provide LSST:UK users with robust matches between LSST and other photometric catalogues. |
Forthcoming meetings of interest
The global pandemic has led to almost all face-to-face meetings being cancelled. However, in light of continued restrictions on travel, Rubin Observatory business has moved online and we aim to maintain a list of relevant/ interesting upcoming meetings on our Confluence site. Some specific highlights include:
The AGN Science Collaboration will hold its Summer 2021 meeting, online, during 12th--14th July 2021. Details are available on the AGN website (login required).
The next DESC Virtual Collaboration Meeting is scheduled for 19th--23rd July 2021. Details provide on the DESC Confluence site (login required).
The 2021 Rubin Project and Community Workshop will take place online from 9th--13th August. Registration is now open at the Workshop page.