Introduction
….
Those with ideas for future newsletter items should contact the LSST:UK Project Managers (George Beckett and Terry Sloan: lusc_pm@mlist.is.ed.ac.uk), while everyone is encouraged to subscribe to the Rubin Observatory Digest for more general news from the US observatory team. Given the Christmas and New Year break, the next newsletter will be a December/January issue, to be circulated towards the end of January.
The LSST:UK Community Alert Broker
R. D. Williams, K. W. Smith, G. Francis, A. Lawrence, S. Smartt, D. R. Young, M. Schwamb, C. Frohmaier, and T. Sloan
The LSST survey is focused on the dynamic sky: a source is not just brightness, but a history of brightness. One major product is the “alerts”, that report new sources and changes in brightness. Most observing nights are expected to produce millions of alerts, each perhaps 50 kB, so the data volume per night can be of order a terabyte. For a scientist to get what they want from this firehose, there will be “community brokers” that ingest the data and allow scientists to utilise it effectively. Only a limited number of brokers can be supported – because of the high data rates – so there is a competition. Thus in December 2020 the LSST:UK submitted a proposal to become a community broker.
Our broker is Lasair[1], the main partners being the University of Edinburgh and Queen’s University Belfast (Lasair means flame or flash in Scottish and Irish Gaelic). Although the LSST survey has not started, there is already a prototype transient stream – ZTF – on which we have built the first versions of Lasair. The architecture of the LSST stream will be similar to ZTF.
Lasair will provide a flexible and powerful platform that will enable worldwide users -- individual users, other projects, and citizen science -- to achieve their own science. Lasair will provide access to a rich variety of added value information and external data sources alongside the alert data, all of which can be interrogated, using queries, filters, watchlists, streaming queries and a programming interface. Lasair uses scalable technology and runs on the STFC-funded IRIS infrastructure. Although powerful, Lasair has an easy on-ramp for scientists from web pages and simple SQL, then on to Jupyter notebooks on their own machines, and then to high-performance mining co-located with the data. Lasair offers direct access with a staged approach: scientists can start with a simple, immediate mechanism using familiar SQL-like languages. These SQL-like queries can be custom made or users can choose and modify one of our pre-built and tested queries. These queries return an initial selection of objects, based on our rich value-added data content, and users can then run their own local code on the results. Users can build up to running their own code on both the stream and the database with high-throughput resources in the IRIS cloud. The SQL filters and code can be made public, shared with a group of colleagues, copied, modified, and excellent examples and their outputs are featured on the Lasair web page.
A broad overview of the Lasair design is shown in the figure below. Alerts arrive from the Rubin Observatory at left, and are cached and saved. Several “tagging” systems add value about sky context (Sherlock), external multi-messenger alerts, user-created watchlists of their own sources, classification engines, and featured of the light curves. User queries and filters are processed and results despatched, and the enriched alert stream kept in databases. The alerts can be utilised in several ways: by web, jupyter-style notebook, a programming interface (API), or received as real-time streams.
[1] https://lasair-iris.roe.ac.uk/
…
…
Recent LSST:UK outputs
LSST:UK has recently produced the following technical reports.
Title | Author | Description |
|---|---|---|