...
Cosimo Q - for Stephen regarding spectroscopic addition to Lasair, confused about the various projects. 4most tides does not do targetted opportunities. So it might be 20 to 100 days after the LSST discovery. Can understand how SOXS can be contribute but not so much 4MOST. So how much can these spectra actually contribute. What spectra information will be available and in what format?
StephenS A - correct that 4MOST pointing cannot be controlled but it is likely to follow the LSST footprints. It is possible to control the fibre pointing within the fields of 4MOST. Anything brighter than magnitude 22 in the field of 4MOST will have a fibre put on it. Spectra of 20K to 30K transients plus 30K host galaxies are expected but cannot control exactly when the spectra will be collected. The challenge is to get objects out of the alert stream and onto the the 4MOST schedule. If fibre can be put on them then when an object is 4MOST observed, then the LSST:UK UK WP3.3 will recognise that the fibre is in place, make estimates and the information will then go back to Lasair as a annotation. All of the data will be public in ESO archives. We will get information back asap into the broker.
Cosimo Q - do you plan the same same thing for SOXS.?
Stephen S A - yes, there will be some link from Lasair.
Jakob Q - How well defined is the UK programme . Have the spectra discussed what they require in terms of triggering from LSST? To what degree has the UK:LSST community specified their LSST plans, such that these can be sued used as Lasair requirements.SJS
Stephen S A- yes these discussions are
happening at
the conceptual level.
The triggers are
classified as supernovae by sherlock (not an agn, not a variable star…). Essentially if bright enough for 4MOST to take a spectra then put a fibre on it.
Jakob Q - sounds like UK will only do supernvoa?SJS
Stephen A - actually mean extragalactic transients
Question from JulienP - Funding context was highlighted, but the focus was on PI/developers. What about the hardware funding status and plan ?
Stephen S answer - This will be discussed in afternoon. Top level funding is secured. Lasair is a small part of the UK computing. LSSTUK is a small part of UK IRIS. So overall Lasair are not the dominant partner and hence we believe Lasair is safe. Andy agrees and explained that the plan for UK expected requirements is being put in to UKRI. This is for the 10 years operation and includes Lasair.
Session-2: ZTF prototype experience (chair: Stephen Smartt)
Lasair-ZTF and how it works (Roy Williams)
query, filter and stream are all the same in Lasair
Sherlock - cross-matches location of transient with other catalogs
Julien Q Julien to Roy - screen annotation - what is the feedback on this? What can you say about user-based annotations?
Roy answer - We have not built the system yet.
Michael will talk later about one of the classifications. We would like to share classifications from other brokers eg FINK, if acceptable.
Sara Q - stream annotation - are there plans to create a collaboation collaboration so people are not competing on the same follow-ups? Will the annotations become public on a “follow-up” list or similar?
Roy answer -
Marshall refers to a
group of
people able to submit opinions on transients and make decisions.
Andy answer -
Marshall building
is beyond our scope to solve this.
Jakob - annotatins annotations are hard. Decided We decided instead tobe to be flexible so people can do what they want but they then have the responsibility to maintain and make useful. On 2nd slide, no arrow to/from Rubin Science platform. Have you thought about jhow how this ealates tp relates to the DB contenscontents. How will eg. updated photometry be incorporated?
Andy answer- grimly aware need to think about this but not yet solved.
Jakob Q to Eric - how frequently can we expect updates and show seriosly seriously to take these.
Eric
answered - let discuss during talk
Science experience with ZTF (Matt Nicholl)
...
Roy will also explain why the architecture is what it is.
Platform maintains ~1-week cache of alerts from ZTF stream
Cassandra and Galera are both scalable. Aim to be able to dynamically scale to deal with changing load
New feature - external annotator - can receive a stream from Lasair and then query DB to draw conclusions. This might be a classifier – e.g., Zooniverse.
Lasair user interface is SQL-based (specifically, simple SELECT), but supports both on-demand and streaming queries
Supports community contributions (queries, classifications, etc.)
Architecture based on scale-up of low-power VMs
1445:Kafka processing (Gareth Francis/Roy Williams)
1455:
Kafka acts as the bus to enable the updates on the Lasair pipeline.
Hardware implementation (Roy Williams)
Julien P asked “Do you make the list of queries public such as others can take inspiration or just play it as is for their own work? Is this something the user that makes the query can choose (public/ private)?
A: Each query has a tick box for making public/ private, as you wish. Public queries can be copied and modified.
Julien P also asked “Is Sherlock doing batch or live processing of the incoming stream?”
A: Sherlock is doing batch processing, as is more efficient, though effectively same outcome as batches are very quick.
Julien noted batch processing may be less similar for LSST, due to data rates. May wish to prioritise processing based on science case. Has Lasair team thought about this?
Dave Y noted that Sherlock consumes whatever it is given. Code is multi-processed and scalable, so believe will be quick enough.
Gareth noted currently running batches of up to 800 alerts (if there are enough). This has been seen to be a reasonable compromise, taking 3--4 minutes according to content.
Andy L asked what was delay in seeing output from an alert
Roy noted around 30 minutes, based on recent sampling
Andy L noted that Rubin aspiration is 60 seconds.
Roy noted some of 30 mins might be within ZTF infra, rather than Lasair. Roy does not see significant use cases for 60-second avalability.
Eric agrees not a lot of science cases need 60-second turn-around. Has been proposed as de-scope opportunity, but has seen pushback from Rubin Directorate for this. Rubin is still aiming to achieve 60 second latency. Once science cases mature, scope to evolve functionality to meet requirements.
Andy noted that, if people need 1-min or 5-min latency, then hopefuly will be achievable somewhere in Rubin landscape – not necessarily in Lasair.
Jakob N noted that idea to have everything evolve around queries and streams is very nice and supports science reproducability. Was intrigued to see Kafka used as a bus, as not something considered by Jakob’s group, to date. Jakob using combination of SQL and No-SQL databases, but for different things. SQL for cut-outs, no-SQL for objects and annotations. Previously proposed to have workshop focusing on database choices. Royt confirmed that this would be interesting. Ken noted that Pit? Google trying to organise workshop on this in late October/ early November (Ken is on Organising Committee). Stephen proposes dedicated session on databases at workshop and encouraged Ken to press for this.
1505: Light curve classification (Michael Fulton)
1515: Discussion