Attendees: RDW, SJS, RGM, MS, Time and venue: Wednesday 18th May 2020, via Zoom
Attendees: Andy L, Roy W, Stephen S, Bob M, Meg S, Ken S, Matt N, Dave Y, Britton S,, Mark H , Stelios V, Michael F, Ally H, Dave R, Gareth F, Terry S
Apologies:
Notes from discussion
DMR found some code for simulated alerts - https://github.com/lsst-sims/sims_alertsim
Roy Williams, Lasair today: what it does, how it works – demo
Jupyter Notebook platform will be integrated into LSST Science Platform (LSP).
Comments on accumulated size of data and of robustness of service.
ZTF has, to date, observed 85 million candidates and 2 million objects.
Believe LSST will be ~50x larger than ZTF.
Sources of unreliability include:
Disk filling up.
Network issue to ROE.
Lasair development undertaken on a development system.
Third system, running on IRIS OpenStack , is target for future platform.
Reliability is strongly linked to level of flexibility we give to users [RDW]
For example, freeform SQL gives users a lot of scope to create resource-sapping queries.
Mike R has significant experience of controlling users' scope for generating SQL, using--for example--maximum number of records, maximum query times.
Risk of malicious or ill-informed query generation.
Andy L noted balance of being conservative versus ambitious with technology choices.
Bob M asked if scalability experiments were needed to test scalability of watchlist function.
Roy noted no hard limit on watchlist, so scope for causing resource starvation. Further, watchlist queries run frequently.
Ken S noted option to scale test watchlists using a long list of variable stars.
Roy W believes Sherlock is better place in which to manage large, community-interest watchlists.
Bob M noted need to work out the size of watchlist that would trigger consideration of moving to Sherlock.