Lasair Cycle-1 Technology Review Meeting Page

This is a self enclosed wiki page containing the agenda and all the supplied review documentation. It is publicly viewable.

Post Meeting

We had a very successful meeting. We didn’t manage to squeeze everything we hoped for in the original planned meeting, but we covered all the key topics thoroughly, and learned a lot. We agreed to debate key decisions and follow-on actions at the Lasair telecon the following week. Key notes:

During the meeting, George kept some detailed notes.
After the meeting, Roy summarised some key questions
Then Andy added and amended: Andy’s version of key questions
Finally, in the Lasair meeting of Mar 25, we debated the key questions and edited the above wiki page to record conclusions and actions.

Update!

Because of the COVID-19 situation, this meeting is now being held online:

Join URL: https://zoom.us/j/294317035
Related Slack discussions https://app.slack.com/client/T4AL8RCP8/C0100SZJQMU

Rather than being spread over Tuesday and Wednesday, will now be 9-5 on Wednesday. Otherwise, the purpose and structure of the meeting is more or less unchanged.

Purpose and Goals of the Meeting

The Technology Review is one of four goals of our Cycle-1 plan for Lasair-LSST development. So far, we have a mature working prototype (Lasair-ZTF); clear science requirements; a high-level technology-free component and flow design; and a long term development plan, broken into six monthly cycles. The next step is to take some key baseline technology decisions, so that we can undertake detailed technical design and development. This meeting is intended therefore as a relatively lightweight, semi-formal preliminary Technical Design Review. The goals are to:

• Identify the key technology decisions to make

• Review the results of related tests and experiments

• Conclude as many key technology choices as possible

• Clarify remaining technology choices to be made

• Set further tests in order to reach conclusions

• Outline a technical design

Place/Date

Wed March 18th 0900-1700 vis Zoom
 https://zoom.us/j/294317035

Session-1: Introduction and Context

For general context, see papers O1-O6 below
Presentation slides will be uploaded on the day

Chair: Bob Mann
09:00 Introduction and goals of meeting (Andy Lawrence) [slides PDF]
09:10 Context: Rubin observatory, LSST and the construction schedule (Slides) (Stephen Smartt)
09:40 Lasair today: what it does, how it works – demo (Roy Williams)[see paper O5 and documentation]
10:10 Sherlock: what it does, how it works (Dave Young) [slide deck]
10:40 Lasair planning process (Andy Lawrence) [slides PDF][see also paper O2]
11:00 Coffee break

Session-2 Discussion of Key Technology Areas

11:30 Technology overview: what needs to be decided? (Roy Williams) (Paper PDF)

11:50 Key Area-1 : Receiving and Processing the Data Flow (led by Stephen Smartt)

• LSST public and private; galaxies, stars and rocks (Stephen Smartt) (Paper PDF) (Slides)

• Lasair ingestion with SSD (Ken Smith) (paper)(slides)

• Handling light curves (Roy Williams) (Paper PDF)

• Parallel ingestion and workflow (Roy Williams) (Paper PDF)

• Streaming queries (Gareth Francis)(Slides)

• Making a super Sherlock (Ken Smith) (Slides)

12:40 lunch break

13:20 Key Area-2: Databases, Storage, Internal Structure (led by Ken)

• MySQL vs Cassandra, query languages (Ken Smith) (paper)(slides)

• Blob storage requirements and technology (Gareth) (Paper PDF)(Slides)

14:10 Key Area-3: User services - querying, extraction, analysis (led by Roy)

• Light curve classification and features (Michael) (slides)

• Lasair and the LSST Science Platform (Stelios) (Slides)

15:00 tea break

Session-3 Key Technologies review (Stephen)

15:20 Recap of Day-1 discussions
15:40 Debate the key technology decisions

Session-4 Conclusions (Andy)

16:20 Conclude as many as possible
16:40 Review remaining technology decisions; set further evaluations
Discuss Outline Technology Design if time permits.
17:00 close

Key area issues

As we step through the presentations on the Key Areas, we should decide and record the key technology decisions needed, but will actually decide them in the closing session. Each discussion will have some presentations to be made available on the day, but the technology evaluation reports should be available in advance.

Supplied Paperwork

Some of the material will be verbal/powerpoint, but we will upload PDF versions here as they become available. Pre-supplied written papers will be as above:

Overview documents
(O1) High-level diagram: detailed version and simplified version
(O1b) Technology Challenges
(O2) Lasair-PhaseB plan
(O3) LSST:UK Phase B proposal
(O4) LUSC-A-02 LSST approach to alerts and variables
(O5) LUSC-A-08 The Lasair Broker (i.e. Phase A prototype)
(O6) LUSC-A-09 Initial science requirements for variable science

These are pre-existing written documents and will be uploaded here in advance of meeting.

Key Links

We intend this wiki page, the talks, and the written papers to be all that is necessary to participate in the review. However, for those who wish, and who have wiki access, here are some other key links:

The Lasair ZTF working prototype
Lasair Phase-B wiki page
LSST:UK Phase B planning pages
LSST:UK Science Requirements

Participants Confirmed