Lasair—User Documentation for Jupyter Notebooks Service

These notes aim to document the process of getting access to and using the LSST UK Jupyter pilot service. They are only intended to cover those aspects specific to this service rather than being a general user guide for Jupyter Notebooks.

Registration and Authorisation

The service uses EGI Check-in for authentication and authorisation. This enables you to sign in using your home institution login.

To access the LSST Jupyter pilot service you need to be a member of the UK LSST group. In order for your login to be added to the group you will first need to sign up with the EGI registry at https://aai.egi.eu/signup. Full documentation on how to complete the signup process is at https://wiki.egi.eu/wiki/AAI_usage_guide.

The signup system will send you an email that you respond to to complete the sign-up process. If you do not see it, PLEASE CHECK YOUR SPAM FOLDER.

Once you are registered in EGI Check-in please email lasair-help@lists.roe.ac.uk and ask to be added to the UK LSST group.

Once you are added to the group go to https://jupyter.lsst.ac.uk/

Having read the accceptable use policy, click on the "Sign in with EGI Check-in" button.

Starting a Server

If you have logged in successfully, the next page allows you to select the image to use and resources to allocate when starting your Jupyter server.

The size options should be fairly self-explanatory. At present these are both guarantees and limits.

A small query will work well on a small server. However, if that query is modified to return a lot of data, the server may JUST STOP DEAD with no explanation! It has run out of memory. The solution is to start a new server which is Large or Xlarge to run the bigger query.

The "standard" and "data science" images are from the public Jupyter docker registry. The ZTF image is based on the standard image, but with a number of additional features described below.

Available Libraries

In addition to the standard libraries, the ZTF image has the following installed by default:

mysql-connector-python
matplotlib
astropy
extinction
mosfit
astrorapid
healpy
wget

If you need to install additional libraries from PyPI you can just use pip. For example to install the package "pip-install-test" you would do:

pip install pip-install-test

If you wanted to make it persist beyond your current session then you would do:

pip install --user pip-install-test

If you think a package should be added to the standard image then please let us know at support@lsst.ac.uk.

JupyterLab Interface

At present the classic Jupyter interface is the default. If you wish to use the JupyterLab interface instead simply replace the /tree at the end of the URL with /lab.

Git

If you are using the ZTF image you can use Git by starting a terminal and using the command line interface as normal. If you are using JupyterLab there are also GUI controls for many common operations (mostly in the Git tab).

For example, to get started with some example notebooks start a terminal and do git clone https://github.com/lsst-uk/jupyter_notebooks.git

ZTF Database

A read-only connection to the ZTF database can be obtained using something like this:

import mysql.connector
from ztf import settings
msl = mysql.connector.connect(\
            user    =settings.DB_USER, \
            password=settings.DB_PASS, \
            host    =settings.DB_HOST, \
            database='ztf')
cursor = msl.cursor(buffered=True, dictionary=True)
query = 'SELECT sgscore1,distpsnr1 from candidates where '
query += 'sgscore1 > 0.0 and sgscore1 <> 0.5 '
query += 'and rb >= 0.65 and nbad = 0 and fwhm <= 5 and elong <= 1.2 and abs(magdiff) <= 0.1 LIMIT 100000'
cursor.execute(query)

Further information

If you have questions or comments, or encounter any issues, you may contact support@lsst.ac.uk.