...
Here are all the moving parts of the system, an architecture that implements the required functionality. Services are set up and waiting, then each node starts consuming Kafka from the MIrrormaker cache. While 4 nodes are shown, it could be any number to achieve sufficient speed. Similarly with the sherlock cluster. Consumers can use the website to build complex filters, they can also use the Jupyter interface, consume a Kafka stream, or code against the Lasair API.
...
The proposed system implements the original diagram.
...