Cone Searching & HTM
...
Decimal | Binary | Base4 | |
|---|---|---|---|
HTM16 | 54680902005 | 110010111011001111000101100101110101 | N02323033011211311 |
HTM13 | 854389093 | 110010111011001111000101100101 | N02323033011211 |
HTM10 | 13349829 | 110010111011001111000101 | N02323033011 |
...
Remember our SSD vs HDD tests? I loaded up the 55 million rows of Gaia DR2 data into Cassandra. It took exactly 25 hours! (About 600 records per second.) Note though that inserts were being done single threaded, and not in “batch” mode.
I also pulled out 1,000,000 random rows based on the HTM16 id triangles from the RA and Dec of the million records. (1 hour and 50 minutes !)- or 6,600 seconds.) This is of course about 3 time slower than the HDD timings for one million rows. BUT we are not yet using Cassandra to its full capacity, and this was done single-threaded.
I’m certain these numbers will massively improve if we build a properly distributed Cassandra system. (To be done.) I’ve seen various statements online indicating that (e.g.) a 15 node cluster can cope with up to 120,000 inserts per second.
Conclusions & Further Work
...