Comets are small, icy bodies left over from the process of building planets. They’re clues to our solar system’s formation and possibly even the origins of life on Earth. Despite being one of humanity’s oldest noted phenomena, comet data and imagery is scattered and not easily accessible, impeding science.
COMA changes that, aspiring to become the global resource for comet imagery, photometry, and molecular production data to help drive science on small solar system objects. Built in conjunction with NASA and the University of Hawaii’s Institute for Astronomy.
On the team, I’m primarily responsible for coding, technical design, and
architecture on our backend database APIs, processing pipelines, and ingestion.
Our ingest pipelines continually process scientific imagery and other data into
a form astronomers can easily use, and the APIs fuel querying the system by
scientists as well as processing from our science backend and sources like
NASA’s JPL. It also feeds our really nice frontend. Ingest pipelines and our
datalake are orchestrated via Airflow, and our
APIs are built on the excellent and highly performant Go
Fiber framework (after an initial MVP with Python’s
FastAPI). It also uses Asynq
for background and intensive, long-running processing tasks.
Bigger plans in 2026 include replacing our batch ingestion with moving to a more flexible streaming ingest system (using Watermill), particularly as we look to take on the massive volumes of data coming from large time-domain sky surveys like the Vera C Rubin observatory (aka LSST).
I’m really enjoying working on this with all its technical and scientific challenges. Learning heaps both code- and astronomy-wise. Very keen to do more work like this and even moar ambitious projects. So, if you found this via search and have interesting Astro/physics problems and projects, are interested in chatting with me about your astro/physics PhD program (I’m actively looking), or happen to be NASA, ESA, JPL, RocketLabs, VG or some other place doing cool space work and throwing objects into space, mail me. We should definitely talk.