Skip to content

Add feasibility study: JPL Scout → Kafka bridge for Rubin ToO alerting - #44

Open
talister wants to merge 1 commit into
mainfrom
scout-kafka-bridge-feasibility
Open

Add feasibility study: JPL Scout → Kafka bridge for Rubin ToO alerting#44
talister wants to merge 1 commit into
mainfrom
scout-kafka-bridge-feasibility

Conversation

@talister

Copy link
Copy Markdown
Collaborator

Summary

Adds docs/scout_kafka_bridge_feasibility.md — a feasibility study for a containerized service that polls JPL Scout and republishes new/updated/cancelled Rubin-ToO-candidate objects as a Kafka stream for the Rubin Observatory ToO system.

Key points:

  • Rubin's first-year ToO operations paper (arXiv:2607.00217) explicitly notes no Kafka stream exists for hazardous asteroids and invites one; Rubin ingests all external ToO alerts via SCiMMA Hopskotch.
  • Broker options evaluated: SCiMMA Hopskotch (recommended, $0) vs self-hosted Kafka on AWS (MSK ≈ $547/mo minimum; single-node ≈ $30–60/mo + ops, and Rubin would need a bespoke consumer).
  • Architecture scoping concluded on a standalone TOM Toolkit project depending on tom_jpl (no logic forking; Postgres transactional outbox; ~1 GB SPICE-free image) over a Django-free micro-poller or building it into FOMO.
  • Deployment via LCO's standard GitOps pattern: app repo lsst-sssc/scout-alert-bridge (public ghcr image) + deploy repo LCOGT/scout-alert-bridge-deploy (kpt/kustomize, sealed-secrets, ArgoCD, 10-min CronJob).
  • Includes a v1 event/message schema proposal, failure-mode analysis, costs, open coordination questions (SCiMMA, Rubin ToO team/SCOC, CNEOS, LCO infra), and prototype milestones M0–M5.

Note: the doc is standalone Markdown (not yet in the Sphinx toctree, which is rst-only); it renders on GitHub and can be MyST-ified later if we want it in the built docs.

🤖 Generated with Claude Code

https://claude.ai/code/session_01EmBXs3EBmEmMZRPZG7pESg

…rting

Assesses a containerized service polling JPL Scout and republishing
Rubin-ToO-candidate events to a Kafka stream (SCiMMA Hopskotch) for the
Rubin ToO Producer, including broker options, a TOM Toolkit-based
architecture reusing tom_jpl and the SSSC NEO ToO filter criteria,
LCO ArgoCD/GitOps deployment fit, event schema proposal, costs, open
coordination questions, and prototype milestones.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EmBXs3EBmEmMZRPZG7pESg
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant