Data Pipeline Observability

Twilio computes data lake freshness from the checkpoint Hudi already writes to S3

Offset lag said the consumers were keeping up while analytics teams found tables hours stale. The fix reads deltastreamer.checkpoint.key from the latest commit, seeks that offset in Kafka, and subtracts the message timestamp from now.