IntegrationAdvancedcdcgap-recoveryreconciliation
Recover missed Change Data Capture events with gap detection batch
Real World Scenario
Subscriber ReplayId gap detected 5000 Account changes missing from warehouse during Kafka broker failure.
Expected Answer
• Hourly reconciliation LastModifiedDate query vs warehouse max modified
• Gap fill Bulk API export missing Id range or timestamp window
• Alert ReplayId discontinuity automated monitoring
• Merge gap fill idempotent upsert warehouse
• Root cause Kafka retention vs subscriber downtime duration
• Document max acceptable gap SLA 1 hour vs 24 hours
• Periodic full reconcile weekly safety net
Follow-Up Questions & Answers
Click to expand — each follow-up includes a direct, interview-ready answer
Main difference: use case and scale. Hourly reconciliation LastModifiedDate query vs warehouse max modified. Gap fill Bulk API export missing Id range or timestamp window. Pick based on your integration pattern and team capability. CDC at-least-once plus gap possible—scheduled reconcile is insurance not optional. Optimize for scale and operational observability.
Architect Perspective
CDC at-least-once plus gap possible—scheduled reconcile is insurance not optional.