All hosts are now running at realtime, future work will segregate high volume hosts traffic to prevent wider system impact.
Posted Oct 16, 2024 - 21:55 UTC
Identified
More event data being sent 'in' that can be processed, specifically, we're being rate limited by some customers Jira instances which is causing a backlog. Infrastructure limits mean this can impact all customers if sufficiently high volumes of hooks are present from such rated limited hosts.
We're going to create dedicated queues for such high volume hosts to solve this and isolate such problems in future.
Posted Oct 16, 2024 - 09:13 UTC
Investigating
Large volumes of webhooks have accumulated overnight for some hosts, that can reduce throughput, we are upscaling resources to help.
Posted Oct 16, 2024 - 07:25 UTC
This incident affected: Enterprise Mail Handler for Jira Cloud (JEMHC) (Outbound notification processing).