The Plugin People Status

App down

Incident Report for The Plugin People

Postmortem

Root cause

As part of our SOC 2 journey we were ‘hardening’ the network connectivity of the app internal infrastructure, whilst cleaning up and documenting the IP connectivity within the app, a change was made which broke outbound connectivity via NAT that caused a cascade failure over all nodes, this took a little time to track down.

Lessons learned

Security Group rules needed fully documenting, that is being done.

Impact

We expect no data loss from this outage, events will be retried (from your Jira to JEMHC) no inbound mail will have been lost.

Posted Nov 03, 2023 - 18:25 UTC

Resolved

Changes to the security settings on the outbound network (due to 'tightening' of settings) broke the internal healthCheck which caused all nodes to become marked as inactive, and this cause. We are documenting this for future reference!

Posted Nov 03, 2023 - 18:18 UTC

Investigating

We are currently investigating this issue.

Posted Nov 03, 2023 - 17:17 UTC

This incident affected: Enterprise Mail Handler for Jira Cloud (JEMHC) (Data Pack purchasing, Inbound mail processing, Outbound notification processing, Licensing, User Interface).