Alert - SSO Service Disruption - August 15th 2019 *RESOLVED*

*RESOLVED*

Summary

On August 14 2019, from approximately 11pm EST until 7:30AM EST August 15th, our SSO service became unavailable for some customers. Any already signed in users or customers using Brightidea Login (Non SSO) remained unaffected but a limited amount of users that attempted to sign in using our SSO service were impacted.

Root cause Analysis

During our scheduled weekly deploy on August 14th, we deployed a change to our SSO service for capabilities to capture more detailed SSO transaction logs. This change resulted in an unexpected fatal error.

At 11:07pm & 11:24pm EST, log files for some of our servers were rotated, the newly created log file did not have correct write permissions and threw a fatal write permission error. This resulted in our SSO service failing.

Resolution

We rolled back the changes made to our SSO service for additional logs. We will also be expanding our automated test coverage in this area to assure this does not happen again.

Apologies for any inconvenience.

Thank you.

The Brightidea Team.