Description: Flexera One UI – APAC, EU & NA – Potential Login Disruption
Timeframe: April 29, 2025, 13:00 PST to April 29, 2025, 14:55 PST
Incident Summary
On Tuesday, April 29, 2025, at approximately 13:00 PST, our engineering teams began investigating reports of intermittent login failures affecting the Flexera One UI across the APAC, EU, and NA production regions. Users attempting to log in experienced incomplete authentication flows, resulting in an inability to access Flexera One services, including IT Asset Management (ITAM), IT Visibility (ITV), and other related applications. The issue was confirmed through internal monitoring, which detected elevated authentication failure rates, as well as customer reports indicating widespread access disruptions. The incident persisted for approximately 1 hour and 55 minutes, during which time customers were unable to perform key operations within the platform. The disruption impacted user experience across all major regions until full service was restored. The suspected configuration change was reverted to its previous stable state to mitigate the impact.
Root Cause
The root cause was traced to a recent configuration update deployed earlier in the day that altered session handling behavior in the authentication layer of the Flexera One platform. This update introduced an unexpected conflict in how user sessions were processed. As a result, valid login attempts intermittently failed due to premature session expiration or misrouted authentication requests.
Although the change passed standard functional tests in lower environments, it was not subjected to the integration tests that would have replicated this edge case production-scale activity. This led to the oversight of the issue prior to deployment. And the issue was declared as restored post validations and customer confirmation At 14:55 PST.
Remediation Actions
· Rollback of Configuration Update: The suspected configuration change was reverted to its previous stable state to halt further impact.
· Service Health Checks and Restarts: Authentication services were restarted across impacted regions to ensure clean session state and refresh the service environment.
· Customer Communication: Customers were advised to clear browser cache and session data to eliminate residual client-side artifacts that may have persisted post-rollback.
· Validation and Monitoring: Engineering teams closely monitored login success rates and customer feedback. By 14:40 PST, metrics indicated full recovery. Final validation was received from customers confirming resolution.
· Restore Declaration: At 14:55 PST, the incident was declared resolved, with login functionality stable across all regions.
Future Preventative Measures
· Enhanced Pre-Deployment Testing: Incorporate stress and load testing as part of the QA process for all authentication and session-related changes.
· Improved Monitoring and Alerting: Strengthen monitoring of authentication workflows with region-specific metrics and alert thresholds for early detection of login anomalies.
· Postmortem Review and Training: Conduct a cross-functional review with engineering, QA, and operations to identify gaps in the deployment pipeline and reinforce best practices.