Flexera One – IT Asset Management – EU – Reconciliation Failures

Incident Report for Flexera System Status Dashboard

Postmortem

Description: Flexera One IT Asset Management – EU & APAC – Reconciliation Failures

Timeframe: September 8, 2026, 1:01 PM PDT to September 15, 2026, 3:21 PM PDT

‌

Incident Summary

‌

On Tuesday, September 8, 2026, at 1:01 PM PDT, following the ITAM 2026 R1.3 release, our teams identified an issue affecting reconciliation processing in the EU Production environment. Some reconciliation jobs were unable to successfully retrieve processing results within the available processing window, resulting in failed or delayed reconciliation results for affected customers.

The issue was identified through monitoring and customer reports. Our teams also detected similar errors in APAC region. During the investigation, our technical teams implemented several service-level and infrastructure mitigations to improve the processing path and reduce reconciliation failures.

The investigation ultimately determined that a change introduced as part of the 2026 R1.3 release altered the timeout behavior for NextGen License Reconcile processing. This caused certain requests to fail earlier when an underlying performance condition occurred, making the issue more visible.

A corrective hotfix was developed and deployed on September 15, 2026, at 3:21 PM PDT. Following deployment, reconciliation processing was validated and the issue was resolved on September 15, 2026, at 10:28 PM PDT post monitoring to ensure sustained stability.

‌

Root Cause

‌

  • A regression was introduced as part of the ITAM 2026 R1.3 release.
  • The release introduced a new timeout mechanism for handling errors returned by NextGen License Reconcile.
  • Previously, the affected error condition could remain in a waiting/retry state for up to approximately eight hours.
  • The updated behavior reduced the timeout for handling the error to approximately three minutes.
  • Under certain conditions, reconciliation requests subsequently exceeded the platform's available processing window and failed before results could be returned successfully.

Contributing Factors

  • Long-running reconciliation requests: Some reconciliation operations required additional processing time and did not complete within the available timeout window.
  • Changed error-handling behavior: The updated release behavior surfaced failures that could previously remain in a retry state for a significantly longer period.
  • Variable processing behavior: The issue did not affect every reconciliation request consistently; some requests completed successfully while others experienced timeouts.

‌

Remediation Actions

‌

  • Hotfix development: Technical teams developed a corrective fix to address the reconciliation failure.
  • Service capacity adjustments: Teams increased capacity for the affected processing service during the investigation to improve processing reliability.
  • Infrastructure changes: A new service cluster was established and the associated service routing was updated as part of the recovery activities.
  • Configuration updates: Service timeout and connection settings were adjusted to improve the reliability of communication between the components involved in reconciliation processing.
  • Post-deployment validation: Teams monitored reconciliation processing following deployment and confirmed successful execution of the affected workflows.

‌

Future Preventative Measures

‌

  • Enhanced release testing: Expand testing of reconciliation workflows where processing times may approach configured timeout limits.
  • Timeout and retry validation: Review timeout and retry behavior for future releases that modify reconciliation error handling.
  • Review monitoring: Review and enhance monitoring for reconciliation processing delays, timeout conditions, and result retrieval failures.
  • Problem Management: Complete a full retrospective and track all identified improvement actions to closure under a problem record.
Posted Sep 29, 2026 - 22:44 PDT

Resolved

The hotfix was successfully deployed, and our teams closely monitored the services overnight to validate the effectiveness of the remediation and ensure continued stability.

No new errors were observed following the deployment, and the affected services remained stable throughout the monitoring period. Based on these results, this incident is now considered resolved.

Our teams will continue to investigate the underlying cause and identify any additional improvement opportunities. Further details, including the root cause analysis and corrective actions, will be shared in a post-mortem report.
Posted Sep 15, 2026 - 22:28 PDT

Update

The hotfix has been successfully deployed to the production environment. Our teams are currently validating the results and closely monitoring the service to ensure continued stability and performance.
Posted Sep 15, 2026 - 04:23 PDT

Update

Our investigation has identified an underlying defect that has the potential to impact any environment upgraded to version 2026 R1.3. To ensure continued service stability and performance, our teams are currently building and validating a targeted hotfix.

As a precaution, deployment of the 2026 R1.3 release to other regions has been temporarily deferred . Based on current progress, we estimate approximately 7 to 8 additional hours to complete the necessary validation and deployment activities.

Our teams remain actively engaged and continue to closely monitor the situation. We will provide further updates as they become available.
Posted Sep 14, 2026 - 23:12 PDT

Update

Our investigation has identified an underlying issue affecting a component involved in inventory processing. Recent platform changes have caused the issue to be detected and surfaced earlier in the processing workflow.

As an immediate mitigation, our teams have developed a hotfix . The hotfix is scheduled for deployment tomorrow morning (AEST).
In parallel, engineering teams are conducting a deeper investigation to implement a permanent resolution and improve the performance and reliability of the affected processing path.
We will continue to provide updates as additional progress is made and will share further details following the deployment and validation activities.
Posted Sep 14, 2026 - 02:56 PDT

Update

Our teams have continued to monitor the effectiveness of the configuration changes and processing timeout adjustments implemented to mitigate the impact of the underlying defect.
While the service remains stable overall, our teams continue to closely monitor processing activity to confirm the long-term effectiveness of these mitigations and ensure sustained service stability. The investigation into the underlying defect remains ongoing as we work toward a permanent resolution.

We will provide further updates as additional information becomes available or if there are any significant changes to service status.
Posted Sep 13, 2026 - 02:11 PDT

Monitoring

Our investigation has identified an underlying defect contributing to the inventory processing issues. To mitigate customer impact, our teams have implemented targeted configuration changes and adjusted processing timeout thresholds in all regions. Initial observations indicate improved processing behaviour; however, we will continue to closely monitor the affected services over the weekend to validate the effectiveness of these measures and confirm service stability.

We remain actively engaged in our investigation and will provide further updates as additional information becomes available.
Posted Sep 11, 2026 - 03:05 PDT

Identified

While the mitigation has reduced the number of failures, a small percentage of reconciliation requests continue to experience issues. Our technical teams continuing to investigate the remaining failures and the underlying cause of the extended processing times. We will provide further updates as soon as additional information becomes available.
Posted Sep 11, 2026 - 02:08 PDT

Update

We are continuing to monitor for any further issues.
Posted Sep 10, 2026 - 22:48 PDT

Update

Our technical teams have identified that some reconciliation requests are taking longer than the configured processing timeout, resulting in intermittent delays or failures for customers in the EU Production environment. As an immediate mitigation, changes have been made to the processing timeout , and early results show improved reconciliation success, with many of the requests that previously timed out now completing successfully. Teams are continuing to investigate the underlying performance issue and are working toward a permanent resolution. Further updates will be provided as significant progress is made.
Posted Sep 10, 2026 - 22:47 PDT

Update

The service continues to operate stably, and error levels remain very low. However, a small number of residual errors are still being observed. Our teams remain actively engaged in investigating these errors and are working to implement further mitigations to eliminate the remaining impact and ensure complete resolution.
Posted Sep 10, 2026 - 04:47 PDT

Update

Following overnight monitoring, our teams have confirmed that while the service remains stable overall, a small number of errors are still being observed. The error rate remains very low, but the issue has not been fully eliminated.
We continue to actively investigate the remaining errors and are working to identify and implement additional mitigations to address the residual impact. Our teams remain engaged and will provide further updates as more information becomes available.
Posted Sep 09, 2026 - 22:54 PDT

Monitoring

Alternate infrastructure has been created and the configuration has been updated to direct processing to the new environment. All executions of the previously affected processing step have succeeded since the change was initiated, with the last observed failure occurring at 2:43 AM PDT on September 9, 2026. Technical teams are continuing to monitor the transition and processing activity to confirm sustained stability.
Posted Sep 09, 2026 - 06:32 PDT

Update

Technical teams have narrowed the investigation to the reconciliation results retrieval process and are evaluating a potential capacity-related issue. As a mitigation measure, service capacity has been increased, and teams are monitoring whether this reduces reconciliation failures. Some reconciliation jobs continue to complete successfully, while others are experiencing timeouts when retrieving results.
Posted Sep 09, 2026 - 02:56 PDT

Update

Technical teams continue to investigate reconciliation failures in the EU Production environment. Some reconciliation jobs are encountering timeout errors while retrieving results from a dependent service, resulting in failed or delayed processing. Infrastructure-related errors are also being investigated, and the root cause has not yet been confirmed.

Technical teams are continuing to analyze service telemetry and infrastructure health data to identify the underlying cause and implement the appropriate remediation.
Further updates will be provided as they become available.
Posted Sep 08, 2026 - 22:38 PDT

Investigating

Incident Description: We are currently investigating an issue impacting the reconciliation process within the IT Asset Management platform in the EU region. Affected customers may experience failures when attempting to run reconciliation jobs. As a result, reconciliation processes may not complete successfully, potentially delaying data updates and downstream processing.

Priority: P2

Restoration Activity: Our technical teams have been engaged and are actively working to identify the root cause and restore services. Further updates will be provided as we continue working toward resolution.
Posted Sep 08, 2026 - 20:40 PDT
This incident affected: Flexera One - IT Asset Management - Europe (IT Asset Management - EU Batch Processing System).
← Current Status Powered by Atlassian Statuspage