Flexera One - IT Asset Management - EU & NAM - Multiple inventory imports are failing

Incident Report for Flexera System Status Dashboard

Postmortem

Description: Flexera One- IT Asset Management- EU & NAM - Multiple inventory imports are failing

Timeframe: March 18, 2025, 12:48 AM PST to March 21, 2025, 10:44 AM PST

Incident Summary

On Tuesday, March 18, 2025, at 12:48 AM PST, our teams detected an issue impacting inventory imports within the IT Asset Management (ITAM) service, specifically in the EU and NAM regions. Although the system remained accessible, certain large inventory compressed with older versions of the Beacon began experiencing failures in inventory uploads following the deployment of the 2024 R2.2 Production upgrade. Analysis revealed that the failures were linked to a change in the file compression mechanism introduced in the R2.2 release. Inventory files compressed with older versions of the Beacon agent could not be processed by the ITAM service, disrupting the import workflow.

Our engineering team developed a hotfix to restore compatibility with the older file format, which was successfully deployed on March 20, 2025, at 8:10 AM PST. Following the deployment, affected customers reported successful inventory uploads and normal operations resumed. The incident was declared as restored on March 21, 2025, at 10:44 AM PST, after validation and confirmation from all impacted cu that the functionality had been restored.

‌

Root Cause

Upon investigation, the root cause was traced to a compatibility issue in the file compression and extraction process. During the 2024 R2.2 upgrade, ITAM transitioned to a new compression library. However, inventory files compressed using the older version of the Beacon agent failed to extract properly using the updated software, while the newer version of the Beacon continued to function. As a result, inventory imports for those tenants could not be processed successfully.

‌

Remediation Actions

· Hotfix Development – The engineering team developed a targeted hotfix to address the compression incompatibility.

· Hotfix Testing – The fix underwent thorough testing to ensure stability and effectiveness.

· Production Deployment – The hotfix was deployed to production on March 20, 2025, at 8:10 AM PST.

· Post-Deployment Monitoring – Post-deployment, teams actively monitored the system and engaged with affected customers.

· Issue Closure Confirmation – The issue was officially marked as resolved on March 21, 2025, at 10:44 AM PST, after receiving confirmation from impacted customers that inventory imports were functioning correctly.

‌

Future Preventative Measures

· Backward Compatibility Validation: Implementing additional validation checks during upgrade testing to account for backward compatibility with legacy components such as older Beacons.

· Enhanced Monitoring: Strengthening observability around import processes to provide early warnings of partial or tenant-specific failures.

Posted Apr 07, 2025 - 04:54 PDT

Resolved

Continuous successful imports have been confirmed in the EU and US regions following our recent hotfix. This issue primarily impacted our US and EU customers and has been thoroughly addressed to restore full functionality of inventory imports.

Initially, we assessed a potential impact in the APAC region. However, these cases have been determined to be unrelated to the incident affecting the US and EU regions. We are actively addressing these isolated instances in APAC separately, and there is no broader impact observed in that region.

We will issue a formal post-mortem report in the coming days. This incident is now resolved.

Posted Mar 21, 2025 - 10:39 PDT

Monitoring

Following the recent hotfix deployment, we have observed successful inventory imports for multiple customers who previously encountered issues. We are closely monitoring the situation to ensure consistent performance across all regions.

Posted Mar 20, 2025 - 17:20 PDT

Update

The hotfix has been successfully deployed across all production environments. We are currently monitoring the situation and conducting additional validations to ensure its effectiveness.

Posted Mar 20, 2025 - 08:51 PDT

Update

Our technical teams have completed initial testing of the implemented fix and are now conducting further validations to ensure its effectiveness and stability. Given the complexity of the issue, we are taking a meticulous approach to minimize any potential risks. We remain on track and will continue to provide timely updates as we progress. Thank you for your patience and support as we work toward a resolution.

Posted Mar 19, 2025 - 04:43 PDT

Update

We have identified that the recent inventory import issues are due to a change in the file compression method used by our IT Asset Management (ITAM) system. Older versions of our Beacon software utilize a different compression method, leading to compatibility issues during the import process.

Our technical teams have implemented a fix, which is currently undergoing rigorous testing to ensure its effectiveness and stability. Given the complexity of the issue, this process is thorough and may span over the next few days. We will continue to provide periodic updates as we make progress. We appreciate your patience and understanding as we work diligently to address this issue.

Posted Mar 18, 2025 - 11:45 PDT

Identified

Incident Description: We are currently addressing an issue impacting inventory imports for IT Asset Management (ITAM) services in the EU & NAM regions. While the system remains accessible, some tenants are experiencing inventory upload failures following the recent 2024 R2.2 Production upgrade.

Priority: P2

Restoration Activity: Our technical teams have identified the root cause and have a workaround in place. They are actively working on a permanent resolution, and we will provide further updates as progress continues.

Posted Mar 18, 2025 - 01:18 PDT

This incident affected: Flexera One - IT Asset Management - North America (IT Asset Management - US Inventory Upload), Flexera One - IT Asset Management - Europe (IT Asset Management - EU Inventory Upload), and Flexera One - IT Asset Management - APAC (IT Asset Management - APAC Inventory Upload).