Update - Initial checks indicate that duplicated licenses are being retired as expected. We will continue monitoring to ensure consistent results across both the EU and APAC environments.
Feb 08, 2025 - 10:11 PST
Update - The rollback has been successfully applied in the EU and APAC environments, following successful testing in UAT. Our team will monitor reconciles over the weekend to ensure data is properly processed before initiating the cleanup in production on Monday.
While minor issues were encountered during testing, they are being addressed. We expect the cleanup to resolve the issue for the majority of customers, though a small subset may require additional follow-up. Further updates will be provided as progress continues.
Feb 07, 2025 - 14:45 PST
Update - The data cleanup process has been initiated in our testing environment, though some results are still under review. Our team is analyzing the outcome to ensure all issues are fully addressed.
Feb 07, 2025 - 12:11 PST
Update - Our team is diligently working to resolve this issue and has initiated the data cleanup process. We will continue to provide updates as we make progress.
Feb 07, 2025 - 02:56 PST
Update - Following successful system tests, we have proceeded with deploying the update. The next step involves data cleanup to address any remaining impacts, which is expected to begin tomorrow. Our team is working diligently to resolve this issue and will continue to provide updates as progress is made.
Feb 06, 2025 - 11:36 PST
Update - Our team has completed the initial build and identified areas requiring further refinement to ensure system stability. We are currently conducting comprehensive system tests to verify these improvements. If these tests are successful, we plan to proceed with the deployment of the update. We understand the impact this issue may have on your operations and are working diligently to resolve it. We will continue to provide updates as progress is made.
Feb 06, 2025 - 08:20 PST
Update - Our team continues to work through validation and testing. While the initial build was completed, additional refinements were necessary to ensure stability before moving forward. We are now addressing the final steps, and if all goes as planned, we anticipate proceeding to the next phase of testing soon. Further updates will be provided as progress continues.
Feb 05, 2025 - 13:55 PST
Identified - Our team is still awaiting the necessary build completion before proceeding with the deployment. Testing and validation efforts are ongoing to ensure a smooth rollout. We will provide further updates as progress continues.
Feb 04, 2025 - 16:01 PST
Update - To ensure comprehensive system stability, our team has been deploying additional hotfixes. However, we have encountered some issues, which we are currently addressing. The cleanup script has been completed and tested. We are actively working to resolve any lingering issues to proceed as planned.
Feb 04, 2025 - 08:37 PST
Update - Our team is currently conducting a thorough validation process to ensure system stability and accuracy. This involves detailed analysis and testing to confirm that all residual issues are effectively addressed. We are also performing a comprehensive data cleanup to remove any remaining impacts. We will continue to provide updates as progress is made.
Feb 03, 2025 - 15:11 PST
Monitoring - A past change that caused the issue has been reverted, and our teams are validating the rollback. Testing is actively underway to ensure everything functions as expected. Additionally, a data cleanup process is in progress to address any remaining impact.
Due to the need for thorough validation, testing, and cleanup across environments, the entire process may take multiple days to complete. If all goes well, the next phase of testing is scheduled for tomorrow. We will continue providing updates as progress is made.
Feb 03, 2025 - 10:00 PST
Identified - We have taken corrective action and are progressing through our testing processes.
Feb 03, 2025 - 09:25 PST
Update - Our teams are actively working on a fix to address the issue and take corrective measures, after which additional steps and validations will follow to ensure accuracy. We will continue to provide updates as progress is made.
Feb 03, 2025 - 07:07 PST
Monitoring - We are actively investigating and evaluating remediation options to resolve the problem.
Feb 02, 2025 - 21:55 PST
Update - We have escalated the priority of the incident to P1.
Feb 02, 2025 - 19:18 PST
Update - We are continuing to work on the fix and actively monitoring the environment over the weekend to prevent further impact. Our team is working to expedite the resolution and will provide updates as progress is made.
Feb 02, 2025 - 16:57 PST
Identified - We have identified an issue related to a past release that is causing duplicate license records to appear. Our team is actively working on a resolution and will provide further updates as soon as more information becomes available.
Jan 31, 2025 - 07:03 PST
Investigating - Incident Description:
Our teams have identified an issue impacting multiple customers in the EU and Australia regions. Affected customers may encounter duplicate records when importing licenses from the SaaS manager.
Priority: P1
Restoration activity:
Our technical team is actively working on identifying the root cause and restoring services. Further updates will be provided as we continue our efforts to resolve the incident.
Jan 30, 2025 - 23:49 PST