Summary
FlexNet Operations applications were intermittently unavailable for brief periods during the maintenance window .
Investigation / Analysis
We made an exception to the month-end change-freeze in an effort to remediate degrading database performance. The execution of the process to re-organize and re-build fragmented indexes on the primary FlexNet Operations database instance led to long-running, blocking transactions. In some cases, the open transactions prevented the execution of other operations leading to timeouts for other processes due to lack of available resources on the database.
Root Cause
The index re-build process that was implemented was inefficient at managing transactions and allowed them to continue for extended periods of time.
Resolution / Corrective Action
The immediate corrective action was to stop the optimization exercise and allow the systems to return to normal operating state.
We have identified an alternate approach from the Microsoft Developer’s Network which allows us to perform the operation in a non-impactful way while the applications are online. We are working through the steps to set up a regularly scheduled job to keep the databases running in an optimal state.