CrowdStrike and its enterprise customers are still recovering from a recent outage caused by a faulty update. The company has hired two security firms to further investigate the issue.
The company announced an external code review in a root causes analysis (PDF). During a post-incident review, it was discovered that a system called ‘Content Validator’ failed to validate content as intended, allowing a faulty IPS Template Instance to validate, leading to crashes caused by out-of-bounds memory reads.
CrowdStrike has announced that it plans to avoid similar update disruptions in the future by deploying updates to devices in a staggered manner. Additionally, its content validator now has runtime bounds to prevent memory issues from occurring. The company also intends to conduct more internal testing, but only time will tell if this will have a significant impact.
Even if you are not entirely sure what a content validator is or how memory reads can exceed their limits, you can probably understand that implementing a phased update rollout system is a good idea for a company with software installed on millions of Windows PCs.
CrowdStrike’s shareholders have filed a class-action lawsuit against the company for failing to implement a system. Delta is also suing for lost revenue over six days.
CrowdStrike claims that Delta’s loss is their own fault. CrowdStrike has stated that they believe the shareholders’ case lacks merit, but it’s difficult to argue against it, as the responsibility for implementing a rolling patch system lies entirely with CrowdStrike.