CASE STUDY
Reclaiming Security Amidst Critical Flaws and OEM Chaos
SecureOps enabled a technology hardware and services provider to overcome devastating zero-day SSLVPN vulnerabilities after an unacceptable response from the OEM.
About
Our client is a technology hardware and services provider that specializes in public safety. This story involves an acute cybersecurity crisis that occurred during our 25 year partnership.
Challenge
- Critical security vulnerabilities in SSLVPN technology due to zero day exploits
- Active exploitation of the vulnerabilities, risking a serious security breach
- OEM response was inadequate, releasing multiple ineffective patches
Solution
- Assigned team of architects to develop custom documentation for effective patching
- All-hands-on deck response to resolve issues as quickly as possible
- Migration to new Palo Alto SSLVPNs after losing confidence in the original vendor
SecureOps Difference
- SecureOps prioritized client’s security, going beyond contractual obligations
- Waived remediation and patching costs
- Accelerated migration to new SSLVPN solutions
Results
- Completed migration in one month instead of four
- Strengthened bonds between companies
- Used as a template example by customer’s CISO for how to do it right
CHALLENGE
Zero Day Exploits with a Confused Response from the OEM Vendor
In 2024, the OEM vendor for the SSLVPN technology that undergirded our customer’s network discovered critical vulnerabilities that left our customer vulnerable to breaches
The zero-day exploits and other issues could enable an attacker to gain a foothold in the network and escalate privileges. Erik Montcalm, Vice President of Services and Technologies at SecureOps, explained: “Our customers were tracing attackers across their network back to their VPNs. Literally they were getting hacked through these VPNs."
This posed an unacceptable risk for the telecommunications company, but remediation was more complex than it seemed.The patching updates and Common Vulnerabilities and Exposures (CVEs) from the OEM vendor were confused and ineffectual. “They released CVEs and patches multiple times. So, nobody felt like they had a good handle on it."
With each passing day, the company risked a major breach, exacerbated by new, confusing updates from the OEM. “We patched it Friday, thinking the problem was solved. Then on Monday, we had to start over, because they reissued the patch. And then on Wednesday, they reissued the patch again."
Active exploitation of the vulnerability made solving the problem a moving target, and the OEM vendor simply could not keep up. It was a frustrating and demoralizing situation. “It doesn't inspire confidence when you release three patches inside a week basically saying this one fixes everything. No, sorry. The one from yesterday doesn't fix it. This one fixes everything."
The company was in serious trouble. They leaned on the expertise of SecureOps to navigate them through the crisis.
SOLUTION
A Partner in the Trenches and a New Approach
SecureOps realized quickly that patching would require more support than usual for this kind of issue.
Support from the OEM kept changing, but each iteration of the patch was also unworkable. “This was so complicated that the patch instructions didn't work outright. If you followed the guidelines, you couldn’t get this done.”
In response, SecureOps brought in their team of architects to support their technicians in resolving the issue. "The OEM called it a patch, but you're basically reinstalling the device." So, the architects wrote custom documentation on how to reinstall the devices correctly, and set to work. “We went all hands on deck, deploying 10 people over a weekend, manning the crisis line until it's resolved."
It was an arduous process, but once patched correctly, the crisis was not yet over. Trust in the OEM was shattered, and the telecommunications company no longer had confidence in the infrastructure. “For the customer it appeared like the OEM had no idea what was going on.” Thankfully, the company had already been testing a PaloAlto SSLVPN solution with a one-year plan to migrate to the new provider. A year was simply too long to wait, so the decision was made to pivot to the new vendor and update the network as soon as possible.
This was no small task. A project of this scope would typically take four months of intensive work to complete. However, SecureOps and the company had a long history together, leveling many of the hurdles that lengthen project time.
“We participate in a lot of architecture level projects with this customer. We've been working with their lead architects for 25 years. They know our architects. They trust them." This familiarity and integration with their team, enabled SecureOps to complete the project in less than a month. “
When they say, ‘How can we do this in a hurry?’ Everybody knows what the process is. So it was pretty easy to fast-track a redeployment project. If this had been a brand new customer, it would have for sure been a multi-month project."
THE SECUREOPS DIFFERENCE
The Customer’s Security Was Top Priority Over Everything Else
This story is a testament to the value of long-standing partnerships, and SecureOps’ ability to put the customer first in a crisis.
Compared to industry standards, SecureOps went above and beyond contractual obligations to be the partner our customer needed when things were at their worst.
- Rapid and proactive response: SecureOps immediately deployed senior architects, normally outside the scope of the contract, to work around the clock to tackle the vulnerability in an all-out-effort.
- Customized solutions and support: The custom documentation outperformed vendor guides, enabling proper remediation of the vulnerability.
- Prioritized the customer over the contract: Installing and reinstalling an SSLVPN could cost as much as $5,000 per device, and doing so three times in one week could have amounted to a serious bill. SecureOps absorbed these costs, focusing on solutions.
- Accelerated migration: Leveraging deep expertise and familiarity with the customer environment, SecureOps fast-tracked a redeployment on an accelerated timeline.
- Constant communication: SecureOps stayed on the crisis line throughout the issue, functioning like a true extension of the team.
RESULTS
Hailed as Heroes
Although disruptions were unavoidable, SecureOps deployed the necessary response to minimize impact on the customer.
SecureOps prioritized the customer from the outset of the crisis and delivered solutions at a speed and level of care beyond what could be expected."This combination of vulnerabilities only happens once every few years."
And SecureOps leaped into action to navigate a particularly complex patching process. Moreover, migrating to a new SSLVPN in a month is exceptionally fast.
The tireless efforts and deep expertise strengthened the longstanding bonds between companies. “Their CISO actually built a story and went out to promote what they did to other leaders internally and other CISOs in the field."
It was a big win for the customer, resolving critical vulnerabilities and leaving them with a stronger, more reliable network. This is part and parcel of the SecureOps philosophy. “There's no phoning it in. Customers often say we go to war together, they're in it with us. That's the type of thing we want to be known for."
Build a security function that furthers your business goals
SecureOps is ready to join a partnership centered on your needs and ensure the protection of your most critical systems.