Rating the Cybersecurity Rating Firms: How Accurate Are They?

  • Post author:

By Russ Banham

Carrier Management

In just a few years, a growing crop of cybersecurity ratings firms has sprouted to assess the vulnerability of businesses to withstand cyber attacks, scoring them on a scale from good to bad. Key markets for the firms are insurance carriers and brokers, each using the ratings for different reasons.

Consequently, insurers have been wary about underwriting cyber risk policies with broad coverage terms and conditions. The complexity of the threat is so large and unwieldy that insurers struggle in modeling and quantifying potential loss frequency and severity. That’s where the cyber risk rating firms enter the picture.

InsurTech startups like Cyence, BitSight, SecurityScorecard, Cybernance, RiskRecon and others have formed to improve insurers’ understanding, identification and measurement of cyber risks. In scoring their risk assessments, the firms typically provide a simple rating using numbers, letters, or red, yellow and green traffic light symbols.

The ratings firms are not to be confused with cybersecurity consultancies that do a deep dive into a company’s network and systems to posit shortcomings. Rather, the firms provide a non-invasive way to assess a company’s exposures, giving a sense of how it might manifest itself to the hacking community, a group that includes nation-states, terrorist organizations, hactivists and old-time hackers seeking bragging rights.

The firms’ utility is wide-ranging. Carriers, for example, use the ratings in their underwriting deliberations and to determine aggregate cyber risk exposures across the books of business. Some insurers also offer the ratings firms’ benchmarking capabilities to their insureds as a service, helping companies compare their cyber risk preparedness and technical defenses to those of peer competitors. Brokers, on the other hand, leverage the ratings to bolster the argument of why a client needs to buy cyber insurance. The ratings also are useful in advising cybersecurity improvements to earn superior insurance treatment.

This is all well and good, assuming the cybersecurity ratings are accurate. As the recent WannaCry and Petya ransomware attacks demonstrated, hackers are in the business of confounding the world’s best cybersecurity professionals. Assessing a company’s risk exposure with a simple letter grade might serve a purpose, but only if the underlying scoring methodology is robust.

How foolproof are the cyber ratings firms? “They’re able to see a lot of information out there to assess relative degrees of vulnerability, but I’m not sure they’re at a stage where they can make accurate predictions,” said Tracy Dolin-Benguigui, director and insurance sector lead at S&P Global Ratings, which rates insurer credit risk.

She added, “Insurers shouldn’t be overly reliant on the scores as the sole basis for underwriting decisions. They’re just another tool in the toolbox.”

Different Strokes

The cyber risk ratings firms are not cookie-cutter service providers. According to their websites, BitSight and SecurityScorecard are focused on assessing the technical defenses of a company, whereas Cyence is more engaged in quantifying the potential financial outcome of a cyber incident. Some overlap is to be expected.

“We don’t really see ourselves competing with BitSight and SecurityScorecard,” said George Ng, Ph.D. (Economics/UC Irvine), Cyence’s chief technology officer and co-founder. “We’re an economic modeling platform. Customers like insurers can do an individual company analysis, looking at various metrics and assessment indicators focused on the organization’s cyber risk in financial terms—its ultimate economic exposure.” Ng formerly worked as a research scientist at DARPA.

BitSight, the first startup in the cybersecurity ratings market, touts the multiple uses of its scores by insurers. “Our product is being used by the largest insurers to develop much-needed cyber risk policies,” said Samit Shah, BitSight insurance solutions manager. “We also help underwriters write a policy at certain limits, terms and conditions. On the reinsurance side, the ratings help them negotiate better reinsurance terms, unlocking more capacity at better pricing for the market, which is a good thing for insurers and their customers.”

Shah makes a good point. All companies are eager to transfer their cyber risks. The challenge has been the wariness of insurers and reinsurers to absorb their exposures. If the insurance industry can get a better sense of cyber risks, the market potential is staggering. A report by Allianz projects double-digit growth figures on a year-by-year basis reaching more than $20 billion by 2025.

Cyber ratings firms are in business to do just that. Nevertheless, some insurers like Chubb are carefully validating. “We’re still in the exploratory phases and continue to refine our use of cyber ratings solutions to address underwriting, enterprise risk management and data analytics,” said Russ Cohen, Chubb vice president of cyber services.

Nevertheless, the giant property/casualty insurer is partnering with the ratings firms to provide a value-added cybersecurity benchmarking service to customers that buy its cyber insurance policies. “Policyholders can view their security scores and their comparative relationship to other companies in their sector for a period of 12 months,” said Cohen.

This service also allows cyber policyholders to monitor third-party vendors, which can be a pathway for hackers to invade a company’s network and systems. (An HVAC vendor was the entry point for the massive Target data breach in 2014.) Scores on three vendors are delivered for a 12-month period. “The outside-in perspective of these solutions can be a virtual canary in the coalmine, giving policyholders insight into what might be happening on their internal network without them even knowing about it,” Cohen said.

Interest and Intrigue

Chubb did not disclose the name of the cybersecurity rating firm with which it has partnered, considering this confidential information. Several other insurers declined the opportunity to be interviewed for this article. Two insurance brokers using the firms’ scores in different ways agreed to an interview, with one, Lockton, preferring not to divulge the name of the provider for proprietary reasons.

“We have relationships with a couple of the ratings firms for their specialized services, which are valuable to our clients,” said Michael Born, vice president and account executive in broker Lockton’s cyber technology practice.

Asked for an explanation of this value, Born cited the broker’s enhanced ability to help clients understand their cyber risk exposure. “Companies want to know the likelihood of a cyber attack and the financial impact, but the data to provide this information is hard to come by,” he said. “Depending on how deep the assessment goes, the ratings firms can tell us how up-to-date a client’s firewalls are to withstand an attack. This gives us an opportunity to reduce their risk.”

By improving the client’s risk profile, the broker is in a better position to place the company’s business with the insurance markets at optimal terms, conditions and pricing. “Underwriters get a better sense of how attractive the client is from a cyber insurance standpoint,” said Born. “When we go out into the markets, we can assure the best deal.”

Added up, the ratings firms help brokers sell cyber insurance to clients. As Born put it, “We use the scores modeling the client’s exposure from a likelihood and financial impact standpoint to say to the company, ‘Here is the potential cost if you don’t insure, and here is the cost if you do insure.’ They now see the benefits of buying the cyber insurance.”

Broker Marsh has had a relationship with Cyence going back two years to help its clients get a better sense of their cyber attack vulnerability. “The firm mirrors how the outside world sees the company, in terms of where its data is traveling to and from and who it does business with,” said Robert Parisi, Marsh managing director and cyber practice leader.

This outside world is the hacking community. “An analogy is you raise a son and send him out into the world, wishing you could see everywhere he’s going, which you can’t,” said Parisi. “In a cyber risk context, that’s what these firms do.”

Marsh relies on Cyence in several ways. For example, the broker receives the same volume of information on a client’s cyber exposures as the insurance markets receive when viewing the company’s risk profile. “When you apply for a mortgage, it’s good to know what your credit score is before you go to the bank,” Parisi said.

Cyence also can run a report on a client’s cybersecurity practices relative to its peers, whose names are anonymized in the document. For example, a financial institution with $1 billion in annual revenue would be compared to other financial institutions of similar size. The report evaluates the motivation for hackers to attack the company and its resilience in financially surviving the incident.

Asked how Cyence comes up with the assessment, Parisi said he was not at liberty to provide it. “That’s very much an internal discussion,” he said. “You’d need to ask the providers.”

Close to the Vest

We did. Unfortunately, the answers were not all that specific. While the ratings firms collect and analyze large swaths of cybersecurity data to score companies, the particular technology and processes to do this work is proprietary. This caution is understandable, given a need to keep this information from the hacking community. Nevertheless, the lack of transparency in how the firms devise their ratings is troublesome from a trust perspective.

Another concern is whether or not the ratings capture the full cost of a cyber incident. “One of the limitations of these models is whether or not they’re really capturing the contingent business interruption of a ‘cybergeddon’ attack, since this is where the really big losses for companies reside,” Dolin-Benguigui said. “Contingent business interruption is not an insured loss in many cyber risk policies. Consequently, the insured portion of a ‘cybergeddon’ attack would add up to mere basis points. There might be a need to broaden the scope of the tool to capture the economic loss.”

To get a better sense of the ratings firms’ value, we reached out to the chief security officer of a software company outside the insurance industry. Max Solonski is entrusted with overseeing the data security of the thousands of global and midsize customers of BlackLine, a provider of cloud-based financial and accounting software. Solonski was familiar with one of the cyber ratings firms, having researched it on behalf of a customer, and cognizant of the others.

“Here’s what I think—cybersecurity risk management must address so many fast-changing risks, all of them important from an insurer underwriting perspective,” Solonski said. “The cyber ratings firms use complex mathematics in their scoring methodologies to offer a perspective that might have some correlation with actual risk, or might not.”

As an analogy, Solonski pointed to the metrics produced by catastrophe modeling firms: “These firms provide useful information to insurance companies, noting that a particular region has a ‘one-in-100-year’ chance of experiencing a major earthquake. However, this doesn’t mean that once an earthquake hits the area another ‘one-in-100-year’ event won’t happen the following year.”

In other words, just because an insured is given a great cybersecurity score today doesn’t mean the company won’t be hit by another major incident tomorrow, particularly if it is a new type of cyber attack.

Solonski also questions if a rating firm partnered with an insurance company or broker may score a company higher on the risk scale to encourage the business to buy cyber risk insurance. “You know these stores that sell mattresses based on a score that tells you just how hard or soft you like the mattress, then you buy it and take it home and still sleep uncomfortably? Well, that’s how you sell a lot of mattresses,” he said.

Still, he sees some value in the scores as a mechanism to improve a company’s cybersecurity. He advised businesses that receive a score to have its veracity checked by a cyber risk consultancy.

Down the line, Cohen from Lockton believes the cyber ratings firms will prove their merit. “The historic challenge in underwriting cyber risks has been the very small pool of exposure and claims data to draw dependable conclusions,” he said. “These new solutions coming into the marketplace, on average, have about a good three years’ worth of decent data to work off of. In time, there will be much more information out there that they can add to their databases, making their scores vastly more reliable in gauging a company’s cyber risks.”

Leave a Reply