Paige St. John: Florida Insurers rely on dubious storm model
Nov 14, 2010
The following article was published in the Sarasota Herald-Tribune on November 14. 2010:
Florida insurers rely on dubious storm model
By Paige St. John
Hurricane Katrina extracted a terrifying toll — 1,200 dead, a premier American city in ruins, and the nation in shock. Insured losses would ultimately cost the property insurance industry $40 billion.
But Katrina did not tear a hole in the financial structure of America’s property insurance system as large as the one carved scarcely six weeks later by a largely unknown company called Risk Management Solutions.
RMS, a multimillion-dollar company that helps insurers estimate hurricane losses and other risks, brought four hand-picked scientists together in a Bermuda hotel room.
There, on a Saturday in October 2005, the company gathered the justification it needed to rewrite hurricane risk. Instead of using 120 years of history to calculate the average number of storms each year, RMS used the scientists’ work as the basis for a new crystal ball, a computer model that would estimate storms for the next five years.
The change created an $82 billion gap between the money insurers had and what they needed, a hole they spent the next five years trying to fill with rate increases and policy cancellations.
RMS said the change that drove Florida property insurance bills to record highs was based on “scientific consensus.”
The reality was quite different.
Today, two of the four scientists present that day no longer support the hurricane estimates they helped generate. Neither do two other scientists involved in later revisions. One says that monkeys could do as well.
In the rush to deploy a new, higher number, they say, the industry skipped the rigors of scientific method. It ignored contradictory evidence and dissent, and created penalties for those who did not do likewise. The industry flouted regulators who called the work biased, the methods ungrounded and the new computer model illegal.
Florida homeowners would have paid more even without RMS’ new model. Katrina convinced the industry that hurricanes were getting bigger and more frequent. But it was RMS that first put a number to the increased danger and came up with a model to justify it.
As a result of RMS’ changes, the cost to insure a home in parts of Florida hit world-record levels.
Hundreds of thousands of homeowners were forced to find new insurers as national carriers fled the state.
Yet the prediction of a more dangerous Florida has not played out.
The new RMS model called for at least 11 hurricanes to come ashore in the United States by the end of 2010, most of them aimed at Florida.
Four hurricanes struck the U.S. None hit the Sunshine State.
RMS stands by its five-year outlook and contends that the risk of hurricanes remains higher than normal. Company officials last week said they would continue to adjust their model as needed, but a single five-year lull does not disprove their results.
Yet a growing number of experts now wonder if the changes spurred by RMS — and the accompanying spike in insurance premiums — were justified.
The woman credited with launching the industry of hurricane modeling questions how near-term models were introduced. She accuses RMS of overselling software that lacked sufficient scientific support, and says insurers accepted the output of that model as if it were fact.
“I’ve never seen the industry so much just hanging on what a handful of scientists or one model would say,” said Karen Clark, founder and former CEO of AIR Worldwide, an RMS competitor.
“They’re just tools,” Clark said.
“They’re models.
“They’re wrong.”
FOUR MEN, FOUR HOURS
The daily papers were still blaring news about Katrina when Jim Elsner received an invitation to stay over a day in Bermuda.
The hurricane expert from Florida State University would be on the island in October for an insurance-sponsored conference on climate change. One of the sponsors, a California-based company called RMS, wanted a private discussion with him and three other attendees.
Their task: Reach consensus on how global weather patterns had changed hurricane activity.
The experts pulled aside by RMS were far from representative of the divided field of tropical cyclone science. They belonged to a camp that believed hurricane activity was on the rise and, key to RMS, shared the contested belief that computer models could accurately predict the change.
Elsner’s statistical work on hurricanes and climatology included a model to predict hurricane activity six months in advance, a tool for selling catastrophe bonds and other products to investors.
There was also Tom Knutson, the National Oceanic and Atmospheric Administration meteorologist whose research linking rising carbon dioxide levels to potential storm damage had led to censoring by the Bush White House.
Joining them was British climate physicist Mark Saunders, who argued that insurers could use model predictions from his insurance-industry-funded center to increase profits 30 percent.
The rock star in the room was Kerry Emanuel, the oracle of climate change from the Massachusetts Institute of Technology. Just two weeks before Katrina, one of the world’s leading scientific journals had published Emanuel’s concise but frightening paper claiming humanity had changed the weather and doubled the damage potential of cyclones worldwide.
Elsner said he anticipated a general and scholarly talk.
Instead, RMS asked four questions: How many more hurricanes would form from 2006 to 2010? How many would reach land? How many the Caribbean? And how long would the trend last?
Elsner’s discomfort grew as he realized RMS sought numbers to hard-wire into the computer program that helps insurers set rates.
“We’re not really in the business of making outlooks. We’re in the business of science,” he told the Herald-Tribune in a 2009 interview. “Once I realized what they were using it for, then I said, ‘Wait a minute.’ It’s one thing to talk about these things. It’s another to quantify it.”
Saunders did not respond to questions from the Herald-Tribune. Knutson said if RMS were to ask again, he would provide the same hurricane assessment he gave in 2005.
But Emanuel said he entered the discussion in 2005 “a little mystified” by what RMS was doing.
He now questions the credibility of any five-year prediction of major hurricanes. There is simply too much involved.
“Had I known then what I know now,” Emanuel said, “I would have been even more skeptical.”
Elsner’s own frustration grew when he attempted to interject a fifth question he thought critical to any discussion of short-term activity: Where would the storms go?
The RMS modelers believed Florida would remain the target of most hurricane activity. Elsner’s research showed storm activity shifted through time and that it was due to move north toward the Carolinas.
But RMS’ facilitator said there was not enough time to debate the matter, Elsner said. There were planes to catch.
In the end, the four scientists came up with four hurricane estimates — similar only in that they were all above the historic average.
RMS erased that difference with a bit of fifth-grade math. It calculated the average.
Thus, the long-term reality of 0.63 major hurricanes striking the U.S. every year yielded to a prediction of 0.90.
Contrary to Elsner’s research, RMS aimed most of that virtual increase at Florida.
On paper, it was a small change from one tiny number to another tiny number.
Plugged into the core of a complex software program used to estimate hurricane losses, the number rewrote property insurance in North America.
Risk was no longer a measure of what had been, but what might be. And for Floridians living along the Atlantic, disaster was 45 percent more likely.
RMS defended its new model by suggesting it had brought scientists together for a formal, structured debate.
Elsner disputes that idea.
“We were just winging it,” he said.
PREDICTING APOCALYPSE
In the Oz of insurance, RMS is the man behind the curtain.
The company is a Silicon Valley prodigy created 22 years ago by four Stanford graduates and their engineering professor, who parlayed a research project into a commodity: calculating earthquake probabilities and selling them to the insurance industry.
It was a short leap from there to run odds on just about every terrible and unlikely event, from Florida hurricanes to Japanese typhoons to European tempests, what RMS CEO and co-founder Hemant Shah calls a “full portfolio of apocalyptic hazard events.”
The company Shah started from his California apartment is now a $200 million-a-year enterprise. Major insurance and reinsurance companies the world over pay annual subscriptions of $1 million or more to lease RMS’ disaster-predicting software.
The impact these private models have on the insurance price homeowners pay is so great that Bob Hunter, insurance director for the Consumer Federation of America, calls them unregulated “rate bureaus.”
For most of the past two decades, risk models have relied on actual hurricane activity recorded over more than 100 years to produce averages and other estimates of storm formation.
But even before Katrina, RMS was under pressure to disband the long-term outlook. Insurance insiders wanted something they believed would be more accurate. And they wanted it to forecast hurricane activity for next few years based on current conditions, not simply assume history would repeat itself.
The pressure came from several places. Some reinsurers sought validation that global warming was increasing the threat of hurricanes. Others in the industry wanted a short-term model to encourage investors, who wanted odds on their returns in the near term.
Shah says he had an obligation to pursue the short-term model because of the belief that hurricanes had gotten more dangerous.
“How are you going to incent people to mitigate their homes if you don’t have the right kind of signaling on what risk really is?” he told the Herald-Tribune in 2008.
An accurate prediction of the near future could save insurers billions of dollars by indicating when to raise rates or drop policies in places most likely to be ravaged. It’s the difference between predicting how many times the number 1 will appear in 100 rolls of the dice, and anticipating what number is expected for the next five rolls.
That, essentially, was what RMS promised.
RiskLink 6.0, RMS chief researcher Robert Muir-Wood wrote in a February 2006 column, “is likely to be the most eagerly awaited model ever introduced into the reinsurance market.”
RUSHING TO RAISE RATES
Records show reinsurers and insurers did not wait.
Using numbers RMS provided in its promotional materials, they began increasing their own hurricane loss estimates 30 to 40 percent, six months before the new model was finished in May 2006.
Florida insurers in turn sought rate boosts in anticipation of what the new model would do to their own costs.
But the yet-unpublished five-year model did not become an industry standard until December 2005, when it was embraced by A.M. Best, the Chicago firm that provides financial ratings for insurance investors.
Best said it would determine an insurer’s soundness by simulating its performance in back-to-back 100-year hurricanes as calculated by the five-year model.
The reasoning was simple.
“Catastrophe is the single largest threat of insolvency to an insurance company,” Devin Inskeep, senior financial analyst at A.M. Best, said in an interview.
According to a confidential presentation one of its officers gave an industry think tank, RMS calculated its new hurricane model raised the expected cost of a major U.S. hurricane by $55 billion.
Plugging that model into A.M. Best’s stress test meant the industry as a whole would need to raise $82 billion to remain solvent.
RMS’ two chief competitors argued there was inadequate scientific grounding to heavily promote a five-year outlook.
Clark, at the time CEO of AIR Worldwide, said she urged A.M. Best to reconsider requiring a model “based on theories.”
Having alternative models available was good, she said, but “I personally was an advocate of not rushing into something that was not tested and would have a dramatic change. Certainly, I had a lot of conversations with A.M. Best.”
The warnings were not heeded. Both Eqecat and AIR eventually produced their own five-year versions, though AIR warned clients it considered the only credible version to be the long-term model.
By January 2006, five months before RMS released its new model, at least half a dozen reinsurers were pricing their contracts based on the new numbers, comments made in quarterly earnings calls show. The pricing triggered a cascade of rate hikes in Florida.
In a calculation Florida regulators learned about two years later, State Farm added a $1.5 billion “frequency adjustment” to its potential hurricane losses. That, in turn, required it to buy more reinsurance from its parent, a cost that resulted in a 47 percent rate increase to its Florida customers.
Allstate increased the loss estimates of its long-term hurricane model by 41 percent, a “climate cycle” adjustment it only briefly noted within its 4,000-page request for a 22 percent rate hike.
By the time the actual model was released in May 2006, it had already reshaped the Florida property insurance market, unleashing the largest spike in premiums in state history.
Florida has a law intended to prevent just such chaos.
A state commission must review and approve catastrophe models before insurers may use them to set rates. No short-term model has ever passed that test.
RMS in 2007 submitted its model for review by the Florida Hurricane Loss Methodology Commission — the only body of its kind in the nation.
Meteorologists, statisticians and engineers for the commission began a lengthy review. But when RMS learned those reviewers planned to reject the model, the company withdrew it from consideration.
A draft report shows the objections centered largely on how RMS had determined its new hurricane rates.
The panel said the model change failed to meet credibility and bias tests, and it questioned how RMS had picked its four scientists and why so few were invited.
Shah later told the Herald-Tribune he believed Florida was “mucking things up,” suppressing a credible view of risk “so pricing can be more affordable.”
“If you artificially constrain your view of risk then you’re not going to have the clarity of insight that suggests what really needs to be done to solve the problem,” he said.
RMS continues to promote its short-term model as the preferred option for its customers. A survey by Bermuda officials shows it is the dominant model for Bermuda reinsurers, the most crucial source of private hurricane protection for Florida.
MONKEYS COULD DO THIS
At the outset in 2005, RMS promised to revisit its forecast at the end of every season. “If there is a material change,” the company said, “rates would be updated.”
So it was in October 2008 that RMS assembled a group of seven weather science experts at the Hotel Victor on Miami’s South Beach.
Rather than produce their own storm predictions, they were asked by an expert in gathering scientific opinion to rank 39 different climate models that RMS would then run to produce a five-year forecast.
The man running the show was Tony O’Hagan, a British statistician who had developed drug trials for AstraZeneca. He came armed with Tiddlywinks, 30 for each scientist, to help them visualize and rank the weather simulators.
What struck University of Colorado environmental science professor Roger Pielke as he played with his pile of green chips was the pointlessness. Pielke, already a critic of the five-year forecast, believed the 39 models were a stacked deck, “biased upwards.”
RMS said it gave its experts the option of sticking with a long-term average. “We were strongly encouraged not to do so,” Pielke said.
Another participant, Georgia Tech climatologist Judith Curry, had her own misgivings. She believed the selection too narrow.
“I thought all of the models were wrong. I didn’t have confidence in any of them,” Curry said.
When RMS averaged the scientists’ choices, the number of expected storms had dropped from the previous finding in 2005.
This time, the number of Category 3 and higher hurricanes expected to strike the U.S. each year dropped, from .9 to .8, a seemingly small change.
That decrease meant the risk of hurricanes had dropped by a third. Presumably, homeowners’ premiums should follow suit.
But there was no rush to adjust homeowners’ bills and no publicity surrounding the new scientific “consensus.”
RMS in December 2008 described the results as “consistent” with past findings. It disclosed the lower numbers six months later in an April 2009 confidential report to clients. By then it was too late to effect that year’s reinsurance rates for many insurance companies.
Company vice president Claire Souch denied that RMS promoted the increase and downplayed the decrease. “Our time lines were the same,” she said.
Even after it was released, brokers said, the revised model was not roundly embraced.
“It is true that many ‘set aside’ the model change when underwriting this year,” said John DeMartini, vice president at the Towers Watson brokerage.
“While they were quick to adopt near-term when it raised loss estimates, they didn’t commit to sticking with it through reductions.”
Following the unusually inactive 2009 season, RMS announced it would skip its annual expert review. By fall 2010, RMS had changed its methodology to remove the human element, Souch said. Souch said a new model will be released in February. It is expected to decrease rates along the coast and increase them inland, RMS officials said.
For his part, Pielke returned to Colorado and set up a random number generator to rank RMS’ 39 climate models from 2008 — akin to blindly throwing darts to choose the best model.
The outcome nearly matched the scientists’ consensus.
“So with apologies to my colleagues,” he wrote in his science policy blog, “we seem to be of no greater intellectual value to RMS than a bunch of monkeys.”
Hurricane Katrina extracted a terrifying toll — 1,200 dead, a premier American city in ruins, and the nation in shock. Insured losses would ultimately cost the property insurance industry $40 billion.
Find this article at: http://www.heraldtribune.com/article/20101114/ARTICLE/11141026/-1/news300?Title=Florida-insurers-rely-on-dubious-storm-model