Combination of Errors Led to Power Loss in San Diego
No new technology is needed to prevent a repetition, only stricter adherence to operating rules, the investigators suggested in a 151-page report.
The failures were “pretty basic things,” said David R. Nevius, senior vice president of the North American Electric Reliability Corporation, which is designated by the federal government to assure grid reliability and investigated the blackout in tandem with the Federal Energy Regulatory Commission.
Still, the investigation revealed new vulnerabilities in the grid. For example, electric companies are supposed to routinely analyze anticipated conditions for the next day. But the Imperial Irrigation District, an electric utility in the Imperial Valley east of San Diego, issued analyses that showed weather, load and generation forecasts but failed to evaluate the underlying power flows, the report said.
Just as in 2003, some utility operators also failed to maintain updated computer models of all the electric system’s components, which help them identify vulnerabilities.
The blackout began on Sept. 8 when a 500-kilovolt east-west transmission line, the Hassayampa-North Gila, failed. A technician was trying to isolate some of the equipment so he could test it and skipped several steps, which led to a short circuit and a shutdown of the line.
Electric grids are supposed to be set up so that no single failure will cause a cascading blackout, by shifting the location of electricity generation in a way that reduces the load on transmission lines. But on that day, because of the way that operators had configured the system, the single failure was enough to cause a blackout at 3:38 p.m.
The blackout affected 2.7 million customers — probably about 9 million people — from San Diego to the Imperial Valley to Baja California. It all but shut down San Diego’s airport and caused traffic jams as drivers tried to navigate roads without stoplights. Power was restored to most areas within 12 hours.
The blackout’s scope could have been limited if operators had been trained to intentionally cut off some areas, the investigators reported. As with the Eastern blackout in 2003, system operators had only poor knowledge of what was happening in neighboring systems, which prevented them from taking proper action until it was too late, their report said.
After the original transmission-line failure, power flowed over parallel paths and overloaded some of them, causing a cascading failure. That cut power to the two-reactor San Onofre nuclear power station, which responded by shutting down, adding to the system’s instability.
A second problem emerged with automatic disconnect systems that are intended to protect power lines, generating stations and transformers from damage, the report said. While some of that equipment helped the system by protecting components from overheating, investigators said, some it may have operated too early, guaranteeing a cascade.
Disconnects on hair-trigger settings also contributed to the 2003 blackout in the East.
Stephen Berberich, the chief executive of the California Independent System Operator Corporation, which runs the grid in most of California, told reporters in a conference call that “we’re going to do as much as we can” to improve computer modeling of the Western grid and coordination between adjacent “risk areas.”
Like the Eastern blackout, the problem was not a shortage of generating capacity or transmission, the investigators emphasized. But when the first line failed, other components became overloaded. While the report made clear standards had been violated, potential penalties will be taken up by the agencies seperately.