Probe: Employee error caused LIRR shutdown

A file photo of LIRR trains. Credit: David Pokress

By ALFONSO A. CASTILLOalfonso.castillo@newsday.comOctober 24, 2011

An LIRR employee on a team trying to fix damage caused by a lightning strike last month at Jamaica Station made matters much worse, essentially shutting down the nation's largest commuter railroad and stranding 10,000 passengers on board trains for hours, an LIRR investigation found.

The 22-page report, released Monday at a meeting of the Metropolitan Transportation Authority's railroad committee, revealed that a "programming error" brought down the new, $56-million switching and signal system east of Jamaica Station about 8 p.m. on Sept. 29.

Less than four hours earlier, the lightning strike at one of Jamaica's two main signal huts, known as Jay, had disabled the signal system west of Jamaica.

With some service from Penn Station restored, a Long Island Rail Road engineering team was attempting to fix the problem remotely using a computer server at Jamaica's other main signal hut, known as Hall, which controls movement to the east. But when the wrong programming was used to transmit information from one place to the other, LIRR president Helena Williams said, the signal system responded by shutting down, with 17 packed trains already en route.

"The system acted correctly. It failed safe," said Williams, who apologized for the nightmarish commute on Rosh Hashanah. "These were very intense circumstances that our troubleshooting team were faced with. They were trying to respond in a way that followed our protocols. . . . It was clearly human error."

It took two hours before members of the team realized the programming error, Williams said, "and then they had to undo it."

Both Penn Station and Jamaica were jammed with commuters for hours. It was the latest of more than a dozen major service disruptions since August 2010, many related to weather.

Ira Greenberg, the LIRR Commuter Council representative on the MTA board, said the mistake may be a sign of inadequate training. Williams said that "may be or may not be" the case, but added that the LIRR has plans to increase training and improve manuals to address the issue. She noted that the signal system, installed at Jamaica just last year, was supposed to have lightning protection and redundancies.

The system's diagnostic software incorrectly reported that the lightning had not damaged its microprocessors, when it did.

"Obviously, the adequacy or inadequacy of the system we bought is an integral part of the review and investigation by the railroad and by others," said MTA board member Mitchell Pally, of Stony Brook.

The report included a number of other actions the LIRR is taking to prevent a recurrence of what Williams called a "catastrophic shutdown." That includes more police involvement, especially to prevent incidents of passengers "self-evacuating" from stranded trains.

SEE PHOTOSLIRR trains and commuters through the years

The LIRR also is looking to tap into a "secondary indication" system that would provide engineers with some of the information they need to move trains even when the system is offline.

To address concerns of crowding and confusion during service disruption, the LIRR is publishing brochures with detailed information on alternative transportation options, including by bus, and may add audio speakers for Jamaica's street level.

The railroad said it will consult with Metro-North Railroad and NYC Transit for ideas on how to protect the system from lightning and electrical surges.

MTA inspector general Barry Kluger, who has launched an investigation into the LIRR's response, called the agency's internal report "a candid appraisal." But, Kluger said, he still has questions, including why it took another major disruption for the LIRR to implement many fixes it now is considering.

Some of the LIRR's new initiatives are included in its just-released "pledge to riders," which Williams said will be posted in trains and at stations. It was developed in response to Sen. Charles Schumer's call for a "commuter bill of rights" for LIRR riders during outages.

Schumer (D-N.Y.) told reporters in Jericho Monday the pledge lacked a necessary system of enforcing the LIRR's promises. He suggested such a component could include refunds and fines "when the railroad's pledge has not been met."

With John Hildebrand

LIRR Sept. 29-30 timeline

4:37 p.m.

Lightning strikes Jay signal hut at Jamaica station

All service suspended between city terminals and Jamaica.

All westbound trains ordered held at Jamaica for eastbound use.

Status of trains between city/Jamaica, 5 p.m.: 8 stranded, 7 standing, 5 reversed to stations.

First customer e-mail alert of temporary suspension.

5-6 p.m.

Troubleshooting in Jay signal hut begins.

LIRR workers manually set switches at Jamaica.

Riders advised to take E train to Jamaica.

Status of trains at Jamaica, 6 p.m.: 3 stranded, 2 standing, 1 goes east through Jamaica, 5 begin at Jamaica and go east, 5 go west to Jamaica.

Full-scale customer alerts warn of significant delays.

6-7 p.m.

Troubleshooting in Jay signal hut continues.

LIRR workers complete manual setting of switches, allowing limited service from city to Jamaica.

Status of trains at Jamaica, 7 p.m.: 0 stranded or standing, 3 go east through Jamaica, 2 go west through Jamaica, 8 begin at Jamaica and go east

Customer alerts continue, riders advised of limited service from city to Jamaica.

7-8 p.m.

Troubleshooting in Jay signal hut continues.

Buses set up to provide service from Huntington and Ronkonkoma to points east.

Status of trains at Jamaica, 8 p.m.: 0 stranded or standing, 7 begin in Jamaica and go east.

Customer alerts continue.

8-9 p.m.

Signal system in Hall signal hut fails.

Service east of Jamaica halted; service between city/Jamaica suspended.

Status of trains at Jamaica, 9 p.m.: 17 stranded, 9 standing, 1 reversed to Atlantic Terminal, 1 begins at Jamaica and goes east.

First train "self-evacuation" reported.

Customers again alerted of service suspension; later, in the hour, customer advised to seek other transportation.

9-10 p.m.

LIRR workers manually set switches, allowing limited service between city/Jamaica.

MTA Bus and emergency personnel begin reporting to Jamaica.

Status of trains at Jamaica, 10 p.m.: 12 stranded, 8 standing, 7 reversed to other stations, 1 begins at Jamaica and goes east.

Customers advised service remains suspended and they should seek other transportation.

10 p.m.-12 a.m.

Jamaica Control Center regains control of Hall Interlocking, 10 p.m.

Manually set switch controls are removed.

Hall signal hut control restored, 10:43 p.m.

Limited service through Jamaica resumes, 11:30 p.m.

Status of trains, 12 a.m.: 0 stranded or standing, 7 reverse to stations, 10 go east through Jamaica, 3 go west through Jamaica, 2 begin at Jamaica and go east.

Customers advised of restoration of limited eastbound service.