Emergency vehicles outside a fire station in Hillsdale County, which was impacted by a statewide network error. Courtesy | Facebook
Hillsdale County suffered a temporary 911 outage on Jan. 10 due to a system error that disrupted emergency calls statewide.
Peninsula Fiber Network, which provides 911 service across Michigan, said in a press release that a network error caused the disruption.
“An error occurred in the company’s optical transport network resulting in database corruption,” the press release said. “This caused a hardware/software mismatch resulting in some 911 calls to not be completed as expected or calls that lacked critical caller and address information.”
The company’s technicians found a disruption in the network at 3:15 p.m. on Jan. 10, the press release said. A backup system also malfunctioned.
Hillsdale County Central Dispatch announced the issue on its Facebook page at 3:42 p.m. the same day.
“Your 911 call may get routed to an adjacent county and they will take the information for us and we will get a response to you,” the center said.
Central Dispatch 911 Director Thomas Whitaker said the issue did not result in any missed calls in the county.
Whitaker said if there is an issue, calls normally roll over to the next county. Hillsdale calls roll to Lenawee County, Lenawee calls roll to Jackson County, and Jackson calls roll to Hillsdale County.
“To my knowledge, that was working,” Whitaker said. “I know Jackson was not 100% impacted as much as we were, but to my knowledge we did not miss any 911 calls. There was nothing reported to us from any other agencies.”
The center said that Text-to-911 was working if residents could not call, and told them to call the number that reaches the center from outside the county.
Text-to-911 is part of PFN’s Next Generation 911 system, which allows photos, video footage, and information from medical devices to be sent along with an emergency call, in addition to more precise location data.
According to Tim McKee, PFN’s Next Generation 911 Program Director, 82 of Michigan’s 83 counties use PFN’s Emergency Services IP Network.
“Not all dispatch centers were impacted but any dispatch center impact is too many,” McKee said in an email.
At 4:53 p.m. the center announced 911 lines were “back up and working.”
PFN General Manager Scott Randall said in the press release that multiple safeguards failed to stop the disruption.
“We take our mission-critical role in delivering consistent and reliable service to Michigan’s 911 operators very seriously and build multiple redundancies into our network. We also work with dispatch center managers to ensure processes are in place to reroute calls between centers when unexpected issues occur,” Randall said. “Despite our best preparations, some calls were still disrupted and for that we are deeply sorry.”
McKee said a very similar issue occurred in November, but it was not as large in scope.
“We identified the issue and are replacing the transport equipment in the network,” McKee said.
According to the press release, PFN is redesigning its network and installing new equipment to make its 911 system more resilient. The company says it is investing $6 million in the upgrades, which will take several months to complete.
“We are now making several upgrades to the whole system to eliminate the possibility of additional network errors in the future,” Randall said.
