We worked to keep our customers updated during Harvey via email, website, and social media. We wanted to also keep record of those posts in our blog in case customers needed to refer to dates.
Hurricane Harvey Network Impact
CUSTOMER UPDATE for Wednesday, August 30th:
We are pleased to report that the sun is shining on most of Houston today and the forecast indicates for that to continue through the Labor Day weekend. The sunshine is a welcome relief from the incessant rains. With the sun, there is obvious relief from adding more water into the heavily flooded areas, but also offering a buoyancy to the morale here at PS Lightwave and the general Houston community.
More roads have become passable, and today we are happy to report that we have 65% of the staff back in the office, another 15% telecommuting, and expect to be at near normal staffing levels on Thursday. We are pleased to report that all of our employees are safe, a few have suffered flooding in their homes, and some are evacuated from their homes. Our staff has exhibited great dedication and commitment to their responsibilities of maintaining critical network services, volunteering and rescuing efforts, and keeping their families safe during this ordeal.
Our field crews have been actively working to restore services since Hurricane Harvey first hit. Although we have been successfully restoring service in some areas, there are a few locations that are completely inaccessible by any route with all roads closed due to high waters. We continue to attempt to drive to those locations and ensure those sites are up as soon as water levels decrease and we are able to reach them. PS Lightwave construction supervisors continue to drive the routes to proactively inspect the condition of the network and initiate any necessary repairs. Although services outages are our top priority, we are already allocating some of our resources back to “business as usual” in areas where it is feasible, turning our attention to new service commitments to ensure that we meet customer expectations for upcoming delivery dates.
As of this morning, there were still more than 250 high-water obstacles on Houston’s major highways according to Houston TranStar. That is down from more than 400 at the storm’s height. Much of the current flooding is due to releases from the reservoirs. According to the Harris County Flood Control district, uncontrolled releases are no longer expected, but water releases will continue for days. Houses and buildings in adjacent areas and roadways that run through the reservoirs are underwater. “The good news is we are getting very close to the peak at both reservoirs”. Mandatory and voluntary evacuations in portions of Harris, Brazoria, Fort Bend, Montgomery, Liberty counties and the cities of Richmond, Rosenberg and Waller are still in place.
The Houston area averages about 50 inches of rain per year, during the last two days, much of Houston has received between 25 and 30 inches of rain. That is the equivalent of what we would normally see in 6 to 8 months within 48 hours. Cedar Bayou, an area in east central Harris County received almost 52 inches of rainfall, setting a new record for the continental U.S. There are still major challenges in the days and weeks ahead, but we are making great strides toward recovery.
CUSTOMER UPDATE for Tuesday, August 29th:
As the Houston area closes its fourth consecutive day following Hurricane Harvey, flood warnings are issued through August 30 covering most of the PS Lightwave seven county footprint. The National Weather Service says flooding isn’t expected to peak until Wednesday or Thursday. “We are not out of the woods yet,” Elaine Duke, Acting Secretary of the Department of Homeland Security, said at a Monday morning briefing. “Harvey is still a dangerous and historic storm.”
In some areas, the roads are open and we are able to successfully respond and repair network issues. We have staff members that are manning generators to maintain power to various hub sites in order to keep services up due to power outages. PS Lightwave’s aerial line crews, fiber optics splicing teams, network operations staff continue to work around the clock to maintain the network under challenging conditions. Large areas of the Greater Houston area are still inundated with floodwaters, which turned streets into rivers navigable only by boat. We are forced to wait to address certain issues until the water subsides.
With almost all businesses and major business complexes remaining closed, Houston’s business community has been paralyzed by one of the largest downpours in U.S. history. With nearly 2 more feet (61 centimeters) of rain expected on top of the 30-plus inches (76 centimeters) in some places, authorities worried that the worst might be yet to come for some areas.
The Houston metro area covers about 10,000 square miles, an area slightly bigger than New Jersey. It’s crisscrossed by about 1,700 miles of channels, creeks and bayous that drain into the Gulf of Mexico, about 50 miles to the southeast from downtown.
The storm was generating an amount of rain that would normally be seen only once in more than 1,000 years, said Edmond Russo, a deputy district engineer for the U.S. Army Corps of Engineers, which was concerned that floodwater would spill around a pair of 70-year-old reservoir dams that protect downtown Houston. The flooding was so widespread that the levels of city waterways have either equaled or surpassed those of Tropical Storm Allison from 2001, and no major highway has been spared effects from flooding.
The city’s normally bustling business district was virtually deserted Monday, with emergency vehicles making up most of the traffic. Although the water has receded in some areas, it has risen and will continue to rise in other areas. Newly identified areas are being advised to evacuate.
Forecasters expect the system to stay over water with 45 mph (72 kph) winds for 36 hours and then head back inland east of Houston sometime Wednesday. The system will then head north and lose its tropical storm strength. Before then, up to 20 more inches (51 centimeters) of rain could fall, National Weather Service Director Louis Uccellini said Monday. That means the flooding will get worse in the days ahead and the floodwaters will be slow to recede once Harvey finally moves on, the weather service said.
The amount of water in Houston was so unprecedented that the weather service on Wednesday had to update the color charts on its official rainfall maps to indicate the heavier totals.
CUSTOMER UPDATE for Monday, August 28th:
With severe conditions expected through at least Wednesday, we are continuing to provide network operations technical support at all levels. Although many roads are impassable, we have been conducting daily inspections of key portions of the network to assess potential damages and access issues. We are grateful for the lulls in the rains, which facilitate repairs by our field staff.
Our PS LIGHTWAVE management and response team has been coordinating with local and county government clients to offer assistance and manage network changes to respond to ever-changing conditions and needs. Our support of those providing critical emergency response services is our highest priority at this time.
Any customers who find it necessary to shut down power for safety reasons, we would appreciate notifying us by either calling our NOC at 832-615-8000 or opening a ticket at firstname.lastname@example.org. This allows us to better isolate actual service impairment issues. We are tracking loss of power in building sites we service and have recorded over 40 at this time.
With the threat from tornados lessened, the torrential rains are causing the greatest current threat to the greater Houston area. The National Weather Service forecast rainfall of 15 to 25 inches through Friday, with as much as 50 inches in a few areas.
A serious threat is the overflow of waters from dams, rivers, creeks, and bayous. The U.S. Army Corps of Engineers reportedly is balancing flooding and damage on both sides of Addicks and Barker reservoirs. They plan to continue releases downstream along Buffalo Bayou via the two dams as long as they can without creating major impact downstream. More than 25 inches of rainfall behind the two mammoth earthen dams have the reservoirs spilling into the surrounding area, primarily suburban neighborhoods. “The volume of water flowing into the reservoirs is unprecedented in the dams’ history,” Edmond Russo with the Corps said.
With more rain coming, officials said the best course is release some of the water along Buffalo Bayou. Designed to handle a 1,000 year storm, Russo said in a Monday morning news conference the reservoirs are teetering on exceeding that level of flooding if worst-case rain scenarios occur. The Corps Engineers are referring to Harvey, not as the storm of the century, but the storm of the millennium for the Houston area.
CUSTOMER UPDATE for Sunday, August 27th:
As flooding conditions escalate in the greater Houston area, we are continuing to provide network operations monitoring and response support to our customers and service restoration, as safety and access permit us to do. We have experienced few service outages, mostly relating to power availability at the customer premises and along the Highway 521 corridor to Freeport, which has been only partially restored due to the power company having to delay any further repairs given current conditions and clear the area for telecommunications companies to make their repairs.
Our NOC (Network Operations Center) will continue to be manned and supporting our customers and our construction and warehouse site is up and running. On Monday, we will be observing the government recommendations and all personnel not involved with NOC support or operational and field restoration activities will telecommute and not be required to come to work. The majority of area schools will also be closed all of next week. Please check online for updates on business and school closings here: http://www.khou.com/closings
With the current heavy rains expected to continue, we are expecting that the next 24 – 36 hours will be the worst impact and continued risk of Hurricane/Tropical Storm Harvey to the greater Houston area. Most all bayous, rivers, and dams are at record high levels and are contributing to the flooding risk. An increased number of streets and areas of the city will likely become inaccessible by vehicles. Local television station KHOU, Channel 11 was forced to evacuate the building and shut down due to flooding along Allen Parkway. Much of downtown and other areas of Houston are impassable. The National Guard has activated support, along with assistance from other state and federal agencies.
If you have any questions please feel free to contact us anytime. The PS LIGHTWAVE phone number is 832-615-8000 and our website is pslightwave.com. Follow us on Twitter, LinkedIn, and Facebook for more frequent updates.
CUSTOMER UPDATE for Saturday, August 26th:
Current conditions in the PS LightWave network are good with minimal issues. We are seeing loss of power in some areas of town, and we are prepared with secondary and tertiary power back up to protect critical hub sites. Road conditions throughout our seven county footprint are not impeding our crews. Tornados have, and may continue to be an issue, causing considerable damage in the Ft. Bend area, but have not been service impacting to our network.
PS LIGHTWAVE is still actively in the Disaster Preparedness and Recovery Plan .
We have crews on call and standing by in the known heaviest hits areas ready to respond to any major damage discovered or reported. Also, our NOC continues to be staffed 24/7 in our primary Network Operations Center and has been monitoring network systems and power outages throughout the night and will continue to do so.
Hurricane Harvey has been downgraded to a category 1 storm and the eye has collapsed, but this should not be minimized in regards to overall conditions in the Houston area. There are several areas still under a flash flood warning and area waterways (creeks, bayous, rivers) are nearing their cresting levels. With the storm stalling or moving very slowly we can still expect heavy rainfall at least through Monday as we are on the “dirty side” of the storm . The circulation of the hurricane will continue to spawn small isolated tornadoes that are powerful and short lived. We will continue to monitor the storm’s progress and the possible effects on our immediate and surrounding areas.
The Harris County Flood Warning System site is a great resource to monitor rising waters: https://www.harriscountyfws.org/
Hurricane Preparedness Plan
August 25, 2017 – At this time, PS LIGHTWAVE management has activated our Disaster Preparedness and Recovery Plan.
We want our customers to know that we are taking this storm very seriously and will do our best to keep you connected.
Our Disaster Preparedness and Recovery Plan is very comprehensive and includes all levels of management and support staff, some of the highlights are as follows:
- All of our in-house and contract restoration crews are on standby. We have secured all necessary restoration materials with a comprehensible access plan.
- The NOC management team and staff will be on site throughout the storm.
- All generators have been tested and fueled, and we have supplemental fuel vendors on standby.
- PS LIGHTWAVE critical staff has been identified and provided with a secondary means of contact.
- Our entire engineering, operations, and restoration teams are on standby with remote capabilities.
- The PS LIGHTWAVE Secondary Command Center has been setup and prepped in case it is needed.
If you have any questions please feel free to contact us anytime. The PS LIGHTWAVE phone number is 832-615-8000. We also encourage you to follow us on Twitter for important updates.