UPDATE - Outages Plague DSL Providers

greenspun.com : LUSENET : Grassroots Information Coordination Center (GICC) : One Thread

[Fair use for education and research purpose only]

April 26, 2000

Title: Outages Plague DSL Providers

By Patricia Fusco

Digital subscriber line broadband access providers have come under fire recently their response to a string of outages.

Both Covad Communications Inc. and Pacific Bell have faced a battery of DSL access issues as they aggressively deploy their broadband services. Their experiences are not different from any other DSL reseller or access provider.

Covad (COVD) has had three separate multi-hour outages in Los Angeles in the past month. In New York, there has been reports of multiple problems over a period of four days resulting in nearly 48-hours of downtime.

SBC Communications Inc. (SBC) subsidiary Pac Bell is still reeling from a DSL service disruption that shut down high-speed connections to Saratoga and up the Peninsula through San Francisco and into the North Bay of California. Although the actual trouble ticket reported that Pac Bell's DSL service outage lasted only eight hours last week, e-mail problems lingered for consumers that connected through its Pac Bell Internet unit.

There are many bugaboos that can bring high-speed Internet access to a crawl, if not break a subscriber's connection entirely. One common problem is when upstream provider completes routine maintenance on its networks and things don't go as planned.

The second most common failure is when one connection reaches its limit on a specific server or switch and the equipment simply fails. The third type of system failure is rooted in the flaky nature of Internet access, where a T3 or DS3 may act up, or bugs in software or hardware can crash a once robust network.

These common types of DSL service disruptions do not include the wrath of deities, which account for "Acts of God" including fiber cuts, inclement weather, or fire.

Anjali Joshi, Covad's vice president of network engineering, said the DSL reseller has experienced its share of service disruptions and has diligently worked to keep its high-speed network flowing.

"I can design a 100 percent reliable network, but I can't predict when a fiber gets cut," Joshi said. "One or two simultaneous outages can be handled and we can get through it because we have build-in redundancy, but sometimes that's not enough."

"We have experienced disruptions to DSL services when a Qwest Communications Corp. (Q) facility went down, we've had AT&T Corp. (T) services disrupted and we've had a public utility backhoe take out as many as three fibers," Joshi said.

"Network outages happen, and a DSL connection can breakdown every step of the way. It's not a matter of prevention, it's a matter of how to deal with it when an outage occurs."

Shawn Dainas, Pac Bell spokesperson, said that its most recent outage was the result of a failed switch to a single central office in California.

"A data switch failed (last) Thursday and the service outage lasted about 8-hours," Danias said. "We're not sure how many customers were effected, but all of them were back up quickly, except for those customers that were the last to re-establish their DSL service on Saturday, only Pacific Bell Internet customers in the San Francisco Bay Area were effected."

The difference between Pac Bell's end-user or consumer-based DSL access and broadband resellers like Covad, which is governed by service level agreements with its business partners, is striking. The difference lies in the frustration-factor that subscriber's and partner's face when DSL services go down.

While Pac Bell customers clog phone lines calling a toll-free number for customer service or technical support to determine what happened to their DSL service or resort to calling a buddy with a different provider to jump on a Web site to review network status, Covad said it takes a more proactive approach.

Beth Lackey, Covad vice president of customer services said it immediately takes four key steps to keep all parties informed as to the status of a service disruption.

"After a trouble ticket has been verified, Covad immediately e-mails its partners to keep them informed on the event" Lackey said. "After partners are notified, we produce a Web-based weather report that's updated every 30-minutes. The weather report includes our diagnosis and estimated downtime."

"From there, we initiate an alpha paging alert to our customer service and technical support to inform them of the outage and put them on high alert so everyone knows what's happening," Lackey continued. "At that point we also provide internal Web updates and contact information to support our staff, incumbent carriers and ISP partners."

For Covad, troubleshooting does not stop when DSL services have been restored. Lackey said the last part of its event process is all about making sure the service disruption does not happen again.

"The last, and most critical key touch-point for Covad is providing all parties concerned with a 'post mortem' report," Lackey said. "The report shows what happened, what we learned and what we plan to do about it. If a switch failed, we would review every single switch made by the same manufacturer in order to deliver on our service agreements. Then, and only then, would we relax."

Technicians for both Covad and Pac Bell work as efficiently as possible to restore high-speed "always-on" service when its DSL lines are disrupted, but that's where the similarities end. If DSL services are going match cable access deployment, consumers will need to be treated like business partners and stay informed when their DSL service provider experiences trouble.



-- (Dee360Degree@aol.com), April 26, 2000

Moderation questions? read the FAQ