November 9, 2009 Leave a comment
This is part three is a series of interviews with Eric Bender, president of Wilshire Connection. Wilshire Connection, or WilCon, is the largest independent local network and neutral fiber infrastructure provider in downtown Los Angeles. In this segment Eric discusses how WilCon managed risk to their network during the initial construction process, the continuing management of critical telecommunications infrastructure, and the role WilCon could play in the event of a major incident impacting the telecom industry in Los Angeles.
Pacific Tier: (on the topic of utility gas and electricity) What risk is there to the infrastructure in downtown LA of an explosion from either electricity or gas, and what would that do to your conduits if it occurred?
Eric Bender: Interesting question… I haven’t really thought much about that, it would, depending on where, you never want to see that. Fortunately with power, the lines typically won’t explode, it is the transformers, which mostly are in the buildings.
The way LADWP (Los Angeles Dept of Water and Power) sets up they bring the high voltage lines into the building, so if the transformer blows up it will be in the building. The transformer is typically not out in the street – but they do have some vaults out in the street, and they have had some explosions, but they have been contained within the vault. We don’t run through any of their vaults so from that perspective we’re OK.
Somebody who is digging and hits a gas line… that’s a different story.
I can say in the roughly 12 years that we’ve had conduit in the ground, and since we were one of the first to dig, I can say that we are lower in the ground, and more protected in many respects, and more of a straight line in routes we’re going without having to jog around any other existing infrastructure that came later.
But we’ve never had one of our conduits damaged or cut. Or interfered with…
The difference between how our conduits or duct banks are typically done vs. many other carriers is that we put in massive amounts of conduit. We use (typically) four inch PVC conduit, and I don’t think we have a single one (trench) which has less than six or eight four inch conduits. Most of them have multiples of that, for example going on Wilshire Blvd between Grand and Figueroa we have an average of conduit in that whole duct bank is probably close to sixty four inch conduits.
Because fifty of them go into One Wilshire, although they kind of peel off in different directions from there, we’ve also got a main ’48 and they lateral off down into other streets into buildings, so there’s some doubling going on.
But in some areas we have up to sixty four inch conduits so that’s the size of a desk, or bigger, and that duct bank is encased in a concrete slurry around it, surrounded by a couple addition feet of dirt and asphalt on top.
So if anybody digging is going to hit (the duct bank) typically there is warning tape on top of the conduit, concrete, or in the concrete they are going to pull it up. They are going to hit (the tape) or something before they hit the conduit. So unless it is some young, uncontrollable (person) with the backhoe who is on a rampage – no matter what you do you can’t control that.
NOTE: Whenever a company opens a street for utility construction or maintenance you will normally have construction observers and safety observers from not only the company opening the street, but also each company with conduit or utility infrastructure in the immediate area of digging.
But some of the other carriers that put in one or two conduits, they are the ones at risk, like for example an MCI, Verizon, or a QWEST in some cases they typically put in just the two conduits… And you could rip through that even if its encased in concrete before you realize what you are doing.
The conduits are just this big, versus this big (Eric shows a note book size to represent two conduits vs. his desktop to represent WilCon’s conduits).
Pacific Tier: Do you consider yourself the only truly neutral facilities-based carrier in downtown LA?
Eric Bender: I think others consider themselves neutral, but they have other motives as well. I don’t care if I sell dark fiber or lit transport. We can do either, and it doesn’t matter to me which one they want. We have so much fiber, and the infrastructure to continue pulling more and more fiber that I’m never worried about running out of capacity for dark fiber.
A Level 3, or a QWEST, or an XO, they run a network, and they’re obviously not neutral. DWP (LADWP), they’re somewhat neutral because they don’t seem to care whether you take fiber or lit services. I don’t know what they do with lit services or on the network side of things – honestly. We’ve leased from them (LADWP) dark fiber to get access to some off-net buildings, and they’re very easy to work with. They are very rigid, and have no flexibility (LADWP is a public utility), but they’re easy to work with.
We’re probably, as a straight, neutral, not really care who you connect with, we may not be the only one (neutral fiber network), but one of the only ones that would be neutral.
Pacific Tier: One Wilshire is traditionally a center of communications in Los Angeles. Some people think it is a high risk location because there is so much on the 4th floor and other parts of the building. Some people think that it is meaningless – that it doesn’t have that much value. Do you think the 4th floor of One Wilshire today is a critical piece of infrastructure, or do you think it is something that is just there, that could be bypassed when and if ever needed?
Eric Bender: I think at this point it would still be considered a critical facility. A lot of carriers and other companies have facilities or locations elsewhere, but because of the way they’ve built their networks from the beginning, One Wilshire has always been the central point for them.
They may not have grown and expanded there, or they may have moved things off to other locations, such as 600 W. 7th, 818 (W. 7th ), or even outside of LA, and use a company like us, or some other carrier to make that interconnection or virtual connection between their two facilities. One Wilshire tends to have been their primary facility.
I think that over the years that’s more applicable to legacy carriers, the bigger carriers, the ones that have been around for a long time.
I think the Internet type of carrier that’s either VoIP or an Internet company, content CDNs (content delivery networks), and those – One Wilshire’s not as important to them at all because in my limited technical knowledge it is easier to reroute that traffic to other servers – they have more mirror facilities than a switch would have on the telecom side of things.
Pacific Tier: Is there a business continuity plan, or disaster plan, in the event One Wilshire or another facility like 60 Hudson (New York), or the NAP of the Americas (Miami) anybody has thought about or put on the shelf in the event one of those critical facilities has a catastrophic failure?
Eric Bender: I am not aware of any common, for the greater good, where all the carriers have participated in developing that, or working out some kind of contingency plan. That’s actually a really good question.
You’d think that after 9/11 where the infrastructure was so significantly damaged that in various other markets such as LA, Chicago, that there would be some kind of a group of organizational effort to have dealt with that. I am not aware of one. They may have one, but I am not aware of it, and they never invited me to participate.
Pacific Tier: Is WilCon positioned, in the case of a worst case scenario in Los Angeles, to assist the community and assist the industry in recovering to an alternate facility if that occurred?
Eric Bender: Sure, I mean, our infrastructure that we built, and that we control and own, is all primarily downtown LA, so… in a worst case One Wilshire becomes untenable, well a lot of our fiber doesn’t all home run into One Wilshire, but a lot of it does go in and turn around, coming back out again.
There would be disruption, but it could be brought out and bypassed. We have diverse paths into most buildings that we connect so we can certainly do it.
Not in LA, but when in the Mediterranean last year when they had the three or four cuts, (several of) our customers were impacted. I sent them an email and said I doubt there is anything we can do here , but if there is anything you need that we can help you with, let me know and we can work with you.
And actually two of the customers said “yeah we need to reroute some connections to put it on a different side of their ring (in the Med and Pacific) that we could do in about five hours with a couple pairs of fiber for them, and they were able to reroute their traffic, or some of their traffic, and lessen the impact of those (submarine fiber) cuts.
Pacific Tier: So WilCon would consider yourselves a very flexible, agile part of a recovery plan, and would not be rigid in your provisioning process, and that you would work with the community to recover from a disaster?
Eric Bender: I agree with that!
This ends the third segment of this series. In the next part, Eric will discuss more of the future of Wilshire Connection, including his visions for expanding WilCon into new markets.
The entire interview is available online.
Previous entries in this series include: