Telecom Risk and Security Part 3 – Human Factors

An employee enters the meet-me-room at a major carrier hotel in Los Angeles, New York, or Miami. He is a young guy recently graduated from high school, hired to do cable removal for circuit disconnects at minimum wage. Although young, he has a wife and child, and has recently been fighting with in-laws over his ability to support a family. Frustration and anger overcome his emotions, and he turns to the ladder rack jammed with cable and starts hammering at the cables for all he is worth.

Network operations centers around the world see circuits dropping, and customers with critical financial, military, Internet, and broadcast news services are shut down. In the space of about one minute our young employee has taken down several thousand individual circuits, creating near chaos in the global telecommunications community.

In their report on Trusted Access to Communications Infrastructure, the NSTAC Vulnerabilities Task Force advises “”it is important to recognize that any one individual with malicious intent accessing any critical telecommunications facility could represent a threat. The threat of insiders performing malicious acts also transcends each type of site discussed in this document.”

Security in TelecomThe event noted in part 2 of this series describing the outages in Northern California following damage to a manhole housing telecom was real. The resulting disruption to regional communications was a wakeup call to the telecom community, law enforcement, and communities affected. It is clear the perpetrator knew what he was doing, and knew exactly what vulnerabilities the major telecom companies had which he could exploit.

There have been many other cases such as Level 3 Communications loss of a major core router in 2006 supporting regional Internet services in London due to theft, a break-in at BT’s switching facility in Birmingham during the same period resulting in the loss of thousands of telephone lines, showing this is not just an American problem, but a global vulnerability.

The message is clear, as an industry our most obvious threat to information and communications security is not a natural disaster, it is people with industry knowledge or access to our critical facilities.

The Telecom and Data Center/Carrier Hotel Industry’s Role in Managing Human Security Risks

Data centers and central offices are in a constant state of change, maintenance, and growth. While facility network operations staff are generally long term employees, with a history of employment and performance, many others entering our data centers are not well known to the landlord.

Janitorial and maintenance staff are normally contracted to vendors, mechanical and electrical workers are contracted to maintenance and engineering companies, and construction contractors often use temporary staff from agencies such as “Labor Ready” and other day labor companies. In most cases data center or landlord employees are given a cursory background check prior to employment, however others entering even critical areas within the data center or central office meet-me-room may be entirely unknown to the facility.

While normally under some level of supervision, or access management, contractors, maintenance people, and even data center tenants are often free to move around the facility without direct security observation. As shown above, it would only take an angry, disgruntled, or undisciplined person seconds to cause a major calamity in our global communications system.

In a worst case, that person may be a terrorist with a detailed plan to cause damage to the facility once given even minimal access. High voltage electricity, water systems from cooling infrastructure, or access to switching equipment and cable interconnections are all exposed within the data center, and any element could be used to cause a major disruption within the meet-me-room or data center.

Most carrier hotels are located in “mixed-use” buildings, in high-rises with additional tenants who may not even be in the data center or telecom industry. This compounds the problem, as those tenants are often reluctant to comply with security and access requirements at the level of a critical telecom facility.

The issue becomes even more acute when we realize that much of the infrastructure supporting carrier hotels transits “risers” between floors, often through floors occupied by non-telecom tenants who may have physical access to riser space within their offices.

Secure Your Manhole CoversThere are a few data centers within the United States where security is comprehensive enough to reduce the risk of malicious intent to a very low level. While many tenants find the access and supervision within the facility extreme, facility resources are protected from all but the most aggressive vandalism or attack.

The NSTAC recommends that in the US the telecom industry establish best-practices guidelines to screen personnel prior to unescorted or unrestricted access to critical facilities, such as carrier hotels and carrier central offices. This may include a national agency check to ensure the person requesting access does not already have a profile indicating they could potentially be a threat to the facility.

The US government may give this additional support, as much of the US government, state, and local communications services are supported either in carrier central offices or commercial carrier hotels.

Recommendations for the Communications Industry

While it is clear not all persons entering a data center or carrier hotel facility can be completely screened, there are tasks each carrier and commercial data center operations should complete. Those could include:

  • Complete background checks for all direct employees
  • Pre-employment screening which would include a personality profile (indicating if they are in a high risk category for emotional stress)
  • Supervision of all contractors on site by a direct company employee who is aware of the risk posed for each type of equipment in proximity to the contractor (such as electrical equipment <UPS, breaker panels, switchgear, chilled water pipes, etc>)
  • Training in situational awareness – being able to identify activities not normal for others in your facility
  • Cooperation with law enforcement and other agencies
  • Working with industry groups to create and follow an industry “best practices” for facility security and human resource management
  • Ensure at least in the streets and areas immediately adjacent to the facility all manhole covers and utility entry points are locked and secured, preventing persons from accessing telecom, electrical, and water infrastructure supporting the building

“Unfortunately our most likely enemies will throw explosives into unguarded cable interconnect rooms or drop cans of petrol into unlocked manholes. End of Cyber War. You might characterize this as the provenance of a 23 year old fundamentalist Skywalker with a cell phone modem and a wild-eyed cousin in Munich figuring out how to blow up the Internet Death Star and stop Predator attacks on his village. Totally asymmetric dude! (From Bob Fonow’s “The Death Star?: Cyber Security vs. Internet security”)”

The commercial operators of data centers and carrier hotels have a tremendous responsibility not only to their owners and shareholders, but also the global telecom community and global economic community. The potential impact, even in the short term to a malicious attack on a meet-me-room at One Wilshire, 60 Hudson, the Westin Building, Telehouse in London, or the NAP of the Americas would be immediate, and extremely disruptive.

Human factors are the threat. Let’s not forget the lessons learned over the past couple years, and keep diligent, have good human situational awareness, and understand the sense of urgency we must apply to ensuring our communications infrastructure is secure.

Let us know your opinions, experiences, and recommendations

John Savageau, Long Beach

Previous articles in this series

About johnsavageau
Another telecom junkie who has been bouncing around the international communications community for most of the past 35 years.

3 Responses to Telecom Risk and Security Part 3 – Human Factors

  1. Bob Fonow says:

    Good turnaround managers live by this mantra:

    “The situation is worse than it seems. The situation will deteriorate”.

    from “Taking Charge”, John O. Whitney.

    • johnsavageau says:

      I sincerely hope we can enlighten the community to force change in our infrastructure management.

      I am never a fan of govt oversight and over regulation, but I am also a fan of best practices and minimum standards, of which we have serious shortfalls in the US telecom infrastructure.

      Thanks for the comment!

  2. This issue is one that is near and dear to my heart, only a couple of rungs up the IT stack.

    I worked for a now defunct company that had a solution ahead of its time, which would mitigate the insider threat on a network. It did this by issuing a ‘key’ to every machine that had access to the network.

    Guests had no key, and by extension, and policy, had little access to network resources. No key = no locks get opened and in many cases as an ancillary function, you couldn’t even see the door. It was akin to walking into a hotel and the lobby was the door to your room.

    You knew there were hundreds of other rooms in the hotel, you couldn’t even see them, let alone try your key in any other door. Five years ago, people thought it was ‘cool’ technology, but there hadn’t been enough damage, and enough quantifiable damage to implement it.

    The point was that the threats are from insiders, and anything you can do to limit their access on a network is the right thing to do.

    We did not focus on the physical security piece, but as you point out – insiders have access to more than we think about in our buildings and on our networks.

    My fear is that it will take something catastrophic to happen, and those of us in the IT world will continue to be reactive vs proactive and adding true value to the organizations we are tasked with supporting. Great suggestions/best practices. They are simple, executeable, and effective.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: