I have so much I want to say, I don’t even know where to start or how much I will get out. Some people have their own impressions of “evil” corporate empires exist out there. I have friends that bash some retail store chains as being evil, or others who can’t stand some or all media empires. Most of those opinions are based on ideals, and some passing interaction with said companies. I on the other think the true “evil empire” out there is one that is partially transparent to people. I am talking about a company I will call the LEC that will not be named. If you don’t know a LEC is better know as “Local Exchange Carriers”. They are the SBC’s, Bell South’s, & Verizon’s of the world. You most likely deal with them with your home phone. The thing is everyday people don’t realize is that if you use an alternate provider for anything like phone service, or even business class T-1’s you still must use a LEC. The LEC owns and operates the physical lines in a given area. They own the copper and or fiber into buildings and houses. in some rare cases you have providers also having their own fiber or copper into an array, but that is uncommon.
So here is the situation most people don’t realize can happen. You have big name internet provider X as your data T-1 or voice T-1 service. They are a huge company but they do not own the local lines in the area you do business. That means between your office and provider X’s POP (Point of presence) you must utilize a LEC. If you don’t know better provider X won’t ever really bring that up, but I know this is the case. it is not that they hide that fact from you, but it is not something that will be highlighted on page one of a contract.
So this is the situation. You take time and effort to design a highly resilient internet backbone for an office that needs high availability on their voice and data setup. You spend the time and money to get multiple POP’s from provider X. You also get multiple routers setup with HSRP & BGP. All bases are covered, right? Wrong. The LEC who will not be named has a problem at their central office. All those nicely diversified circuits all go through the same LEC. Remember provider X will give you diversity, but they don’t own or control the lines into your office. If your area is serviced by 1 LEC, all your lines go through the same conduit out to the same CO (central office). Well if that CO has a problem with lets say some hardware, all your network diversified circuits are down. The LEC is the pinch point in most situations. Now if you have provider X, they will try to fix the issue. If they can’t figure it out, they will escalate the issue with the LEC who will not be named. Here is one problem. YOU or I are not a customer of the LEC who will not be named. Provider X is. You are considered a wholesale customer of the LEC who will not be named. To them you are the least important person. Now, they may say otherwise, but if you ever negotiated a T-1 between provider X and the LEC who will not be named, the LEC all but says you are not important if you use provider X. Is it true? I think so, but it may be a negotiation tactic. In any case when you have a problem, the LEC who will not be named seems to not care. Even if they ARE quick to respond they have the attitude of “we are big LEC who you have to use. We will get to you whenever we want”. I have had that feeling several times over the years. That is why I have come to the conclusion that the LEC who will not be named is the true evil empire.
So in a vague (or not so vague depending on if you know the true story) I have vented about my technical woes today. I know shit happens. When people ask why something is down (when telecom circuits and major network gear is down, not the minor stuff) I tell them I honestly don’t know how the stuff works the 99.999% of the time it does. Really. People think I am kidding, but if you ever had to spend 12+ hours trying to get the LEC who will not be named on the phone when you have an outage and when you finally do get them to do something it is fixed in 20 minutes you will begin to think like I am. I mean come on at least look like are trying to care about my problem!
And that reminds me of what Howard said to me when he stopped by my office today in the middle of the disaster that was my day. He was like, at least you are not having capacity problems like Sixaprt and their Typepad service was having. I had to laugh. He didn’t realize I use them. I think he thought I was using Gus’ server still. The funny thing was I gave them prop’s for how they were handling the situation. I sent an email to their CEO commenting on their outages and how they are communicating the issues to customers and I got a response in like 2 hours. How then can billion dollar LEC who won’t be named take 3 hours just to get one of multiple tickets into their system when calling the emergency support line? I am not talking about calling the “hi, my home phone isn’t working” number. I am talking about the “I spend allot of money and many circuits are down” number. It is really scary how a small company can be so responsive and a large one just plain old suck. Of course I should know that by now, since my company is not that big and we are SO much more responsive to issues than people we deal with. Yes small plug for my own tech group, but it is true.
Ok, by this point in my writing tonight I think I am just rambling. It has been a long day, and by now I am sobering up, but yes I did have a few drinks before I got home and started writing tonight. Hey it is Halloween and I went out for a few with friends from the office before coming home. I can’t remember if I have been over my opinion on the evil empire? If not I think I made myself clear tonight. Am I asking for too much? All I want is to get a person on the phone when I have a problem, and have them seem interested in solving my issue and get me back up and running. That person should also speak clear understandable english, and must understand when others speak clear and concise english. Forgot to mention that issue. Not sure what was worse, the FULLY automated ticketing system of one company today, or the get an offshore support person who cannot fully understand what you are saying number? Then there was the automated update system that called every 30 minutes with a message telling us nothing has changed. that would have been ok, if it wasn’t for the fact that we had 4 issues open at the same time, so Jayson had calls every few minutes. And in their attempt to be good about contacting people on alternate numbers if you didn’t answer your primary one, they would call Jay’s cell phone if he didn’t pick up his work one. The issue was he didn’t want to get more calls from auto response guy, but they kept calling.
Really I have come to the conclusion that I should not write about things that really bother me right after they happen. When I do, I write allot of stuff that is true, but when I am calmer I might not have written. For my own safety names of companies and details of issues have been deliberately modified in this post. The general issue is true, and yes I had a bad day today. I need to take a vacation day soon! On that note, I am going to stop writing. If I have more things to say about the LEC who shall not be named I will write later. On a semi positive note, provider X was not as bad as the LEC who shall not be named. They sucked allot, but at least their sales guy who I deal with allot was able to get some escalations in for me. That is saying something, in a day full of problems. Did it help? I don’t know, but it made me feel a bit better.