Work

VoIP Day T-Minus 0.5

by

Completed work today without a stroke. This deal making is stressful. Up until now I have been very vague at what sort of hardware deal I am talking about. My company runs a call center. We are opening a new call center in upstate New York since our current office is at capacity. We have been shopping around since like January for a solution that will work for us. We needed a phone system, network gear (switches, routers, firewalls) and a software solution that will run the call flow for the call center. We wanted to do this with as few vendors as possible. In addition to the above listed items we also need server, desktops, laptops, cabling for the floor (that is not done by itself:), setup and testing of the gear, etc.

Because there was so much stuff we looked at no fewer than 6 vendors to do this for us. Each offered a variety of options. We narrowed down the call center software package that we will use to 2 products. this we did early on. we had a new product produced by Cisco, and the current one we use by a company called zeacom. We built our configuration around either of these products. Now months later we are almost at the final choice. We have narrowed down the playing field from Avaya magix or difinidy PBX switches. We looked at NEC VoIP switches, and also a few lower end systems that we didn’t like. Now we are down to the wire. We have 2 vendors we like, one more than the other. I can’t say specifics because things are not done yet.

In a nutshell that is what has been going on as far back as december 2002. I have a background in data networks. That being said I am a fan of the Cisco hardware. It is Cisco, support is good. They make (or buy companies that make) good hardware. I work for people that are VERY cost aware. Being the best does not fly with these people. We have to justify what we spend. At first it drove me crazy. It still does, but I appreciate it, and it is now a challenge to do everything we want and spend small amounts doing it. As much as working a deal sucks in my opinion (i hate sales people), it is nice to shave off $17,000 off a quote in 20 minutes of a phone conversation.

We meet tomorrow to finalize and hopefully sign contracts. I await final paperwork tonight. I just hope all we asked for is in writing for tomorrow. This is the biggest decision I have ever made for a company. Yes, I am nervous. I think we thought it through and are making the correct decision. It was a collaborative effort. Keith, Gus, & I reading up on the technology. Gus working the deal. I hammered out the design and worked the deal a bit. But even after that, you still are nervous until you take that first call, and that 100,000 call…

Once this is done I can focus on actually building what up until now has only been a concept written on my white board in my office. That and quotes piled 3 feet high on my desk.

The big remaining obstacle is lease terms. We don’t want to pay tons of money cash to pay for all this equipment now. Being a relatively new company we need to work on getting good terms. That is ongoing but I don’t want it to affect our deal.

OK, stressing still. I just want this to be over. As I write this I got emails with some revised numbers that I don’t like. Working on correcting. Dam sales guys…

VoIP Day T-Minus 1

by

This year my company is trying out a softball leagure. It is a place where people with no athletic skill get together and compete in softball games with other out of shape people at other companies. Generally I am told this is done while consuming large amounts of alcohol. I got arm twisted into joining the team. I don’t like softball, and I am not very good at it. We will see how many games I actually make. Especially since we are in the middle of prepairing a new office. We got our tee shirts yesterday. I don’t even know if it will fit me, but I got the number I wanted. Lucky #3. I still have to fill out the release form to play, but don’t tell paula that or she will kill me. Guys from work went out yesterday and hit some balls at the batting cages. Today is their first official game. I am still working on hardware quotes so I don’t know if I will make it.

T-minus 1 day for our hardware decisions for the new office. We decided that papers need signitures by friday on our telecom purchases. We need to order T-1’s, and get a hardware intergrator picked. We are narrowing down things now, and hopefully will be putting the finishing touches deals friday. Of course we have said this before and things have slipped. This time feels different. I will know tomorrow.

I spent almost all day yesterday dealing with planning for kingston. We also had our weekly technology staff meetings. Those are always interesting.

I got some nice unexpected words from David yesterday. Dave is our call center manager, and an all around good guy. After our meeting with our CEO on our project plan he took me aside and said he was very impressed with how I know and explain correctly what I do to people. He was like he does not think that about most people. I was very touched. Dave is a hard guy to please, and to get those words from him was a welcome surprise. That goes to show you that I cannot read people, since I thought I fumbled a bit durring our planning meeting, but according to him it was well done. He said he has seen our CEO rip people appart for not having answers, but I got him to go along with almost all our proposals. Feels good, now all I have to do is make things actually happen.

On another note as I said in an earlier post this week, we were having problems with some database transfers throughout the night that killed our internet connection. Well, it is still happening. We are nowhere near close to stopping it in my opinion. Sometimes I get the felling that some people could not navigate themself’s out of a paper bag. Those are probibly the same people that would over engineer a wheel if you know what I mean?

Well I am train tunnel bound so I will close this entry out. Hopefully today brings most of these deals to a close, but I am not sure just yet.

MRTG never lies

by

As you may know or guessed by these posts I actually work in the technology field. More specifically I deal with network & server operations for my company. Because we do that we monitor lots and lots of things on our systems. Anywhere from network traffic to temperature of equipment, to performance counters. This gives us an idea of what is going on when we are not watching the systems. Some of this is manual (stuff gets logged but to understand what is going on we need to look at it) and some things are automated (something is not working we get paged about it). That being said late this morning I took a look at our network monitoring pages for the weekend. I was surprised to see huge utilization between two of our sites. More specifically a database server in one site is sending data to two servers in another site. We use MRTG to watch bandwidth. It is a free linux tool. It is great. I for one am just getting into linux, but stuff like MRTG gets me wanting to learn more. So I bring the problem to everyone’s attention. Now here is the problem, half the people dealing with our database servers are saying nothing is going on at the time we see a problem. The other half agrees something is wrong but not sure what. Turns out, oh yeah we run some stuff late at night and an anomaly happened to cause it to send out 600 meg files and not 20 meg files, or something like that. So some of these same fun guys are saying that nothing is going on, but our T-1 pipe is flooded for 4 hours with traffic because some servers are trying to transfer well over a gig of data.

The moral of this story is that tools like MRTG don’t lie. We knew (well, Gus, Keith and I knew) from the first second we saw this what was causing the problem. We just didn’t know why it was happening. If it wasn’t for some sherlockian tactics from the man “Gus” we would be shooting in the dark. Never doubt the tools you use to figure out problems. When you doubt their ability it is time to get new tools, or test them. MRTG has not failed us yet, but we still have skeptics claiming “everything is ok”. Wake up and smell the coffee. Or wake up and drink some coffee to stay awake.

This whole problem boils down to a sherlock holmes type mystery. That is how I think of it. You have a problem, and you have to logically deduce what it is, and why it happens. Don’t ignore the obvious. Coming up with a complicated explanation as to why something is going on is stupid when a simple one fits the same criteria. There is a term for that, that I cannot spell so I won’t mention it here. I sometimes think that is all I really do when it comes to fixing problems. I try to use deductive reasoning. Probably why I like sherlock holmes so much.

This blog may seem like a rant, but it really is not. it is to say pay attention to the obvious, and think things through. Some people forget to do that, or just never think.

In other work news, we are still working to get a deal for voice service in our new facility. Dealing with telecom people is a nightmare. I thought dealing with routers was hard. And I remember when I thought programing a switch was a daunting task. Boy was I naive.

I spent the entire day dealing with telecom quotes or the above mentioned bandwidth issues. What is crazy is I know we have tons more in monitoring to deploy and watch. Right now we are finding tons of stuff. I wonder what we are letting slip through the cracks? Maybe nothing, but you never know.

I am left work at like 7pm tonight so I won’t get home till like 8pm. I need to make dinner, and relax a bit. today was very stressful day.

More work, less hacking, unless it is for work

by

It is monday again, and I am off to work. Today we meet with another telecom provider to see if they have a good deal. We need to make a decision this week.

OK, time for me to gripe about something. I am on the train, and cannot stand it when people receive cell phone calls (or make them) and have an entire conversation. In addition to that these people usually talk like a plane is flying by. Now I am all for cell phone use. I have 2 of them, and I don’t even have a home phone (another story). When I am out and about I will receive calls but only talk for a min if it is not important. If it is a work emergency (which I get more often than I would like) I talk. What is with this casual talking on a train with 100 other people. We don’t care what you have to say to your friends or family. We don’t need cell phone restriction laws, we need some etiquette from people. Ok, I am done ranting.

Speaking of cell phones I am kind of bummed that my main phone (an AT&T GSM Provided Sony Ericsson T68i) won’t work in our office in Kingston. Actually it wont work in kingston at all. I am not sure what will work well up there. I know verizon works, and t-mobile, but not sure if either of them provides good signal at our site. I have a verizon phone but I wanted to cancel it. No I am paying the monthly fee to wait and see. I also have a T-Mobile SIM card, but no phone. I am holding onto that for a bit longer also.

I gotta tell you, I go through phones (just like PDA’s) fast. I just upgrade to the latest and greatest allot. I was on ebay last night looking for a phone for the CEO of my company. He is going to europe and wanted a phone. He has a SIM already, but no phone. Since I didn’t have any extra he said to look for one. Well after looking I don’t see any GSM phones that do everything I want, except for the one I already have. For my CEO we are just going to rent one, but I started looking for myself also. Like I said I don’t have a phone for my T-Mobile SIM. It is weird. For the first time in a while I am ready to spend money.

OK, back to work talk. This post was supposed to be about work. I ended up sending back the low end Compaq 1U server we got. FYI, 1U is the size of it. Computer racks are broken down by U’s. Each standard rack has 42U. A 1U server takes up 1U of that 42. They are really small. It turns out the compaq server we bought does not have a RAID card for IDE like were were told 3 times. The were going to send me one for free, but I decided to send it back instead. Part of my reasoning is that we were going to get 2 of these. And I want to get them and see if I like them. if I do like them I want to standardize on them later on. If we got a free raid card, I would have to pay for them in the future. That would put the cost well over what I wanted to budget for these low end machines. I ended up ordering a super micro box and saved like $600 off my original quote. It is a no name brand, but we already use no name VA Linux machines, and this thing has a 3 year warrantee. I will see if they are any good.

Bidding wars, no its not a movie

by

So we have the bidding going on for services at my new office in kingston. We got people asking to provide Internet service that I never even heard of. We also have two big voice providers coming in to bid this week and next. This week is the contender. (no names just yet), next week is the guys who have our biz, but need to work to keep it.

I also got news that we have a new hire for monday. I ordered his computer last night and should have it today. I hope justin can build it in time. We are out of phone ports so that will be interesting. we also have no place to put the guy so a phone is no big deal. it is now starting to get fun with space.

I am in the subway tunnel now, so I will hopefully write more later.

Kingston project is a go!

by

Even though officially our project was a go for a while, we finally sat down and ironed out time table’s, and things. Our CEO wants everything ready at the facility by June 6. June 6, by the way was the date we moved into our current office last year. That poses a problem for me since I have vacation plans for that week. I am in barbados with my family. My boss Gus says to chill and wait and see if things actually shape up next week. Realistically I don’t think we can make that date, but I am told to aim for that date, I have to do everything possible to do it. Even if doing all that won’t change the fact that we wont make that date. it is an interesting position.

I also got the cd-rom / hard drive for the new server we bought. I believe I talked on monday about the discount server we got from comaq that didn’t ship with a cd-rom or floppy drive. Well that problem is resolved. I need to build the dam thing now.

It just started raining in NYC. I wore light khakis thinking it would be nice out. They are going to get messed up by the dirt and water kicked up when I walk around now. Oh, well.

I am on my own today, well I have justin but he can’t do everything I need him to yet. Keith is off due to the holiday. Not really sure why since he is not very observant, but I think he said something about having to travel to jersey or something. Should be an interesting day. We shall see…

It might be summer, & an anniversary of sorts!

by

Today is the first day this year that I left my house without a coat. It feels like summer. It is like 65 degree’s now going up to 80. I am happy the way it is right now. It is perfect. The AC in our office is not working yet. I think there is a problem. If I had this blog last summer I would have written about all the great problems we had with AC in our office. But I didn’t so use your imagination. It was hot. This year is not shaping up very good either right now. On a plus note the AC in my office works, but it dies under the strain of keeping air cool everywhere. even with the door closed it will get hot on the first 90+ day. we shall see what happens.

OK, due to me taking a late train yesterday night I actually had 3 blog entries yesterday. that will not happen that often. I really should have had 2, but I broke up one into personal stuff, & work stuff.

I am working on figuring out how feasible it is to move my entire photo web site over to my apache server. I want to get it to work, but I don’t want to have to do allot of work. there are tons of pages, and about a gig of images. it will be a big job if I can’t automate it or get frontpage extensions to work on apache. Hopefully I will have time this weekend to look at it some more.

I am taking Dan to bestbuy today to purchase wireless gear for his new apartment. Dan is the director of operations at the company I work for. He is really cool, and we get along great. That said, I don’t normally do computer work for people outside of work. I can’t stand it. Dan understands that so we are going to buy the stuff only. I like buying things! Actually some people I don’t mind helping, but others who have no business asking for help or other people who just expect it, I don’t like doing stuff for. He is in neither category so it will be fun to setup some wireless stuff. Wireless is good.

It dawned on me this morning that yesterday was an anniversary of sorts for me. I started at MFS network technologies (a division of worldcom at the time) 6 years ago on tax day. That was 1997 if people can’t do math. That really marks my beginning of my technology career. My first official help-desk job didn’t come till AuGust 1997, but I did plenty of technical and help-desk work at worldcom so I consider that my start. I figure when you are upgrading file servers, building new computers for people and auditing software you are in the technology field. That was just a bit of history about my tech roots.

Today’s work agenda, tech meeting, then tech management meeting, then trip to bestbuy, then kingston move meeting. Just full of fun meetings today! I also need to call and harass some vendor’s (see yesterdays posts).

Tonight’s the hockey game. I can’t wait. I broke out my Islanders jersey and it is in my bag for later. I am at a point where I cannot wear jersey’s of any kind to work. it sucks but I will deal. At least I can still wear tee shirts. I think it is funny to sit in a meeting with vendors in a tee shirt and khakis while they are in suits. it is very amusing.

Well I feel us descending into the train tunnel so I will stop here. Details on the game and other stuff tomorrow.

Tuesday With vendors, and web problems.

by

Well I am starting to write this on the east bound train home tuesday. Not sure when this post will be finished but I am stuck leaving work at 7:30PM tonight. It is daylight out so I don’t even realize that it is late anymore. There is still work to do and it is light out, you forget it is 7pm.

We had some problems some web sites today. It was not fun. We think it is related to a bunch of different problems ranging from running out of disk space on one web server to huge amount of searching on a database server. It probably will take days to sift through the data.

I ended up meeting with our telecom vendor today. It was a meeting that was planned when I was out on friday so I was sort of surprised. I think it went well. We told them point blank that we were not happy with them, and we need problems fixed now or we wont use them on in our new office. they seemed eager to do what we want (of course who wouldn’t when you dangle money to someone). We are on sales guy number 4. This guy seems to have a clue so we will see how things go from there. it is tough. I like what these guys say, but I have heard speeches before and then get no results. if our new sales guy can do what he claims I will be happy. the question is will he do what he claims? I am betting allot on that answer.

In other news, Keith is officially a dumb-ass. His problem with the RAID array I spoke about before was that the card came loose and he didn’t check it while he tried to fix it for 2 1/2 hours yesterday. At least he figured it out on his own. FYI, I use the term dumb-ass lovingly:) If I really don’t like them I would just call them an asshole.

Sales guys suck, & other Tuesday news

by

Have I mentioned that sales guys (people, gotta be politically correct) suck? They do. I am trying to sift through all the quotes and crap I am getting while preparing to build this new office, and people don’t have a clue. I have one phone provider call me last week going they can’t provide local phone server in the area we are moving to. I was like, ok sorry. I guess we can’t use you then since I want one company to do everything for me. Then monday I get a call from that guys boss going, well we kinda can do local service in that area. I was like make it simple and make it a good price and I will look at you. Until then these guys are talking BS. I cannot wait to get the bidding out of the way and just build the thing.

Our weekly management meeting was canceled yesterday. I think there was a board meeting or something. It will happen today instead.

So I got my new low end compaq rack mounted server. I mean low end when I say it. This thing is an IDE drive machine with RAID in it. it is so cheap that it didn’t even come with a CD-ROM & Floppy drive. Normally I assume a computer will ship with at least 1 input device such as a floppy drive. got a call out to those people to get something so I can load the OS onto it. Like I said sales people are moron’s.

The only other things of note on monday were we had some patches of problems with one of our database servers. Our engineers are looking at it. That means I need to get my crazy 8 ball out and help them with the diagnosis. Also Keith is having problems with an IDE RAID system. I will look at it with him today.

Work again, Monday Meetings & hardware purchases

by

It is monday again. The most hated day of the week. I used to not like monday’s just because it was the first day back from the weekend. My new issue with monday is it is the day of meetings. Actually it used to be the day of meetings, now it just feels like it. Monday is the day that all the managers in my company get together to let everyone know what is going on. That means writing up agenda’s, and other fun meeting stuff.

Monday is also the day in most companies where new hire’s start. We are no different. Thankfully we don’t have any today. I actually have someone new starting tuesday, but we probibly will have like 5 new people on next monday. Not sure yet. No one tells me till the last minute normally.

This week I need to finish setting up the temporary firewall I will use to test the new network for our kingston office. For setup purposes I am creating a new isolated network for all the gear we are getting for that office. I am using an old firewall we stopped using in february. Once that is done I will need to begin getting quotes for servers. The next step in the design would be file servers and a domain to put all the computers in. We currently use a mix of compaq and dell servers. Most of our machines are dell, with some database machines being compaq’s. Alot of our older equipment is VA Linux machines. I personally don’t like the VA Linux equipment. The company stopped making hardware, and the machines are not hot swapable anything in them. Granted their track record is ok. I have only had problems with 3 of them that I can remember, and we have over a dozen of them. For this new office I have not yet figured out if we will go compaq, or dell. It will depend on price most likely. I also have an IBM guy wanting to sell us stuff. They say that they can meet the other guy’s prices, but I haven’t seen that happen yet. Sales guys, are sales guys.

Another cool project we have going on is upgrading our data center to a gigabit backbone. We have the design down, and now we need to buy and deploy the hardware. I probibly will purchase that stuff this week. that will be cool to deploy.

Obviously since I am talking about work stuff I need to be vauge enough so I can write it here, but also descriptive enough since I want this blog so I can write about what I am doing and thinking. That will be a fine line I walk when doing this.