Work

VoIP Day T-Minus 1

by

This year my company is trying out a softball leagure. It is a place where people with no athletic skill get together and compete in softball games with other out of shape people at other companies. Generally I am told this is done while consuming large amounts of alcohol. I got arm twisted into joining the team. I don’t like softball, and I am not very good at it. We will see how many games I actually make. Especially since we are in the middle of prepairing a new office. We got our tee shirts yesterday. I don’t even know if it will fit me, but I got the number I wanted. Lucky #3. I still have to fill out the release form to play, but don’t tell paula that or she will kill me. Guys from work went out yesterday and hit some balls at the batting cages. Today is their first official game. I am still working on hardware quotes so I don’t know if I will make it.

T-minus 1 day for our hardware decisions for the new office. We decided that papers need signitures by friday on our telecom purchases. We need to order T-1’s, and get a hardware intergrator picked. We are narrowing down things now, and hopefully will be putting the finishing touches deals friday. Of course we have said this before and things have slipped. This time feels different. I will know tomorrow.

I spent almost all day yesterday dealing with planning for kingston. We also had our weekly technology staff meetings. Those are always interesting.

I got some nice unexpected words from David yesterday. Dave is our call center manager, and an all around good guy. After our meeting with our CEO on our project plan he took me aside and said he was very impressed with how I know and explain correctly what I do to people. He was like he does not think that about most people. I was very touched. Dave is a hard guy to please, and to get those words from him was a welcome surprise. That goes to show you that I cannot read people, since I thought I fumbled a bit durring our planning meeting, but according to him it was well done. He said he has seen our CEO rip people appart for not having answers, but I got him to go along with almost all our proposals. Feels good, now all I have to do is make things actually happen.

On another note as I said in an earlier post this week, we were having problems with some database transfers throughout the night that killed our internet connection. Well, it is still happening. We are nowhere near close to stopping it in my opinion. Sometimes I get the felling that some people could not navigate themself’s out of a paper bag. Those are probibly the same people that would over engineer a wheel if you know what I mean?

Well I am train tunnel bound so I will close this entry out. Hopefully today brings most of these deals to a close, but I am not sure just yet.

MRTG never lies

by

As you may know or guessed by these posts I actually work in the technology field. More specifically I deal with network & server operations for my company. Because we do that we monitor lots and lots of things on our systems. Anywhere from network traffic to temperature of equipment, to performance counters. This gives us an idea of what is going on when we are not watching the systems. Some of this is manual (stuff gets logged but to understand what is going on we need to look at it) and some things are automated (something is not working we get paged about it). That being said late this morning I took a look at our network monitoring pages for the weekend. I was surprised to see huge utilization between two of our sites. More specifically a database server in one site is sending data to two servers in another site. We use MRTG to watch bandwidth. It is a free linux tool. It is great. I for one am just getting into linux, but stuff like MRTG gets me wanting to learn more. So I bring the problem to everyone’s attention. Now here is the problem, half the people dealing with our database servers are saying nothing is going on at the time we see a problem. The other half agrees something is wrong but not sure what. Turns out, oh yeah we run some stuff late at night and an anomaly happened to cause it to send out 600 meg files and not 20 meg files, or something like that. So some of these same fun guys are saying that nothing is going on, but our T-1 pipe is flooded for 4 hours with traffic because some servers are trying to transfer well over a gig of data.

The moral of this story is that tools like MRTG don’t lie. We knew (well, Gus, Keith and I knew) from the first second we saw this what was causing the problem. We just didn’t know why it was happening. If it wasn’t for some sherlockian tactics from the man “Gus” we would be shooting in the dark. Never doubt the tools you use to figure out problems. When you doubt their ability it is time to get new tools, or test them. MRTG has not failed us yet, but we still have skeptics claiming “everything is ok”. Wake up and smell the coffee. Or wake up and drink some coffee to stay awake.

This whole problem boils down to a sherlock holmes type mystery. That is how I think of it. You have a problem, and you have to logically deduce what it is, and why it happens. Don’t ignore the obvious. Coming up with a complicated explanation as to why something is going on is stupid when a simple one fits the same criteria. There is a term for that, that I cannot spell so I won’t mention it here. I sometimes think that is all I really do when it comes to fixing problems. I try to use deductive reasoning. Probably why I like sherlock holmes so much.

This blog may seem like a rant, but it really is not. it is to say pay attention to the obvious, and think things through. Some people forget to do that, or just never think.

In other work news, we are still working to get a deal for voice service in our new facility. Dealing with telecom people is a nightmare. I thought dealing with routers was hard. And I remember when I thought programing a switch was a daunting task. Boy was I naive.

I spent the entire day dealing with telecom quotes or the above mentioned bandwidth issues. What is crazy is I know we have tons more in monitoring to deploy and watch. Right now we are finding tons of stuff. I wonder what we are letting slip through the cracks? Maybe nothing, but you never know.

I am left work at like 7pm tonight so I won’t get home till like 8pm. I need to make dinner, and relax a bit. today was very stressful day.

More work, less hacking, unless it is for work

by

It is monday again, and I am off to work. Today we meet with another telecom provider to see if they have a good deal. We need to make a decision this week.

OK, time for me to gripe about something. I am on the train, and cannot stand it when people receive cell phone calls (or make them) and have an entire conversation. In addition to that these people usually talk like a plane is flying by. Now I am all for cell phone use. I have 2 of them, and I don’t even have a home phone (another story). When I am out and about I will receive calls but only talk for a min if it is not important. If it is a work emergency (which I get more often than I would like) I talk. What is with this casual talking on a train with 100 other people. We don’t care what you have to say to your friends or family. We don’t need cell phone restriction laws, we need some etiquette from people. Ok, I am done ranting.

Speaking of cell phones I am kind of bummed that my main phone (an AT&T GSM Provided Sony Ericsson T68i) won’t work in our office in Kingston. Actually it wont work in kingston at all. I am not sure what will work well up there. I know verizon works, and t-mobile, but not sure if either of them provides good signal at our site. I have a verizon phone but I wanted to cancel it. No I am paying the monthly fee to wait and see. I also have a T-Mobile SIM card, but no phone. I am holding onto that for a bit longer also.

I gotta tell you, I go through phones (just like PDA’s) fast. I just upgrade to the latest and greatest allot. I was on ebay last night looking for a phone for the CEO of my company. He is going to europe and wanted a phone. He has a SIM already, but no phone. Since I didn’t have any extra he said to look for one. Well after looking I don’t see any GSM phones that do everything I want, except for the one I already have. For my CEO we are just going to rent one, but I started looking for myself also. Like I said I don’t have a phone for my T-Mobile SIM. It is weird. For the first time in a while I am ready to spend money.

OK, back to work talk. This post was supposed to be about work. I ended up sending back the low end Compaq 1U server we got. FYI, 1U is the size of it. Computer racks are broken down by U’s. Each standard rack has 42U. A 1U server takes up 1U of that 42. They are really small. It turns out the compaq server we bought does not have a RAID card for IDE like were were told 3 times. The were going to send me one for free, but I decided to send it back instead. Part of my reasoning is that we were going to get 2 of these. And I want to get them and see if I like them. if I do like them I want to standardize on them later on. If we got a free raid card, I would have to pay for them in the future. That would put the cost well over what I wanted to budget for these low end machines. I ended up ordering a super micro box and saved like $600 off my original quote. It is a no name brand, but we already use no name VA Linux machines, and this thing has a 3 year warrantee. I will see if they are any good.

Bidding wars, no its not a movie

by

So we have the bidding going on for services at my new office in kingston. We got people asking to provide Internet service that I never even heard of. We also have two big voice providers coming in to bid this week and next. This week is the contender. (no names just yet), next week is the guys who have our biz, but need to work to keep it.

I also got news that we have a new hire for monday. I ordered his computer last night and should have it today. I hope justin can build it in time. We are out of phone ports so that will be interesting. we also have no place to put the guy so a phone is no big deal. it is now starting to get fun with space.

I am in the subway tunnel now, so I will hopefully write more later.

Kingston project is a go!

by

Even though officially our project was a go for a while, we finally sat down and ironed out time table’s, and things. Our CEO wants everything ready at the facility by June 6. June 6, by the way was the date we moved into our current office last year. That poses a problem for me since I have vacation plans for that week. I am in barbados with my family. My boss Gus says to chill and wait and see if things actually shape up next week. Realistically I don’t think we can make that date, but I am told to aim for that date, I have to do everything possible to do it. Even if doing all that won’t change the fact that we wont make that date. it is an interesting position.

I also got the cd-rom / hard drive for the new server we bought. I believe I talked on monday about the discount server we got from comaq that didn’t ship with a cd-rom or floppy drive. Well that problem is resolved. I need to build the dam thing now.

It just started raining in NYC. I wore light khakis thinking it would be nice out. They are going to get messed up by the dirt and water kicked up when I walk around now. Oh, well.

I am on my own today, well I have justin but he can’t do everything I need him to yet. Keith is off due to the holiday. Not really sure why since he is not very observant, but I think he said something about having to travel to jersey or something. Should be an interesting day. We shall see…

It might be summer, & an anniversary of sorts!

by

Today is the first day this year that I left my house without a coat. It feels like summer. It is like 65 degree’s now going up to 80. I am happy the way it is right now. It is perfect. The AC in our office is not working yet. I think there is a problem. If I had this blog last summer I would have written about all the great problems we had with AC in our office. But I didn’t so use your imagination. It was hot. This year is not shaping up very good either right now. On a plus note the AC in my office works, but it dies under the strain of keeping air cool everywhere. even with the door closed it will get hot on the first 90+ day. we shall see what happens.

OK, due to me taking a late train yesterday night I actually had 3 blog entries yesterday. that will not happen that often. I really should have had 2, but I broke up one into personal stuff, & work stuff.

I am working on figuring out how feasible it is to move my entire photo web site over to my apache server. I want to get it to work, but I don’t want to have to do allot of work. there are tons of pages, and about a gig of images. it will be a big job if I can’t automate it or get frontpage extensions to work on apache. Hopefully I will have time this weekend to look at it some more.

I am taking Dan to bestbuy today to purchase wireless gear for his new apartment. Dan is the director of operations at the company I work for. He is really cool, and we get along great. That said, I don’t normally do computer work for people outside of work. I can’t stand it. Dan understands that so we are going to buy the stuff only. I like buying things! Actually some people I don’t mind helping, but others who have no business asking for help or other people who just expect it, I don’t like doing stuff for. He is in neither category so it will be fun to setup some wireless stuff. Wireless is good.

It dawned on me this morning that yesterday was an anniversary of sorts for me. I started at MFS network technologies (a division of worldcom at the time) 6 years ago on tax day. That was 1997 if people can’t do math. That really marks my beginning of my technology career. My first official help-desk job didn’t come till AuGust 1997, but I did plenty of technical and help-desk work at worldcom so I consider that my start. I figure when you are upgrading file servers, building new computers for people and auditing software you are in the technology field. That was just a bit of history about my tech roots.

Today’s work agenda, tech meeting, then tech management meeting, then trip to bestbuy, then kingston move meeting. Just full of fun meetings today! I also need to call and harass some vendor’s (see yesterdays posts).

Tonight’s the hockey game. I can’t wait. I broke out my Islanders jersey and it is in my bag for later. I am at a point where I cannot wear jersey’s of any kind to work. it sucks but I will deal. At least I can still wear tee shirts. I think it is funny to sit in a meeting with vendors in a tee shirt and khakis while they are in suits. it is very amusing.

Well I feel us descending into the train tunnel so I will stop here. Details on the game and other stuff tomorrow.

Tuesday With vendors, and web problems.

by

Well I am starting to write this on the east bound train home tuesday. Not sure when this post will be finished but I am stuck leaving work at 7:30PM tonight. It is daylight out so I don’t even realize that it is late anymore. There is still work to do and it is light out, you forget it is 7pm.

We had some problems some web sites today. It was not fun. We think it is related to a bunch of different problems ranging from running out of disk space on one web server to huge amount of searching on a database server. It probably will take days to sift through the data.

I ended up meeting with our telecom vendor today. It was a meeting that was planned when I was out on friday so I was sort of surprised. I think it went well. We told them point blank that we were not happy with them, and we need problems fixed now or we wont use them on in our new office. they seemed eager to do what we want (of course who wouldn’t when you dangle money to someone). We are on sales guy number 4. This guy seems to have a clue so we will see how things go from there. it is tough. I like what these guys say, but I have heard speeches before and then get no results. if our new sales guy can do what he claims I will be happy. the question is will he do what he claims? I am betting allot on that answer.

In other news, Keith is officially a dumb-ass. His problem with the RAID array I spoke about before was that the card came loose and he didn’t check it while he tried to fix it for 2 1/2 hours yesterday. At least he figured it out on his own. FYI, I use the term dumb-ass lovingly:) If I really don’t like them I would just call them an asshole.

Sales guys suck, & other Tuesday news

by

Have I mentioned that sales guys (people, gotta be politically correct) suck? They do. I am trying to sift through all the quotes and crap I am getting while preparing to build this new office, and people don’t have a clue. I have one phone provider call me last week going they can’t provide local phone server in the area we are moving to. I was like, ok sorry. I guess we can’t use you then since I want one company to do everything for me. Then monday I get a call from that guys boss going, well we kinda can do local service in that area. I was like make it simple and make it a good price and I will look at you. Until then these guys are talking BS. I cannot wait to get the bidding out of the way and just build the thing.

Our weekly management meeting was canceled yesterday. I think there was a board meeting or something. It will happen today instead.

So I got my new low end compaq rack mounted server. I mean low end when I say it. This thing is an IDE drive machine with RAID in it. it is so cheap that it didn’t even come with a CD-ROM & Floppy drive. Normally I assume a computer will ship with at least 1 input device such as a floppy drive. got a call out to those people to get something so I can load the OS onto it. Like I said sales people are moron’s.

The only other things of note on monday were we had some patches of problems with one of our database servers. Our engineers are looking at it. That means I need to get my crazy 8 ball out and help them with the diagnosis. Also Keith is having problems with an IDE RAID system. I will look at it with him today.

Work again, Monday Meetings & hardware purchases

by

It is monday again. The most hated day of the week. I used to not like monday’s just because it was the first day back from the weekend. My new issue with monday is it is the day of meetings. Actually it used to be the day of meetings, now it just feels like it. Monday is the day that all the managers in my company get together to let everyone know what is going on. That means writing up agenda’s, and other fun meeting stuff.

Monday is also the day in most companies where new hire’s start. We are no different. Thankfully we don’t have any today. I actually have someone new starting tuesday, but we probibly will have like 5 new people on next monday. Not sure yet. No one tells me till the last minute normally.

This week I need to finish setting up the temporary firewall I will use to test the new network for our kingston office. For setup purposes I am creating a new isolated network for all the gear we are getting for that office. I am using an old firewall we stopped using in february. Once that is done I will need to begin getting quotes for servers. The next step in the design would be file servers and a domain to put all the computers in. We currently use a mix of compaq and dell servers. Most of our machines are dell, with some database machines being compaq’s. Alot of our older equipment is VA Linux machines. I personally don’t like the VA Linux equipment. The company stopped making hardware, and the machines are not hot swapable anything in them. Granted their track record is ok. I have only had problems with 3 of them that I can remember, and we have over a dozen of them. For this new office I have not yet figured out if we will go compaq, or dell. It will depend on price most likely. I also have an IBM guy wanting to sell us stuff. They say that they can meet the other guy’s prices, but I haven’t seen that happen yet. Sales guys, are sales guys.

Another cool project we have going on is upgrading our data center to a gigabit backbone. We have the design down, and now we need to buy and deploy the hardware. I probibly will purchase that stuff this week. that will be cool to deploy.

Obviously since I am talking about work stuff I need to be vauge enough so I can write it here, but also descriptive enough since I want this blog so I can write about what I am doing and thinking. That will be a fine line I walk when doing this.

The scrub haircut guy & other misc. items

by

Yesterday was a good day, I think. Got a lot done at work. We still are no where with picking a hardware vendor, but at least we getting other stuff done. Got construction requirements done for the space. I was bad and had burger king for lunch. I don?t do it so often anymore so I think it is ok. Keith was going down there, what was I to do? I also took care of my metro card problem. I never got my mail and ride ticket for April so I had to fill out an affidavit and get it notarized. It was a pain, but it is all sent out and I hopefully will get my money back and my may ticket on time.

Got my hair cut also. The guy I always goto was on vacation so his dad did my hair. He is good, but never cuts it short enough. I will probably have to go back in like 2 weeks. I like those guys. I goto an old fashioned barber. It is father and son who run it. These two Italian guys just cutting hair. they are cheap also. My friend jayson showed me this place years ago. I was paying double what I pay now for a haircut at the ?stylist? and it was not as good.

After I got my hair cut I picked up food at the omega diner and got my mom. We went over to my sisters to eat dinner. My dad was already there babysitting. Everyone is all worried because Morgan cry?s right after eating. I guess that is a bad thing for babies. Got to play with her for a while. Then I got to hold her while she slept. I love doing that. It is so peaceful. I had to go home though because it looked like we were having problems at work. Anyone at work reading this, when I get 20 pages while holding my niece I am not a happy camper!!! Besides playing with Morgan I also sent all of her baby pictures to myself via ftp. My sister is running out of hard drive space, and she does not have a cd burner. My next project is to find her one.

I got home around 9pm. I installed my new hard drive for my desktop computer. 160 gigs of stuff. I started to move stuff off my old drives. Got one of them done and found out the one that broke was the data drive with all my personal files on it. I am so happy I made a backup of most of it to DVD before I did this upgrade. It is bad, but not too bad. I gave up on moving my mp3?s since I have a more recent copy on the powerbook I am using. I will back it up later. The only thing I need to do now with my ?modified? not upgraded computer is to reinstall windows xp. I have been using the same install for over a year. That is like a record for me. I tend to reinstall operating systems every few months. Now I am busier I do it less often. Hopefully the mac will require less reinstall?s of the OS also.

Spoke to Steve. We are still on for Thursday. Him and tom are driving down around 4pm. That means they will probably be late for the 8pm concert. We shall see.

I am running late today. I just got up and could not get going. So I missed the 2 regular trains I take. I should be like 20-30 minutes late for work. Still well ahead of the rest of the department. Actually looking at my pager just now I realized I may be a little later than that. I don?t think we are doing a staff meeting today since 4 people are out of the office this week. I just paged Gus asking him what we will be doing.

Well we are now entering the famous train tunnel to NYC. It is rainy today, again. My hands also hurt and it is only 9am. I hope they get better because I type and mouse click all day! More later, maybe?