Data Center

The Work Weekend

by

I had to work this weekend. It wasn’t just work, but I had to be onsite at a data center for all of Saturday. I can’t and won’t really talk about what I did, but it was allot of waiting, and then allot of work. During a lul in the work I left a little before 9PM. I got home before 10 and was back on a conference call regarding the work that was going on. Unfortunately things took allot longer than originally anticipated and I had to get on another call at midnight. It didn’t end until after 4AM.

I vaguely remember getting to bed around 4:30 and was up by 9:30 freaking that I slept past an alarm I had at 9AM for another call. Thankfully that call never happened anyway. I ended up working for the majority of the day until after 5. Thankfully everything worked out ok, but things were dicy for a while. I really would like to talk in more detail but I am mindful of the sensitivity of what I work with. So nothing more than the vague descriptions I gave.

Thankfully I was able to get a little rest in tonight before sitting down to write this. Now it is off to bed.

A Release in Progress

by

I am eagerly waiting for one of the guys on my team to finish a software release on one of the applications my team supports. They have been working on it almost all day. I don’t have anything really to do with the release again until tomorrow morning but I know I won’t sleep right until I hear that everything went OK. I am hoping I get that nice all OK email in the next 90 minutes! Fingers crossed.

The Busiest Day of The Year

by

Today was the busiest day of the year for me at work. As it turns out this statement is true on several levels. My current company has a bi-annual event that generates the highest volume of web traffic on one of our sites. We literally began planning for today back in June during our last event. Because of the work shifts we ended up doing, and problems creeping up throughout the day today was also probably the hardest work day I have had since I started working at this job last February.

The odd thing is I kind of liked it regardless of the craziness. There is something about being in the thick of issues with the pressure mounting and you having to make split second decisions that has an odd attraction. I guess I am still sometimes an adrenaline junkie from my days volunteering as an EMT in College. The other alluring aspect of today is you see six months of work put to good use (hopefully), and for me all my skills as an engineer and a manager put to the test. The result is exhausting physical and emotionally, but in the end very rewarding. During our last event in June I commented afterward that if you are in an operations group and you don’t secretly enjoy such days you are in the wrong profession or lying to yourself. I still think that to be true.

The day did have its high and low points. In the end I know now that I have to handle a few personnel issues, and our systems performed better than anticipated. All in all a mixed ending for a roller coaster ride of a day.

A Very Long Day

by

It feels like monday night but it is fact Tuesday night in France. I am in Lille, France. The red eye last night was ok. I took some stuff to help me sleep and it worked a little but not completely. The bose noise canceling headphones also helped but it was still a long flight.

Once we got to Paris, we were able to goto a hotel and get a Day room. It is basically a room you pay for during daylight hours so you can shower, sleep, etc. We landed like 7AM local time but didn’t have meetings until noon so we needed a place to go and relax. That room made all the difference. I was able to take a shower and just lie down for an hour. I didn’t sleep but just able to relax on the bed was all i needed to get through the day and even end up at dinner at 10PM local time.

We had our meeting in Paris after we got a bit lost walking around to find our office. We then toured a data center for a bit and then our host took us to the train station so we can catch the TGV to Lille. The TGV was awesome. We got biz class on that also, but it probably wasn’t needed for the hour long trip. That train is fast, and quiet. I always like trains but this one was really cool.

Once in Lille we hit the hotel and then went out for some dinner. We ended up meeting up with some other people from our office and had dinner together. I didn’t get back to the hotel until late. It is weird when it is like 10PM and there is still some daylight out.

I am off to a much needed sleep in a bed.

Redcats Paris Trip Tuesday - 001 Redcats Paris Trip Tuesday - 003 Redcats Paris Trip Tuesday - 016

, ,

Network Issues

by

Today started early with a network circuit outage, and closed the day with another network circuit outage. All told i think we had three distinct circuits go on the fritz. We had backups, but one needed to be manually switched and the outages caused us to stop everything and work on the problems. It was a long tiring day. I passed up going out for my dad’s birthday because i was so tired and i had a minor head ache.

Looking Forward

by

I wanted to write a bit about some of the cool new projects i am working on at my new job. I am not going to go into great detail but it is something.

One of the biggest things i am doing is working on a plan to move an existing self hosted data center to a Co-Location facility. We just need more space. Thankfully instead of physically moving all of our gear we are looking at getting an all new setup built out and retire most of the old gear. The timing works out perfectly for us so we are giving it a try. We will be moving some stuff, but not a whole lot. Another thing we are trying to do is make this data center site the model (for us at least) for Virtualization. We are going to build out as much if not everything with the exception of SQL server as Virtual Machines running on VMware’s Virtual Infrastructure 3. If all goes according to plan it will be really cool setup. The TCO will also be allot less than doing it physically, it will be quicker to build out and deploy, and it will consume allot less power! The way i think about it is if we get a request for new servers to be build we just see how much Virtual capacity we have and just provision a new machine. If we don’t have extra space, we just purchase an additional node and more SAN storage and add to our VI3 cluster. The goal is to stay far enough ahead in available capacity (minus the N+1 for redundancy) that we always can accommodate even the largest provisioning requests. I can’t wait to see how this turns out! It will be interesting.

Technorati Tags: , , ,

The Wet Mini Road Trip

by

Today I left the island of Manhattan and my reliance on the public transit system to go out to Long Island for a data center tour. In theory everything should have been fine even in the rain. My car was late showing up, but I still made it to the site a few minutes early. That should have been a good thing, but apparently there is no reception area at the place I went to so I was standing out in the rain with no way of getting inside.

Thankfully my boss drove to the site so eventually he showed up and I had the dryness of his car. We tried to sort out the whole situation. Apparently there is no one typically onsite at this facility that can greet people or give tours. Some how they messed up the fact that we were showing up for a our so we were out of luck.

Since the tour was a bust we hit a local diner for some early lunch, and then I called a car to take me back home. With the amount of time lost going back and forth to the island, this was the perfect day to work from home. And that is exactly what I did. Once I got home I got online via the wonderful world of VPN and worked the rest of the day from the dry confines of my apartment.

I ended up not going out to meet Danny after work even though we had tentative plans. I was no where near the office, and I was tired and didn’t really want to go out after a crazy wet morning. So laziness kicked in and I hung out at home.

Technorati Tags: , , ,

Data Center Tour

by

Today was busy with meetings and an offsite tour of a data center. We are looking to take some of our self hosted gear and put it at a hosting facility. We looked at one in the city today. It was a decent facility but not the nicest I have ever seen. It does meet our needs, but we need to audit what we want to put there and get a real space and power estimate. Other than that, nothing else went on that was worth noting.