Spooks Series 4

Spooks (MI-5 in the US) has completed its series 4 (season in the US). Thanks to the joys of the internet I was able to watch the entire series while living in the US even though it has not been broadcast here yet. The 10th and final episode was crazy. I for one can’t wait to see what happens to everyone next year. Isn’t that what good TV is about? Keeping you wanting more? I still haven’t found an american show as good as this in the genre.

Since I was able to get all the episodes so quick I now have to wait until the end of 2006 for more. Let the waiting begin…

Poker Wednesday Again

We actually got together to play poker again today. I fared better than I did last week. I still didn’t win, but I was back to my old self. I had $40 on the table and came away with $38.50. Not bad when only 2 other people out of 8 walked away with anything. And only one of them had a profit. It was a John taking everyones money night. I was down for a while and happy that I came back in the end. If people would play hands I stayed in on in the end I would have had a profit. I had 2 (count them 2) full houses near the end and no takers even after I tried to slow play them. I won’t complain about losing a buck fifty. To me that is a wash.

I personally don’t like the style of play that went on. Everyone seems to bully people out of the game before the flop. John does it, and since he is effective at it everyone seems to emulate it. With 2 cards you have nothing, don’t bet $3 and before the flop and then bet nothing after it. It makes you look like a dick. That is just my opinion.

Overall it was a decent night. Hopefully we can play again before December!

Mac Sync

So I am kind of ticked at Mark/Space. Not for a bad product, or support or anything. For releasing a product that solves all my problems but not announcing it until it is out. A little heads up on the new Missing Sync 5.0 would have been nice. it would have saved me the $149 I spent on Daylite and then have to go out and buy the upgrade for the Missing Sync. Daylite is a great product, but the integration the Missing sync now gives me with the Address & iCal apps make it hard not to use them. I can get invites again without having to manually add them. I get all the fields from Address book that I couldn’t before. And the biggest advantage is that calendar categories move back and forth with iCal now. Daylite was a bit limiting on allot of those features, but it had the best task lists, and ability to assign or associate people, projects, etc to tasks and appointments. The question is are those nice advanced features worth not having basic features like what Address book and iCal can now give me with the Missing Sync? Honestly I don’t know, but I am once again using the Mac PIM apps and the new Missing Sync 5.0. Will I get fed up with them in a few weeks and switch back to Daylite? Maybe.

But for now I am happy with what I am using. Now if only Marketcircle could integrate better with the Missing Sync and have iCalander support for invites. The best of both worlds would be nice.

Chop Sticks

I went out to dinner at a Japanese restaurant last night (thanks Zagats for 2 nice options, we chose the less crowded of the 2). I Kristen attempted to show me how to use chop sticks. I had it working for a little bit, but I still suck at it. If my dad has been unsuccessful for 2 decades to show me, does that make me a lost cause?

After dinner we hit a nearby bar. Ended up meeting a bunch of Kristen’s friends from school. They seemed cool. We drank. Fun had by all. It was a nice end to the work week. If only I would stop coughing. I don’t know what my problem is, but for the past week I have had sinus issues, then this weird cough. I don’t feel sick, yet I cough.

Poker Loss

Yesterday at Brian’s request I got a last minute game of hold-um setup. We did it at my place. We had a good turn out. I think 8 showed by the time we were done. I lost $40. I have never lost when we don’t play tourniment. I broke even, or won but never flat out lost all my money I played with. It felt weird. it was also weird not being able to leave in shame since it was my apartment. It wasn’t that bad actually, but I would have liked to play a bit more.

We all want to get together more regularly, but who knows who will take it upon themselves to schedule it. Even with the loss I had a good time.

Best Way To Send A Large File?

Do you know the best way to send a large file? Lets say a gig or bigger for argument sake? The answer may surprise you. SomeoneĀ once told me they heard a CEO of a big bandwidth company ask that question. The answer wasn’t some whizzing technology. It was sneakernet. Don’t get the reference? Take the file and walk it wherever you need it to go. Part of our operations tasks is couriering around drives with updated data between offices. It is cheaper to do that then send over the internet a 100 gigs of data. I sometimes find it funny we do it, but it makes sense. With many servers of the same hardware configuration we can also build servers and send the drives to their intended location and plug them into a different shell. Old idea, but it never ceases to amaze me how simple and effective it can be.

Crazy the things you think about when your mind isn’t preoccupied with something!

ESX Server?

I have praised VMWare’s products for some time on this blog. As things look now, I will continue to do so. I have used their workstation product for almost 2 years. I have used GSX server for about a year. Now comes ESX server. It looks awesome. I have a project that requires allot of new hardware to be built. ESX server looks like an good alternative to server sprawl. Instead of building dozens of servers, the same solution done virtually can be done with a few (not many) ESX servers. The cost benefit is real. I ran some figures over the weekend and it can save us a nice amount of money by going virtual. Then there are the saving space, and energy issues. And the flexibility we get by having a virtual setup. I am really excited about what we can do.

I have been trying out ESX on a test box we setup (reminds me I have a VM building right now that probably needs some click next help). The install was quick. I now have to read the dozen or so white papers spread across my desk (my computer desktop and physical desk). This software solves problems I didn’t even think about until I got to reading about the solutions. Multi vlan support is nice, as well as NIC teaming. I have allot of reading and testing to do in the short amount of time I have until we have to deploy the solution. I don’t think have been this excited about a technology since Call Manager. More info as I play (i mean test).

Technorati Tags: , , ,

The Evil Empire

I have so much I want to say, I don’t even know where to start or how much I will get out. Some people have their own impressions of “evil” corporate empires exist out there. I have friends that bash some retail store chains as being evil, or others who can’t stand some or all media empires. Most of those opinions are based on ideals, and some passing interaction with said companies. I on the other think the true “evil empire” out there is one that is partially transparent to people. I am talking about a company I will call the LEC that will not be named. If you don’t know a LEC is better know as “Local Exchange Carriers”. They are the SBC’s, Bell South’s, & Verizon’s of the world. You most likely deal with them with your home phone. The thing is everyday people don’t realize is that if you use an alternate provider for anything like phone service, or even business class T-1’s you still must use a LEC. The LEC owns and operates the physical lines in a given area. They own the copper and or fiber into buildings and houses. in some rare cases you have providers also having their own fiber or copper into an array, but that is uncommon.

So here is the situation most people don’t realize can happen. You have big name internet provider X as your data T-1 or voice T-1 service. They are a huge company but they do not own the local lines in the area you do business. That means between your office and provider X’s POP (Point of presence) you must utilize a LEC. If you don’t know better provider X won’t ever really bring that up, but I know this is the case. it is not that they hide that fact from you, but it is not something that will be highlighted on page one of a contract.

So this is the situation. You take time and effort to design a highly resilient internet backbone for an office that needs high availability on their voice and data setup. You spend the time and money to get multiple POP’s from provider X. You also get multiple routers setup with HSRP & BGP. All bases are covered, right? Wrong. The LEC who will not be named has a problem at their central office. All those nicely diversified circuits all go through the same LEC. Remember provider X will give you diversity, but they don’t own or control the lines into your office. If your area is serviced by 1 LEC, all your lines go through the same conduit out to the same CO (central office). Well if that CO has a problem with lets say some hardware, all your network diversified circuits are down. The LEC is the pinch point in most situations. Now if you have provider X, they will try to fix the issue. If they can’t figure it out, they will escalate the issue with the LEC who will not be named. Here is one problem. YOU or I are not a customer of the LEC who will not be named. Provider X is. You are considered a wholesale customer of the LEC who will not be named. To them you are the least important person. Now, they may say otherwise, but if you ever negotiated a T-1 between provider X and the LEC who will not be named, the LEC all but says you are not important if you use provider X. Is it true? I think so, but it may be a negotiation tactic. In any case when you have a problem, the LEC who will not be named seems to not care. Even if they ARE quick to respond they have the attitude of “we are big LEC who you have to use. We will get to you whenever we want”. I have had that feeling several times over the years. That is why I have come to the conclusion that the LEC who will not be named is the true evil empire.

So in a vague (or not so vague depending on if you know the true story) I have vented about my technical woes today. I know shit happens. When people ask why something is down (when telecom circuits and major network gear is down, not the minor stuff) I tell them I honestly don’t know how the stuff works the 99.999% of the time it does. Really. People think I am kidding, but if you ever had to spend 12+ hours trying to get the LEC who will not be named on the phone when you have an outage and when you finally do get them to do something it is fixed in 20 minutes you will begin to think like I am. I mean come on at least look like are trying to care about my problem!

And that reminds me of what Howard said to me when he stopped by my office today in the middle of the disaster that was my day. He was like, at least you are not having capacity problems like Sixaprt and their Typepad service was having. I had to laugh. He didn’t realize I use them. I think he thought I was using Gus’ server still. The funny thing was I gave them prop’s for how they were handling the situation. I sent an email to their CEO commenting on their outages and how they are communicating the issues to customers and I got a response in like 2 hours. How then can billion dollar LEC who won’t be named take 3 hours just to get one of multiple tickets into their system when calling the emergency support line? I am not talking about calling the “hi, my home phone isn’t working” number. I am talking about the “I spend allot of money and many circuits are down” number. It is really scary how a small company can be so responsive and a large one just plain old suck. Of course I should know that by now, since my company is not that big and we are SO much more responsive to issues than people we deal with. Yes small plug for my own tech group, but it is true.

Ok, by this point in my writing tonight I think I am just rambling. It has been a long day, and by now I am sobering up, but yes I did have a few drinks before I got home and started writing tonight. Hey it is Halloween and I went out for a few with friends from the office before coming home. I can’t remember if I have been over my opinion on the evil empire? If not I think I made myself clear tonight. Am I asking for too much? All I want is to get a person on the phone when I have a problem, and have them seem interested in solving my issue and get me back up and running. That person should also speak clear understandable english, and must understand when others speak clear and concise english. Forgot to mention that issue. Not sure what was worse, the FULLY automated ticketing system of one company today, or the get an offshore support person who cannot fully understand what you are saying number? Then there was the automated update system that called every 30 minutes with a message telling us nothing has changed. that would have been ok, if it wasn’t for the fact that we had 4 issues open at the same time, so Jayson had calls every few minutes. And in their attempt to be good about contacting people on alternate numbers if you didn’t answer your primary one, they would call Jay’s cell phone if he didn’t pick up his work one. The issue was he didn’t want to get more calls from auto response guy, but they kept calling.

Really I have come to the conclusion that I should not write about things that really bother me right after they happen. When I do, I write allot of stuff that is true, but when I am calmer I might not have written. For my own safety names of companies and details of issues have been deliberately modified in this post. The general issue is true, and yes I had a bad day today. I need to take a vacation day soon! On that note, I am going to stop writing. If I have more things to say about the LEC who shall not be named I will write later. On a semi positive note, provider X was not as bad as the LEC who shall not be named. They sucked allot, but at least their sales guy who I deal with allot was able to get some escalations in for me. That is saying something, in a day full of problems. Did it help? I don’t know, but it made me feel a bit better.

Backup of Files

I am trying to weekly rsync my files from the firewire drive on my Powerbook to the mini. I am using my Mac Mini and its Firewire drive as a backup set of my laptop. I have had trouble with the rsync choking on large initial copies. Not sure what is up with that. Some of the replications needed to be ran 3-5 times before they would complete. I have had better luck with the weekly updates. I guess with less file coping I get less errors. It kind of works for now, so I wont go crazy trying to debug it more.

I have a few more items to replicate but most of the important stuff is done.

Virtual vs Physical

At work I am deploying a growing number of servers for both production use and development & testing. One of the things we are doing is paying more attention to replicating our production environment at every step of the development and deployment environments. That means allot more servers. How much is allot? We added 25% more gear enterprise wide in the month of August. We have been busy, and we are not yet done. I have filled one computer room at a facility and we are bringing online another at that location sooner than originally planned. We also moved into a larger cage at our data center earlier this summer. Then we added cabinets to the new cage to accommodate the larger growth. In yet another office we added a cabinet, and are working on beefing up the power in the room to accommodate yet more gear. Then we need to look at more HVAC. I never had to deal with ancillary issues such as not having enough power to run gear. 5 years ago I would never have thought I would be in a situation like this. It is very interesting to me to look at how many rack U’s a server takes up when quoting them out and determining what to buy based on the cost of a 2 vs 4U server and how much it would cost to just add another cabinet if you got the bigger gear. As a plain old engineer I would just recommend buy this server because it did the job. In my current position I need to look at the entire picture.

The next few months will bring another burst of server sprawl. One of the things we are looking at is the cost of buying hardware for every server role we need to fill, or the cost to do the same amount of computing power in virtual machines. I must sound like a broken record talking about one of my favorite software companies, VMWare. Our GSX server has served us well, and is a great proof of concept to show how we can expand the use of VM’s. I don’t think we will deploy VM’s en-mass at our data center to do production work, but we have plenty of other uses for the technology elsewhere that makes looking into GSX or ESX server a viable alternative to buying more gear.

To me it boils down to 2 factors. 1 is of course cost. How much does it cost us to buy and deploy a dozen servers, power them, get KVM’s, and rack space for them, vs purchasing hardware for a VM server (or 2) that can handle the same amount of work load.

The 2nd factor is ease of use. How quickly can we get build a physical server for use? Restore it from a preset level of configuration for use in dev and qa? How fast can we buy and deploy hardware when a need comes up for a new server? The same questions apply for virtual machines.

I am a bit biased. I want to virtual machines. The flexibility they give you is amazing. I also know that I have SLA’s to keep, and costs to consider. So if the per server (or server instance) costs are too high we can’t do it. For now I spoke with my boss late this week to identify what applications need homes in what environments. The next step is to crunch the numbers to get all of our options. The VMware user groups have been helpful in figuring out realistically how many VM’s you can get per GSX and ESX server. More news as the project unfolds.