2 Days At Kingston

I did my 2 days in Kingston this week. I am on the train (Amtrak) heading home. Got allot done, and still more to do. We just brought up 10 computers from our NYC call center and are setting them up in Kingston. We now have over 32 stations setup with 10 more for training. We are well on our way to 48, the target for phase 1 of our build out. Then we move up to 85 stations soon after that.

I also just got a new help-desk person in Kingston. Kai started monday for us. He was originally hired as a call center supervisor but he had IT background and we liked him so we transfered him over. Sorry dave. the call center’s loss is our gain.

I will be in NYC the rest of the week, and Keith will be in Kingston Wed, & Thursday. I want all the building out to be done so we don’t have to spend so much time up there. It takes time to transfer the knowledge we have to the new people at the same time as we are deploying so much gear.

We just got 30 new phones. We need to program them. We also had 8 new people start Monday. It is busy, and I wouldn’t have it any other way. Of course a break now and then would be nice!

The Phone Traffic Transfer

Last night I successfully completed tranfering almost 50% of our call traffic to our new Call Center. At about 8:45PM Sunday AT&T redirected the traffic and it immediatly came up at the new center. It was actually very painless. Knock on wood nothing goes wrong.

Today I got to work at 7:30AM to reboot the call center software server, and from 8AM on the office was busy. We maxed out all the computers we had setup by 1PM. We even had 3 users logged in from our NYC office using the VPN tunnel. As of 10PM tonight it was still busy and they needed to get people back on the cisco phones because the queue was backed up alot.

Surprisingly (or not) we didn’t have that many major issues after the transfer. We had some computer that had some apps not working. We also had 5 machines that didn’t have Windows XP activation completed. The new computers ran out of the 30 days for activation and some of them coudln’t startup. To make matters worse the microsoft activation servers were not working ok. After some work, we got them online but it was a pain in the ass.

Service Observe And Phone Stuff

Today I dealt with finalizing a solution for “service observe” in our new call center. We want the ability to listen in on our rep’s calls so we can QA them. Cisco’s VOIP solution does not offer that out of the box. I don’t know why, but it doesn’t. We have a company we are looking at, but we want to be sure things work correctly before we order it.

Also today I configured up 4 Cisco 7940 IP phones to use in our NYC office. We need some in case we are overwhelmed with calls on next monday. We should be able to have a total of 5 phones ready to go if needed.

Configuring the Cisco phones is really cool. They are programable via the Call Manager web site, and you can reboot the phone remotely. To get them to work, all you need to do is set the phone to DHCP and have the proper TFTP server config in the scope. It is not bad once you get the hang of it.

I was using my Cisco phone all day to talk to Joe, Justin, & David in our Kingston office.

We are planning a test of our T-1 redundancy on friday. The work to provide roll over and redundancy was completed and we will test it friday to make sure it works for mondays move of voice traffic. We need 3 voice T-1’s to work together so if calls come in and one T-1 is full, they roll over to the next one. This is also helpful in redundancy in case one circuit fails to work. We won’t need all 3 next week, but we will probably use up one and goto a little on the second one.

Training For A Call Center

Yesterday a new class of trainee’s started at our Kingston Call Center. We have 24 customer service reps who will “hopefully” start taking calls full time on 9/22. The class was so big that we had to put the working reps in the training room and the people to be trained sat in the call center. Joe our Tech guy in Kingston is overloaded right now. Justin went up for 2 days to give him a hand. Justin was eager to go up, so it worked out ok. I need to get up there one day next week. I will also need to be there on the 22nd.

Besides the ongoing call center build out I have hardware swap out’s to deal with in our Data Center Thursday morning. Tomorrow will be interesting since it will be a long day.

I just received my first Penguin Computing Server. These guys make cheap linux servers. The joy of it is, I can put Windows OS’s on it without a problem. The Super Micro servers I have been buying for windows will not handle Linux. I have some issues with the hardware, but for the price we will deal with it. A decent server can be had for around $1600. I can get a really cheap one for under a grand. We are actually looking to get some cheap windows box’s and may actually use these guys.

I am still awaiting a heat sync for a Super Micro test server we are testing. I got a trial of a serial ATA server. Actually we had the financial ok to buy it but instead of buying a regular server we got this to test. If we like it, we will keep it. If we don’t like it, we will send it back and get a normal IDE server. Serial ATA looks good, and this machine can do hot swappable drives, where IDE RAID cannot.

I need to also start working on our Exchange 2003 test server. Gus and I want to get one working for the Information Systems team and see if it is easy to admin and troubleshoot. If it is we may want to make a larger scale deployment. The difference this time will be we don’t outsource it and we do it ourselves. In December of 2002 we outsourced our mail and put it on Exchange 2000. It was a horrible failure. Too much latency with accessing exchange over the WAN. We abandoned that option and returned to our original mail provider with just POP/SMTP mail. We have been happy since.

We Have A Call Center, Well Sort Of!

Wednesday we finally officially started taking calls at my companies new call center. I was on a conference call with AT&T while they made the first test call. It was exciting, but melodramatic. After a few small issues the Customer Service Rep’s (we call them Part-solver’s) were taking calls. Not many calls mind you. We wanted to make sure the system worked before we send lots of traffic to it. We maybe got 100 or so real calls. Now that the system is up and running, it is time to begin hiring lots of people to take calls. It will take months to get the new center fully staffed. For now I am focusing on cleaning up things, virtually and physically. Joe doesn’t yet have a desk in his office so our stuff is sprawled all over 8 call center desks. I still have lots of work to do, but it is nice to have started taking calls.

I was able to leave to go home with Dan Wednesday evening. It was nice to be home. I rode back with him and 3 Rep’s who he brought from NYC. They were there so the new hires could see how things should go.

The Last Big Kingston Trip, Hopefully!

I am off to Kingston NY again for a few days. This week we officially activate this office. Tuesday we turn on the phone system and begin taking calls. I am going up today so I can do any last minute changes. Keith got allot done for me last week, but I need to finish up any loose ends. Joe is feeling better and will also get make sure we are good to go. I am scheduling myself to be “upstate” as we NYC people call it for 3 days, but if all goes well I may be back tuesday night.

Hopefully once the call center starts taking calls I can limit my trips to one and two day stints up there. that way I can catch up on issues in NYC.

The New Office Is Open, Well Sort Of

The new office I have been working on for months is finally open. Well it is open for training at least. The first group of call center rep’s started today. We still don’t have all the phones working, the patch panel is not done, and I still don’t have a ghost image or security profiles for the users. On the other hand we had enough computers and people were able to train. In the end things are not perfect, but they are working. As we speak our tape system we took out of our NYC office is restoring applications that I could not move to the new network via my laptop.

Tomorrow I have our intergrator return to mount and finish testing on my call center software and my phone system. Things are working now, but the phone system is not mounted yet. Allmost all my data gear is mounted, and looks great. The only thing left is the patch panel. The rest of the work to be done is all virtual. Setting up users and security. I am in Kingston till tuesday evening then I am back to NYC. I need to spend some time in NYC doing other work besides this move. The move preocupies almost all my time recently. I need to spend more time in kingston, but just can’t be away. That is how I feel right now, that tons of things are getting overlooked because I am not in NYC. I will send Keith up on thursday and have him spend 2 days getting kingston in order while I clean up issues in NYC. Then next monday I am back to Kingston for our first queue move. This is when we really put the new system into production.

Today is looking like it was a record day for my company. We maxed out our 3 voice T-1’s with calls durring peak hours. We also look like we are going to make a sales record for today. This is most likely due to the black out, but it still is a record day. The new office doesn’t come fast enough.

Wow, Crazy Stress Filled Week

I have been really busy, and out of town most of this week. This will be quick since I am off to bed. I was in Kingston NY working on getting our new office ready Tuesday – Thursday. Things are coming together but still having some issues. The voice T-1’s all work. But the phone system needs some new hardware. My companies current phone system had problems tuesday night causing problems for hours. Dam windows NT 4.0. More issues on Wednesday, but not as bad. Now I have issues with requests for a redesign of how our call center is going to operate. I need to map out the changes tomorrow. I am so stressing, and tired right now. I hope I can get some sleep. Friday will be interesting. it always is when you are out for a few days. I have racked up allot of hours on the mobile phone this week.

The good news is the Cisco Voice Over IP (VOIP) phone system we put in actually works. Sounds great and everything. We are just having some issues with getting the wrong type of routers. All sorts of capacity issues now. We are having our integrator fix the problem.

More tomorrow, I hope.

Queue This

So the more I work in the technology field, the more I realize what I don’t know. I learned about a totally new type of product last night. Now I need to become an expert in how it is tied into Windows 2000 Active Directory. The product is MSMQ or Microsoft something or other messaging queue. it is really interesting stuff, but I never dealt with it before. This is one of the many new projects I am learning about.

I am knee deep in setup stuff for our new Kingston office. I have the Active Directory model set. Now I have to build it. I ordered the first batch of 25 desktops today. They wont ship till August but we locked the price and model in. We are waiting on some better deals for flat panels before we order them.

Our cable guy’s finally called us back today. Joe is going to kingston on thursday to meet with them and make sure the plan is set and the work begins and finishes on time. I am still waiting for an install date on our long distance T-1’s. We keep getting the run around from our carrier. I am freaking out that the LEC verizon goes on strike in August. And yes, they will strike. both sides of that argument are no where near an agreement. That will get messy.

Staff meeting day

Another mid week staff meeting day. The mornings are always shot doing this stuff, but it is needed. I have allot to bring up today. We (keith) is about done with the new switch configuration for our data center. We need to schedule a deployment of that. It will be a big deal since so many network paramiters are changing.

I also have a conference call with our VOIP intergrator regarding our project. I think we are about ready to set an install date. Our CEO wants us able to test calls on Augist 15. I am not sure if that will happen, but the big bossman wants us to try, so we try!