Networking

Fresh New Circuits

by

We just got our new 3700 series Cisco router today. it will be used for our new 4.5 megabit internet connection we are getting for NYC. This replaces our 2 redundant T-1’s. this connection is basically 3 T-1’s plugged into 1 big router. less redundancy, but triple the bandwidth. We figured our corporate office can have some acceptable downtime. Since our uptime stats have been great on our current circuit it seemed like a good gamble. I hope that once this is done I can use my VOIP phone without getting terrible choppiness sometimes.

We hope the circuit goes live in about 1-2 weeks. Verizon will be out tomorrow (had to cancel for some reason today) to do the smartjacks. Like I always say, you cannot have enough bandwidth.

Busy and Sick

by

Got one of the 2 circuits we were going to cancel taken care of. our provider is working on canceling the other line. The new bigger circuit for my office is on order now.

Jayson is working on finishing up the hardware for our VMware GSX server. We had issues with the poweredge we bought so we won’t have 4 gigs of ram. We will settle for 3. That means less VM’s we can run. I need to cofigure GSX when I am in the office Friday.

Jay is off next week to goto sell his house. he is going to fix it up and put it back on the market.

I am working to get hardware for our Biztalk 2004 deployment. I need to buy a server for QA and 2 for production. Working out pricing issues now. Hope to have them by next week.

New Switch

by

We just upgraded our network in my companies call center office. We put in a new Cisco blade switch. it will act as our primary core. We did some reconfiguration on the old core’s and will use them as backup. this is phase 1 of our expansion plans.

When I say “we” did the work. I mean we hired a contractor, and told them what to do. We then looked over the configs and did testing. It is nice to leave the heavy lifting to others.

The upgrade actually went well. I am always hesitant to say that because I am fearful that something out of the blue will go wrong. But actually it went as good as to be expected. no major problems. I was back to the hotel and in bed by 3AM. I would have been in bed sooner but I had some cell phone problems I had to play with to fix.

Friday it is back to my normal office. I miss my chair:(

AD is New Again

by

We had problems with one of our active directory servers the other day. No big deal since we had more than 2 of them. A redundant back died. We rebuilt it, but that got us looking at our other AD servers wondering if we wanted to rebuild them. I tinkered with the new box today, and looked at our AD layout to see how we can juggle servers a bit.

This change in box’s also got me to start our project of moving our servers to a new ip block. This is mainly for tidiness. Our other office is newer and has a better internal ip layout. We are just taking our office and getting it up to standards with the call center.

A Single Minded Day

by

Some days I cannot remember what it was that I did at work. Today I only did one thing almost the entire day. I cannot forget what I did if I tried. I worked on fixing our Call Manager all day. Call Manager is the Voice Over IP version of the PBX. It is basically the computers that control our phone system. Last night Kai and Jayson upgraded our Cisco Call Manager’s to version 3.3.3 from 3.3.2. When you read the version number change you (I) think it is a minor fix. Well the latest version (it is the latest version we are using, but there is a 3.3.4, and a 4.0 now) does some things differently. Kai and Jay got the upgrade done, and tested all the phones in our call center. Everything seemed to work. What they couldn’t test was our remote phones. The phones in our NYC office didn’t work. We didn’t know this until hours after they went off to sleep. They left word that we may need to upgrade the firmware on the phones, but that was it. Turns out the firmware upgrades automatically when you reboot the phone and their is a newer version of the firmware.

The problem was the phones could make outbound calls, but could not receive calls. when you called the phones in a remote office the phone would ring but if you picked it up the Call Manager didn’t recognize that the call started. this is not good if we want to our reps to take calls. So off to call Cisco I went. This all happened before or as I got to the office. What a way to start the day that I knew I was going to be short handed anyway. Kai and Jay were both off to sleep because of their overnight. So after a long and drawn out troubleshooting session with Cisco we figured out our problem. I actually spoke to 2 guys over there. the first guy gave me the right answer but could not explain why we had to do what he asked. Since it required me to change firewall rules and we upgraded a phone system not the firewall I was skeptical. So the second Cisco guy came into the picture. He walked me through the same troubleshooting process (a bit quicker than the first guy). he then made some phone calls and got back to me. Turns out the first cisco support guy was right we needed to change some rules on our Cisco Pix’s. Why? Well in the new version of our CCM (Cisco Call Manager) they changed how some protocol’s operate. So what worked in older versions of CCM didn’t work in the newer version. We had to remove to fix-up protocol lines on all of our Pix’s that are involved in the VPN that makes up our WAN. Sure enough Kai was right. Kai as in Kai the cisco rep I spoke to, not Kai the guy I work with. I made the two firewall rules and the phones started working. Elapsed time on the whole saga, 8 hours. I got the phones working exactly at 5PM.

During this adventure I had other fun things to think about. Sean and danny dealt with problems with an index on a database somewhere that was causing one of our websites to be slow. Word of advise to people I work with. When I am fiddling with one phone, on another phone and talking to someone on a nextel and you know a system is down, don’t come and tell me about another problem that you need me to work on. I can only do 4-5 things at once. Thankfully Sean was able to get a handle on the index problem and fix it with little to no help from me.

Danny was helpful in my network trouble shooting saga today. Everyone else was surprisingly not bothersome. usually when I have a major problem people come out of the woodwork to bug me about minor issues, or that is how it feels.

I was crazed today. Am I glad it is over? Of course I am, but there is something to be said about days like this. First, it goes by so quick. Second, it is the type of day that you earn your salary. You get a few of the each year, and when you live it you hate it, but after you live through those days you are a better person for surviving them. On a personal note I am glad I got through the issue mostly by myself. Danny did give me great assistance in the network trapping, but allot of it was second opinion from what we got from Cisco. It is good having someone else around who you can sound off ideas to. I think Danny and I work great in that respect. On a whole I had to tackle most of this issue by myself. In the past that is not a big deal. Recently I have been delegating allot of the day to day technical responsibilities. It is something that I have to do, but I feel like I get rusty by not doing hands on work all the time. Days like today keep me sharp and lets me prove to myself that I am still in the game and can get dirty with the best of them, or so I think.

Pix Configuration Problems

by

I am having a problem with the new Pix. I think it is a minor issue, but I just cant figure it out. I am trying to make a second static VPN tunnel, and the pix keeps telling me the crypto map is not complete. I am doing nothing different from the first tunnel I created but it still wont work. I am going to call Cisco and try to figure out what the deal is.

New Cable

by

I got a call from the cable company that provides service in the building I am moving to. They said that the management company sent them my name. I currently use Time Warner, and for the most part I am happy. This place uses RCN.

First impression is that they are more money for less services. I get 4 premium packages, digital cable and a cable modem for under $100. These new guys offer you digital cable, a cable modem, and 1 premium package for $101. I am loosing 3 premium packages. To get them, I need to pay $13 more per channel That is a racket. Extra cable box’s are also more money per box than I currently pay. The only plus side is that I can buy a static IP address for $4 per month. I don’t think I can do that now. Of course my current provider only changes your address if you leave your modem off for extended periods of time. I have had my same IP for over a year.

In a nutshell I think I am getting a worse deal than I currently have. Only time will tell, but as I like to say “i think this will end badly”.

Network Problem Fixed

by

I have spoken about our potential network issue with a file server. Last night we decided to move the old and new file servers over to a different switch. Jayson this morning moved both servers to our core switch. He then began to try and copy files back and forth between machines exactly like we were doing before. Where we got errors before, the files transfered normally. It looks like it was either the port on the switch, or the switch itself. My fear is that another fiber module went bad. I hope it is on the 24 port rack switch and not on our Core.

We are going to test copying files to another server that is connected to the 24 port switch in question. That will help us narrow down our area of concern.

VPN Changes, Man I Am Out Of Practice

by

I really need to brush up on my firewall rules knowledge. It took me 20 minutes to figure out how to modify a rule. Once I realized how it was setup it was trivial to fix, but that initial learning curve to remember how it was originally configured is the hard part.

When things settle down I need to sit in front of the Cisco simulator software we got and play with some ideas I have regarding new changes I want to make on our system.

Remote Agents And Terminal Server

by

While I was awake and not sleeping last night, I thought about my remote agent design issues. Currently we have schematics of different electronic and home appliance parts on CD’s we use to look up stuff. The problem is they take up a huge amount of drive space. We get updates every month that need to be added to the existing data. This is fine when a computer is on our network, we just run the update from our file server. What do you do when you have remote computers over a VPN WAN link trying to get the same update? Pushing out 500 megs to 2 users is possible, but what about 20, or 200? It becomes unfeasible due to bandwidth limitations.

One option is to not give the remote agents this software, but they rely on it. My thoughts then turned to Terminal Services. What if all the updates are on the central server, and users just connected to the server via terminal services. That way I don’t have to worry about anything on the local machine except for an IP address and terminal client working.

This is contingent on our phone control software working with Terminal Services. I believe it does, but an answer to that question is easy enough to get.

The next problem is cisco soft phone. I doubt it would work through terminal services. We may need to go with a physical cisco IP phone at a person’s house. It adds cost per user, but it may be necessary. Also going with a hardware VPN solution is looking better than a software solution. If anything goes wrong with the remote computer we would be responsible to manage it. We cannot troubleshoot software issues like vpn over the phone, and we are not equipped to send someone onsite to fix problems like that.

A hardware VPN device that we can remote into and verify it is working, along with terminal services is looking like a good solution. it is not the cheapest but maybe the most feasible. We would require someone to have their own computer, and all they would need to put on it is terminal client. That takes 5 minutes to setup. Wire up a VPN router from cisco or linksys plug in a phone and they are up and running.

Now all I have to do is sell everyone on the solution and make sure it works from a technical standpoint.

, , , ,