I’ve recently encountered an issue on CUE as it pertains to user mailboxes. Whenever someone tried to leave a message for a particular user, they were told by the AutoAttendant that there was no mailbox associated with the user’s extension. Before we go any further, let’s get some more information on how the environment is currently setup.
CME is configured to use SIP Trunking back to the ISP in order to make calls. Each user has Direct Inward Dial (DID) extensions. Here’s a sample config of what the Ephone-DN on CME and the Voicemailbox on CUE looks like.
CME Ephone-DN config for Userephone-dn 1 dual-linenumber 7890 secondary 1234567890 no-reg bothlabel Johndescription 123-456-7890name John Doecall-forward busy 1000call-forward noan 1000 timeout 20
CUE config for Userusername John create
username John phonenumber “7890”
voicemail mailbox owner “John” size 1772description “John mailbox”end mailbox
voicemail notification owner John enable
Reviewing the configuration above you would think that your voicemail mailbox configuration for this user is correct, right? Wrong! If you placed a call to this user from an internal phone to their primary extension of 1234, you’d be able to leave a message however, if an external user placed a call to the DID number of 1234567890, they would be told that there is no mailbox associated with the number. The reason for this, is that when an external user places a call to the DID number, the AutoAttendant forwards the entire DID number and tries to find a mailbox associated with this number. Because the primary number associated with the mailbox is 7890 and not 1234567890, the mailbox lookup fails. To correct this issue we would need to change the phone number associated with the mailbox in CUE to the DID number of 1234567890.
username John phonenumber “1234567890”
I was able to verify this by enabling call forwarding to voicemail on the user’s phone, enabling the “debug voice dialpeer” command and then placing a call to that user’s DID number.
Checkpoint provides three methods for backing and restoring the operating system and networking parameters.
- Snapshot and Revert – Snapshots can only be performed on Splat and backs up everything including the OS drivers; can be used to backup both gateway and management server. File sizes for these backups are usually very large and can only be restored to devices having the EXACT OS, Checkpoint version of Splat and patch level. Command used to perform a snapshot is snapshot_ and must be run from expert mode. By default the snapshot file is stored in the /var/CPsnapshot/snapshots directory. To perform a restore, issue the revert command from expert mode.
- Backup and Restore – The Backup utility is only available on Splat and backups up your firewall configuration as well as networking parameters such as routing. The file size is usually smaller than that of a snapshot because it doesn’t contain any drivers. Can be used to restore to a machine having the same OS, Checkpoint version and patch level. Backups are performed using the backup command; the default location is /var/CPbackup/backups. On UTM-1 and Power-1 appliacnes the default location is /var/log/CPbackup/backups. Restoring is done by issuing the restore command from export mode. Backups are generally performed via the WebUI however restores must be done via the CLI.
- Upgrade_export/Export – Upgrade tools backs up all configuration independent of hardware, OS and Checkpoint version. Migrate utility is used for uprades/migration of database information and can’t be used when downgrading to an earlier version of Checkpoint. File size usually depends on the size of your Policy. Usually this can be done on a live system provided that the CPU isn’t overloaded. Can be run on Splat, Linux and Windows. Upgrade tools on R75 can be found at $FWDIR/bin/upgrade_tools.
Saving Interface and Routing Information
- Windows: netstat -rm > routes.txt – saves route information to text file.
- Windows: ipconfig -a > ipconfig.txt – saves interface information to tex file.
- Splat: ifconfig > ifconfig.txt – saves inferface information to text file.
- Splat: copy /etc/sysconfig/network.C <location>– copies files containing route information to a location defined.
Always upgrade the Security Management Server first before the Gateways.
Migration steps for SMS
- Prepare source machine for export by performing a migrate export which creates a backup of all configurations. Once this is completed, export the file using SCP on Splat or by copying it from its directory on Windows.
- Perform clean install on new server
- Import the database on the new server using the migrate import command.
- Test to make sure everything works before putting into production.
As most of you may already know by now, I recently relocated to the beautiful Island of Bermuda and started a new gig. Part of my job role will be to deploy and support Checkpoint firewalls. This means that I would need to get up to speed on how these firewalls work pretty quickly. Given this, I figured it would be a good idea to blog about my experiences with these devices as a way to both understand and store my thoughts for future references.
Building a home lab for my studies turned out to be a whole lot easier than I had first anticipated. Everything I needed to be able to study for the exams and also practice deploying the devices could be virtualized. Below you can find my home lab blue print.
- Lenovo ThinkPad with 8GB of RAM and lots of disk space.
- VMware Workstation 7
- Two Windows Server 2003 VMs
- One Windows 7 (host install on my laptop)
- Two Security Gateways
- One Security Management Server
- Checkpoint SecurePlatform R75.20
One of the 2K3 VMs was configured with the RRAS role to act as a router while the other was used just as a general client sitting behind the firewall. Any client OS such as Windows XP/Vista/7 can work but I was too lazy to install another OS, so I copied the VM for the server 2K3 🙂 . I also used my windows 7 host OS as a client sitting behind one of the gateways; this way I was able to do ping tests to the remote site when testing my VPN configurations.
As you can see, it’s a very basic setup, but should allow me to test most of the stuff relevant to the exams.
Being in Networking and dealing with Cisco gears on a daily basis requires the use of a terminal application in order to configure and manage your devices. I’ve used both SecureCRT and Putty but I find myself using Putty a whole lot more. It’s a small application that doesn’t even require an install before you can use it. You simply download and run it. It’s also freeware which is an added bonus.
Due to the number of problems I’ve had with Windows and my laptop, I found myself having to reload quite a number of times. Each time resulting in me losing all my device profiles I had configured and saved in Putty. Because of how the program is written there is no configuration file(that I know of) that I can save and then import whenever I do a new install. This totally sucked, as I was able to backup my configs using SecureCRT. Doing some research led me to this article that showed exactly how to export/import your configurations in Putty. To avoid me having to google around again for such an article just in case I need it in the future, I’ve decided to simply make a copy of the steps required here for my convenience. All credit goes to the author of the article mentioned above for writing such a detailed post on the how to accomplish this.
- Open Regedit.
- Look for “SimonTatham” by using the find option (ctrl f).
- Export the reg file by clicking Export from the File Menu.
- Choose the path to store the registry key and hit enter.
- To import the settings in a new Desktop simply copy the registry key to that system and double click to apply the registry setting. At this stage you simply download Putty on the new system and run it, you will notice that all your devices are already populated in the list.
This will definitely benefit me in the future and hopefully those of you who aren’t yet aware of how to backup your configs.
Over the past couple of weeks I’ve been experiencing a strange problem with one of our client’s network. They payed for a 5Mpbs connection but had the ISP opened up the pipe to 100Mbps temporarily to their DR site to allow for the initial replication of servers to move a lot faster. The problem was that we weren’t getting anything close to 100Mbps whenever we did speedtests. Results averaged in the 40Mbps range and sometimes it would even get as low as 18Mbps.
Before I go any further, let me give you a brief overview of the current design. The client has their own /24 block and ASN. The same ISP that is providing their primary internet at HQ is also providing internet at the DR site along with the lease line that connects them. eBGP relationships have been established to the ISP at each site along with an iBGP peering between the HQ and the DR. They’re also using EIGRP as their IGP.
After the above configurations were completed I performed some initial tests. We were receiving a partial table at the DR site while only getting a default route at HQ. To prevent the upstream ISP sending the full table unexpectedly I configured a prefix list at both sites to only allow a default route in. Further tests were performed to ensure IGP was routing properly and that both sites were learning routes from each other. The issue arose when we were ready to test replication of the servers at HQ to the DR site. The ISP informed us that they had opened up the pipe to 100Mbps and that we could begin replication. However, conducting a number of bandwidth tests on speedtest didn’t concur with their confirmation. Speeds were fluctuating greatly but didn’t get anywhere close to 100Mbps. This was where the troubleshooting process began and what highlighted my BGP design #FAIL.
Now there could be any number of reasons why I wasn’t getting the desired results. So how do you go about troubleshooting such an issue? A carefully planned process and my good old buddy google. That’s how! The next step I did after testing on my laptop was to test on some other desktops and servers in the network. This would allow me to eliminate my POS laptop that freezes a zillion times a day out of the equation. The tests performed on the servers yielded the same results. Ok, so now we’re moving on to the next set of tests to perform. Before performing anymore tests internally I decided it would be more productive to test from the ISP’s fibre switch and walk my way into the network. It really didn’t make any sense performing a bunch of tests on the internal network when the problem might be with the ISP’s switch, the border router or even the PIX(shoot me now) that are all inline before hitting the internal network. The link comes in from the ISP via fibre, into their switch and then connects to the border router via ethernet. The border router then connects to the PIX. To ensure I was getting the desired bandwidth from the ISP I connected my laptop the to the fibre switch, placed the IP address used to peer up and then perform tests. Results were good. I was getting 98Mbps on average. Ok, so this proves that we are indeed getting the full bandwidth. The next step was to connect back the fibre switch to the border router then connect the router to my laptop. The results of these tests weren’t promising. So now I’m baffled. Could it be the border router? It can’t possible be! It’s a brand new 2901 ISR generation2 and as you know, the new ISRs are built to handle a lot more bandwidth. Time to hit up my friend google. I found another user with the exact same model router with the exact same design using a fibre switch from their upstream ISP, who was having the exact same problem. A lot of users were saying it could be a duplex/speed issue with the ports however this wasn’t the case for me as both the speed and duplex negotiated correctly. I saw @jastorino on skype around the same time I was researching the issue so decided to pitch my problem to him see what input he had. One idea that came up was that it could be possible that the ISP was doing rate limiting via sourced traffic. This made sense as when I connected directly to the ISPs switch using their IP I got full bandwidth, but when I connected to the router using IPs from the client’s block we didn’t. They denied that they were rate limiting as I was suggesting. At this point I knew I must be on to something. Why would I get full bandwidth when traffic was sourced from the ISPs IP but didn’t when it was sourced from the client’s IP. Something’s not right. Moving on, I checked to see how the client’s block was being seen as reachable from a couple of BGP looking glass. Performing a traceroute to an IP at the HQ revealed something quite interesting. The path traffic was taking to reach HQ was through the DR site. This meant that traffic was not taking the same path to return into the HQ network as the path used by exit traffic. This lead to a loud outburst of ooooooooooooooooooooh while at my desk :). Everyone around me wanted to know what happened. So now I’ve found the problem! The internet facing link at the DR site wasn’t opened to 100Mbps while the lease line was. Return traffic was actually being limited by the DR’s internet link. How do I fix it? My first thought was to split the /24 block into two /25 then reconfigure BGP to advertise those blocks to the upstream. This would allow return traffic to take the same path a the exit traffic depending on which block it was coming from. Unfortunately because ISPs wouldn’t advertise any prefix smaller than a /24 this was no longer an option. Using AS prepending was the next fix for this scenario. Prepending the client’s AS 3 times at the DR site made the AS path appear much longer than that of the HQ and because of BGP’s path selection process, return traffic would choose the HQ site.
I know this probably isn’t the best solution but it worked just fine for me. I’m by no means an expert at BGP. However, working on this project exposed me to a lot of good factors about the protocol that I will be sure to take into consideration during future designs.
Today’s task was to deploy a couple of 7962 Phones along with some 7915 Expansion Modules. This shouldn’t be too bad. All I need to do is head over to the client with the gear and configure their CME to accept the new phones and modules. How hard could this be? This should take me probably 2 hours max! Heh! Don’t know who I was kidding. By now I should be well aware that nothing goes as planned in networking, more so in Voice :-).
So I’m over at the client and begin by unpacking the phones, connect them up to the switch and begin my CME configurations. Ephones created, phone registers just fine and hey! I was even able to successfully place a test call. Things are progressing as planned. Next up is to test the expansion modules. Connected up the 7915 modules but for some reason they wouldn’t power up. Did a quick search on google and turns out that these modules need power supplies if the phones are power via POE which was the case here. If the phones are powered via power supply, then you don’t need a separate power supply for the first module. However, for the second module you most definitely require a power supply. Good points to know. So after heading back to the office to acquire a power supply, I was able to get the modules powered up. Now they powered up just fine but didn’t display the lines they were configured for. They just sat there stuck with the amber lights on. Time to hit up my buddy google again. He pointed me to some really cool forums which informed me that the firmware on my phone needs to be at 8.4( they’re currently 8.3 at this stage). Five minutes later and I have ver 9.2 downloaded and copied to the UC560. Rebooted the phones so that they can begin the upgrade process only to be hit with more good fortune. For whatever reason these phones weren’t upgrading. The process kept failing. Eventually I think I tried so many different stuff that the phone would not complete the boot process anymore and load it’s configuration. It would just stay stuck at “Upgrading” and then reboot, going through this cycle repeatedly.
At this point I figured da hell with it! Reset this baby to factory default and start over. If you head over to Cisco’s site for instructions on how to restore this model phone to factory default, they state this. After ensuring that I had everything in place as advised, I proceeded with the reset sequence. This proved to be unsuccessful. After entering 123456789*0# the phone simply went back to the state of “Upgrading”. My frustration level at this point was of course reaching max. Ok, so time to turn on some debugs and go from there. Enabling debugs for tftp enabled me to see what files the phone was looking for. First file I noticed it was not finding was “term62.default.loads”. Rechecking my tftp-server configurations revealed that I had the alias configured incorrectly, leaving out the .loads which prevented the phone from finding the file. I corrected this and proceeded with the factory reset procedure once again only to be greeted with the same error. This time the file that couldn’t be located was “jar42sccp.8-3-3ES2.sbn”. This one really puzzled me as the tftp configurations for this file was correct! This was where the engineer in me came to life! Comparing the firmware files I noticed that the very first file the phone looks for “term62.default.loads” was a lot smaller than the other files. It was 1KB to be exact, when compare to others that were 2439KB. This lead me to believe that this file probably just contained instructions. Opening the file with notepad revealed some gibberish followed by an orderly list of files. The files in this list were the same firmware files but in a particular order. Comparing the order the files were listed in to the order I had my tftp-server statements showed that my statements were in a different order from that of the list. This got me to thinking that maybe I should reorder my statements to the same order that was in the file. This indeed proved to be the problem because as soon as I reconfigured my tftp-server statements the phone successfully restored to factory default. Ok, so I’m some what relieved at this stage. I got the phones back to a working state however, the 7915 module still doesn’t work as we’re back to the 8.3 firmware. I proceeded to use the same logic used to perform the factory restore and retried upgrading the firmware with ver 9.2. This too was successful and the module booted up and dispalyed it’s configuration correctly.
I highly doubt this is how this procedure is supposed to work, though. What happens if your tftp server isn’t the CME device and it’s actually a server? How does it know the order? Maybe this was just a bug in my case as I haven’t seen any other reports of this while researching the issue. In anycase, it was a learning experience, one that will surely help me in the future should I encounter this problem again.
I’m currently working on a VOIP project consisting of a pair of UC560s and some SPA508G Phones. Configuration for the project is pretty straight forward as no fancy or complex translation rules need to be created. The client is basically replacing their current antique Nortel system with a modern Cisco solution.
The steps required for me to get the system up to a basic state to perform tests were as follows:
- Configure a DHCP Pool for all IP Phones.
- Copy tftp files for the SPA508G phones to flash.
- Configured UC560 to serve up the files so that the phones are able to download their configuration.
- Basic ephone-dn and ephone configurations to test phones.
- Create and assign IP to loopback interface.
- Configure Service-Engine interface to use loopback interface
- Configure IP address and default-gateway of CUE
- Test connectivity to CUE by pinging interface and also establishing connection.
- Configure user accounts in CUE and assign Voicemail mailboxes to users.
- Perform Tests.
As you can see from the list, this truly is a simple deploy.
All configurations were proceeding just fine until I arrived at step 8. At this stage I should be able to both establish a connection to CUE and also receive replies from pings. However this wasn’t so. For some reason all my ping tests were failing. I checked, rechecked, checked again, then checked again to make sure my configs were correct.
Here’s an example of what the configuration would look like.
!Loopback interface created to use with our Service Engine interface. (Cisco's recommended way of configuring it) interface Loopback0 ip address 192.168.1.1 255.255.255.0 !Service Engine interface used to communicate with the Cisco Unity Express. interface Integrated-Service-Engine0/0 description Loopback interface used to manage integrated application module ip unnumbered Loopback0 service-module ip address 192.168.1.2 255.255.255.0 service-module ip default-gateway 192.168.1.1
Did I miss anything? From the configuration we see that the IP Address of the CUE is 192.168.1.2 and the IP address used to communicate with the CUE is 192.168.1.1 which is the IP of the Loopback interface.
For reasons unknowing to me, this configuration was not working. I was able to successfully establish a session to the CUE, logged in using the same credentials used to login to the UC560 and I was able to proceed with further configurations tasks. I was able to create user accounts, assign them mailboxes, assign Voicemail number so that the application is activated whenever that number is dialed etc. Unfortunately ping tests were still failing and as such when I made a test call to Voicemail, it did not work. There is one step I forgot to include in the list above, and that was to create a dial-peer so that whenever users dialed the Voicemail number it would use that particular dial-peer to transfer the call to the correct destination thereby activating the Voicemail system. This was also completed and the configuration for this can be seen below.
dial-peer voice 2 voip destination-pattern 600 session protocol sipv2 session target ipv4:192.168.1.2 dtmf-relay sip-notify codec g711ulaw no vad
I spent hours researching Cisco’s documentation and reading through different forums. Eventually I saw a user on IPexpert’s OSL was having the same issue. He was able to access and ping CUE whenever he used a physical interface or a SVI as opposed to using a Loopback interface. I decided to test this out for myself by using an SVI, Vlan1 and changed the configuration on the Service Engine to use Vlan1 instead of Loopback0. Of course this worked flawlessly! I was able to ping the CUE IP which enabled me to proceed with my further configuration and testing.
I found the whole thing really interesting, as no where on Cisco’s site mentioned there being instances where the Loopback interface wouldn’t work. Have any of you VOICE folks out there experienced this problem before? Did I miss something with my configs? And please don’t tell me that I needed to reboot/reload/reset CUE as I did this so many times till at one point I had to do a complete factory reset to get basic communication again. 🙂