Tuesday 21 May 2013

testing beyond your Cisco ASA

I needed to change some NAT's and DNS entries. The DNS end of things was going to be handled by a 3rd party. The network looked like this:

120.180.240.224 /28
120.180.240.224
Network ID (unusable)
120.180.240.225
HSRP IP Address
120.180.240.226

120.180.240.227
firewall primary outside interface
120.180.240.228
firewall standby outside interface
120.180.240.229
NAT to internal device
120.180.240.230
NAT to a BI Server
120.180.240.231
NAT to a test app server
120.180.240.232
NAT to production app server (www.cust.com)
120.180.240.233
NAT to standby app server
120.180.240.234

120.180.240.235

120.180.240.236

120.180.240.237
Router 2 IP Address 
120.180.240.238
Router 1 IP Address
120.180.240.239
Broadcast address (unusable)

I copied the NAT entries and replaced them with the new public IPs. I copied the existing access-list, replaced the old IPs with the new IPs and applied the new ACL to the outside interface. I did a clear xlate. I assumed that everything was correct but it was not working. I couldn't browse to www.cust.com


Here is a list of steps that were used to resolve the issue:
Confirm the service is up by testing locally on the app servers with localhost or the private IP.
Confirm the public and private IPs are correct.
Look at the NAT entries again "sh run | i static".
Look at the current translations and arp entries "sh xlate" and "sh arp"
Run a packet-tracer command, make sure your NAT and ACL are being hit as expected.
Check your ACL. When I checked there were no hit counts on it the ACL, not 1. However the packet-tracer said everything should work.
Ensure an ACL allows ICMP to all public IPs. Test it with packet-tracer.
Create a packet capture to capture all incoming ICMP traffic.
Create a script to ping all of the public IPs. You should see the traffic coming in on all of them.
For me traffic was only appearing on the firewalls interfaces.
The traffic wasn't making as far as the firewall. 
I assumed wrongly that something else (another firewall) was blocking it.
I contacted the 3rd party. They tried to ping the new public IPs from one of the routers, there was no response and no arp entry.
The issue was on my firewall. For some reason it was not responding to the arp.
The 3rd party was kind enough to put in static routes to the new IPs and everything started working. Except for the BI server which was running on a non standard port. This port did have to be unblocked on a 3rd party firewall.
Next attempt is to reboot the Cisco ASA.
If that fails upgrade.
If that fails leave the temp fix in place. Call Cisco support.







No comments:

Post a Comment