vpn filter ACLs not working as expected

Had issues with VPN filter ACLs

Had some thing like

permit ip host x.x.x.x any

This should have allowed my traffic but it was not working

I needed to change this to which was essentially the same but it worked. It seems you need to make rules for the vpn filter ACLs in this way

permit ip 192.168.1.0 255.255.255.0 10.150.200.0 255.255.255.0

mtu issues in wireshark

Small packets are working like ping

TCP 3 way handshake is setting up

However when we move into setting TLS / https connection its failing. This can be the MTU is too big for the MTU on the path. The TCP window will scale up as the connection goes on.

Things to try:

tracert -d x.x.x.x

See what network devices you pass through and check the MTU on those. (if you can)

On the client:

ping –l 1490 -f 8.8.8.8

Reduce 1490 by 20 until you find a working value where the pings respond.

Remember the value when the ping command is working isn't the MTU but ICMP payload. This confuses a lot of people. MTU 1500 == ICMP payload 1472 (20 Bytes for IP and 8 Bytes for ICMP, I think). You can also check the MTU's on the path

PathMTU discovery is not reliable, will investigate MSS clamping, which can be enabled on your firewall / VPN endpoints.

CMD to check the MTU in windows

netsh int ipv4 show subinterface

CMD to set MTU in windows

netsh interface ipv4 set subinterface “Local Area Connection” mtu=1458 store=persistent

ASA should set MTU 1380 by default to account for IPsec tunnels

Palo needs to be configured

https://www.cisco.com/c/en/us/support/docs/security/asa-5500-x-series-next-generation-firewalls/113393-asa-troubleshoot-throughput-00.html#anc14

upgrade cisco firepower FMC and FTD

Cisco docs:

https://www.cisco.com/c/en/us/td/docs/security/firepower/upgrade/fpmc-upgrade-guide/upgrade_firepower_threat_defense.html#id_64765

https://software.cisco.com/download/home/286291275

https://software.cisco.com/download/home/286306503/type/286306337/release/6.7.0

Old method was sensor first then FMC. 

Figure our your upgrade path (can save a lot of time):

https://www.cisco.com/c/en/us/td/docs/security/secure-firewall/upgrade/management-center/740/upgrade-management-center-741/planning.html#r_ftd-upgrade-path

7.4 (max version for 2140 hardware, will still get sec updates until EOL)

6.4 can jump to 7.0

6.6 can jump to 7.2  (user id issues + DH groups on S2S VPNS, PBR is merged in FMC)

7.0 can jump to 7.4

7.1 can jump to 7.6

7.2 can jump to 7.7

step 0

check pre-reqs and docs

Some times you may need to clear space before doing upgrades:

df -h

delete old file, leave the 2 most recent

/var/sf/SRU

/var/sf/sru

cd /var/common/

/usr/loacl/sf

New method is FMC, deploy, sensor, deploy.

Step1
Update your VDB and geolocation to the latest.
Run a backup for both FMC and FTD (tick box to bring back to FMC)

Download those backup files

You can export the policy from policies > import/export (top right)

Good idea to also take a dump of running config from FTD's (pri and sec)

Especially if you have a stand alone FTD

You can also take a snapshot before FMC upgrade but Cisco seem to flip/flop on weather this is supported. If you take snapshot make sure to merge/delete after upgrade is completed. Don't want a snapshot growing, it will cause an issue later.


Since 6.2 you need to upgrade FMC first, then sensor.

To upgrade from the web GUI the FMC sh upgrade file can be downloaded here
Downloads Home > Products > Security > Firewalls > Firewall Management > Firepower Management Center > Virtual Appliance > FireSIGHT System Software-6.2.0

See here

Network sensor sh files available here
Downloads Home > Products > Security > Firewalls > Next-Generation Firewalls (NGFW) > ASA 5500-X with FirePOWER Services > ASA 5525-X with FirePOWER Services > FirePOWER Services Software for ASA-6.2.0

see here


In later versions you can run a readiness check first. Most logs are found in /var/log/sf

From version 6.3.0 you can upgrade direct to major versions
Lets say we are on 6.2.1 and  want to go to 6.4.0.2
We can upgrade directly to 6.4.0 and then up to 6.4.0.2
Remember you need to deploy after each install.

Other commands:

DBCheck.pl

attempt to resume an upgrade

install_update.pl --detach --resume /var/sf/updates/Cisco_Secure_FW_Mgmt_Center_Upgrade-7.2.10

update roll back

/var/log/sf/Cisco_Secure_FW_Mgmt_Center_Upgrade-7.2.10/upgrade_roll back

Dealing with flex config

When flex config is brought into FMC it can cause issues

One example I had some policy based routing (PBRs)

After upgrade it wasn't working even thought the code was there in ASA CLI.

I had to delete the flex config off FMC

Deploy

Config the PBR via the FMC

Deploy

Check the CLI so that the PBR matched what I had before in ASA CLI

This was done by doing a win merge on the config before and after so its crucial to have that show run backup for fixing flex config issues.

All was working.

The process will work the same for other flex config.

setting http source interface on cisco switch

This command was needed to copy files from a http server to a switch

ip http client source-interface G0/0

Similar to ip tftp source-interface

http transfers should be faster and more reliable for large files

linux tools

Shells to consider

Bash (on basically everything)

zsh (auto complete etc)

fish (friendly interactive shell)

DHCP scope full of BAD_ADDRESS Entries

This one comes up a lot. I've come across the following causes:

1 - Malware just filling up your DHCP scope to break it. 

2 - Rouge DHCP server

3 - Broken NIC on some device, never completes the DHCP steps and keeps taking IP's (some times fixed with driver update)

3 - Proxy arp issue. Similar to above. The client gets DHCP offer but before it accepts it will arp the IP to see if anyone is using it. The firewall responds to the arp so the client rejects the IP and starts DHCP request again multiple clients fill up the DHCP rules. Check NAT rules and make sure they have "no-proxy-arp" even if the NAT rule matches different traffic. Look for any any nat rules.

You can look in DHCP logs on the DHCP server

Check arp -a and wireshark looking for dhcp request/offers on the DHCP server.

If the DHCP is being forwarded by an IP helper you'll need to take the capture on the LAN and can check arp on the device that is doing the IP helper

If its just one mac address a quick work around is to make a reservation for that MAC so it always gets the same IP. That way it won't fill up your DHCP scope. Then clear out the DHCP scope.

You also have the option of setting the DHCP lease times down lower. You can get a script/scheduled task to stop/start the DHCP service (clearing down the scope). If you have scripting skills you can look for "BAD_IPADDRESS" entries and delete those with a script.

The longer/better fix is to find the device(s) that are causing the issue and sort them out. 

You may also have the option to enable DHCP snooping on your switch. You can configure the switch ports to your DHCP servers so they are the only ones allow to respond to DHCP requests.

https://www.petenetlive.com/KB/Article/0001651

Internet browsing and webpages loading slow for no reason

I have hit an issue where internet browsing is slow but when looking at wireshark the round trip time on TCP 3WHS is always good. Not seeing delays in packets. Network graphs show the connection is not being maxed out. Network speed looks good yet the users report that page load times are slow.

I had to allow google and facebook tracking IP block/ranges

The traffic I saw was UDP 443

Google URLS

admin.google.com

ssl.google-analytics.com

*.google.com

www.google-analytics.com

www.google.com

code.google.com

update.googleapis.com

talk.google.com

*.gstatic.com

clientservices.googleapis.com

www.googleapis.com

accounts.google.com

clients4.google.com

*.googleapis.com

oauthaccountmanager.googleapis.com

clients2.googleusercontent.com

update.googleapis.com

Have also seen windows updates using up all the bandwidth so windows updates can be blocked during business hours or controlled with WSUS etc.

MS URLs

login.microsoftonline.com

aadcdn.msauth.net

*.microsoft.com

autologon.microsoftazuread-sso.com

IP's

FB ranges

31.13.64.0/18

66.220.144.0/20

69.171.224.0/19

69.63.176.0/20

Google Ranges

74.125.0.0/16

172.253.0.0/16

209.85.128.0/17

216.58.192.0/19

172.217.0.0/16

64.233.160.0/19

108.177.0.0/17

66.102.0.0/20

173.194.0.0/16