This article is a list of the top 20 most common network configuration errors and how to avoid them. User configuration error is a common cause of network downtime and outages costing thousands of dollars per minute.
1. Duplex/Speed Mismatch
This is primarily something that occurs with Ethernet interfaces for connected neighbor switches. Cisco Gigabit interfaces default to autonegotiation of duplex and speed. The recommended best practice is to match autonegotiation configuration setting on connected interfaces of all devices. It is common to have problems when connecting to ISP equipment or non-Cisco devices.
This also applies to Cisco switches with older interface hardware. ISP equipment could implement hard-coded duplex and speed that does not work with autonegotiation. Some trial and error is often needed to identify what settings will work as a result. Symptoms include interface counters with a high number of collisions and slower network performance.
2. Switchport Trunk Allowed VLAN
There is a difference between allowing VLANS and adding VLANs that will affect trunk interfaces between switches. For example, switchport trunk allowed vlan command will ONLY allow VLANs specified with that command. You would have to include all existing VLANs currently permitted as well. This command is only recommended with the initial configuration of a trunk. All VLANs are permitted on a trunk interface by default. You would use switchport trunk allowed vlan add or remove command to add or remove VLANs after initial configuration.
3. Firewall Port Blocking
Stateful firewalls have a zero-trust security operational policy that denies all traffic unless explicitly allowed with a firewall rule. This is a common problem when no request is made to open firewall port/s or the wrong ports are specified. Host-based firewalls are managed by SysAdmins and they would open ports on Linux servers for example. The correct TCP/UDP or other port/s have to be opened for network connectivity.
4. MTU Mismatch
Most network traffic exists within data centers where Ethernet interfaces are prevalent. WAN traffic is also increasingly transported across Ethernet WAN and fiber services. Ethernet has a default global MTU size of 1500 bytes that is implemented on all LAN/WAN Ethernet interfaces. This is the maximum frame size that includes all headers and payload. MTU mismatch on an interface with lower MTU setting causes IP packet fragmentation that affects network performance.
Ethernet frames that arrive with a higher MTU must be fragmented and reassembled. It also causes packet loss when a network device does not permit fragmentation since packets are dropped. The most common problem within a data center is Ethernet jumbo frames that support MTU 9000 bytes. This causes extensive fragmentation and reassembly when they arrive at an interface with MTU 1500 bytes. Most of the MTU mismatch problems will occur with ISP connection, VPN tunnels, and MPLS network.
There are serial interfaces that implement smaller MTU than Ethernet based on the WAN protocol. ISP should notify you what MTU they will support for connectivity to their equipment. VPN tunnels add headers that reduce the MTU size supported and by extension amount of payload per packet. It is common to configure MTU 1400 on Cisco VPN tunnel to allow for IPsec and GRE headers. MPLS packets add labels that will reduce MTU size supported across the ISP network. PMTUD is usually enabled on all hosts to identify lowest MTU in forwarding path and modify MTU accordingly. This is done after manual configuration of any interface or tunnel with custom or non-default MTU size.
5. Spanning Tree Root Bridge Election
Cisco switches have a default configuration where the priority setting is 1. During spanning tree root bridge election, the switch with lowest MAC address is elected root bridge for all VLANs when priorities are equal. The problem is an access switch could be elected root bridge and cause havoc with network performance. It is even worse when the access switch is older and has outdated IOS software. The normal flow of traffic is from access layer to faster aggregation (distribution) switches within a switching domain. The primary root bridge for VLANs should be assigned to an aggregation switch for best results. Troubleshooting and applying network policies are also much easier with this layered configuration.
6. SSL/TLS Expired Certificate
The number of internet and private web servers that are based on HTTPS is currently around 90%. This also includes a variety of network management applications. You will notice with a Wireshark trace that TCP handshake connection works. The problem occurs with SSL/TLS handshake and certificate negotiation. The most common problem is an expired certificate. Consider the sheer number of certificates installed and setup management alerts to avoid application downtime.
7. BPDU Guard Missing
BPDU guard provides STP topology protection from errors such as plugging in an unauthorized access switch. The switch detects BPDUs sent from the unauthorized switch and transitions to err-disabled state when BPDU guard is enabled. This feature should only be configured on an access mode port where PortFast is also enabled. It is only hosts that are connected to access ports and they do not send BPDUs. The result of plugging in a switch could include broadcast storms and/or unwanted root bridge election. This will disrupt network traffic and cause an outage until resolved.
8. Native VLAN Mismatch
The purpose of native VLAN is to forward untagged management traffic across a trunk link. This essentially isolates and extends management traffic (CDP, DTP, LACP, STP, etc) between switches. There is also VLAN tagged data traffic sent across the trunk interface. Cisco assigns VLAN 1 by default to the native VLAN consistent with the management VLAN. The result of native VLAN mismatch between neighbors could cause a variety of network problems. For example frame drops, MAC address flooding, and security issues from VLAN hopping. You will notice that traffic is dropped from the specifically affected VLANs and network connectivity fails. This will trigger system error messages generated from STP and CDP that can detect native VLAN mismatches. Cisco recommends assigning a non-default native VLAN as well and verify they match between connected switches.
9. Wireless Co-Channel Interference
One of the most common causes of degraded wireless signal strength is co-channel interference. This occurs when neighbor access points are assigned to the same channel causing more cell contention and collisions. The result is higher latency and reduced throughput on what is already half-duplex transmission. This translates to really slow performance that particularly affects voice and video traffic. Wireless co-channel interference often affects the 2.4 GHz frequency band where there are only three non-overlapping channels. The most effective solution is enable wireless controller dynamic channel assignment. The other option is to reduce transmit power on access points and reposition to minimize overlap.
10. Wildcard Masks
Wildcard masks are configured with routing protocols such as OSPF and EIGRP to advertise specific subnets or range of subnets. They are also used with access control lists (ACL) for packet filtering. It is easy to make mistakes when inverting binary 1s and 0s of a standard subnet mask to calculate wildcard masks. Any errors would either prevent network connectivity or cause security lapses from incorrect filtering. The preferred method is to use a calculator or wildcard mask table. My video for CCNA students includes an easy method to calculate any wildcard mask in seconds.
11. Switched Virtual Interface (SVI)
The configuration of VLAN interfaces (SVIs) on switches enable Layer 3 connectivity across the network. The problem is VLAN interfaces are not operational (up/up) until the associated VLAN is created, assigned to a switch port, and has a cable connected. Traffic from that VLAN is not routable until the SVI is operational. Most of the time network engineers start with configuring an SVI and forget to create the VLAN and/or assign it to switch ports. The status of a VLAN interface is easy to verify with show ip interface brief command and show ip route command.
12. VLAN Missing From Trunk
Cisco switch interconnects can be complex to configure with multiple VLANs assigned to static or dynamic trunk interfaces. Etherchannels are also common on trunk ports and interface settings must match between physical member ports. Layer 2 VLANs must be explicitly added to a switch trunk interface unless it has a default configuration. This can be verified with show interfaces trunk command.
13. AAA New-Model
TACACS+ and RADIUS servers provide centralized management of security access to network devices. Anytime you issue aaa new-model command on a Cisco device, it will automatically delete login local command on VTY and console lines. This will cause lockout if you do not finish the AAA configuration and logout for some reason. The problem is worse if you are connected remotely and cannot access the device. There are instances as well where the connection drops or device hangs for some reason. Plan on some backdoor solution such as reload command for example and set for more than enough time to complete your configuration. Do not forget to cancel reload after testing and do not save configuration until testing is done. Another option is to open a second VTY line to the network device for backup purposes.
14. Authentication Key Mismatch
The purpose of authentication keys or pre-shared keys is to validate the source of packets. Network protocols such as VPN, WPA2-PSK, NTP, OSPF, EIGRP, AAA, and SNMP support authentication keys. Based on the protocol, pre-shared keys either authenticate a network server or a device. You will get an authentication error when there is a key mismatch between network nodes and this will disable network connectivity. For example, OSPF neighbor adjacency will fail along with routing, or VPN tunnel won’t be operational. OSPF authentication keys must match between directly connected neighbors only. By contrast, all network devices must be configured with the same pre-shared key as AAA server or SNMP server.
15. Layer 3 Interface Shutdown
Cisco default is to shutdown all Layer interfaces until they are explicitly configured and enabled. This is a repeated problem when network engineers forget to issue no shut command on an interface. In fact, when configuring multiple interfaces with a new deployment you can waste a lot of time troubleshooting. It could initially seem like a routing problem or firewall issue for example.
16. Debug All
Cisco engineers often use debug commands when testing or troubleshooting more difficult network problems. There are many debug commands available that are used to analyze traffic between network devices. It is helpful as well to collect debug output and sent to Cisco TAC for analysis. The recommendation is to only enable one debug command at a time. CPU load on the control plane is significant and using multiple debug commands could crash your network. This applies when you are troubleshooting off-hours as well.
17. Access Control Lists (ACL)
The prevalence of access control lists (ACL) affect packet filtering and by extension network connectivity. It is easy to make mistakes with port numbers and other syntax when configuring ACLs. The result is unwanted packet drops or security lapses from allowed traffic. The order of ACL statements is also as important as command syntax. You could use an open port testing tool or Cisco ACL checker to identify errors. Cisco ASA firewall packet trace is well recommended for testing firewall rules since they are ACLs as well.
18. Local Account Security Privilege Level
Cisco default settings are often unknown considering the number of protocols available. The local account is configured for device security access and as failover when AAA is enabled. Cisco assigns user exec mode security access to all local accounts unless explicitly configured. It is common for example to create username admin account with privilege level 15 and assign a password. There is no access to enable mode with user exec mode (privilege level 1) unless an enable password is configured. If you do not know the enable password or none is configured then only some basic show commands are available. The show running-config command is also not available.
19. ISP Stripping
ISP stripping occurs when QoS is dropped from packets traversing the ISP network. Sometimes your ISP will also apply rate limiting and/or throttling to network traffic. The result is slower network performance or dropped packets across what could be a congested network that you do not manage. Isolate your troubleshooting to the ISP network only and analyze how inbound and outbound packets are affected.
20. Static Routes
Route selection is based on the dynamic and static routes installed in a routing table. Since static route have the lowest administrative distance, they are preferred over any dynamic route or default route. Configuring excessive static routes can lead to problems since dynamic routing can learn best and alternate routes to a destination. Static routes could also cause suboptimal routing since they take precedence. They could forward packets to a destination that no longer exists called blackhole routing or cause routing loops. it is a common mistake not to delete an older static route when configuring or fixing a static route. Cisco does not automatically delete static routes to the same destination.