Friday, January 27, 2017

Layer 2 loop detection configuration and script on EXOS

ELRP configuration
The Extreme Loop Recovery Protocol (ELRP) is used to detect network loops in a Layer 2 network. A switch running ELRP transmits multicast packets with a special MAC destination address out of some or all of the ports belonging to a VLAN. All of the other switches in the network treat this packet as a regular, multicast packet and flood it to all of the ports belonging to the VLAN. When the packets transmitted by a switch are received back by that switch, this indicates a loop in the Layer 2 network.
After a loop is detected through ELRP, different actions can be taken such as blocking certain ports to prevent loop or logging a message to system log. The action taken is largely dependent on the protocol using ELRP to detect loops in the network.

Configuring standalone ELRP

The ELRP client (standalone ELRP) must be enabled globally in order for it to work on any VLAN. To globally enable the ELRP client, use the following command:
enable elrp-client
The ELRP client can be disabled globally so that none of the ELRP VLAN configurations take effect. To globally disable the ELRP client, use the following command:
disable elrp-client

Configuring Periodic Requests

To start periodic ELRP packet transmission on specified ports of a VLAN using a particular interval, use the following command:
configure elrp-client periodic <vlan_name> ports [<ports> | all] interval <sec> [log |
log-and-trap | trap] {disable-port {{duration <seconds>} | permanent}
This command starts periodic ELRP packet transmission on the specified ports of the VLAN using a specified interval. If any of these transmitted packets is returned, indicating loopback detection, the ELRP client performs the configured action of logging a message in the system log file and/or sending a trap to the SNMP manager.
When the option of disabling a port is configured, you choose the duration, in seconds, as a hold time or you disable the port permanently. When ELRP disables the port, the operation is not persistent. When the switch is rebooted, the port is enabled when the switch comes up.
NOTE.
ELRP detects loops on a per VLAN basis. When the disable port option is selected, keep in mind that the entire port will be disabled. This may affect connectivity for other VLANs configured on that port that did not have any data loop problems. ELRP also does not distinguish between uplink ports and host ports. When the disable port option is selected and ELRP detects a loop, any and all ports where the loop was detected will be disabled, including uplink ports.

Configuring Exclude Port List

To configure an ELRP exclude port list, use the following command:
configure elrp-client disable-ports [exclude | include] [ <ports> | eaps-ring-ports]
To disable an ELRP exclude port list, use the following command:
unconfigure elrp-client disable-ports

Technical TIP –  Troubleshooting Procedure for loop

1. Isolate the problem segment.

Isolate the problem segment. Due to the nature of a network loops, it should ONLY affect a single VLAN (single broadcast domain). Therefore, first try to identify which VLAN is impacted
“Show L2stats” (L2 VLANS)
  Which VLAN has the highest packets to the CPU?
  Which VLAN has the highest IGMP snooped or switched?

2. check physical port utilization.

If unable to identify storm impact by VLAN, then check physical port utilization. Pay close attention to the ports with the highest bandwidth utilization (up to 100%) and the traffic direction (Tx – transmit, Rx - receive).  
“Show port utilization”
  • Which ports have the highest % utilization bandwidth?
  • “Clear counters” (repeat) then which ports have the highest packets/sec.

3. Check packet count of the port.

“Show port statistics” – (Tx and Rx counts, broadcast vs. multicast)
 
Narrow down the source port/s:
  • Which ports have the highest packet counts?
  • “Clear counters” to see rate of increase.
  • Are broadcast or multicast packet/byte counts higher?
  • Back trace all ports which are receiving 100% traffic utilization.

Useful script for loop detection.

Gathering the statistics by script.

By using following script, data required for analysis (IPSTAT/PORT STAT/CONGESTION/L2 STAT/STP etc ) can be gathered automatically.
Ø  How to run.
1.       Save following script as data.xsf and save it on tftp sever and download to switch.
2.       save the log of telnet on your PC.
3.       Run following command on the switch.
* ds2.4 #  tftp get 10.120.40.45 vr "VR-Mgmt" data.xsf data.xsf
* ds2.8 # load script data
4.       Stop the script with Ctrl+C button if you want to abort.
[ data.xsf ]
disable clipaging
#### wtime variable determines sleeping time between two successive commands to
#### get difference value.  It's based on mili-seconds
set var wtime 2000
set var cli.out 0
set var msg "GET IPSTATS"
set var msg1 "=================== 1st Run =================="
set var msg2 "=================== 2nd Run =================="
set var smsg "SLEEP $wtime mili-seconds"
set var msg "GET IPSTATS"
show var msg
show var msg1
show ipstats
show var CLI.out
show var smsg
set var v $TCL(after $wtime)
show var msg2
show ipstats
show var CLI.out
set var msg "GET L2STATS"
show var msg
show var msg1
show l2stat
show var CLI.out
show var smsg
set var v $TCL(after $wtime)
show var msg2
show l2stat
show var CLI.out
set var msg "GET PORT STATISTICS"
show var msg
show var msg1
show port statistics no-refresh
show var CLI.out
show var smsg
set var v $TCL(after $wtime)
show var msg2
show port statistics no-refresh
show var CLI.out
set var msg "GET PORT CONGESTION"
show var msg
show var msg1
show port congestion no-refresh
show var CLI.out
show var smsg
set var v $TCL(after $wtime)
show var msg2
show port congestion no-refresh
show var CLI.out
set var msg "GET STPD INFORMATION"
show var msg
show stpd detail
show var CLI.out
show stpd
set var sv $TCL(split ${cli.out} "\n")
set var e $TCL(lsearch $sv *(C)*)
set var i 8
set var e ($e - 3)
while ($i < $e) do
  set var cli.out 0
  set var v $TCL(lindex $sv $i)
  set var vn $TCL(lindex $v 0)
  set var msgs "show stpd $vn port"
  show var msgs
  show stpd $vn port
  show var cli.out
  set var i ($i + 1)
endwhile
set var msg "IPARP STATISTICS"
show var msg
show var msg1
debug fdb show iparp statistics
show var CLI.out
show var smsg
set var v $TCL(after $wtime)
show var msg2
debug fdb show iparp statistics
show var CLI.out
set var msg "GET VRRP Information"
show var msg
show vrrp
show var CLI.out
set var msg "GET CPU Utilization"
show var msg
show cpu-monitoring
show var CLI.out
set var msg "GET DEBUG HAL CONGESTION"
show var msg
show var msg1
debug hal show congestion
show var CLI.out
show var smsg
set var v $TCL(after $wtime)
show var msg2
debug hal show congestion
show var CLI.out
set var msg "GET VLAN VPIF INFORMATION"
show var msg
debug vlan show vpifs
show var CLI.out
################# CLEAN UP VAR ######
delete var msg
delete var smsg
delete var msg1
delete var msg2
delete var wtime
delete var v
delete var vn
delete var msgs
enable clipaging

ELRP SCRIPT

If ELRP not configured on the switch, with the following script, we can detect the loop, by running following script.
Ø  How to run.
1.       Save following script as elrp.xsf and save it on tftp sever and download on switch.
2.       save the log of telnet on your PC.
3.       Run following command on the switch.
* ds2.4 #  tftp get 10.120.40.45 vr "VR-Mgmt" elrp.xsf elrp.xsf
* ds2.8 # load script elrp
4.       Script return the loop status per vlan on the cli
5.       Stop the script with Ctrl+C button if you want to abort.
[elrp.xsf]
disable clipaging
enable elrp-client
set var sv "#ELRP POLL STARTED#"
show var sv
set var cli.out 0
show vlan
set var sv $TCL(split ${cli.out} "\n")
set var e $TCL(lsearch $sv *(B)*)
set var i 5
set var e ($e - 1)
while ($i < $e) do
set var cli.out 0
set var v $TCL(lindex $sv $i)
set var vn $TCL(lindex $v 0)
set var z $TCL(regexp {Mgmt} $vn)
if ($z == 0) then
conf elrp-client one-shot $vn port all print-and-log
set var p $TCL(split ${cli.out} "\n")
set var p1 $TCL(lsearch $p *NO*)
if ($p1 == -1) then
set var p1 $TCL(lindex $p 2)
set var p2 $TCL(lindex $p1 9)
set var p1 $TCL(lrange $p1 0 6)
set var p1 $TCL(concat $p1 $p2)
set var p2 $TCL(lindex $p 3)
set var p2 $TCL(lrange $p2 5 7)
set var p $TCL(concat $p1 $p2)
show var p
else
set var p1 $TCL(lindex $p 2)
set var p2 $TCL(lindex $p1 10)
set var p1 $TCL(lrange $p1 0 7)
set var p1 $TCL(concat $p1 $p2)
show var p1
endif
endif
set var i ($i + 1)
endwhile
disable elrp-client
enable clipaging
set var sv "#ELRP POLL COMPLETED#"
show var sv



No comments:

Post a Comment