+-----------------+ +-----------------+
| HP A7500 | | HP A7500 |
| | IRF | |
+-----------------+ Links +-----------------+
| 10Gb +<--------->+ 10Gb |
| Module +<--------->+ Module |
+-----------------+ +-----------------+
| | | |
| | | |
| | | |
| | | |
+-----------------+ +-----------------+
In our environment, we have an HP IRF (Intelligent Resilient Framework) switch cluster running between two HP A7500 chassis connected via 2 10Gb Fiber interfaces.
1 Comment
I used to post in Blogsome couple of years ago before until the site closed down. In here, I have posted blogs about RANCID which is a set of scripts (PERL) used to monitor network devices configurations and maintain history of changes via CVS (Concurrent Version System). Here are some of my post about RANCID (version 2.3.2) which are now archived in the Internet which might be of used to someone interested in RANCID and customizing it.
RANCID under the hood (posted October 23, 2011) This post show the internal working of RANCID and understand how it works and which scripts are executed and dependent on each other RANCID additional commands (posted November 2, 2011) Normally, RANCID executes some commands when connecting to network devices, this post discusses how to include additional commands. RANCID Customization (posted November 9, 2011) This post is discusses how to poll different groups of network devices with different sets of commands I hope in the future I can update my knowledge with the current release of RANCID. This is continuation post of the usage of Python Paramiko network device configuration script (netscript.py) I am currently testing. Let us say you have around 100 network devices accessible via SSH and you need to update their configuration. For example you need to add a new syslog server and remove the current one. In Cisco IOS, you would need to run these set of commands: # Enter Configuration mode We had this problem with one of our Internet links which was an ADSL/PPPoE connection. Every now and then this Internet connection seems to be unstable and it affected Internet access through this line. One solution we came across is resetting the ADSL connection either by pulling the cable off or shut/unshut the interface. Once the ADSL/PPPoE gets reconnected, Internet access becomes stable again. While waiting for the service provider to fix the issue (if it gets fixed), we needed to every now and then reset the interface connecting the ADSL/PPPoE line. This can be during after office hours (i.e. 12MN) but should be automated (I can't be doing this manually) Here comes my Python Network Script (netscript) discussed here in my post along with cron. Below are my steps in automating this task. First we create the command file (interface_reset.txt) which will shutdown and enable the network interface. #!/bin/bash As a network engineer, I have always been learning how to go about with scripting with network devices. Since most network devices are capable nowadays some kind of CLI via SSH remote access, there should be a better way in network scripting commands into these devices. As such, I needed a network script which will connect to the network devices via SSH and execute commands similar to running these in the command line.
Here I have used Python Paramiko as my preferred approach. There have been several articles discussing how to use Paramiko but I am more focused in using it for network scripting. After reading these articles, I have managed to create a working script and uploaded in my github repository (netscript). Kindly note that this a work in progress. To get this script working, you need to install Python and Paramiko on Linux (preferred). Check with your preferred Linux distribution in installing these. In my environment which is Debian Linux, these are installed with apt-get. This post will discuss Password Recovery method for HP Comware network devices. This password recovery procedure was tested on HP 5120 48-Port Network Switch with BootROM version 171 (released February 24, 2015).
In my last post, I have discussed in accessing the Extended BootROM of the HP Comware network device. In order to perform password recovery, we need to access the Extended BootROM menu by pressing CTRL+B and selecting the option to skip the current system (or startup) configuration. (Note: This is done thru the console port of the network device) This post is in continuation of the previous post on HP Comware BootROM. In this article, I will display the menu options for the Extended BootROM for a HP 5120 Network Switch. This mode can be accessed by pressing CTRL+B while the device is booting and this keypress option is displayed (see below).
This post will highlight accessing the boot loader of an HP Comware network device. This is called BootROM in most HP Comware devices (or BootWare in the H3C models). This post will show accessing the bootloader (BootROM) for an HP 5120 Switch. The HP 5120 Switch has two modes in accessing the BootROM. First mode is the Basic Boot Mode which can be accessed by pressing CTRL+D while the device is booting. Note that this can only be performed while accessing the network device through the Serial Console. Starting...... The Basic BootROM menu provides limited options in updating the BootROM software for both basic and extended upgrades. The Menu display might be different for other BootROM versions. In most options, it requires selecting the baudrate prior to downloading the BootROM software via XMODEM protocol.
In summary, HP Comware supports local logfile. This means logs generated by the network device can be stored in a local file within the device. This feature is very convenient in case the network devices encounters a disruption (i.e. power, hardware fault) and its logs are not able to reach the configured SYSLOG host. The locally stored logs will be useful in conducting post mortem network analysis (i.e. before the network device rebooted). HP Comware local logfile is disabled by default. To enable local logfile support Below are configuration options for local logfile management <HP-TEST-SWITCH>display logfile summary <HP-TEST-SWITCH>system-view These are basic file management commands for HP Comware which are handy when working with the contents of a Comware network device. dir - Displays contents of the current working directory dir /all - Displays contents of the current working directory including hidden files dir /all-filesystems - Displays contents on all storage (i.e. cfa0:/ if exists) dir <storage|directory> - Displays contents of a target storage or directory cd <storage|directory> - Change current directory pwd - Display current working directory Note: Some network device have more than one storage other that the default Flash storage (flash:/) such as Compact Flash (cfa0:/). # Display contents of current directory Other relevant file system commands. copy <source_file> <target_directory|target_file> - Copies the source file to the target directory or target file (if renaming) move <source_file> <target_directory|target_file> - Moves the source file to the target directory or target file (if renaming) rename <target_file> <new_filename> - Renames the source file to the new file name delete [/unreserved] <target_file> - Deletes the target file and moves it to the recycle-bin. The unreserved keyword deletes the file permanently. undelete <target_file> - Restores the target file located in the recycle-bin. reset recycle-bin [/force] - Deletes files in the recycle-bin. The force keyword deletes all files without confirmation. more <target_file> - Displays contents of target file . Recommended only for text files. mkdir <target_directory> - Creates a new directory on the current storage. rmdir <target_directory> - Deletes an existing directory. Note: only empty directories can be deleted. format <target_storage> - Formats the target storage. Useful for formatting mounted Compact Flash (cfa0:/). Warning: Formatting the flash is not recommend unless firmware is corrupted or planned to reset the device. mount|umount <target_storage> - Mounts or unmounts a storage device particularly the Compact Flash (cfa0:/) fixdisk <target_storage> - Examines and repairs the target storage (i.e. similar to chkdsk) Kindly refer to this URL for more details on file system management configuration. This is simple configuration for Policy Based Routing for HP Comware (version 5.2) we have implemented in our environment where we required all Internet access routed to a transparent proxy. in this setup, we have these conditions:
First we defined two access-lists, one for the traffic exempted from the PBR acl number 2000 name PBR-Traffic-Exemption Second ACL for the traffic to be routed by the PBR to the transparent proxy acl number 2001 name PBR-to-Transparent-Proxy Create the PBR configuration with these ACLs included policy-based-route PBR-Transparent-Proxy deny node 10 Kindly note that the keyword "deny" indicates that any matched traffic will be handled by the device's routing table where as the "permit" keyword indicates any matched traffic will be handled by the corresponding apply sub-command. Appy the PBR configuration on the desired interface interface Vlan-interface 100 To inspect and verify the PBR configuration on the device: # Check the PBR configuration I hope this simple configuration would help anyone. In this post, I will not discuss in detail QoS (Quality of Service). I will however discuss specific in configuring a HP Comware network device to mark specific traffic with DSCP code point. This marking of traffic is important for QoS/CoS aware network devices in order for them prioritize these traffic. I have tested this configuration with a HP MSR 3020 Router with Comware version 5.2. In this setup, I have installed 2 probes both running Linux where one (10.10.10.251) is located locally next to the router and the other (10.0.0.200) located over a WAN connection. On the MSR router, we configured an access lists (ACL) to match all traffic to/from the remote probe (10.0.0.200) acl number 3001 name probe_traffic Configure a traffic classifier to match these traffic to/from the remote probe traffic classifier class_probe_traffic Configure a traffic behavior to mark a traffic with a specific DSCP code point value. Below we use AF11 traffic behavior mark_dscp_af11 Below are the other DSCP values we can use on HP Comware INTEGER<0-63> DSCP (DiffServ CodePoint) value Then configure a QoS policy combining the traffic classifier and behavior qos policy qos_test_remark This QoS policy can now be applied to an interface both inbound and outbound traffic interface GigabitEthernet0/0 Verify the QoS policy applied on the interface <HP-WAN-TEST-RTR-3020>display qos policy interface g0/0 Test the policy from the local probe by pinging the remote probe here we sent 3 ICMP echo request root@probe:/# ping -c 3 10.0.0.200 Verify if the traffic is matched, classified and marked by the policy <HP-WAN-TEST-RTR-3020>display qos policy interface g0/0 We can see in the above results, 3 packets are matched and marked both inbound and outbound. From the remote probe, traffic is capture via tcpdump to check in detail the packets arriving at the remote probe. root@probe200:/# tcpdump -n -vvv host 10.10.10.251 The important highlight here is the packet trace confirming the TOS HEX value 0x28 which is equivalent to DSCP code point of AF11 is detected. This is a continuation of the interface range configuration capability of HP Comware network devices, see Part 1 for the "port-group" command. HP managed to introduce the same "interface range" command in HP Comware network devices. Unfortunately I have only seen it available in some models. So far I have seen this command in the EI Switch Series, A5800 and A7500. Someone can correct me here as I don't have the resource to verify all of their product line. To give an understanding on how the commands works, below is a session example on a HP A5800 switch # Show the HP Comware version As we can see, the HP Comware "interface range" command is very similar to Cisco IOS and no longer has limited interface sub commands. These means we can now use interface sub commands such as description, shutdown, port-security, and so forth. Another option with the "interface range" command is to name the range and save in the configuration. # Enter system-view mode As we can see, this feature allows us to save a range of interfaces which can be configured altogether again and again without redefining them. Saves some NetOps time :) Similar to Cisco IOS interface range command, HP Comware has options to configure multiple ports. Depending on the HP Comware network device model, not all of these options might be available. I will break this post in two parts in order to focus in each commands. The port-group command is common to all HP Comware network devices which is used to configured multiple ports. First create a name for a group of ports and then use the group-member sub-command to assign a port into the port group. This can be a continuous range of ports or not. Afterwards, these group of ports can be configured at the same time with the available interface commands. Below is a session example using the port-group command. # Enter system-view mode HP Comware has provided network administrators the port-group command as an equivalent to the interface range option of Cisco. However, it still has limitations particularly the available interface commands while in the port-group mode. In my daily network operations, commands like description, port-security and even shutdown are not available in this mode which provides limitations. In Part2, HP Comware introduced another command which addressed these. Everyone who have dealt with HP Comware should be familiar with the "display" command which is the "show" equivalent in Cisco IOS. This post is a recap on the command and its features. The most useful display command is the "display this" which serves its effectiveness while in system-view mode. <HP-TEST-SW-5800>system-view Then let us focus on the "display current-configuration" and its options <HP-TEST-SW-5800>display current-configuration ? Here I find very useful the by-linenum in tracing and debugging in detail the device configuration. Below we look at the pipe "|" options. <HP-TEST-SW-5800>display current-configuration | ? Those who use Cisco IOS should be familiar how to use these begin, exclude, and include preceeding a string to be searched. Then we have this option while in the "display current-configuration" where we have the "---- More ----" paging. <HP-TEST-SW-5120>display current-configuration While in this view, you can press any of these keys:
Here I will discuss the minimum configuration required to enable sFlow for HP Comware (v5.2) network devices which supports it. First thing is to check if sFlow is supported by the HP Comware network device. Below is the output for the command "display sflow" on a HP MSR 3020 Router. <TEST-ROUTE-MSR-3040>display sflow As you can see, this router supports sFlow with default settings such as sFlow version 5 and the sFlow agent IP address (192.168.0.128). This configuration can support 10 sFlow collectors with default UDP/port 6343. Collectors are hosts which receive/process and/or analyze sFlow data. In this test router, we configured sFlow with a collector IP 10.0.0.251 and sflow collection on the interface G0/0. Below are the minimal configuration. # Reconfigure the sFlow agent IP (optional) Once sFlow is configured, we will its final configuration. <TEST-ROUTE-MSR-3040>display sflow We can see in detail the sFlow configuration particularly the collector IP address and its assigned interface. Finally, we will verify if we are receiving sFlow traffic on the collector. Here I have a tcpdump packet capture on Linux host which is our sFlow collection. # tcpdump -n port 6343 For more details and options in configuring sFlow with HP Comware, kindy refer to this link from H3C. We have been running HP Comware in our environment for almost four years now. I just want to share notes on the interconnection of Cisco IOS network devices and HP Comware network devices which both have link aggregation. In Cisco terminology, these are EtherChannel called while in HP Comware these are called Bridge Aggregation. In connecting two devices with link aggregates, they need to support the same protocol. The IEEE standard is LACP (Link Aggregation Control Protocol). There are already a number of articles in configuring link aggregation for Cisco and HP Comware. I will only share the key points in this article. HP Comware Bridge Aggregation supports LACP. To enable LACP, execute these commands on the Bridge-Aggregate interface with mode dynamic. interface Bridge-Aggregation <X> Cisco EtherChannel support LACP and its proprietary PAgP. To enable LACP, execute these commands to each EtherChannel member interface with mode active. interface GigabitEthernet <X/X/X> For more detailed discussion on configuring HP Comware Bridge Aggregatation, refer to this link. With Cisco EtherChannel, these are not difficult to find. We will now continue where we left off in Part 1 of this guide. C. SNMPv3 with level AuthPriv Configure HP Comware network device with SNMP group V3AuthPriv and username snmp1user, authentication passwd snmpauthpass and privacy passwd snmpprivpass using hash mode MD5 and encryption mode AES128 snmp-agent group v3 V3authPriv privacy Verify configuration with these display commands # display snmp-agent group Test the configuration using the snmpwalk tool # snmpwalk -v 3 -u snmp3user -l authPriv -a MD5 -A snmpauthpass -x AES -X snmpprivpass 192.168.200.11 system If you get a response similar to the one above, this confirms SNMP version 3 with AuthPriv is working. Restricting access to the SNMP agent Define the ACL (Access Control List) acl number 2000 name SNMP-Management-ACL Appy the ACL to the SNMP group, in this example its V3authPriv snmp-agent group v3 V3authPriv privacy acl 2000 Test the SNMP configuration from unauthorized hosts should give you a similar response below # snmpwalk -v 3 -u snmp3user -l authPriv -a MD5 -A snmpauthpass -x AES -X snmpprivpass 192.168.200.11 system Debugging SNMP connections Enable console to display debugging logs # terminal monitor Enable SNMP debugging, below are the recommended ones # debugging snmp agent packet header Check debugging status # display debugging Sample console debugging logs *Jan 21 08:35:20:542 2015 HP-TEST-SWITCH-5120 SNMP/7/TXRX_INFO: I hope this guide will assist anyone working witn SNMPv3 with HP Comware 5.2. This guide is to configure, enable and test SNMP version 3 with HP Comware (version 5.2). I have broken this into parts to make it a manageable reading. For detailed SNMP command guides for HP Comware, you can refer to this link from H3C (Note: HP is really bad in keeping their documentation easy to locate), In this guide, we will enable the essential SNMP settings for HP Comware, below are the commands. !# Enable SNMP Agent In summary for SNMP version 3, there are 3 security levels namely:
We will configure all 3 security levels on the HP Comware network device with IP address 192.168.200.11 and use the snmpwalk tool to test the SNMPv3 configuration. A. SNMPv3 with level noAuthnoPriv Configure HP Comware network device with SNMP group V3noAuthnoPriv and username snmp1user snmp-agent group v3 V3noAuthnoPriv Verify configuration with these display commands #display snmp-agent group The display command outputs the SNMP group and username configurations Test the configuration using the snmpwalk tool # snmpwalk -v 3 -u snmp1user -l noAuthnoPriv 192.168.200.11 system If you get a response similar to the one above, this confirms SNMP version 3 with noAuthnoPriv is working. B. SNMPv3 with level AuthnoPriv Configure HP Comware network device with SNMP group V3AuthnoPriv and username snmp1user and authentication passwd snmpauthpass using hash mode MD5 (or SHA) snmp-agent group v3 V3authNoPriv authentication Verify configuration with these display commands #display snmp-agent group The display command outputs the SNMP group and username configurations Test the configuration using the snmpwalk tool # snmpwalk -v 3 -u snmp2user -l authNoPriv -a MD5 -A snmpauthpass 192.168.200.11 system If you get a response similar to the one above, this confirms SNMP version 3 with AuthnoPriv is working. We will continue in Part2 for the SNMP configuration for the 3rd security levels, security ACLs and debugging. Most network administrators know that traceroute packets requires TTL expires and ICMP unreachables to be enabled on all network hops in order to effectively trace the routes of the packets from source to destination. In HP Comware, these are disabled by default, they can be enabled with these commands: ip unreachables enable Some fair warning if the network is not fairly congested, enabling these should not have that much impact on the CPU on the network device. I have managed to configure two years ago a HP Comware network device to authenticate users via RADIUS connecting via SSH. I am posting here the configuration for both the network device side and RADIUS server side. This setup is tested working with HP Comware version 5.2 and FreeRADIUS version 2.1.12 installed on Debian Linux version 7.7. For the HP Comware network device, below is the configuration template. system-view Please be guided that you need to replace all variables (all within "<>") with the correct ones for your environment. Moreover, it is recommended to be connected via console on the network device to avoid a device lockout. For the FreeRADIUS configuration, these are the files needed to be configured and the lines to be added. Clients Configuration File (clients.conf) Insert the lines below and change all variables above with the appropriate values. client <network_device_management_ip> { Users Configuration File (users) Insert the lines below with example username "netadmin" and password "netadmin" which has administrator privileges (Level 3). netadmin Cleartext-Password := "netadmin" Kindly refer to this link in using other than clear text password. The FreeRADIUS service should be restarted to have these settings take effect. If all are working properly, these logs should appear showing the account connecting to the network device via SSH. Jan 17 19:00:01 radiussvr01 freeradius[52280]: Login OK: [netadmin] (from client HP-TEST-SWITCH port 0) Kindly let me know if anyone have problems or issues with this post. I will post next time testing on HP Comware 7 with the latest FreeRADIUS. Update: Check this post for support for HP Comware 7. Most network administrators familiar with Cisco are aware that you can schedule reboot (reload). This link clearly discusses how to execute this. For HP Comware (version 5.2), this facility is also possible with the command "schedule". Below are two options for the command. !# Reboot system at exact time where hh is for hours and mm is for minutes. To display status of scheduled reboot display schedule reboot In case you change your mind for the scheduled reboot undo schedule reboot Below is sample session on the test switch where I scheduled a reboot in 10 minutes but later cancelled it. <HP_TEST-SW-5120>schedule reboot delay 10 This is a simple guide to enable the HP Comware network device to lockdown a specific switchport to a MAC address. This is a restrictive Layer 2 security measure however not very flexible. This method is recommended for areas where there is minimal network movement particularly the Server Room. When in the Comware CLI, enable first "port-security" in system-view. You should see "Done" once enabled. <HP-SWITCH-5120>system-view While in system-view, proceed to the specific interface (i.e. G1/0/1) and enable these commands interface G1/0/1
Below is log generated when we connected a device initially on the switch port %Jan 14 08:52:41:334 2015 HP-SWITCH-5120 PORTSEC/6/PORTSEC_LEARNED_MACADDR: When we check the interface configuration, we see the MAC address registered. <HP-SWITCH-5120>display current-configuration interface g1/0/1 Below is sample log for port-security MAC Address violation %%10PORTSEC/4/VIOLATION(t): Trap1.3.6.1.4.1.25506.2.26.1.3.2 An intrusion occurs! IfIndex: 9437208 Port: 9437208 MAC Addr: B8:88:E2:EC:32:24 VLAN ID: 1800 IfAdminStatus: 1 You can refer to this link for more options on Port Security on the HP/H3C Comware Platfrom _ I am posting here my simple configuration template for hardening the HP Comware network devices. This template currently works well with Comware Version 5.2.
Configuration template !##### Enter System-View mode ##### In the template, all lines which begin with "!" will not be processed by the Comware CLI (command line interface). Again, ensure all variables "<>" are replaced with the correct values in your environment. Example <admin_workstation_ip> is replaced with 192.168.1.201 (note: no more <>). After applying the configuration, review the configuration display current-configuration Onced confirmed, save the configuration save force For more information and other options in hardening HP Comware network devices, kindly refer to this URL. Tcpdump is very famous tool in network traffic capture and packet analyzer. It has so many options and one of them is the ability of writing captured traffic to a file (normally called a PCAP file which is short for Packet Capture). This is invoked with the -w <capture_file> option. The command below instructs tcpdump to write the network traffic to the file "trace.pcap". tcpdump -w trace.pcap Another option for tcdump is that it can capture network traffic every N seconds and write it to a different file. This can be done with the -G <num_secs> option and using time format string on the capture file. Let see an example. tcpdump -G 60 -w 'trace_%Y%m%d-%H%M%S.pcap' This command instructs tcpdump to capture network traffic every 60 seconds and write it to trace file which has time format directives. When you cancel the tcpdump, the listing files will see network capture every 60 seconds (1 minute). # ls -l For the time format directives, refer to this link but these are the ones used above
Recording and rotating capture files every interval in tcpdump is very useful particularly for very heavy network traffic. Big capture files are difficult to process as such breaking them down to smaller and manageable ones is recommended. Moreover, if you tcpdump process crashes while capturing network traffic (i.e. memory exhaust), all the effort is not lost if your using rotated capture files. |
Certified Geek
A Certified Geek who blogs anything geeky he comes across mostly in Linux and Networking. Archives
February 2024
Categories
All
|