"Brocade devices support authentication using up to eight RADIUS servers, including those used for 802.1X authentication and for management. The device tries to use the servers in the order you add them to the device configuration. If one RADIUS server times out (does not respond), the Brocade device tries the next one in the list. Servers are tried in the same sequence each time there is a request."
If you have multiple RADIUS servers used for remote management (SSH/Telnet/Console), the order it is configured/added is the order they are contacted. The issue appears when adding support for 802.1x/NAC on the switches which needs to authenticate devices to RADIUS/EAP. If the same lists of RADIUS servers are to be used for 802.1x/EAP, then this should not be an issue. If you use a different set of RADIUS servers for remote management and a another set of RADIUS servers for 802.1x/NAC, we would need different approach.