In the 1990s, many Cisco's applications used direct-connected storage (DAS). As the company expands its size, mergers and acquisitions of DAS for other companies cannot meet the needs of the company. At the end of 2001, Cisco began to move from DAS mode to a shared SAN environment. As of September 2002, Cisco IT has deployed 55 McDATA and Brocade switches in three large enterprise data centers, with a total of 1,400 ports.
Inspired by the successful porting of DAS-SAN in enterprise data centers, Cisco Engineering Data Center administrators began to shift their storage to SAN environments, using McDATA 6064 enterprise director switches at the core and McDATA Sphereon 3032 switches at the edge. But obviously Cisco IT departments are not satisfied with this result. They hope to build a larger storage system, improve the utilization rate of servers and storage systems, and optimize management. At this time, Cisco's own Cisco MDS 9000 series multi-layer switches were launched. This product was designed to foresee the need for SAN for larger scale, stronger management and higher availability. Cisco IT intends to use Cisco MDS 9509 multilayer director switches – the industry-leading ability to provide up to 224 Fibre Channel port switches on a single cabinet.
The high availability of SAN requires that the migration process minimizes the impact on the application. On the other hand, many potential customers of Cisco have the problem of investing in and gradually moving switches to protect their original small-scale SAN switches. Therefore, Cisco storage switch R&D department also hopes to understand what problems will be faced when Cisco's MDS9509 works with low-end McDATA switches to gain practical experience. Our plan is to run the engineering SAN in interoperable mode in the first few months to understand the many details of the customer’s transplant process. Cisco's specific plan is: In the summer of 2003, IT will install two Cisco MDS 9509 multi-layer director switches at the Cisco Engineering Building network core in San Jose, California and connect them to the original McDATA 3032 edge switches and storage arrays. After two months of running SAN in operating mode, Cisco will replace the McDATA 3032 edge switch with the Cisco MDS 9120 multi-layer matrix switch for more advantages. In order not to affect the normal operation of Cisco's critical business applications that rely on SAN during the transplantation process, many technical guarantee measures need to be taken.
Rigorous testing is done step by step
Cisco's IT department has developed rigorous testing plans, migration plans and backup plans for migration and interoperability efforts. For example, in order to verify the problems in interoperability, Cisco IT department selected the IBM Rational ClearCase application to test whether interoperability will affect the application, because thousands of Cisco developers use it as the source code base for developing Cisco IOS? Software, which requires very high storage performance.
Cisco successfully transplanted the first core switch in June 2003 and completed the transplantation of the second core switch in September of the same year.
Thanks to a careful planning, the process of transplanting to a hybrid Cisco MDS and McDATA SAN has no impact on the service. "When we shut down a core switch, the host was not affected in any way, and the customer didn't even feel the change," Angulo said.
Effectiveness
From June 2003 to January 2004, Cisco Engineering SAN operated normally in interoperable mode. In this case, Cisco announced the successful conclusion of interoperability testing and replaced the McDATA 3032 edge switch with the Cisco MDS 9120 multi-layer matrix edge switch to take advantage of the SAN capabilities only available on Cisco MDS switches. During the interoperability testing phase, Cisco not only improved scalability and utilization, but also achieved availability goals. As stated in Next Step, Cisco soon took advantage of the powerful VSAN capabilities of the MDS 9000 series multilayer switches.
Scalability and utilization
The high port density of Cisco MDS 9509 multilayer director switches—32 ports per blade, up to 224 fiber channels—can not only reduce costs, improve utilization, but also save space in the data center. In fact, the Cisco Engineering Group not only plans to add 50 Cisco MDS 9216 multi-layer matrix edge switches, but also intends to use only two Cisco MDS 9509 multi-layer director switches instead of 4 to 6 McDATA 6064 switches to support all 60 edge switches.
Angulo said the scalability of the MDS 9509 multi-layer director switch also ensures the highest resource utilization. He said: "Suppose the ERP environment is not enough ports and the engineering environment has surplus ports, we were unable to share resources in the past. But with the MDS 9509 multi-layer director switch, we just need to plug the ERP host into the spare port and configure it as part of the ERP VSAN." To join VSAN, the Cisco MDS 9509 multi-layer director switch can reside anywhere on the Cisco Campus Network Metropolitan Area Network (MAN), including different data centers.
High availability
Since the date of deployment, the two Cisco MDS 9509 multi-layer director switches have been running normally. Its high availability comes from redundant switching management engines, fully stateful switching management engine failure recovery, redundant connections, smooth software upgrades, single process restarts and process management within VSAN. During the migration process, the service of the ClearCase host was never interrupted and the application was always running normally, thanks to the Cisco MDS 9000 series multi-layer switches. "When the second matrix was ported, all devices quickly went into steady state," Finley said.
Finley notes that the high availability of Cisco MDS 9509 multilayer director switches is particularly important when companies move from DAS to SAN environments. In a DAS environment, if the Fibre Channel connection between the host and the storage is interrupted, it will generally only affect a set of applications in a business unit. In contrast, in a SAN environment, if the connection between the host and the storage is interrupted, it will affect the application and business units of the entire data center.
VSAN
Currently, all hosts and storage in Cisco Engineering SAN are placed on the same VSAN, and Cisco has not yet fully utilized the VSAN capabilities. If management traffic starts to affect performance, Cisco can set up multiple VSANs on the same switch, allowing each VSAN to have its own broadcast domain. For Cisco customers, the advantage of leveraging VSAN in a hybrid MDS-McDATA environment is that one Cisco MDS 9509 multi-layer director switch can support certain VSANs in interoperable mode and others run normal functions. This is important because certain features of the Cisco MDS 9509 multilayer director switch cannot operate in interoperable mode, such as the relay function that uses aggregation I/O and thus multiple cables appear to be one. For McDATA switches, either all use interoperable mode or all do not.
Experiences and lessons
"It is very important to develop a switch plan because once there is a problem, it will affect many hosts." For example, Cisco has developed a fiber network topology plan to replace two core switches one by one rather than at the same time, so that even if one of the matrix has problems, the transplant process can be completed smoothly.
Finley stressed that for interoperability mode, it is necessary to check whether the McDATA domain ID is within the allowed range and whether there is a partition conflict between the Cisco MDS switch and the McDATA switch. Additionally, when the Cisco MDS 9000 series multilayer switches run in interoperable mode with hosts produced by other manufacturers, the intersection of these two switch functions can only be used. Obviously, the advanced VSAN and network management features in Cisco MDS switches cannot be used in interoperable environments. After replacing the remaining McDATA edge switches with the Cisco MDS 9120 multi-layer matrix edge switch in 2004, Cisco gained unlimited VSAN capabilities, higher scalability and superior network management capabilities.
postscript
Cisco IT plans to deploy MDS series switches on a larger scale to form larger SANs, including more MDS9509 and MDS 9120, using VSAN to improve SAN virtualization
Inspired by the successful porting of DAS-SAN in enterprise data centers, Cisco Engineering Data Center administrators began to shift their storage to SAN environments, using McDATA 6064 enterprise director switches at the core and McDATA Sphereon 3032 switches at the edge. But obviously Cisco IT departments are not satisfied with this result. They hope to build a larger storage system, improve the utilization rate of servers and storage systems, and optimize management. At this time, Cisco's own Cisco MDS 9000 series multi-layer switches were launched. This product was designed to foresee the need for SAN for larger scale, stronger management and higher availability. Cisco IT intends to use Cisco MDS 9509 multilayer director switches – the industry-leading ability to provide up to 224 Fibre Channel port switches on a single cabinet.
The high availability of SAN requires that the migration process minimizes the impact on the application. On the other hand, many potential customers of Cisco have the problem of investing in and gradually moving switches to protect their original small-scale SAN switches. Therefore, Cisco storage switch R&D department also hopes to understand what problems will be faced when Cisco's MDS9509 works with low-end McDATA switches to gain practical experience. Our plan is to run the engineering SAN in interoperable mode in the first few months to understand the many details of the customer’s transplant process. Cisco's specific plan is: In the summer of 2003, IT will install two Cisco MDS 9509 multi-layer director switches at the Cisco Engineering Building network core in San Jose, California and connect them to the original McDATA 3032 edge switches and storage arrays. After two months of running SAN in operating mode, Cisco will replace the McDATA 3032 edge switch with the Cisco MDS 9120 multi-layer matrix switch for more advantages. In order not to affect the normal operation of Cisco's critical business applications that rely on SAN during the transplantation process, many technical guarantee measures need to be taken.
Rigorous testing is done step by step
Cisco's IT department has developed rigorous testing plans, migration plans and backup plans for migration and interoperability efforts. For example, in order to verify the problems in interoperability, Cisco IT department selected the IBM Rational ClearCase application to test whether interoperability will affect the application, because thousands of Cisco developers use it as the source code base for developing Cisco IOS? Software, which requires very high storage performance.
Cisco successfully transplanted the first core switch in June 2003 and completed the transplantation of the second core switch in September of the same year.
Thanks to a careful planning, the process of transplanting to a hybrid Cisco MDS and McDATA SAN has no impact on the service. "When we shut down a core switch, the host was not affected in any way, and the customer didn't even feel the change," Angulo said.
Effectiveness
From June 2003 to January 2004, Cisco Engineering SAN operated normally in interoperable mode. In this case, Cisco announced the successful conclusion of interoperability testing and replaced the McDATA 3032 edge switch with the Cisco MDS 9120 multi-layer matrix edge switch to take advantage of the SAN capabilities only available on Cisco MDS switches. During the interoperability testing phase, Cisco not only improved scalability and utilization, but also achieved availability goals. As stated in Next Step, Cisco soon took advantage of the powerful VSAN capabilities of the MDS 9000 series multilayer switches.
Scalability and utilization
The high port density of Cisco MDS 9509 multilayer director switches—32 ports per blade, up to 224 fiber channels—can not only reduce costs, improve utilization, but also save space in the data center. In fact, the Cisco Engineering Group not only plans to add 50 Cisco MDS 9216 multi-layer matrix edge switches, but also intends to use only two Cisco MDS 9509 multi-layer director switches instead of 4 to 6 McDATA 6064 switches to support all 60 edge switches.
Angulo said the scalability of the MDS 9509 multi-layer director switch also ensures the highest resource utilization. He said: "Suppose the ERP environment is not enough ports and the engineering environment has surplus ports, we were unable to share resources in the past. But with the MDS 9509 multi-layer director switch, we just need to plug the ERP host into the spare port and configure it as part of the ERP VSAN." To join VSAN, the Cisco MDS 9509 multi-layer director switch can reside anywhere on the Cisco Campus Network Metropolitan Area Network (MAN), including different data centers.
High availability
Since the date of deployment, the two Cisco MDS 9509 multi-layer director switches have been running normally. Its high availability comes from redundant switching management engines, fully stateful switching management engine failure recovery, redundant connections, smooth software upgrades, single process restarts and process management within VSAN. During the migration process, the service of the ClearCase host was never interrupted and the application was always running normally, thanks to the Cisco MDS 9000 series multi-layer switches. "When the second matrix was ported, all devices quickly went into steady state," Finley said.
Finley notes that the high availability of Cisco MDS 9509 multilayer director switches is particularly important when companies move from DAS to SAN environments. In a DAS environment, if the Fibre Channel connection between the host and the storage is interrupted, it will generally only affect a set of applications in a business unit. In contrast, in a SAN environment, if the connection between the host and the storage is interrupted, it will affect the application and business units of the entire data center.
VSAN
Currently, all hosts and storage in Cisco Engineering SAN are placed on the same VSAN, and Cisco has not yet fully utilized the VSAN capabilities. If management traffic starts to affect performance, Cisco can set up multiple VSANs on the same switch, allowing each VSAN to have its own broadcast domain. For Cisco customers, the advantage of leveraging VSAN in a hybrid MDS-McDATA environment is that one Cisco MDS 9509 multi-layer director switch can support certain VSANs in interoperable mode and others run normal functions. This is important because certain features of the Cisco MDS 9509 multilayer director switch cannot operate in interoperable mode, such as the relay function that uses aggregation I/O and thus multiple cables appear to be one. For McDATA switches, either all use interoperable mode or all do not.
Experiences and lessons
"It is very important to develop a switch plan because once there is a problem, it will affect many hosts." For example, Cisco has developed a fiber network topology plan to replace two core switches one by one rather than at the same time, so that even if one of the matrix has problems, the transplant process can be completed smoothly.
Finley stressed that for interoperability mode, it is necessary to check whether the McDATA domain ID is within the allowed range and whether there is a partition conflict between the Cisco MDS switch and the McDATA switch. Additionally, when the Cisco MDS 9000 series multilayer switches run in interoperable mode with hosts produced by other manufacturers, the intersection of these two switch functions can only be used. Obviously, the advanced VSAN and network management features in Cisco MDS switches cannot be used in interoperable environments. After replacing the remaining McDATA edge switches with the Cisco MDS 9120 multi-layer matrix edge switch in 2004, Cisco gained unlimited VSAN capabilities, higher scalability and superior network management capabilities.
[1][2] Next page
Article entry: csh Editor in charge: csh
postscript
Cisco IT plans to deploy MDS series switches on a larger scale to form larger SANs, including more MDS9509 and MDS 9120, using VSAN to improve SAN virtualization
Previous page [1][2]
Article entry: csh Editor in charge: csh