faz1.com blog

Swordfish Beta 1 Delivered – July 2008

Before this program, I never realized how much work is involved in getting customers to actually test and provide feedback on a new product, especially a software add-on to another company’s product.  Swordfish’s success was dependent on a successful beta cycle of VMware’s next generation product in the 2nd half of 2008 and 1st part of 2009 (to be known as vSphere) and customer’s desire to evaluate the new features.  To make matters even more difficult, because of customer confidentiality agreements already in place, the 2 companies could not share beta customer information.  Since the Swordfish beta was to be run as a “add-on” beta program to the vSphere beta, we (Cisco) needed to approach customers one by one who we knew were interested in Swordfish and possibly participating in the vSphere beta program already.  Sometimes we got lucky, sometimes we struck out, but overall, this “beta on a beta” proved to be very difficult from a logistics perspective.  Looking forward, my goal would be to find a way to run subsequent beta programs on a version of vSphere that has already GA’d to dramatically simplify the customer engagement process.  This wasn’t an option with Virtual Infrastructure 3 because of the lack of the vNetwork Distributed Switch functionality (which was known at the time as “distributed virtual switch”).

In parallel to figuring out the beta process with VMware, the team was busy with nailing down the the business arrangement to ensure that 1) both Cisco and VMware could embrace this co-development effort as a win-win (not very easy when you have two 800lbs gorillas sitting in the same room, neither wanting to move away or give on the terms they normally require in any agreement) and 2) the solution would result in something easily deployed and adopted by our mutual customers.  Flexibility, lots of hard work, many late nights and some ingenious engineering efforts on both sides went into pulling this off, and it was certainly aided (from my perspective) by the initial feedback we were starting to get from joint customers.

So what was the initial feedback from customers?  It was extremely positive.  There were only a few features supported in the beta 1 program, but one of the key innovations that the Swordfish engineering team had developed (a feature called Port Profiles) was starting to catch on with customers.  This feature allows network administrators to configure/set network policy for virtual machine environments and then expose a collection of these network policies to the server administrator through VMware’s vCenter tool.  Port Profiles would enable network, security and server admins to embrace server virtualization like never before, allowing the functional data center teams to respect each other’s configuration and operational boundaries.  This is a critical capability if customers are going to attempt to virtualize 100% of their applications in the next few years.

As the positive customer feedback continued to come in, we worked with VMware to figure out if we wanted to introduce this new concept at the upcoming VMWorld show in Las Vegas in September of 2008.  The goal was to create awareness for the new technology and pave the way for an even broader phase 2 beta program (the biggest in Cisco’s history) and successful product launch.  The 2 companies agreed that this would be a great step to take and the planning began for a September technology launch.

The next chapter

Chapter 4: August 2008 – Swordfish officially named the Nexus 1000V

Swordfish Starts Swimming – January 2008

In January of 2008, the Swordfish engineering team delivered the first NX-OS based proof of concept for what would ship as the Nexus 1000V for VMware’s vSphere 4.0 almost 1.5 years later.  This proof of concept, named “Sailfish”, leveraged the same OS that was being used to run the new Nexus 7000 Data Center Switch, known as NX-OS.  NX-OS (Nexus Operating System) is a data-center-class operating system that combined the best of Cisco’s IOS and SANOS with a data center focused feature set to meet the demands of the virtualized data center.

Sailfish, from an architectural perspective, leveraged NX-OS to provide a Cisco CLI for the existing VMware ESX 3.5 vSwitch.  This basically allowed the operator to use the familiar Cisco IOS interface to configure and provision the VMware vSwitch.  A key feature of this Sailfish was the ability for a single Cisco NX-OS instance to manage many vSwitches across separate ESX 3.5 hosts, a feature that would be delivered more than a year later in vSphere 4.0 known as the vNetwork Distributed Switch.

For the next 3 months, we used Sailfish as a demonstration with key customers around the world to validate that what we were doing with Swordfish was something that they would actually deploy if a product was ever actually built.  The feedback was largely positive, but also a little mixed.

Our original assumption was that customers would be attracted to this type of product because Cisco could provide more features (QoS, ACLs, NetFlow, ERSPAN, etc) than the basic vSwitch and we found this to be accurate.  What we also found was that customers were looking for more operational tools (management, monitoring, diagnostics etc) as the virtual access layer was continuing to grow both in size and in complexity as the percentage of virtualized workloads continued to grow.

With overwhelming positive response from our proof of concept vehicle, it was time to get back to work on delivering the first beta of the final product to support VMware’s next major release.

The next chapter…

Chapter 3: July 2008 – Swordfish Beta 1 Delivered

Two exploratory programs are merged at Cisco – June 2006

VN-Link “the name” was born in the summer of 2008, but the concept was developed several years earlier and formally established as a program at Cisco in June of 2006.  Two separate groups inside of Cisco had been working on ideas to introduce Cisco networking capability to the virtual machine environment.  At the time, these two groups were concerned that server virtualization would lead to the eventual reduction in the number of data center switch ports that the market would need and move the span of control of the largest volume of ports (the access layer) to something Cisco didn’t provide.  This would be bad for Cisco and all network vendors who did not find a way to monetize the “virtual network” evolution.  What was interesting at the time was that we were actually seeing the opposite effect…data center access ports were growing FASTER than expected due to the fact that virtualized hosts required 2-3 times as many physical NICs as a non-virtualized servers and in 2006-2008, customers were still investing in both models.  This trend slowed dramatically in  2008 as economic downturn pushed customer’s focus more exclusively on highly optimized virtualized servers and spending literally stopped on non-virtualized servers.  But I am getting ahead of myself, how did VN-Link start?

In early summer of 2006, the 2 programs inside of Cisco which were both exploring this space (and for a while didn’t know about each other…common for a big company with lots of R&D efforts) were merged into 1 program with the purpose of formalizing a plan to develop a proof of concept for a Cisco “softswitch”.  For a while, the program was simply knows as “softswitch” but we eventually decided we needed a proper name.  Ironically, the eventual name of the program (Swordfish) was actually thought up by mistake in the hallway on the way to the meeting which was to decide if the “softswitch” program would be funded or if it would die.  That conversation went something like this:

Ram: “Hi Tom, are you going to the softswitch meeting?”

Tom: “Swordfish? What Swordfish meeting?

Ram: “Not Swordfish, softswitch.”

Tom: “Oh yes, the softwswitch meeting, yes I am going.”

Ram: “I think we just came up with our program name — Swordfish!”

As a result of that meeting, the program got a formal name and actual funding.  What was interesting was how the program was actually funded and staffed.  Initially sponsored by then Cisco SVP Jayshree Ullal (now CEO at network startup Aristra Networks), Cisco SVP John McCool (who currently runs the Catalyst 6500, 4500, NX-OS, MDS & Nexus 7000 products), Cisco SVP Tom Edsall (hardware mastermind behind Catalyst 6500, MDS 9500 and Nexus 7000 products) and former Cisco NMTG CTO Paul Gleichauf (who is now working on lots of top secret things over at Apple), Jayshree decided to implement a R&D tax on all of the major business units she ran to get the Swordfish program funded.  With that, we were off to build a proof of concept for our newly funded “internal startup”.

In an attempt to keep our engineers from being poached, I am going to stick with first names as I introduce the team.  Folks familiar with Cisco will know who I am talking about.  The team started as 2 physically diverse groups (team San Jose and team Minnesota – yes, there are good engineers in Minnesota, really good!). Team San Jose was made up of Saravan, Michael, Paul, Ram, Michelle and me.  Team Minnesota was made up of Mark, Dave and Tim.  Over the course of 2006 the team got really good at using collaboration technologies to get stuff done.

So now that we had a team to build a proof of concept, we needed a hypervisor platform to run this new softswitch on.  It was a very easy decision at the time, and I would argue that it still would be an easy decision today (mid 2009).  The VMW ESX 3.0 solution was on the market and customers where deploying it faster than any other new data center technology.  From my perspective, this was when x86 server virtualization really began going mainstream and 80% or more of the market was going with ESX 3.0 Enterprise.  We had our hypervisor, now we needed to convince VMware that this was a good idea for them and their customers.

Why would VMware want our softswitch in their hypervisor?  After all, they already had one of their own already (the VMware vSwitch).  Well, with the introduction of ESX 3.0, server administrators started facing push-back from network administrators.  This wasn’t because the network administrator didn’t support the use of the new technology, but rather was a result of some of the infrastructure and configuration changes required to support the new hypervisor deployments.  For instance, network administrators have been driven for years not to trunk multiple VLANs to server NICs and this was a common deployment best practice for ESX.  Also, the network admins were usually held responsible for application connectivity issues and ESX vSwitch didn’t offer the operations, management and diagnostics toolkit needed to operate the virtual network access layer like the physical.  These types of issues kept popping up during VMware sales calls and their account teams didn’t have a solution.  We initially dubbed customers with these types of objections “the CCIE types” but both companies quickly realized that the challenges previously described were not isolated issues.  I was seeing this same sort of feedback in during customer briefings on the Catalyst 6500 (prior to working on Swordfish, I was the data center hardware product manager for that platform).  At some point, our and VMW’s respective light bulbs went off and we agreed that this co-development effort would be mutually beneficial.

Over the next few months, Cisco and VMware developed our budding business relationship around the Swordfish program.  This turned out to be an interesting and often challenging relationship.  Both companies were used to getting exactly what they wanted from partners (in other words, having it their own way) and this co-development effort, if it was to ever take off, needed to be one of equals.

The next chapter…

Chapter 2: January 2007 – Swordfish Proof of Concept Delivered (Sailfish)