For whatever reason, I’ve been spending a pretty fair amount of time lately with vCloud Director both at home as well as at the office. It’s a great product. It always has been, beginning with its Lab Manager roots. Like my last blog post, this writing will exhibit another vCloud Director database editing exercise which stemmed from a problem I encountered in the lab.

I was attempting to get away from my VLAN-backed Network Pool by configuring vCloud Director’s Provider vDC-VXLAN-NP Network Pool which is much more dynamic and powerful in nature. The Provider vDC-VXLAN-NP Network Pool is installed by default in vCloud Director but to configure and use it for Organization and vApp networks, one must follow a set of instructions which basically involves configuring upstream physical switch(es) with jumbo frames, a transport VLAN, and multicast settings, preparing the hosts by installing an agent on each of them using vShield Manager, adding VMkernel ports, Network Scopes, Virtual Wires, and so on (Mike Laverick and Rawlinson Rivera both have easy to follow tutorials. The VMware VXLAN Deployment Guide is also a great read). Once it’s all set up and working, VXLAN is pretty effing cool. Anyway, it sounds like a lot of steps and admittedly it requires some reading and attention to detail, but much of it is automated by vCloud Director, with some bumps along the way.

I did run into a few snags which ultimately lead me to going through the configuration process start to finish a few times. In the end I had to configure the Network Scope in vShield Manager manually when normally this step is performed automatically by vCloud Director via the Enable VXLAN Provider VDC right-click menu item.

Once I got beyond the installation hurdles, there was some residual impact left in the vCloud Director database and vShield Manager such that it all looked to be working properly, except that at the very end I could not power on a vApp with an isolated vApp network which relied on the use of the VXLAN-backed Network Pool. The error message was:

Cannot deploy organization VDC network  (uuid for that network)
com.vmware.vcloud.fabric.nsm.error.VsmException: VSM response error (202): The requested object : vdnscope-1 could not be found. Object identifiers are case sensitive.

[ bb505f5e-27f1-419e-9b05-da0d38a7788f ] Unable to deploy network “vApp net1(urn:uuid:7d813867-d3f1-420d-a0a8-a65263369327)”.

com.vmware.vcloud.fabric.nsm.error.VsmException: VSM response error (202): The requested object : vdnscope-1 could not be found. Object identifiers are case sensitive.

– com.vmware.vcloud.fabric.nsm.error.VsmException: VSM response error (202): The requested object : vdnscope-1 could

not be found. Object identifiers are case sensitive.

– VSM response error (202): The requested object : vdnscope-1 could not be found. Object identifiers are case sensitive.

An object named vdnscope-1 seems to be the obvious problem.

I was not able to make use of the Network Pool Repair function as it was unavailable:

Fortunately I was able to locate a related thread in the VMware Communities which more or less explained what might have happened and what I could try to fix the problem (credit to IamTHEvilONE). This is my interpretation.

Each time a Network Scope is created in the vShield Manager, an underlying object reference is tied to the Network Scope with a naming convention of vdnscope-x where x begins at 1 and is incremented at each create iteration. So the first Network Scope created in vShield Manager by vCloud Director is going to be called vdnscope-1. This object is stored in the vCloud Director database and is referenced each time an Org or vApp network is spun up which leans on the VXLAN-backed Network Pool. This is formally handled at vApp power on. The object is also stored somewhere in the vShield Manager although I was never able to locate it. What happened here is that Network Scope object known by both vCloud Director and vShield Manager were not sync and didn’t match. vCloud Direct dials up vShield Manager and says “I need that vdnscope-1 you have” and vShield Manager responds with “I have no idea what that object is”. Obvious problem.

The solution is fairly simple: Edit the vCloud Director database with the correct Network Scope object reference. But a small problem still remains: I was never able to locate the correct object name in vShield Manager. However, going back to the VMware Communities discussion, I’ll eventually be able to find the correct object name by incrementing the vdnscope-x object reference in the vCloud Director database by 1 until the two sides agree and the vApp powers on successfully.

I’ll borrow the same disclaimer from the previous blog post: An obligatory warning on vCloud database editing. Do as I say, not as I do. Editing the vCloud database should be performed only with the guidance of VMware support. Above all, create a point in time backup of the vCloud database with all vCloud Director cell servers stopped (service vmware-vcd stop). There are a variety of methods in which you can perform this database backup. Use the method that is most familiar to and works for you.

So after stopping the vCloud Director services and getting a vcloud database backup…

Step 1: Open Microsoft SQL Server Management Studio and navigate to the [vcloud].[dbo].[network_pool] table. Under the vdn_scope_id column, increment the vdnscope-1 value from 1 to 2.

Step 2: Start the vCloud Director service in all cell servers (service vmware-vcd start) and verify in vShield Manager the Virtual Wire has been created and the vApp can power on successfully. If it fails, stop vCloud services and repeat Step 1 above while incrementing the vdnscope value to 3, then 4, and so on. In my case, vdnscope-5 did the trick.

vCloud Director is awesome. VXLAN with 16 million networks capability kicks it up a notch.

Updated 8/22/15: I received a tip from Jon Hemming in the form of a blog comment. Jon states he has written a VMware KB article titled Creating an isolated network in VMware vCloud Director reports the error: vdnscope-x does not exist (2065485) which documents a process to get the correct VDN Scope ID via the REST API of vShield as well as update the vCloud Director database. Thank you Jon! I did find the syntax for the curl statement to be slightly off. The KB article calls for the following syntax:

curl -k -u admin:default -X GET https://vshield.boche.lab/api/2.0/vdn/scopes/

The result is HTTP Status 404 The requested resource is not available.

What did work was:

curl -k -u admin:default -X GET https://vshield.boche.lab/api/2.0/vdn/scopes

The only change was removing the trailing forward slash on the URL.

• Previous Entry: vCloud Director Error Cannot delete network pool
• Next Entry: Error 1603: Java Update did not complete