One of the greatest things about this industry and a primary driver for me personally is exploring new technology that tackles some of the hardest business and technical challenges. VMware’s NSX product is one of those technologies that I truly believe will change the way IT infrastructures and cloud deployments are done. Software-defined networking has made huge strides over the past year and I believe within the next few years that growth will increase even more.
I’ve been within the EMC family for the last 3 years and it’s been an amazing and career-changing journey. I am extremely thankful for all my friends and teammates that I have gotten to work with. Most recently, as a member of the vSpecialist team, I have been on the front lines with EMC’s Hybrid Cloud solution along with some of the smartest people I have met. This has been, without a doubt, the best team I’ve ever been a part of. Thanks to all those that made this job so fun and also taught me so much.
While it was a very hard decision for me to make as I will miss my EMC and vSpecialist family, I am very excited to announce that I will be attending VMworld next week as a VMware employee. My new role will be within the NSBU where I will be an NSX specialist on the pre-sales side with an added focus on security and compliance. I’m really looking forward to digging in to the product and working with the awesome teams at VMware.
So I’m not going too far and am very excited to see my extended network of friends next week in San Francisco. I’ll be floating around the VMware booth, HoL, and sessions so please stop by and say Hi!
I’m a few weeks behind on posting this up due to travel, but I’m excited to announce that my latest course for Pluralsight, titled VMware vSphere Data Protection, has now been published. This course covers everything from the basics of backups/restores all the way to advanced features and demonstrations of vDP and vDPA. One of the nice things about this course was that I was able to make it mostly lab based, getting the foundational material in at the beginning and then spending the bulk of the course in a live environment.
You can check out the course at the following URL on Pluralsight’s website: http://pluralsight.com/training/Courses/TableOfContents/vmware-vsphere-data-protection
I’d appreciate any comments or feedback on the course and welcome any questions on the content after you go through it. I hope the the content is useful and that you enjoy the course!
Happy New Year everyone! This is just a quick post to announce the release of my second Pluralsight course, VMware vSphere Security, which was just published today. If you have a current subscription (or want to sign up for a new one), please check it out:
Hope the course is helpful and any comments/feedback/suggestions are welcomed.
What was supposed to be a afternoon of host memory upgrades, cluster re-balancing and DRS changes, as well as an upgrade from vSphere 5.1 to 5.1U1, turned into quite the troubleshooting exercise. A few people asked me on Twitter to document the experience so hopefully this post saves a few of you some time if this issue comes up. Here we go!
Before I started, the environment looked something like this:
- vCenter Server 5.1 installed on a Windows Server 2008 R2 standard VM. This same machine also had vSphere SSO, vSphere Web Client, vCenter Inventory Service, and vCenter Update Manager on it. All running 5.1 unpatched.
- vCenter Server database is stored on an external SQL 2008 server. vCenter SSO database is stored locally on the virtual machine within a SQL 2008 Express instance.
- 9 ESXi 5.1 hosts all in a single cluster with HA enabled and DRS set to manual
- Active Directory authentication was enabled on both the ESXi hosts and vCenter Server.
- AD/DNS/DHCP are hosted on separate physical servers
Given that I needed to power off the hosts to upgrade the memory and they all needed a reboot to patch to ESXi 5.1U1, this meant downtime. Additionally, the current cluster did not have EVC enabled and there are a mix of different processor families here, so we scheduled downtime for the entire environment to shutdown each VM and move the hosts into two, EVC-enabled clusters.
Now that we have some background, the first thing on my list was to upgrade vCenter Server to 5.1u1. Now if you’re not familiar with the Windows installer, when you run the autorun program that comes with the vCenter Server iso, you install the components in the following order:
- vCenter Single Sign On
- vCenter Inventory Service
- vCenter Server
- vSphere Client
- vSphere Web Client
- vCenter Update Manager
There is a simple install option which automates a lot of this, but that is not available for upgrades. It only works if you are doing a new install. So I proceeded with the install order and completed the SSO, Inventory Service, and vCenter Server pieces. Everything installed just fine. Next I updated the vSphere Client and after that installed I attempted to login to vCenter Server using active directory credentials. This is where things went downhill…
Active Directory authentication did not work. I verified AD was actually working properly so this was not the issue. Not being able to login with AD, I tried the default administrator@System-Domain account which let me in. I also updated the vSphere Web Client, in hopes that it would let me in, but the installer wouldn’t let me past the part where you connect it to vCenter SSO. Even though I was typing the correct lookup service URL and username/password, it would come back with “password incorrect or blank”. Now I’m not going to list out all my troubleshooting steps here as it was lengthy, but suffice it to say that I had a corrupt SSO installation/database. Database repairs failed, so my only option was to re-install SSO. And this is where the fun begins!
So after uninstalling vCenter SSO and attempting to reinstall, it came back with an error saying unable to re-create database users. Now this gave me a clue that uninstalling SSO doesn’t actually wipe out the current database configuration. So what you’ll want to do here is use the SQL Server 2008 Management Studio which should be installed on the VM to browse to the local database instance. The default name of the instance is VIM_SQLXP, so the full server name looks like: localhost\VIM_SQLXP or .\VIM_SQLXP
The next thing you need to do is delete both the database used for SSO and the database users. In my case I backed up the database before deleting if, for some reason, I needed something inside of it. The default database name is RSA. After deleting that, I deleted the two DB users: RSA_User and RSA_DBA. Once that was completed, vCenter SSO installed properly.
Now after re-installing SSO, you are left with an environment that is no longer linked to vCenter SSO. In my case, this meant that the vCenter Inventory Service, vCenter Server, and vSphere Web Client all needed to be repointed.
You will find the following VMware KB very helpful if you ever run into this issue: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2033620
The first step in my case was to repoint vCenter Server to the new SSO instance. You do this by performing the following steps (I’m assuming in all these steps that vCenter Server is installed in the default location and ports):
- Use Windows explorer and navigate to: C:\Program Files\VMware\Infrastructure\VirtualCenter Server\ssoregtool
- Locate the sso_svccfg.zip file and extract it to a folder here
- Open a command prompt and CD to that folder you just unzipped the files to
- Run the following command, updating your vCenter SSO URL, and user/pass as appropriate:
repoint.cmd configure-vc –lookup-server https://vc5.corp.com:7444/lookupservice/sdk –user “admin@System-Domain” –password “SSO_pw1!” –openssl-path “C:\Program Files\VMware\Infrastructure\Inventory Service\bin/”
If you try to start vCenter Server at this point, it will try to, but fail. You need to re-populate the certificate file names within the VPXD.conf after re-pointing to the new SSO instance. The following VMware KB describes this and I’ve also included the steps below: http://kb.vmware.com/kb/2048753
- Locate the vpxd.conf file which is located in:
- Create a copy of this file in case anything goes wrong. Now open this file in Notepad
- Search for “null” and you’ll see two fields that look like this:
- On both of these fields, change the null values to match below
- Save the file and close it
Now restart both the vCenter Server and vCenter Management Webservices services. Your vCenter Server should now be linked to the new SSO instance and should start up properly.
Next we need to re-link the vSphere Web Client to the new SSO instance. To do that, follow this procedure:
- CD to C:\Program Files\VMware\Infrastructure\vSphereWebClient\scripts
- Run the following command, replacing your vCenter Server name, admin username, and password:
client-repoint.bat https://vc5.corp.com:7444/lookupservice/sdk “admin@System-Domain” “SSO_pw1!”
Restart the vSphere Web Client service and you should be able to login to v Center Server with the user admin@System-Domain and the password you specified during installation. The default URL for the vSphere Web Client is: https://vc5.corp.com:9443/vsphere-client/
At this point, you should be able to login, but you should now see a message about vCenter being unable to connect to the inventory service. We’ll tackle that next…
To fix the Inventory service and re-link it to SSO, we need to perform a similar process:
- Open a command prompt and CD to C:\Program Files\VMware\Infrastructure\VirtualCenter Server\isregtool
- Run the following command, replacing your vCenter Server name in:
Now you can restart the Inventory Service and it should be re-linked to SSO. You’ll need to restart vCenter Server as well to pickup this change. We’re almost there!
After vCenter Server restarts, login to the vSphere Web Client using the admin@System-Domain credentials again. Although we have fixed all the links to SSO, in my case the Active Directory groups and permissions had been blown away, however the AD identity source was still there. So to add those back, use the following steps:
- On the left-hand panel, click on vCenter Home
- Click on vCenter Servers until Inventory List. Then click on your vCenter server name
- Click on manage along the top row and then choose the Permissions subsection
- Click on the + to add a new permission
- In my case, I was granting the AD group Domain Administrators the Administrator role in vC. So if you click the add button on the left hand pane, it will let you select your domain and then you can search for and add the group.
- Choose whichever role you would like to assign and make sure to do propagate to children
Once that is complete, log back out of the web client (or the vSphere client) and you should be able to login using Active Directory credentials! After that, in my case, I updated vCenter Update Manager and was able to proceed with my host updates.
So that was a bit of a long post, but I wanted to outline what happened and all of the steps I had to go through. Hopefully this helps out one of you if you ever hit this upgrade bug! This whole afternoon made me appreciate the SSO implementation in vSphere 5.5, which was completely re-written, as it is much easier to install and administer!
It was a whirlwind October in my new role as a vSpecialist and I didn’t have a chance to write up a post about my new course being published. So a few days late, but I wanted to let everyone know that my first course that I wrote/recorded for Pluralsight was published last week. This course is designed to be a primer on vSphere networking and give real-world lab scenarios and tips for the vSphere administrator. It aims to hit the most important features and provides foundation knowledge for working with and configuring networking in a vSphere 5 environment. It also covers some networking 101 basics to lay some groundwork.
If you have a PluralSight subscription, please check it out:
I welcome any comments, feedback, or follow-up discussions on the topic and the course. I’ve also started work on my next course, so look out for that one coming soon!
This post is a bit of a resurfacing as I have been disconnected from the community lately due to the craziness that has been work and my personal life for the past 6 months. I’m back now though and looking forward to reconnecting with everyone, the community, and the technology!
It’s hard to believe that I’ve been at RSA almost 2.5 years now in my current role as a technology consultant. I have learned a lot, worked with some great people, and most importantly, spoken with hundreds of EMC/RSA customers. I’ve also been on the leading edge of security technologies both for authentication and advanced analytics and threat detection/prevention. An exciting space to be sure! A big thanks goes out to all those on my direct team and countless others that have helped me throughout this journey.
All that being said, a recent opportunity has come along that I could not pass up. I am very excited to announce that starting October 1st, I will be moving over to a new role within EMC as a vSpecialist for the Northeast region. I am extremely excited to get back to my virtualization and storage roots, focusing on everything EMC and VMware. I am also honored and proud to be joining a team of this caliber. Between the technology, the customers, the partners, and the internal teams that I will be involved with, I can’t wait to get started. I’m also looking forward to participating in many more local VMUG’s and other events and hope to see everyone there!
As part of this new role, I will also be resuming my journey down the VCDX road and aiming for a defense at PEX 2014. So stay tuned for updates, blog posts, and more information around that as things progress.
More to come, but just wanted to share the news with everyone right now!
This post is a little bit overdue, but I wanted to report on my experience with the VCAP5-DCA exam that I took at VMworld US. The exam, much like its DCD counterpart, is 210 minutes. This one has 26 lab questions that you have to complete and each question can have multiple parts. There is partial credit so read each question (and its parts) thoroughly! A passing score is 300.
For studying, the first thing you’ll want to do is download the official blueprint from VMware:
This was a tough exam just in the sheer amount of tasks you are asked to do in a short amount of time. It basically compresses what you know as a VMware administrator and troubleshooter into 3.5 hours. Repetition of tasks on the blueprint and knowing where to go really helps here! If you don’t have the experience from your job, make sure you lab a lot here. You need to be quick! I ran out of time on the exam as I was running a command. I’m not sure it actually finished when the timer hit zero, but I went up until the last second here.
This exam definitely would have benefited from a dual monitor setup or even alt-tab! You end up switching between the exam questions and then an actual live terminal which is a jump box of sorts. This has all of your vSphere clients, remote desktop connections, SSH, and other connectivity tools. Additionally, the vSphere documentation set is on this VM. Familiarize yourself with the tools on this VM when you start and I would also recommend opening connections to all your important sources. This will save you time later on!
I’ve read mixed thoughts on looking at the documentation during the exam. I’m taking middle ground on this. If you know what to do or how to do something, but need to reference the syntax of a command, open the PDF! This happened to me twice and I actually planned on this before. There are some longer commands that I knew sometimes I don’t remember the exact argument order on, but I knew where to look! You don’t want to spend much time in the docs, but if it’s just for a quick reference and you know where to go, it can be very beneficial. However, if you don’t know how to do an objective, you will waste a lot of time going through the PDF’s trying to find a more general solution. Don’t fall into this trap!
One thing I did notice is that resources on the jumpbox VM were a little slow. I had to close down a few of the PDF’s and I had some duplicate RDP windows which I also closed, this helped to speed up performance. Thankfully I didn’t have any interface crashes this time like on DCD, but a few times switching between questions/lab was a bit slow and it made me a little nervous!
Overall I enjoyed the exam and thought it was one of the more fun formats for cert tests that I’ve taken. It’s definitely a mad rush to the finish, but I really liked how well the live lab format worked and the way you can structure the tasks around a scenario (building out a new datacenter, deploying a new app, etc.).
You will walk out of this one tired! I was mentally drained after focusing and running through the tasks over the 3.5 hours. It is a demanding test for sure!
Tips and Tricks
- Know each way (PowerCLI, command line, GUI) to do a task on the blueprint, but also have your preferred way. Yes, some things can only be accomplished via GUI or command line, but if not, do it your preferred way and do it fast!
- A great tip suggested by Tim Antonowicz (@timantz) to me at VMworld was to go through all the questions when you start and write down their objectives. Some tasks will build upon others you’ve previously completed and there are others that require you to wait while something completes. This way, you can group tasks together or know what you can move onto next while waiting.
- Be able to troubleshoot quickly. I’ve had practice with this in a production environment when your manager is breathing down your neck to get a server back up, but know where to look when things go wrong! Even if you haven’t had the job experience, do this in your lab. I’d even suggest letting a friend remote in and break a few things for you, then methodically go back and investigate/fix them.
- Write down the password for your system account. The password will be the same across all the different resources and it’s also displayed on the desktop. However, I found a few instances where I wasted time by having to move a window out of the way to see the password. If you’re like me, the stress of the exam makes you forget the password or you try typing in your own lab password which doesn’t quite work. Easy solution, write it down on your whiteboard!
- Make sure you know the other components (networking, storage, etc.) that can also affect a VMware environment. Familiarity with these and knowing a lot of the corner cases or lesser-used features will help you here!
- Chris Wahl (@chriswahl) put together an awesome DCA study sheet that you can check off each objective as you complete. This and the blueprint are a great way to start your studies:
- Gregg Robertson (@GreggRobertson5) has a good DCA5 objectives overview: http://thesaffageek.co.uk/vsphere-5-study-resources/vcap5-dca-objectives/
- Ed Grigson (@egrigson) has a good overview on changes between the DCA4 and DCA5 exams: http://www.vexperienced.co.uk/2012/04/12/vcap5-dca-whats-new/
- Jason Langer (@jaslanger) has great info up on different objectives: http://www.virtuallanger.com/vcap-dca-5/
- Josh Coen (@joshcoen) also goes into really good detail on the different objectives: http://www.valcolabs.com/vcap5-dca/
- Tim Antonowicz (@timantz) has a great post up on exam tips (including the one mentioned above) : http://whiteboardninja.wordpress.com/2012/09/19/vcap5-dca-testing-strategy-and-tips/
- As always, the crew at ProfesionalVMware put on a great series for VCAP5-DCA. Definitely watch/listen to all of the episodes to help prepare: http://professionalvmware.com/brownbags/
- Bas Raayman (@basraayman) has a good post up on his exam experience: http://basraayman.com/2012/08/30/vcap5-dca-my-test-experience/
One word. Patience When I finished up the exam, a note said results would come within 15 business days. A little over 40 business days later and mine finally came! I was very excited to receive the e-mail saying I had passed! It looks like there is a bit of a backlog over at the certification team right now though, so don’t be surprised if your results are delayed.
The Advanced track of the VMware datacenter virtualization certification ladder has been a fun one! I really enjoyed this exam and DCD. Now it’s onto VCDX, which I hope to be defending a defense at PEX in February 2013!