Recent Articles

VMM Error 23008 - The VM Server1 cannot be migrated to Host Host2 due to incompatibility issues

I was recently doing a Live Migration of some VMs from a Windows Server 2012 Hyper-V cluster to a Windows Server 2012 R2 Hyper-V cluster using Virtual Machine Manager 2012 R2 UR4 and got the following error message:

Error (23008) 

The VM Server1 cannot be migrated to Host Host1 due to incompatibility issues. Virtual machine migration operation for Server1 failed at migration source Host1.

Recommended Action

Resolve the incompatibility issue and try the migration again.



There are other reports that state that they had the same issue. One solution (from Benedict Berger) pointed to a conflict with the Hyper-V virtual switch name of the source and destination host. In my situation, the name of the virtual switch on the destination host was different then the name on the source host.

When doing a live migration between hosts, VMM gives you the possibility to select the VM Network and the virtual switch of the destination host. However, choosing a correct switch did not make a difference.


Renaming Hyper-V Virtual Switches, specially when using Hyper-V Clusters and Logical Switches is not really fun. But there is another option. I created a Private Switch on the destination host using the same switch name as on the source Hyper-V host. After that, I was able to migrate the VM between the 2012 and 2012 R2 Hyper-V cluster.

Once the VM was on the other side, the VM lost it’s network connection. Checking the VM network configuration, every thing was configured correctly and the correct Hyper-V switch was configured on the destination host. To get the VM connected again, I had to select a standard Hyper-V switch for the VM and apply the setting. Checking the VM network configuration again, I saw that the correct logical switch was selected again. Strange… However, the network connectivity of the VM was restored.


System Center Service Manager 2012 R2 – Edit Service Offering is blank

The System Center Service Manager 2012 R2 Console with UR4 still has issues when running in full screen mode. When you try to edit a service offering in full screen mode, you will see an blank dialog box:


When you resize the Console, you will be able to see the content of the service offering.


Connection issues to Azure Operational Insight

When configuring the connection from SCOM 2012 to Azure Operational Insight, there are some steps that are not that obvious. SCOM will alert you that the connection has issues, but it’s not that clear why. Here are some alerts I noticed when I initially configure Azure Operational Insight:


Description: Advisor is failing to send operational data to service.

A proxy server or your network settings might be misconfigured or there is a temporary network problem. Check the Knowledge Tab for possible resolutions. The context of where the alert appears might indicate whether the problem is due to a proxy server configuration or some other


Alert: Advisor failed to download the latest advisor knowledge management packs

Description: Event Description: Failed to synchronize the latest Management Package information from Advisor Cloud service. Wait for the next cycle to retry


Alert: Advisor connector failed to send analysis data to management server

Connector Framework Alert Write Action Last modified time


Description: One or more data batch was dropped dropped after exceeding retry time.

This indicates that some data was not uploaded to Advisor for some time and was lost ; there should be additional Alerts pointing at the root cause as to why data was failing to be sent in the first place. Typical reasons include but are not limited to problems with DNS name resolution, failed SSL connection, network timeout and proxy settings. Check the Knowledge Tab for possible resolutions.

SatyaVel from Microsoft postet some very use information on how to troubleshoot Azure Operational Insight connection issues here:


Upgrading a Windows Server 2012 Hyper-V Cluster to Windows Server 2012 R2

Ok, it´s a bit late for such a post since Windows Server 2012 R2 is out for over six month now, but still here are some rough steps from the upgrade of my demo 2012 Hyper-V cluster to Windows Server 2012 R2.

High Level Steps

To do this, I need to remove a existing node from the current 2012 cluster, install 2012 R2 on it and build a new single node cluster. Then move the VMs from the old cluster to the new cluster. Once finished, destroy the old cluster, install 2012 R2 on the free node and join it to the R2 cluster. Since I use VMM, I will try to leverage VMM as much as possible to make the transition easy. These steps are done in VMM:

  • Add the R2 server as a host to VMM, which also installs the Hyper-V role
  • Setup the virtual switch using a logical switch in VMM, which configures all the converged network settings for me
  • Setup the new R2 cluster using the VMM cluster wizard

It´s important to notice that you need additional shared storage for the new cluster. At least one drive for Quorum and one for a CSV disk.

The cluster notes use a converged network of three NICs for Live Migration, Cluster CSV, Management and VM traffic. iSCSI is connected via separate physical NICs.

In this environment I am also using System Center Virtual Machine Manager 2012 SP1. So this needs to be upgraded as well. This post captures the high level steps I performed:


Upgrade of System Center Virtual Machine Manager 2012 SP1 to 2012 R2

  1. Uninstalled VMM 2012 SP1 using Retain Database from the VMM server
  2. Uninstalled the ADK 8 from the VMM server
  3. Upgraded the VMM server to Windows Server 2012 R2
  4. Installed ADK 8.1 on the VMM server (Deployment Tools and Windows Preinstallation Environment features)
  5. Install VMM 2012 R2 using the retained VMM database
  6. Follow the post-upgrade tasks found here:


Fresh install of Windows Server 2012 R2 on the first cluster node (HV01)

  1. Evicted cluster node1 from the cluster (in my case the rest of the cluster run on just one node)
  2. Remote the cluster node1 from VMM
  3. (Remove the virtual switch and the virtual management OS network adapter from the host)
  4. Started Windows Setup from the DVD
  5. Renamed the network adapter to make sense and set up TCPIP correctly on one of the NICs
  6. Renamed the server and joined the server to the domain.
  7. Run Windows update
  8. Configured iSCSI NIC adapter  (for the new cluster

Bring first cluster node (HV01) under control of VMM

  1. Added the new Windows Server 2012 R2 cluster node to the same VMM server as the other node, so the previous VMM server. Now, I have a cluster with one node and a non-clustered Hyper-V node.
  2. Configured the networking on the host. Since I used logical switch configuration I could apply the already created logical switch to the new Hyper-V host quite easily. First I configure the NICs in the team, then I configured the virtual network adapters:

    image         image

    In the Logical Network view I can now see that the HV01 host has been configured with the Logical Network configuration and that the host is compliant.


    On the host itself, there is now a NIC team and the virtual network adapters as specified in the Logical Switch.

    image     image
  3. Note, that although I name the host management adapter in the logical switch configuration “HostManagement”, in the Network Connection panel it´s named vEthernet (Converged Switch), which is the name of the switch not the adapter. The other adapters like ClusterCSV or LiveMigration are named according to the name that I specified in VMM. This is by design, I guess.

    Also, it´s interesting how VMM 2012 and VMM 2012 R2 name the team adapter in Windows. In VMM 2012 the team adapter is named “ConvergedSwitch6b858dbe-fefc-4ecc-994e-e1587551422b” in VMM 2012 R2 the adapter is simply named “ConvergedSwitch”. I always thought that the VMM 2012 naming was a bit strange.

Create a New Hyper-V Cluster from VMM

  1. I created two new LUNs on the storage for the new cluster (called R2Cluster) that I am going to create using HV01.
  2. Connected the HV01 host to the iSCSI target and set up a LUN for the Quorum and a LUN for VM storage (I called it CSV3)
  3. Run the create cluster wizard from VMM. Add HV01, the R2 server to the new cluster. Configure a IP address for the cluster and configured the two storage LUNs for the new cluster. I only marked the CSV drive as CSV. The other drive will be used as a witness drive.
    image        image   image
  4. After this I have now two clusters in Hyper-V, the old cluster (HVCLUSTER01) and the new cluster (R2CLUSTER):
  5. On the new cluster I renamed the network according to there usage:


Copy the VM Role from the current Cluster to the new Cluster

  1. I went to the cluster node of the new cluster and started the cluster manager. I used the Copy Cluster Role link to start copying the VM roles.
    image      image

    The copying of the roles is very quick. At this point the roles are still running on the current Cluster.
  2. Now I stopped the VMs on the current cluster. I put the CSV where the VMs are located in Offline mode. Next I had to connect the new cluster node to the iSCSI Target where the CSV was located. Once that was connected, I brought the CSV online on the new cluster node.

Once all the rules have been moved to the new cluster, I started removing the cluster service from the old cluster and clean up all the relevant VMM and Hyper-V components. Then the old cluster node was reinstalled with Windows Server 2012 R2 and joint the new R2 Cluster.


VMM 2012 R2 - Error (2941) / Error (2927)

I recently got the following error in a VMM job when creating a new VMM template or when deploying a VM from a VM template.

A Hardware Management error has occurred trying to contact server hvhost02.demo.lab  .

WinRM: URL: [], Verb: [INVOKE], Method: [GetError], Resource: [{26F5564E-4816-4F87-90D5-913B12E9C48E}]

Unknown error (0x803381fa)

Recommended Action
Check that WinRM is installed and running on server hvhost02.demo.lab. For more information use the command "winrm helpmsg hresult" and .

Or following error:

VMM is unable to complete the request. The connection to the agent on machine has been lost.
Unknown error (0x80072efe)

Recommended Action
Ensure that the WS-Management service and the agent are installed and running and that a firewall is not blocking HTTPS traffic.

To fix this, first check the normal network connection issues to the VMM server and the Hyper-V host. Use following link to further dig into the issue. One think that is not mentioned in this KB is the required VMM certificates. In my case the issue was a missing/deleted certificate on the VMM server itself. Following blog post explains who to correct this:

Hope that helps other that experience similar issues.


Windows Azure Pack–Creating a Plan for VM Clouds

After you set up the connection from the Windows Azure Pack to the Service Provider Foundation and the the Virtual Machine Manager (VMM) you need to create a plan that your customers can subscribe to. The plan defines the capacity boundaries and specifies on which VMM cloud the VMs should be placed.

To create a plan start the WAP Admin Portal go to Plan and click on Create a new Hosting Plan


Enter a name for the plan and select the previous configured VM cloud service.


Finish the Plan wizard. Now the created plan appears on in the plans list.



Next the plan needs to be configured. Click on the plan name, here VM Server Plan. This will open the dash board for the plan.  In the middle of the page you find the plan services:



Click on the name of the plan service, in my example Virtual Machine Clouds. This will open the virtual machine clouds configuration page where you select the VMM server and the VMM cloud that you want to include in the plan.




In the usage limit section you can configure the usage limit for the plan:




In the networks section you need to add the network that the plan will use:



In the hardware profile section add the hardware profiles from VMM that you want to offer in the plan:



In the template section add the template you want to make available. You can offer multiple templates:


Finally you can add additional settings that define what action on the VMs can be executed:



Now save the plan.


The final step in your virtual machine service is to create a plan where you configure the capacity and capability settings of the virtual machine you want to offer to your users. You should name the plan in a way that is self describing to your users or customers to help them choose the right plan. Plan should make it easy to your customers to find the right service for their needs.


Windows Azure Pack–Setting up the VM Cloud Service

Once the basic WAP installation is completed, the various services need to be configured. In the post I will look at the VM Cloud Service configuration.

The VM Cloud Service requires some additional installations and configurations. The following three steps are required.

  1. Service Provider Foundation Installation and Configuration (SPF)
  2. System Center Virtual Machine Manager Installation and Configuration
  3. WAP VM Cloud Service Configuration

Service Provider Foundation Installation and Configuration

The Service Provider Foundation requires a SQL Server. So make sure you have one available where you can put the SPF database. Before you install SPF you need to enable some Windows features and you need to install two prerequisites. To enable the windows features use the following power shell command:

Install-WindowsFeature Web-Server, Web-WebServer, Web-Common-Http, Web-Default-Doc, Web-Dir-Browsing, Web-Http-Errors, Web-Static-Content, Web-Health, Web-Http-Logging, Web-Request-Monitor, Web-Http-Tracing, Web-Performance, Web-Stat-Compression, Web-Security, Web-Filtering, Web-Basic-Auth, Web-Windows-Auth, Web-App-Dev, Web-Net-Ext45, Web-Asp-Net45, Web-ISAPI-Ext, Web-ISAPI-Filter, Web-Mgmt-Tools, Web-Mgmt-Console, Web-Scripting-Tools, NET-Framework-45-ASPNET, NET-WCF-HTTP-Activation45, ManagementOdata, WAS, WAS-Process-Model, WAS-Config-APIs

Then install the following software.

Create two users that are required for the SPF installation:

  • domain\srv_spf_app (domain user account used for the SPF application pool)
  • spfserver\srv_spf_local (local user on SPF server used for the WAP connection)

Create four domain groups:

  • spf_Admin
  • spf_Provider
  • spf_VMM
  • spf_Usage

Once this is all set up start with the SPF installation. SPF can be installed from the System Center 2012 R2 Orchestrator setup program. Select Service Provider Foundation under Service Management and follow the wizard.

  • Define the database settings for the SPF database
  • Define the certificate for the SPF service. For a lab environment you can use a self-signed certificate.
  • Configure the web services. Use the srv_spf_app user account for all application pools. And use the SPF domain groups for the different web service. 
    image   image   image   image
  • After the installation is completed, check the application pools in IIS on the SPF server. The :


Add the local srv_spf_local user account to the following local groups on the SPF server:

  • spf_Admin
  • spf_Provider
  • spf_Usage
  • spf_VMM

System Center Virtual Machine Manager Installation and Configuration

Now to the Virtual Machine Manager configuration. If you have not installed Virtual Machine Manager yet check out the Microsoft installation guild for that. For the configuration of VMM do the following:

  1. Add the domain srv_spf_app user account to the VMM administrator role using the VMM console.
  2. Create or have a VMM Cloud and do not select any of the Capability profile. Assign a logical network to the cloud. Assign a VMM Library to the cloud.

  3. Create or have a VM template. The OS image of the template must have the remote desktop connection enabled. The VM template must have a selected Guest OS Profile and no Capability profiles are selected.

WAP VM Cloud Service Configuration

The last thing to do is to configure the VM Cloud service in the Windows Azure Pack. Let’s start with the following:

  1. Log in to the WAP Admin portal
  2. In the VM Clouds Window select Register System Center Service Provider Foundation. Enter the URL of the SPF server, the local user account (srv_spf_local account on the SPF server) and it’s password. 


    Check that the connection has been registered successfully:

  3. Once the SPF connection is successfully established, add the VMM server to the VM Cloud. In the Clouds tab click USE AN EXISTING VIRTUAL MACHINE CLOUD PROVIDER TO PROVISION VIRTUAL MACHINES and enter the name of the VMM server.


    After the connection is completed, you will see the VMM server and it’s clouds in the VM Cloud.


So far, we have configured the Windows Azure Pack to connect to the System Center Virtual Machine Manager. In the next post we will look at the creation of a Windows Azure Pack plan for VM Clouds. The plan will define what resources we what to provider to our WAP customers.


The Service Provider Foundation is a service that sits between the Windows Azure Pack and the System Center Virtual Machine Manager that allows the Windows Azure Pack to talk to Virtual Machine Manager. To make this happen we need a Service Provider Foundation service account that has administrator rights in Virtual Machine Manager and the Windows Azure Pack requires a local account on the SPF server to have access to the SPF service.


Windows Azure Pack – Installing the WAP Basis

As we have see over the last posts on Windows Azure Pack, the whole Windows Azure Pack system consists of multiple components. There are the components that face our consumer or customers. There are the high privileged components and there are the Provider components and supporting components like the database and the federation service. In the post I want to look at installing the initial set of components that make up the basis for the WAP system.

The WAP system is installed by using the Web Platform Installer. The Web Platform Installer is a tool where you can select which components you want install and the tools is automatically downloading all the required software and is also installing and configuring the required settings. There are only little configuration settings that you need to specify for your environment. You can download the Web Platform Installer here:


Now, when you install the WAP system, you have basically two option. You can either install all initial components on a single server, or you can install each components on separate servers. The single server option is only for testing or proof of concept installations.

Before you start the Web Platform install, make sure that you have a Windows Server 2012 R2 server where you can install the WAP components and that you have a SQL 2012 database server that WAP will use.

To install all components on a single server, select Windows Azure Pack: Portal and API Express from the Products selection.


The Web Platform installer will also download and install all the prerequisites for the WAP system. Once started it shows the progress of the download and the installation.

image     image

Once the wizard is completed, you will be redirected to a configuration page. There you need to specify the parameters for the SQL server. You can use SQL authentication or Windows authentication for this step. However, other providers will only use SQL authentication. So make sure that SQL authentication is also enabled on the SQL server.

The passphrase is used to securely store the WAP data in the database. Make sure that you note down the passphrase. You need the passphrase on each server where you install one of the WAP components. Note, if you loose the passphrase, there is no easy way to recover the data in the database, and probably no hard way either.


After the wizard is completed, you must sign out and sign in again so that your user can be configured correctly with WAP.

Then, open internet explorer and use the admin port 30091 for your local server, e.g.: https://wap01.demo.lab:30091

After a few introduction dialogs you will land on the main administration portal page. It should look like this:


On the left hand side of the page you see the menu for the services and settings. The ALL ITEMS sections shows you all the services that are created, which is empty because we have not configured any service yet. Below that you see the providers that are already installed out of the box. There are the Web Sites service, the VM cloud service, the Service Bus service, SQL server service and the MySQL server service. It’s a bit strange that Microsoft is implementing MySQL Servers as a provider out of the box with WAP.

AUTOMATION, PLANS, USER ACCOUNTS and USER COSTS are internal WAP services that help to run and configure the WAP system.

When you look at the providers that are installed, you see that there is still some configuration work to be done before they can be used. Here’s the example of the VM cloud service:


Now, check the database server where you installed the WAP databases. There you will find a couple of WAP databases:


Also start the IIS manager console on the server where you installed the WAP system. There you can see the web sites for the components we discussed:


In the next post I will look at configuring the VM clouds service.


Thanks to the Web Platform Installer method, it’s really easy to install the WAP system. There are two methods, the single server installation and the distributed installation. The single server installation however is only for lab environments. After the initial installation there are a couple of service providers out of the box implemented. However, these providers still need to be configured first.


Windows Azure Pack–The Components

In the previous posts about the Windows Azure Pack, we looked at what the Windows Azure Pack is and how it compares to Windows Azure.

Now, I am going to look at the components that make up the Windows Azure Pack. The components are the pieces of the Windows Azure Pack software that you need to install. As we previously saw, there are internet facing services that the WAP offers. Because of this, Microsoft designed the WAP in a way that allows to separate it’s services into components that can be place in different parts of you network. This is done mainly for security reasons. If you don’t offer the WAP service directly to the internet, you could also put all the components into your intranet. But still for scalability reasons, you would place the components onto multiple server. Here’s a diagram of the components:



Let’s briefly look at each of these  components:

SQL Server: Of course you will need a database server that holds all the management information and configuration of you WAP environment. There will be multiple databases that the WAP setup will create for you.

WAP Admin Site and WAP Admin API

The WAP Admin Site and WAP Admin API allows the WAP administrator to administer and to maintain der WAP infrastructure. There the Admin would add the VM clouds, the web site clouds, the mySQL or SQL server services and would manage them via Site and the APIs. The admins create bundles of these services and put them into plans, which they can offer to consumers as subscriptions. The plans contain the services with their restrictions and quotas.

The WAP admin Site is the WAP Admin portal talks directly to the WAP Admin API and exposes this management functionality.

Tenant API

The Tenant API provides the functionality that the Tenant Site needs to execute the required management tasks. It’s also used to create the subscriptions for the tenants, where the they define what services they want to subscribe to. Therefore, it’s placed into the high privileged service area.

Tenant Site

The Tenant Site basically is the portal for the end user that allows the end user to subscribe to your plans and to create and manages their  resources. This Tenant Site portal talks directly to the Tenant API in order to do this.

Tenant Public API

The Tenant Public API allows API access to create and maintain resources in the realm of a particular subscription. So you can use these APIs to create applications that manages your resources directly from the application without the need to use the Tenant site.

Tenant Authentication Site

The Tenant authentication Site is a build-in mechanism to allows ASP membership users to authenticate against this site. You can simple create users based on email addresses for the authentication into the WAP.


So far, we have looked at the internal components of the WAP system. These components build the framework for the workloads or services you want to offer using the WAP. In the previous posts we have looked at the different services that you can offer with WAP which are called Providers in WAP terms. In the diagram above, I listed Web Sites, Service Bus and VMs. Some of these Providers are installed out-of-the-box when you install the WAP system, other must be installed manually.

Identity Federation

Identity Federation is the key components to make the WAP enterprise ready. With ADFS you can plug in different identity providers that can be used for the WAP system. In an enterprise, you would use Active Directory so that your customers can use their Active Directory credentials to log in to the tenant site and the administrators to log in to the admin site.

Scale-out and High Availability

Now that we have very briefly looked at all the components that make up the WAP system, there is still the question about scale-out and high availability. Generally, you should scale-out the components that gets the most traffic. These are normally the tenant components. Additionally for high availability, all tiers of the WAP system should be made highly available if the services are critical to your business.


There are many components to the WAP system. Theoretically you can install all of them on the same server. However, scalability reasons, you should distribute them onto multiple systems. Additionally, if you want to provide WAP services to the internet, you would separate the tenant components from the admin components. If you want to provide WAP services only to your internal enterprise customers, you want to set up ADFS and connect your Active Directory to the authentication site.

In the next post I will look at the installation of the WAP system.


Windows Azure Pack–Architecture Comparison to Windows Azure

In the article I will look at the high level architecture of the Windows Azure Pack compared to Windows Azure. This will help to understand how Microsoft was able to provide their vision of 1 consistent Cloud OS platform. So here’s a nice picture that explains this:


On the left hand side you see the Widows Azure stack and on the right hand side the Windows Azure Pack stack. Let’s look at Windows Azure first.

Windows Azure

At the bottom of the Windows Azure stack is the Windows Azure fabric, the hardware, OS, and services that Windows Azure runs on. Microsoft does not provide too much information how this look like. However, Microsoft started discussing their cloud server hardware designs:

We can assume, that Microsoft is using some form of Windows Server with Hyper-V to power the Azure cloud. Microsoft does say that the Hyper-V version that is in Windows Server 2012 R2 is the same version that is used in Windows Azure. This does not really surprise since most of the new features in Hyper-V seems to be created for Windows Azure on purpose. Windows Azure also uses the same format VHD disk format then Hyper-V which allows you to copy VHDs that you created on-premises to Windows Azure and use them there (VHDX is not jet supported).

On top of the fabric sit the service and workloads that Windows Azure offers: Web Sites, VM IaaS, SQL, Service Bus and many more. We don’t know how these are implemented and if these run on standard or customized Microsoft products. However, it does not really matter. What we do know is, that Microsoft is updating their services and workload quick frequently so.

The next layer is the important one: The Service Management API layer. This is a REST API that allows you to get programmatic access to the services below. On top of that sits the Windows Azure Self-service portal. This is the portal that you see when you subscribe to Windows Azure and where you can consume the different Azure services. The portal allows you to configure and manage the available services.

Windows Azure Pack

On the Window Azure side, the foundation for the Windows Azure Pack is of course Windows Server with Hyper-V and System Center. That’s why Windows Server is also called the Cloud OS. Many of the new features in Windows Server 2012 and 2012 R2 are specially target to support cloud functionality. The System Center tools provide the fabric management and monitoring that is required for the services above that layer.

The service and workload layer reveals the familiarity of the Windows Azure Pack to Windows Azure. The Windows Azure Pack offers a subset of the services and workloads that Windows Azure offers. So far there are Web Sites, VMs, SQL and Service Bus. But that’s not it, there is room for more. The WAP is build in a way that allow you to plug more and more services into the Windows Azure Pack.

Now, the Service Management API is where it get’s interesting. The Service Management API is an implementation of the Windows Azure Service Management API that provides a consistent consumer API that talks to the WAP fabric underneath. Event if the services running in the WAP are not implemented identically as in Windows Azure, the access through the Service Management API is consistent. The Service Management API abstracts the service running underneath. Having this Windows Azure consistent API allowed the reuse of the Windows Azure portal in the Window Azure Pack. It’s almost the same portal as used in Windows Azure. Without this consistent API I would not have been possible to move the Azure portal over to the WAP.

The portal in the WAP looks and feels very identical as the portal in Windows Azure. However, as a service provider you are still able to adjust the portal the way you want it. Or if you don’t like the portal at all, you can use or build your own portal and use the required Management Service APIs.

Additionally to the end user portal view, there is a provider portal view. The provider portal allows the WAP administrator to configure the WAP infrastructure and to define offers (or in WAP terms: plans) that is assigned to an end user.


Looking at the high level architecture of Windows Azure and Windows Azure Pack, you can see that the foundation of the two is different, however the closer you go to the end user experience the close the two become. Most importantly, it’s not just about the user interface, but also about the management APIs that will allow the transition of the offered service and workload between the two. I would expect, that Microsoft is building more and more service and workloads into WAP and that the consistency between the two is getting closer and closer, even in the fabric level. I would not be surprised if the WAP will sometime be a role in Windows Server.