Warning – this post is over 3800 words long and perhaps should have been split into a series – however I felt it best to keep it together – Make sure you have a brew (or 2) to keep you going throughout reading this
In this post we will be looking at how you can build a VM Lab environment from pretty much scratch. This maybe for testing SharePoint applications, SQL Server, Exchange or could be for additional peace of mind when deploying troublesome patches.
Our requirements for this include
- Machine capable to Run Client Hyper-V – Needs SLAT addressing (most machines released in last 3 years are capable of this)
- Windows 8.1 / 10 / Server 2012R2 / Server 2016 TP* – In this post I will be using Windows 10 build 14925 – ISO download is available from here
- If using Windows 8.1 then you will need to install PowerShell PackageManagement – you can use the script in my previous post to do this as detailed in here
- A Secondary/External Hard Drive or Shared Drive – this is to store all Lability Files including ISO’s, Hotfixes & VHDX files
Where do we begin?
Obviously you need to install your version of Windows as detailed above and once you have done this you can crack on!
Time Taken – ??? Minutes
However as mentioned I’m going to Use Windows 10 – This is just personal preference and is for my ease of use.
As you hopefully know by now Windows 10 comes with WMF5 and therefore we have PackageManagement installed by default. We will use this to grab any PowerShell Modules that we need from the Gallery. I personally have a Machine Setup Script that lives in my Onedrive as you can see below. As this is a Windows 10 Machine I am logging into it with my Hotmail credentials – this then means that I am able to straight away pick the folders that I want to sync to this machine (joys of the integrated ecosystem)
This takes about 5 minutes for OneDrive to finish syncing and then we are ready to go onto the next step.
Time Taken - 5 Minutes
In this stage I will Open ISE with Administrator Privileges – this is required as I need to change the Execution Policy from Restricted to RemoteSigned as well as run other scripts that require elevation.
Once I have done this I can move onto the next step. This includes setting up my PowerShell Profile and Environment Variables and then setting up all the required functionality for me to continue working on this new machine.
This includes setting up the ability to install programs via Chocolatey like VSCode & Git and installing Modules from the PowerShell Gallery a few examples being ISE_Cew, ISESteroids, & importantly for this post Lability . Also It is worthwhile to note that at this point I am not downloading any DSC Resources as part of my setup script – this is because we will cover this later on as part of the workings of Lability.
As an additional note it is worth mentioning that the Version of Lability at the time of writing this article is 0.9.8 – however this is likely to change in future with more features being added as required. If you have a thought or suggestion (or issue then head over to the Github Repo and add your suggestions / issues.
I am also in this script enabling the Hyper-V Windows Feature to enable me to carry on with this Lab. I then initiate a System Shutdown. Overall this whole section takes maybe about 10 minutes to complete & yes I intend to build this as a DSC Resource in the near future, however it is worth while to note that Lability has a Function that will ensure that the Hyper-V feature is enabled & your are not awaiting a System Reboot for you – more on this a little later on.
Time Taken - 15 minutes
Once the reboot has completed we can then get on with the Lability bits and that is the real interesting part of this post.
Lability Functions
Lability has 38 public functions and 6 Aliases as can be seen below.
I wouldn’t worry too much on the aliases as these are built in for continued support from prior versions of the Lability Module and will likely be removed on the 1.0 release.
We will be using a number of these functions throughout and as is always best practice have a read of the help for the functions and Yes they do include some great comment based help.
There are a number of additional private functions in the Lability module that have comment based help too but again I wouldn’t be worrying about these too much, unless you need to do a lot of debugging or want to help add to the module.
The Key Lability Functions that you will need are and likely in the below order
- Get-LabHostDefault
- Set-LabHostDefault
- Reset-LabHostDefault
- Get-LabVMDefault
- Set-LabVMDefault
- Reset-LabVMDefault
- Start-LabHostConfiguration
- Get-LabHostConfiguration
- Test-LabHostConfiguration
- Invoke-LabResourceDownload
- Start-LabConfiguration
- Start-Lab
- Stop-Lab
- Get-LabVM
- Remove-LabConfiguration
- Test-LabConfiguration
- Import-LabHostConfiguration
- Export-LabHostConfiguration
These are just a few of the Functions available in Lability and we will cover most of these functions in greater detail as we head through this article.
Lability Media Files
Lability has a number of different configuration files all in JSON format, and these are HostDefaults, VMDefaults & Media. All of these files are in the Config folder of the Lability Module which on your new Machine will be C:\Program Files\WindowsPowerShell\Modules\Lability\0.9.8\Config The HostDefaults file contains all the settings that we associate with the Lability Host Machine. These include the paths where we will be looking for any ISO’s, VHDX, Hotfixes and any additionally required Resource Files for in our Lab.
The VMDefaults file contains all the default settings that we associate with the created VM’s. This includes Media used to create the Machine, Startup RAM, Number of Processors and which virtual switch we can expect the VM’s to use. This can be useful to have just like the HostDefaults but as we will see later on in this post we are most likely to override this in our configurations.
The Media file contains the settings for any media that we we want to use. As Lability in its nature was was built for building Labs it uses the Evaluation Licensed media for the VM’s.
The benefit of this is that the items already in this file allows you to get on with building Labs almost straight away on a brand new Machine.
This file has some included Hotfix Download links for getting the DSC updates on WMF4 for Server 2012R2 & Windows 8.1, but don’t worry Lability uses these to download the hotfixes and embed them into the VHD files for you. 1 Less job to worry about 
LabHost Defaults
Firstly we need to get the LabHost Defaults setup correctly for our environment – this is important and also is great for being able to move Labs between machines if required ( I’ve had to do this a fair amount myself ) and is why I recommend that all the core Lability bits are installed on a Separate Drive.
Personally I’m using an External Hard Drive but that is because my Lab is portable. I have not tried this with a Shared Drive however there shouldn’t be much that needs to change to get it working that way.
On my External Drive I have the following Setup - I have a folder called Lability and in this I have all the Folders required by Lability as detailed in LabHost Defaults as we will see below – however I also have another folder – Lability-Dev as this was from the Zip that you can download of a repository from GitHub as this was prior to Lability being made available on the PowerShell Gallery. In essence this means that I have copy of Lability that I can edit as required – especially the 3 Lability Configuration files detailed in the previous section but also allows me to do additional debugging as required.
Firstly we will Run Get-LabHostDefault and this should return the below by default – this is because the File HostDefault.json is stored in the C:\Program Files\WindowsPowerShell\Modules\Lability\0.9.8\Config location (remember 0.9.8 is the current version – yours may vary)
As this is the default and I’ve been using Lability on a few different machines I have a copy of it on my External HDD in the Lability Folder. Lets see what that file says it should be.
Well – That’s not good! As you can see on my last machine the external drive had been the D Drive but on this machine its the E Drive. A simple (yet annoying) thing that we can easily change. Now I could be Done Manually but I decided that I wanted to wrap this all together so that I don’t have to think about it again. This is simple enough so I just wrapped it in a very simple function as seen below
This allows me to Update this as I move it between machines quite easily. This isn’t an ideal scenario but it works at at least.
The benefit of this is that it will update the HostDefaults file on both my C: Drive and the External Drive at the same time – Which further means that this will be easier to be portable.
We can then run the function Reset-LabHostDefault and we should get something similar to the below
We can also do the same thing for the VMDefaults file however I find this is less likely to be a requirement as we can override the defaults in the configuration data files that we will work with and this is my preferred method.
Once we have done this we are ready to run the following function Start-LabHostConfiguration – this will on a new machine go and create the required directories as specified in the HostDefaults.json file that I have shown you how to amend and the output from Start-LabHostConfiguration is below
We would then use Test-LabHostConfiguration to confirm that this is all correct and we can see that this is the case below
Building your First Actual Lab
Wow that was a fair bit of setup required though a lot of it may be completely ignored depending on your own set up or if your re-visiting this post.
Now we move onto the real meaty part of the post and I’m going to use 2 examples for this – The Bundled TestLabGuide and one of my own for a SQLServer install.
So starting with the TestLabGuide.ps1 file there is only 1 small modification that I have made and this is at the end of the File and that is to include the following 2 lines
This allows me to build the configuration for these VMs as if it was a script and this is how I am personally doing it.
However on a Machine with No DSC resources we have an issue if we are building VM’s that are Dependant on these DSC Resources.
Well within Lability there is a Function called Invoke-LabResourceDownload and this has the ability to download all the required resources that we need as defined in our configuration data file.
Within the Configuration Data file shown below, the key section for us to look at in here at this point is the NonNodeData section where we have a subsection for Lability configuration Items, this can include EnvironmentPrefix, Media, Network, Resources & most importantly for us DSC Resources.
So far I have found that we only need to run this for pulling the DSCResources as defined in our configuration data file as shown below – this is because we require them to be on the machine before we can build the mof files.
I found that it best to have the DSCResources as RequiredVersion and not MinimumVersion as it is by default in the TestLabGuide.psd1 file – This is by preference but with the amount of changes happening to the DSC Resources its worthwhile being extra cautious here.
The output from Invoke-LabResourceDownload can be seen below and this as we can see has downloaded only the DSC Resources that we specified in the Configuration data file (TestLabGuide.psd1)
This also means on a clean machine you will be sure that you have the right required versions. This is especially useful when building Labs in my opinion.
However if you have multiple Labs running concurrently then the next bit may be an unfortunate blow to you.
Within the Configuration keyword we have a Dynamic Keyword defined – this is Import-DSCResource – which you may have thought was a function.
With it being a Dynamic Keyword it works a little differently to a normal Function/Cmdlet and therefore we are limited as to what we can do with it – for example we cannot use Splatting with it and we also cannot pass the required DSC Resource Modules to it from outside the current file. This is required for the syntax highlighting that we get as part of the parser. If you want to learn more about the Import-DSCResource Dynamic Keyword then read up this article by the PowerShell Team – be wary it is from 2014 and there hasn’t really been any better content come out on this since (that I can find anyway)
My thoughts on this is that we should be able to pass the required DSC Resources through from the Configuration data file like we have already detailed prior – however this isn’t currently possible.To me it would be beneficial (& logical) to be able to extract this away from the configuration as it is really a part of the configuration data, especially seeing as we already have to pass configuration data to our outputted configuration keyword – in this case TestLabGuide. However this is where we are at and at this time we will need to mirror the DSC Resources between both the configuration itself and the configuration data file.
However that aside lets look at the Node data and especially the All Nodes section which is where the NodeName = *
As we can see in here we have a few settings for the all the nodes in this configuration that will share items from Lability and these include the items we had available in the VMDefaults file as well as some other items too that we would want shared between the VM’s like DomainName etc
Further down we can see that for the Client VM’s in this Lab we are specifying different Lability_Media values for each- so it looks like we will have both a Windows 8.1 & a Windows 10 Client Machine in this Lab.
That’s enough about the configuration and configuration data side of things – lets go and build our Lab.
At this point what we want to do is just do the below.
At this point you will be prompted for an Administrator Password and once that has been given as we can see above it will go and create all the mof files that we need for this lab. The next step is to go and kick off the actual build of the lab which can be done as shown below
This Function Start-LabConfiguration is the key function of this module as it will go and
- check that the Lability host is correctly setup – by calling Test-LabHostConfiguration - if not it will throw an error (possible update here)
- download any ISO’s that are required as we have expressed in configuration data if that image matches one we have listed in the Media file. It will match these to the Checksum value given in the Media file for the Image
- download any Hotfixes that are detailed in the Hotfix section of the matched Media in the media.json file.
- build a Master VHDX File from the ISO & Hotfixes as detailed for the media type for the Lab VM’s as downloaded above – it is worthwhile to point out that this is built of lots of smaller functions that are essentially based off of the Convert-WindowsImage script.
- build a Lab Specific VHDX file – this is currently setup as a 127GB Dynamic Differencing disks
- build and inject a Lab VM specific unattend.xml file into each Lab VM VHDX
- Inject into Lab VM VHDX all required certificates
- download & Inject any resources into that are defined in the Lability Section of the NonNodeData section of the Configuration data file – I Will show more on this in the SQL Example later on. These are injected into the Lab Specific VHDX file
- Inject all required DSC Resources into the resulting Lab VM Specific VHDX file.
- Inject the mof and meta.mof files for each Lab VM into corresponding VHDX file.
Seriously though - Wow - that 1 Function is doing a lot of I would call tedious work for us and depending on your internet connection speed can take anywhere between maybe 30minutes to a day to complete - 1st time I ran it I think it took about 7 hours to complete for me due to slow Internet & I was also watching Netflix at the time
You can see the final output from this Function below
Note – If you have your own Media you could always create new entries in Media.json for these to save the download time – Especially if you have a MSDN License
Now this is where the fun bit really starts and it also involves more waiting but hopefully not as long as the last bit took you.
All we need to do at this Point is run Start-Lab like shown below and let DSC do its thing – note that I’ve used Get-VM and not Get-LabVM – this is a small issue that I have faced and have reported it on the Github Repo
And here is an image of all the VM’s running and getting started
This part can take anywhere from 10minutes to a few hours depending on your VM Rig setup and the amount of ram allocated to each VM as part of your configuration Data and whether there is requirement to wait for other machines to have be in their desired configuration as well as the complexity of the configurations being deployed.
Under the hood Lability has injected the DSC Configuration into the VM VMDX and has setup a Bootstrap process which in turn calls Start-DSCConfiguration and passes the path of the mof files to this. You can have a look at how this is setup in a VM’s in the following folder C:\Bootstrap\ if you are interested.
Once that is done you’ll have your first fully deployed set of VM’s using DSC & Lability – Pretty amazing isn’t it!
SQL Server install – Showing some of the other features of Lability
In this section I’ll try and keep the content to a minimal but still add in some additionally useful screenshots.
My ConfigurationData file is as below for the SQL Server Node, notice how we have the required properties to be able to install SQL, SourcePath, InstanceName, Features and the Lability_Resource.
As this was taken from a previous configuration this is using the xSQLServer DSCResource – take a look at cSQLServer here as this will likely be the version that gets ported to replace the xSQLServer & xSQLPS resources ass it is relatively close to being usable in place of the two resources. Expect news on this after PSConfEU.
Also note that in the Configuration Document we are specifying an additional item in the NonNodeData Section – Resource
This allows us to specify further resources that are stored in the E:\Lability\Resources\ folder (E being in my case of course)
I’ll let you decide what you want to put in that folder but any items for installation from the VM could be candidates, things like SharePoint Media or SQL media or other installable programs etc. You could always add your personal script library in a zip file and then get this Lability to unzip it into the right directory. Choices are up to you on this one – so be creative
For this Lab I didn’t have the installation media already downloaded so this has had to be downloaded as part of the Start-LabConfiguration Function – however if your remember there was a Invoke-LabResourceDownload Function.
This has some additional Parameters that allow you to download any of the required items for the LabConfiguration to succeed. This can be useful for example if you happen to have a few hours where the Internet Connection is much better than that of your own – especially if you are using this for personal testing and not professional lab testing as it was originally designed to be for.
One of the other great things with this module is that you can make use of it for your lab environments regardless of whether your shop is using WMF5 or not. If your still running WMF4 (with the essential DSC Updates) then you can still build labs using this.
Wrap up
Well I hope you’ve enjoyed reading this 3800+ word post of mine and this helps you get to grips with building out Labs in an easy and repeatable way whilst having the chance to play with DSC to do it.
Remember that this Module DOES A LOT behind the scenes – if it didn’t there wouldn’t be the need for this post - and there is more functionality being introduced as appropriate all the time.
Lability is built for building labs – however you could easily use this for building production like environments – if you dare that is and I can see the benefit to doing so, I mean why re-invent the wheel when Lability will do a lot (most) of the work for you.
Like with getting to grips with most new modules always start with the Help files. This Module has a number of about_* help files and almost all the functions (even the internal ones) have Comment Based Help.
This is a module where you need to RTFM to really understand all the workings of it. Spend a few hours looking through it and understanding it a best as you can. It will be so worth it in the long term even after reading this post a decent number of times.
My do however take my hat off to Iain Brighton (@iainbrighton) on creating this module and for me it is the only module to use when building Lab Environments – So lets gather some momentum as a community to suggest enhancements and improve it even more over on the Github Repo.
My example files that I have used (especially the SQL one) will be made available in due course once Iain has decided on a scalable way forward for being able to share Lab Configurations. We have discussed a number of options (yet to be added to this issue) and if you have an Idea please add it via the Lab Sharing Issue on Github.
This is just the first in a series of posts that I intend on doing on Lability – although future ones will be much shorter but will focus in depth around the functions that I haven’t covered in this post along with some of the more interesting parts in more depth. However I expect that this will be a good starting point for you to get to grips with the Lability Module and start building and running test labs.
As per usual please let me know your thoughts on this post whether it’s via Twitter or via the below comments section and I hope you have enjoyed the read.