Functional / Non-Functional Pester Tests and why I think you really should have a form of both.

So in this blog post I’m going to cover why there is a need to create Functional & Non-Functional Pester Tests for your PowerShell Modules but before I get into the nitty gritty into the whys behind creating both let me explain what the real differences are between the two because it may not be something that you have previously thought about or considered in your journey up until this point.

 

Functional

  • Used to test the code’s different use cases
  • Can be either be a form of Unit or Integration Test
  • Where we “Mock” the functionality to confirm it works as expected
  • To Determine the level of code coverage that your tests actually hit
  • Makes Functionality changes simpler and easier going forward as long as you write more Functional tests
  • Should save headaches as code moves between environments as part of a Build/Release Pipeline
  • Provides a Documentation Mechanism to catch either bugs so these can be fixed
  • Provides a Documentation Mechanism to potentially highlight where you may be able to make possible improvements

 

Non-Functional

  • Can be more referred to as “Traditional Documentation”
  • Aids Newcomers to the code base by being suggestive that you provide some useful help documentation
  • This can also aid newcomers in learning how to understand some of the more advanced functionality
  • We get Validation on the Functions Parameter types – i.e should the parameter be a String for input
  • Confirmations on whether the Parameter a Mandatory Parameter or not ?
  • Gives us a basic form of ParameterSet Validation
  • Gives us a basic form of Parameter Position Validation
  • Does the Parameter Accept Pipeline Input ?
  • Does the Parameter Accept Pipeline Input by Property Name ?
  • Does the Parameter use Advanced Validation at all ?
  • Does the Parameter have at least some help text defined ?
  • Does the Function have at least a basic level of Comment Based Help ? – lets leave the pro’s & con’s for another topic shall we.

 

So with the additional amount of tests that we may have to write from looking at the above why should we spend the time writing these tests?

This is where the story for Non-Functional tests becomes a little hazy in some ways but it really depends on the situation on how you’ve ended up with this module.

 

These possibilities can include

You’ve Inherited or downloaded someone else’s code and you have no clue what its doing because it’s

  • Not well documented with little or no help
  • Difficult to read because of the formatting
  • Uses a number of privately scoped functions
  • All the functions are either in a single ps1 or psm1 file
  • Just needs to be refactored to make it easier to manage, maintain & update going forward

Or it may just be that

  • It almost does what you need but you need to extend the functionality
  • You want to dig a little deeper into how it works
  • You are possibly continuing a discontinued open source project
  • Or you are looking at your own older code and want to give it a much needed update considering you’ve become a more experienced scripter than you were when you originally wrote it

 

If you were to go and create all the Non-Functional Tests that I’ve listed above then this will give you a lot of additional tests (& I mean a lot) that you would then have available to you to provide you some more trust in your code whilst you refactor or just understand how all the bolts fit together.

However I will point out that from this is really meant to provide you with a Singular Set Baseline on what is included in the module and not how the Module actually functions as that’s the role of the Functional Tests to do so.

 

In my next post I will show you how we can automagically create these Non-Functional Tests for each function included in an existing Script Module, including those functions that are defined as private/internal functions to give us a better chance of being able to manage, maintain & update it going forward.

Creating a set of simple Pester Tests for existing or old PowerShell Modules & making them easier to update in future.

I have long thought of a way to Automagically create some Pester Tests for the Functions contained in a module that perhaps was developed before Pester was really well known.

At that Point we may have been creating psm1 files that contained a number of nested functions within them. I know for one that I am one that did this / added to existing modules that were built this way – have a look at SPCSPS on Github or aka SharePointPowerShell on CodePlex as one of the first projects that I got involved with in the Open Source world.

*Please note I would highly advise to check out the OfficeDevPnP team work for any real SharePoint PowerShell work instead of the example I have given at PnP-PowerShell *

However this is where we are at with a number of older modules and as expressed prior about SPCSPS this was a way that was exceptionally common to run into.

However this isn’t a very scalable way of working with existing codebases and as very frequently found in the PowerShell community an author will not be able to spend the time reviewing the code and accepting pull requests from others. I have previously blogged about the need to “Pull The Community Together” to remove this totally unneeded & actually quite ridiculous barrier to better Modules for the benefit of the community. From a personal stand point –  A PR to an Open Source repository that is open with no input at all for more than a month shows that there is no value in adding to that prior Repository as it shows the Repo Owner has little/no time to do the needed Code Reviews etc.

Now one of the ways that we as a community can negate this issue is to build a stronger collaborative platform for these modules and build teams of people that can be relied on to perform cohesive reviews on various aspects of the code being added. By a Platform I mean a collective of ALL of the community to work out who all are the right people to get involved in the differing areas of the PowerShell Language.

Funnily enough GitHub within Organisations has this model already defined – called Teams. This allows us as a community to have an overarching organisation that will allow us to add the right people to get involved in discussions about certain semantics as we move on in time.

This essentially is a massive change to how we as a community do things however at this point in time really is the best way forward to minimize duplicated effort across multiple codebases and to ensure that we have the best & fully functional modules out there for the others in the community to work with.

Again please read my previous post “Pull The Community Together” on my thoughts on this.

Anyway back to the actual topic of this post. And from this point on I will be using the SPCSPS module as a good example that could be worked with as we find modules can currently be across Repo’s etc

So with SPCSPS I have 103 Functions that are put together in 11 psm1 files. Here are a few Screenshots just to back this up.

Although this “Works” its not great when there maybe a number of additions to 1 file (New Functions, Removing existing functions or Rewriting functions completely) and this can be an easy way for merge conflicts to occur – which we do not want.

So to get round this I realised that the only way was to write a function that will Export all the Functions (not Cmdlets) from a Module and whilst doing this will create a basic pester test for each of the exported functions into a User Specified Folder. My Reason for choosing to do it this way was to allow users to check the exported code before merging this into their existing codebases even though I am actually quite confident that this will work as expected.

This will allow users to refactor any of their existing code much easier going forward and will allow them to also benefit from having some basic pester tests that they can then expand upon.

 

The key component of this is the Export-Function function which has 2 parameters

  • Function – As a String
  • OutPath – As a String

Under the Hood the Export-Function function will when passed the Function Name & the OutPath will get the Function Definition and all the parameters from the Function and will then create the below files based on the following structure.

OutFilePath\FunctionVerb\FunctionName.ps1

OutFilePath\FunctionVerb\FunctionName.tests.ps1

Technically this isn’t actually difficult for us to do at all (hey we are using PowerShell right) but will allow us to quickly and easily add tests to existing (or new) code with little amount of effort and as a PowerShell Enthusiast this is exactly why I started working with PowerShell in 2013.

As a small note this will only work with public functions – though if you were to explicitly load private functions into the current session in a way they become public then you could use this to do the same for those as well.

The module is available on the PSGallery called PesterHelpers and is available on Github under https://github.com/PowerShellModules/PesterHelpers

The benefit of this module is that it can allow a quicker way to move away from modules that contain multiple functions in 1 psm1 file (or nested ps1 files) and can be used to help start to build a test suite of Pester Tests when used with the accompanying PesterHelpers.psm1 & PesterHelpers.basic.Tests.ps1 files for other modules. Is is possibly by modularising as much of the code in both of these files as possible.

A shoutout must go out to Dave Wyatt for a section of code that was contributed to ISE_Cew a while back that on a review whilst looking to expand that module lead me onto creating this Module.