Azure Chaos Studio and PowerShell
In this post
📣 Dive into the code samples from this blog on GitHub!
I had some spare time last month and decided it was time to take a closer look at Azure Chaos Studio, which is a managed service that allows you to orchestrate fault injection on your Azure resources, or inside of virtual machines, in a controlled manner. It helps you to perform chaos engineering on the Azure platform.
💡 Chaos engineering involves experimenting on a distributed system to build confidence in its capability to withstand turbulent conditions in production.
Azure Chaos Studio functions similarly to other Azure PaaS services and does not require you to perform any management or maintenance tasks. It also integrates with other Azure services such as Azure Policy, Azure Active Directory and the Azure Resource Manager. It has a fault library, which comes with a set of pre-defined fault actions that can introduce a disruption in a system. Best of all, Azure Chaos Studio is capable of rolling back an injected fault, to a certain degree. To top things off, it comes with a resource onboarding and permissions model which ensures fault injection can only be done by “authorized users against authorized resources, using authorized faults”.
Before my encounter with Azure Chaos Studio, I had also tinkered with Chaos Toolkit. It is open-source software that can be used to perform chaos engineering experiments.
Chaos Toolkit uses a JSON-based DSL (declarative style language) for you to define your experiments with, and it aims to be the simplest and easiest way to get started with chaos engineering. It also comes with additional “drivers”, plugins all written in Python, to connect to various cloud vendors, applications and environments. You can invoke just about any executable that is available on your system, as part of your experiment, and check its exit code via regex or JSONPath.
“Quick Question”…
A friend of mine asked me whether I could combine PowerShell, Pester 5 (a test and mock framework for PowerShell) and Azure Chaos Studio, in order to perform chaos experiments in a similar fashing to how Chaos Toolkit does it. I shrugged and gave him the only suitable answer to his question, that being “probably”.
If I wanted to tackle this I figured I’d have to go back and see what a Chaos Toolkit experiment consists of. When you run a Chaos Toolkit experiment, the following actions take place:
- A first check (or pass, as I refer to it) of the steady-state hypothesis.
- Defines what the working state of our app should be.
- For example: does this call to our web page return a status code 200?
- A method block.
- Changes the conditions of our system.
- For example: block incoming traffic on the firewall.
- A second pass of the same steady-state hypothesis.
- This is to validate that everything is still working as intended.
- A rollback block
- Does what it needs to, to roll back the changes introduced by the method block.
After reading through all that, wasn’t quite sure whether I could build something similar with PowerShell and Pester 5… But I think I’ve managed to come up with something that can work.
Pester 5 refresher
Let’s step back for a moment and take a look at some of the things you’ll probably want to know before we proceed any further. Pester is a test and mock framework for PowerShell, to quote its own documentation page:
“Pester provides a framework for writing and running tests. Pester is most commonly used for writing unit and integration tests, but it is not limited to just that. It is also a base for tools that validate whole environments, computer deployments, database configurations and so on.”
When I first started to use Pester 5, I went in unprepared thinking that most of how it works would be similar to how I wrote tests for Pester 4. For the uninitiated, in Pester 4 you could write top-down scripts that are very similar to how you typically write PowerShell script, however, this has drastically changed in Pester 5.
Pester 5 works in two distinct phases:
- 🔍 Discovery phase:
- Pester walks through “*.tests.ps1” file and
- Does execute statements inside of “
BeforeDiscovery”, “Context” or “Describe” blocks, - Does not execute statements inside of “
It”, “BeforeAll”, “BeforeEach”, “AfterAll” or “AfterEach” blocks. - Parameters that are provided to “
It” blocks are evaluated.- This is to determine: the name of the test, additional data passed to the test via the “
-ForEach” parameter and whether specific tests should be skipped during the run phase.
- This is to determine: the name of the test, additional data passed to the test via the “
- Does execute statements inside of “
- Pester walks through “*.tests.ps1” file and
- ⚡️ Run phase: Pester executes the “
BeforeAll”, “BeforeEach”, “It”, “AfterEach” and “AfterAll” blocks.- It blocks execute in a new scope!
⚠️ Hold on, what are all these blocks for?
Validation of test results takes place inside of an “It” block, which can be part of “Describe” and optional “Context” blocks. Also, the name of the “It” block should expressively state the expectation of the test.
By performing assertions with the “Should” command you can assert whether a value matches the expected value, if it doesn’t the “Should” command will throw the exception which will cause a test to fail. You can actually have many different “Should” assertions in an “It” block, in case you need to verify more than one thing. When you do this, a test will only pass once all assertions have been met!
An example might clear up any confusion, so here is one.
BeforeAll {
function Get-Hello {
return "Hello world"
}
}
Describe -Name "Get-Hello" -Fixture {
It -Name "return 'Hello world'" -Test {
$result = Get-Hello
$result | Should -Be "Hello world"
}
}
When you run the Invoke-Pester -Output 'Detailed' command, your tests will be run and you will get the following output:
# Describing Get-Hello
# [+] return 'Hello world' 10ms (7ms|3ms)
# Tests completed in 87ms
# Tests Passed: 1, Failed: 0, Skipped: 0 NotRun: 0
This makes sense when you think about it. Our “Get-Hello” function’s value should indeed be “Hello world”.
💡 The “
Should” command has many different assertion operators, which are plain ol’ PowerShell parameters, that allow you to validate a value in different ways. Currently you have the following assertion operators at your disposal: Be, BeExactly, BeGreaterThan, BeLessOrEqual, BeIn, BeLessThan, BeGreaterOrEqual, BeLike, BeLikeExactly, BeNullOrEmpty, BeOfType, BeTrue, BeFalse, Contain, Exist, FileContentMatch, FileContentMatchExactly, FileContentMatchMultiline, HaveCount, HaveParameter, Match, MatchExactly, Throw, InvokeVerifiable and Invoke. If you want to learn more about them, check out Pester’s documentation.
Pester 5 -AND variable scoping
Initially, I really tripped over the way variables are scoped, but now not so much! You see, if you define a variable in “BeforeDiscovery”, “Context” or “Describe” blocks, you will not be able to access the variable inside of an “BeforeAll”, “BeforeEach”, “It”, “AfterEach” and “AfterAll” by simply referencing it like you normally would.
We can see how this plays out with a simple test:
BeforeDiscovery {
$hello = "hello there"
write-host "#1 Discovery phase: "$hello
}
write-host "#2 Discovery phase: "$hello
Describe "some description" {
write-host "#3 Discovery phase: "$hello
Context "some context name" {
write-host "#4 Discovery phase: "$hello
BeforeAll {
write-host "#1 Run phase: "$hello
$hi = "hi there" # ←----------------- During the run phase, because of the Before* block
} # | $hi is available to all other tests
It "tests something" { # | within this Context block
write-host "#2 Run phase: "$hello # |-------------------------------
write-host "#3 Run phase: "$hi # ↲ |
$hello | Should -Be "world" # |
} # |
It "tests something" -ForEach @(@{ hello_as_test_case = $hello }) { # |
write-host "#4 Run phase: "$hello # |
write-host "#5 Run phase: "$hello_as_test_case # |
write-host "#6 Run phase: "$hi # ←------------------------------- Here too!
$hi | Should -Be "hi there" # |
} # |
AfterAll { # |
write-host "#7 Run phase: "$hello # |
write-host "#8 Run phase: "$hi # ↲ And here as well! 🥳
}
write-host "#5 Discovery phase: "$hello
}
write-host "#6 Discovery phase: "$hello
}
write-host "#7 Discovery phase: "$hello
Again, when you run the Invoke-Pester -Output 'Detailed' command, you will get the following output:
Pester v5.3.1
Starting discovery in 1 files.
#1 Discovery phase: hello there
#2 Discovery phase: hello there
#3 Discovery phase: hello there
#4 Discovery phase: hello there
#5 Discovery phase: hello there
#6 Discovery phase: hello there
#7 Discovery phase: hello there
Discovery found 2 tests in 12ms.
Running tests.
Running tests from '/workspaces/blog-2021-12-azure-chaos-studio-pester/chaos-experiments/hello.tests.ps1'
#1 Run phase:
Describing some description
Context some context name
#2 Run phase:
#3 Run phase: hi there
[-] tests something 18ms (16ms|2ms)
Expected 'world', but got $null.
at $hello | Should -Be "world", /workspaces/blog-2021-12-azure-chaos-studio-pester/chaos-experiments/hello.tests.ps1:19
at <ScriptBlock>, /workspaces/blog-2021-12-azure-chaos-studio-pester/chaos-experiments/hello.tests.ps1:19
#4 Run phase:
#5 Run phase: hello there
#6 Run phase: hi there
[+] tests something 14ms (12ms|2ms)
#7 Run phase:
#8 Run phase: hi there
Tests completed in 94ms
Tests Passed: 1, Failed: 1, Skipped: 0 NotRun: 0
By performing that test, you can see how the two phases deal with variable scoping. The second phase, the run phase, cannot easily access variables from the first phase, the discovery phase unless we pass them through in a particular manner. When I first noticed that, it was time for me to stop with whatever I was doing so I could take a breather and read through Pester’s “Discovery and Run” documentation.
🤷♂️ In retrospect, it turned out that this was actually all I needed to do, to begin with.
The thought process
I think that running an Azure Chaos Studio experiment is kind of similar to executing the “method” and “rollback” sections in a Chaos Toolkit experiment. Azure Chaos Studio experiments inject failures into your system and can roll them back, to an extent. But what about checking the steady-state hypothesis, i.e. whether my application is functional?
In my post, “Azure Chaos Studio - Public Preview”, I wrote this:
⚠️ It is important to highlight that Azure Chaos Studio only orchestrates fault injection, it is up to us to create additional tests to verify that the system is still working as intended.
This still holds true, so what do we do about it? You can write these tests in a number of different ways, with a number of different tools… But since I was asked whether I could do it with Pester, I decided to try just that.
As with Chaos Toolkit, I thought it would be a good idea to have Pester perform two passes of the steady-state hypothesis testing logic: a first pass runs before the chaos experiment is triggered and a second pass right after. Should the first pass fails, though, there is no point in triggering the experiment or performing the rollback operation. I think that having this capability becomes really important if you’re in a scenario where you are already suffering some form of outage because then you could do without these tests triggering additional chaos experiments to exacerbate the outage even more…
Figuring things out
I really had some difficulty with figuring out a workable way forward, initially, I thought of adding a “global:” variable that I could use to perform some checks against. I could reuse that variable in multiple scripts if I wanted to. I must admit that, as I was creating this, I had the feeling that I was going against some of the best practices that Pester expects you to follow, though I am not 100 per cent certain. After reading some of the docs and several posts online it seems like the only rule I should keep in the back of my mind is the following:
In Pester 5 the mantra is to put all code in Pester controlled blocks. No code should be directly in the script, or directly in Describe or Context block without wrapping it in some other block. There is a special block called BeforeDiscovery that shows intent when you need to put code directly in the script, or directly in Describe or Context block. This is useful when Data-driven tests.
I decided that if that was the only thing to worry about, I might as well try to build something.. Regardless of how many conventions it could potentially break:
BeforeDiscovery {
$global:IsSteadystateOk = $true
}
Describe "A chaos experiment 1" {
Context "Steady-state hypothesis" {
BeforeAll {
$webResponse = Invoke-WebRequest -Uri "thomasvanlaere.com/fail" -MaximumRetryCount 1 -TimeoutSec 3 -SkipHttpErrorCheck
}
It "should return status code 200" {
$result = $webResponse.StatusCode | Should -Be 200
if ($result -eq "?") { # This won't ever work
$global:IsSteadystateOk = $false
}
}
AfterAll{
Write-Host "`$global:IsSteadystateOk: $($global:IsSteadystateOk)"
}
}
}
# Describing A chaos experiment
# Context Steady-state hypothesis
# [-] should return status code 200 19ms (15ms|4ms)
# Expected 200, but got 404.
# at $result = $webResponse.StatusCode | Should -Be 200, /workspaces/blog-2021-12-azure-chaos-studio-pester/chaos-experiments/test.tests.ps1:12
# at <ScriptBlock>, /workspaces/blog-2021-12-azure-chaos-studio-pester/chaos-experiments/test.tests.ps1:12
# $global:IsSteadyStateOk: True
As far as I know, a “Should” assertion does not return any output, though it will throw an exception. I figured I could catch the exception, do whatever it was I wanted to do and eventually rethrow it.
BeforeDiscovery {
$global:IsSteadystateOk = $true
}
Describe "A chaos experiment 2" {
Context "Steady-state hypothesis" {
BeforeAll {
$webResponse = Invoke-WebRequest -Uri "thomasvanlaere.com/fail" -MaximumRetryCount 1 -TimeoutSec 3 -SkipHttpErrorCheck
}
It "should return status code 200" {
try {
$webResponse.StatusCode | Should -Be 200
}
catch {
$global:IsSteadystateOk = $false
throw $_
}
}
AfterAll {
Write-Host "`$global:IsSteadyStateOk: $($global:IsSteadystateOk)"
}
}
}
# Describing A chaos experiment
# Context Steady-state hypothesis
# [-] should return status code 200 16ms (12ms|4ms)
# Expected 200, but got 404.
# at $webResponse.StatusCode | Should -Be 200, /workspaces/blog-2021-12-azure-chaos-studio-pester/chaos-experiments/test.tests.ps1:30
# at <ScriptBlock>, /workspaces/blog-2021-12-azure-chaos-studio-pester/chaos-experiments/test.tests.ps1:30
# $global:IsSteadyStateOk: False
# Tests completed in 224ms
# Tests Passed: 0, Failed: 1, Skipped: 0 NotRun: 0
This “sort of” works, but it feels a little clunky to me. Regardless, I figured I’d flesh this idea out a little more:
BeforeDiscovery {
$global:IsSteadystateOk = $true
}
Describe "A chaos experiment 3" {
Context "Steady-state hypothesis" {
BeforeAll {
Write-Host "Pass 1"
# 👇 Url has changed
try { $webResponse = Invoke-WebRequest -Uri "thomasvanlaere.com" -MaximumRetryCount 1 -TimeoutSec 3 -SkipHttpErrorCheck }catch {}
}
It "should return status code 200" {
try {
$webResponse.StatusCode | Should -Be 200
}
catch {
$global:IsSteadystateOk = $false
throw $_
}
}
It "should have content" {
try {
$webResponse.Content | Should -Not -BeNullOrEmpty
}
catch {
$global:IsSteadystateOk = $false
throw $_
}
}
AfterAll {
Write-Host "`$global:IsSteadyStateOk: $($global:IsSteadystateOk)"
}
}
Context "Method" {
BeforeAll {
if (!$global:IsSteadystateOk) { return }
Write-Host "Injecting some faults.."
$someResult = @{Id = "hello there!" }
}
BeforeEach {
if (!$global:IsSteadystateOk) { Set-ItResult -Skipped -Because "steady state failed" }
}
It "should have an id" {
$someResult.Id | Should -Not -BeNullOrEmpty
}
}
Context "Rollback" {
BeforeAll {
if (!$global:IsSteadystateOk) { return }
Write-Host "Rolling back faults.."
$someResult = @{StatusUrl = "https://some.url.com" }
}
BeforeEach {
if (!$global:IsSteadystateOk) { Set-ItResult -Skipped -Because "steady state failed" }
}
It "should have a StatusUrl" {
$someResult.StatusUrl | Should -Not -BeNullOrEmpty
}
}
Context "Steady-state hypothesis" {
BeforeAll {
Write-Host "Pass 2"
if (!$global:IsSteadystateOk) { return }
try { $webResponse = Invoke-WebRequest -Uri "thomasvanlaere.com" -MaximumRetryCount 1 -TimeoutSec 3 -SkipHttpErrorCheck }catch {}
}
BeforeEach {
if (!$global:IsSteadystateOk) { Set-ItResult -Skipped -Because "steady state failed" }
}
It "should return status code 200" {
$webResponse.StatusCode | Should -Be 200
}
It "should have content" {
$webResponse.Content | Should -Not -BeNullOrEmpty
}
}
}
AfterAll {
$global:IsSteadyStateOk = $true
}
# Pass 1
# Describing A chaos experiment
# Context Steady-state hypothesis
# [+] should return status code 200 15ms (11ms|4ms)
# [+] should have content 7ms (4ms|3ms)
# $global:IsSteadyStateOk: True
# Injecting some faults..
# Context Method
# [+] should have an id 12ms (8ms|3ms)
# Rolling back faults..
# Context Rollback
# [+] should have a StatusUrl 8ms (5ms|3ms)
# Pass 2
# Context Steady-state hypothesis
# [+] should return status code 200 10ms (7ms|4ms)
# [+] should have content 7ms (4ms|3ms)
# Tests completed in 514ms
# Tests Passed: 6, Failed: 0, Skipped: 0 NotRun: 0
Here is the result I got when setting the “-Uri” to “thomasvanlaere.com/fail”, causing the first pass to fail.:
# Pass 1
# Describing A chaos experiment
# Context Steady-state hypothesis
# [-] should return status code 200 22ms (11ms|11ms)
# Expected 200, but got 404.
# at $webResponse.StatusCode | Should -Be 200, /workspaces/blog-2021-12-azure-chaos-studio-pester/chaos-experiments/test.tests.ps1:52
# at <ScriptBlock>, /workspaces/blog-2021-12-azure-chaos-studio-pester/chaos-experiments/test.tests.ps1:52
# [+] should have content 7ms (4ms|2ms)
# $global:IsSteadyStateOk: False
# Context Method
# [!] should have an id is skipped, because steady state failed 7ms (3ms|4ms)
# Context Rollback
# [!] should have a StatusUrl is skipped, because steady state failed 10ms (4ms|7ms)
# Pass 2
# Context Steady-state hypothesis
# [!] should return status code 200 is skipped, because steady state failed 7ms (3ms|3ms)
# [!] should have content is skipped, because steady state failed 6ms (3ms|3ms)
# Tests completed in 334ms
# Tests Passed: 1, Failed: 1, Skipped: 4 NotRun: 0
Again, this sort of works. I could certainly optimize some things here or there, it’s certainly not adhering to the “don’t-repeat-yourself” principles, but conceptually this is what I want it to do.
I’ve tried replacing the try/catch with “$webResponse.StatusCode | Should -Be 200 -ErrorVariable hasError -ErrorAction SilentlyContinue” which works as well, provided that you remove the “try/catch” and put an additional “if” statement in the test block. As soon as I start to introduce multiple “Should” assertions in a single “It” block, I think that both approaches will become difficult to maintain.
💡 By the way, when an assertion throws an exception it halts the test immediately, though you can override this default behaviour to continue the remainder of the test. If I did this I could omit “
-ErrorAction SilentlyContinue”.
I might require a boatload of code to check whether the steady-state is valid, so let’s refactor this code a bit.
BeforeDiscovery {
$global:IsSteadystateOk = $true
}
BeforeAll {
function Test-SteadyStateHasError ($errorVar) {
if ($null -ne $errorVar -AND $errorVar.Count -gt 0) { $global:IsSteadystateOk = $false }
}
function Skip-IfSteadyStateInvalid {
if (!$global:IsSteadystateOk) { Set-ItResult -Skipped -Because "steady state failed" }
}
}
Describe "A chaos experiment" {
Context "Steady-state hypothesis" {
BeforeAll {
Write-Host "Pass 1"
try { $webResponse = Invoke-WebRequest -Uri "thomasvanlaere.com" -MaximumRetryCount 1 -TimeoutSec 3 -SkipHttpErrorCheck }catch {}
}
It "should return status code 200" {
$webResponse.StatusCode | Should -Be 200 -ErrorVariable hasError -ErrorAction SilentlyContinue
Test-SteadyStateHasError $hasError
}
It "should have content" {
$webResponse.StatusCode | Should -Be 200 -ErrorVariable hasError -ErrorAction SilentlyContinue
Test-SteadyStateHasError $hasError
$webResponse.Content | Should -Not -BeNullOrEmpty -ErrorVariable hasError -ErrorAction SilentlyContinue
Test-SteadyStateHasError $hasError
}
AfterAll {
Write-Host "`$global:IsSteadystateOk: $($global:IsSteadystateOk)"
}
}
Context "Method" {
BeforeAll {
if (!$global:IsSteadystateOk) { return }
Write-Host "Injecting some faults.."
$someResult = @{Id = "hello there!" }
}
BeforeEach {
Skip-IfSteadyStateInvalid
}
It "should have an id" {
$someResult.Id | Should -Not -BeNullOrEmpty
}
}
Context "Rollback" {
BeforeAll {
if (!$global:IsSteadystateOk) { return }
Write-Host "Rolling back faults.."
$someResult = @{StatusUrl = "https://some.url.com" }
}
BeforeEach {
if (!$global:IsSteadystateOk) { Set-ItResult -Skipped -Because "steady state failed" }
}
It "should have a StatusUrl" {
$someResult.StatusUrl | Should -Not -BeNullOrEmpty
}
}
Context "Steady-state hypothesis" {
BeforeAll {
Write-Host "Pass 2"
if (!$global:IsSteadystateOk) { return }
try { $webResponse = Invoke-WebRequest -Uri "thomasvanlaere.com" -MaximumRetryCount 1 -TimeoutSec 3 -SkipHttpErrorCheck }catch {}
}
BeforeEach {
Skip-IfSteadyStateInvalid
}
It "should return status code 200" {
$webResponse.StatusCode | Should -Be 200
}
It "should have content" {
$webResponse.Content | Should -Not -BeNullOrEmpty
}
}
}
AfterAll {
$global:IsSteadystateOk = $true
}
I suppose that the “Test-SteadyStateHasError” and “Skip-IfSteadyStateInvalid” functions could even be moved into a .ps1 file that can be dot sourced in the top-level “BeforeAll”.
Custom Should Assertion Operator
Even though the last test worked fine, it was then that I stumbled upon something interesting and decided to settle for a slightly different approach.
Pester gives me the option to register a custom"Should" assertion operator! Assertion operators are used to perform assertions on an input value, this meant I could do the following:
It "should return status code 200" {
$webResponse.StatusCode | Should -BeAHealthyStatusCode
}
That “-BeAHealthyStatusCode” flag is actually a function that is declared in a custom module and will handle the custom assertion logic.
function BeAHealthyStatusCode($ActualValue, [switch] $Negate)
{
# 👇 the actual check
[bool] $succeeded = $ActualValue -in 200, 201, 202, 203, 204, 205
if ($Negate) { $succeeded = -not $succeeded }
if (-not $succeeded)
{
if ($Negate)
{
$failureMessage = "{$ActualValue} is a healthy status code."
}
else
{
$failureMessage = "{$ActualValue} is not a healthy status code"
}
}
return New-Object psobject -Property @{
Succeeded = $succeeded
FailureMessage = $failureMessage
}
}
# 👇 Tell Pester about the new should operator! (Don't do this twice though)
Add-ShouldOperator -Name BeAHealthyStatusCode `
-Test $function:BeAHealthyStatusCode `
-Alias 'BAHSC'
If you would like to learn more about the custom Should assertion operators, feel free to take a look over on the Pester docs.
A Steady-State tracking Should operator
With that knowledge I started to think, why not track some state inside of the module? I could probably keep it scoped to the module itself as long as I don’t do anything crazy or create any race conditions in my actual Pester tests.
All I need the operator to do is keep track of whether the current pass has failed at all. If the assertion fails at any point, that’s enough for me to call it quits and consider that particular pass to be in a failed state.
Let’s try this…
enum SteadyStateHypothesisPassType {
First = 0
Second = 1
}
# Track the state in a small array, one value for each of the two runs
$script:SteadyStatePasses = @($true, $true)
# When the module is loaded set the current pass to 'First'
[SteadyStateHypothesisPassType]$script:CurrentPass = [SteadyStateHypothesisPassType]::First
function Should-BeSteadyStateHypothesisValue ($ActualValue, $ExpectedValue, [switch] $Negate, [string] $Because) {
<#
.SYNOPSIS
Checks whether the provided value matches the expected value.
.EXAMPLE
$greeting = "Hello world"
$greeting | Should -BeSteadyStateHypothesisValue "Hello world"
Checks if the greeting variable matches "Hello world" . This should pass.
.EXAMPLE
$greeting = "Hello world"
$greeting | Should -Not -BeSteadyStateHypothesisValue "Hello world"
Checks if the greeting variable does not match "Hello world". This should not pass.
#>
# 👇 Good enough for now
[bool] $succeeded = $ActualValue -eq $ExpectedValue
if ($Negate) {
$succeeded = -not $succeeded
}
$failureMessage = ''
if (-not $succeeded) {
if ($Negate) {
$failureMessage = "Steady-state hypothesis failed: Expected $ExpectedValue to be different from the actual value,$(if ($null -ne $Because) { $Because }) but got the same value."
}
else {
$failureMessage = "Steady-state hypothesis failed: Expected $ExpectedValue,$(if ($null -ne $Because) { $Because }) but got $(if ($null -eq $ActualValue){'$null'} else {$ActualValue})."
}
# 👇 If it fails once, toggle the variable
$script:SteadyStatePasses[$script:CurrentPass] = $false
}
return [PSCustomObject] @{
Succeeded = $succeeded
FailureMessage = $failureMessage
}
}
function Get-SteadyStateHypothesisStatus {
<#
.SYNOPSIS
Get the current steady state hypothesis pass's status.
.DESCRIPTION
Get the current steady state hypothesis pass's status.
$true when succesful.
$false when failed.
.PARAMETER Pass
'First' or 'Second'
.EXAMPLE
Get-SteadyStateHypothesisStatus -Pass First
.EXAMPLE
Get-SteadyStateHypothesisStatus -Pass Second
.OUTPUTS
[bool]result
#>
[CmdletBinding()]
param (
[Parameter(Mandatory = $true)]
[SteadyStateHypothesisPassType]
$Pass
)
if ($null -eq $script:SteadyStatePasses[$Pass]) {
throw "No steady state hypothesis value was found."
}
else {
return $script:SteadyStatePasses[$Pass];
}
}
function Skip-SteadyStateHypothesisWhenFirstPassFailed {
<#
.SYNOPSIS
Set an It block's result to 'skipped'.
.DESCRIPTION
Set an It block's result to 'skipped'.
.EXAMPLE
Skip-SteadyStateHypothesisWhenFirstPassFailed
#>
[CmdletBinding()]
param ()
if (!(Test-SteadyStateHypothesisFirstPass)) {
Set-ItResult -Skipped -Because "the first pass of the steady-state hypothesis did not complete succesfully."
}
}
function Test-SteadyStateHypothesisFirstPass {
<#
.SYNOPSIS
Return true or false to indicate whether the first pass has succeeded.
.DESCRIPTION
Return true or false to indicate whether the first pass has succeeded.
$true when succesful.
$false when failed.
.EXAMPLE
Test-SteadyStateHypothesisFirstPass
.OUTPUTS
[bool]result
#>
[CmdletBinding()]
param ()
return (Get-SteadyStateHypothesisStatus -Pass First)
}
function Test-SteadyStateHypothesisSecondPass {
<#
.SYNOPSIS
Return true or false to indicate whether the second pass has succeeded.
.DESCRIPTION
Return true or false to indicate whether the second pass has succeeded.
$true when succesful.
$false when failed.
.EXAMPLE
Test-SteadyStateHypothesisSecondPass
.OUTPUTS
[bool]result
#>
[CmdletBinding()]
param ()
return (Get-SteadyStateHypothesisStatus -Pass Second)
}
function Get-SteadyStateHypothesisCurrentPass {
<#
.SYNOPSIS
Returns the current pass number.
.DESCRIPTION
Returns the current pass number.
.EXAMPLE
Get-SteadyStateHypothesisCurrentPass
.OUTPUTS
[SteadyStateHypothesisPassType]
#>
[CmdletBinding()]
param ()
$script:CurrentPass
}
function Set-SteadyStateHypothesisCurrentPass {
<#
.SYNOPSIS
Sets the current pass number.
.DESCRIPTION
Sets the current pass number.
.EXAMPLE
Set-SteadyStateHypothesisCurrentPass -Pass First
.EXAMPLE
Set-SteadyStateHypothesisCurrentPass -Pass Second
#>
[CmdletBinding()]
param (
[Parameter(Mandatory = $true)]
[SteadyStateHypothesisPassType]
$Pass
)
$script:CurrentPass = $Pass
}
function Complete-SteadyStateHypothesisPass {
<#
.SYNOPSIS
Sets the current pass number to the next available pass.
.DESCRIPTION
Sets the current pass number to the next available pass.
.EXAMPLE
Complete-SteadyStateHypothesisPass
#>
[CmdletBinding()]
param ()
switch ($script:CurrentPass) {
([SteadyStateHypothesisPassType]::First) {
Set-SteadyStateHypothesisCurrentPass -Pass Second;
break;
}
([SteadyStateHypothesisPassType]::Second) {
break; <# Don't do anything 🤷♂️ #>
}
Default {
throw "Uhm"
}
}
}
function Reset-SteadyStateHypothesis {
<#
.SYNOPSIS
Sets both passes to their default value ($true) and sets the current pass to first.
.DESCRIPTION
Sets both passes to their default value ($true) and sets the current pass to first.
.EXAMPLE
Reset-SteadyStateHypothesis
#>
[CmdletBinding()]
param ()
$script:SteadyStatePasses = @($true, $true)
$script:CurrentPass = [SteadyStateHypothesisPassType]::First
}
# Add all ShouldOperators
@(
@{Name = "BeSteadyStateHypothesisValue" ; InternalName = 'Should-BeSteadyStateHypothesisValue'; Test = ${function:Should-BeSteadyStateHypothesisValue} ; Alias = 'BSSHV' }
) | ForEach-Object {
try { $existingShouldOp = Get-ShouldOperator -Name $_.Name } catch { $_ | Write-Verbose }
if (!$existingShouldOp) {
Add-ShouldOperator -Name $_.Name -InternalName $_.InternalName -Test $_.Test -Alias $_.Alias
}
}
That should do the trick, right? I’d say it’s pretty much the same concept as I had before. But, since this custom “Should” assertion operator is in a module I decided to expose a few functions for my Pester test to use.
- “
Complete-SteadyStateHypothesisPass”- Sets the current pass number to the next available pass.
- “
Reset-SteadyStateHypothesis”- Sets both passes to their default value “
$true” and sets the current pass to the first pass.
- Sets both passes to their default value “
- “
Skip-SteadyStateHypothesisWhenFirstPassFailed”- Set an It block’s result to ‘skipped’.
- “
Test-SteadyStateHypothesisFirstPass”- Return true or false to indicate whether the first pass has succeeded.
I will end up using these in the next bit…
The pester chaos experiment
In my first blog post on Azure Chaos Studio, I created an ARM template that deploys a demo setup that can serve as a jumping-off point for this demo. Here is what the complete ARM template rolls out for you:
- Deploy a web server virtual machine that serves a static web page over port 80.
- Hooks up an Azure network security group (NSG) to a subnet.
- Onboards the NSG as a Chaos Target extension-resource, in Azure Chaos Studio.
- Allow Chaos Studio to run the “SecurityRule-1.0” fault action against the NSG, this is done by activating a capability on the target extension resource.
- Creates an Azure Chaos Experiment resource that adds a new inbound NSG rule, that blocks all traffic coming in through port 80 (and 443 for completeness’ sake).
If deploy this ARM template, you should be able to see the newly created chaos experiment in Azure Chaos Studio. When you run the experiment, it will block the inbound traffic for about five minutes, before rolling back the changes that were introduced by the “SecurityRule-1.0” fault action on the NSG.
The final two scripts
I usually wrap the “Invoke-Pester” command in another script, just so I have a little bit of additional control of what goes on. Plus this script is written in such a way that you could potentially use it in either a deployment pipeline or while you’re developing on your local machine. All this script will do, is execute the Pester tests and set some configuration settings that I like to use.
Most importantly, it will include the module that we created, so our scripts can have access to the custom “Should” assertion operator.
When you run the following script, please make sure that you replace “” and “http://”. Ideally, this is something that you pull out of a template file and fill up dynamically.
#Requires -Version 7
<#
.SYNOPSIS
Runs Pester tests.
.DESCRIPTION
Runs Pester test that perform chaos experiments.
.PARAMETER ChaosExperimentsPath
Provide the path to the directory containing the .Tests.ps1 files.
.INPUTS
None. You cannot pipe objects to Invoke-Pester.
.OUTPUTS
None.
.EXAMPLE
PS C:\> .\Invoke-Pester.ps1 -ChaosExperimentsPath "/workspaces/blog-2021-12-azure-chaos-studio-pester/chaos-experiments"
#>
#Requires -Version 7
[CmdletBinding()]
param (
[Parameter()]
[string]
$ChaosExperimentsPath = (Join-Path -Path $(if ([string]::IsNullOrEmpty($PSScriptRoot)) { "." } else { $PSScriptRoot }) -ChildPath "chaos-experiments")
)
Import-Module -Name "Pester" -RequiredVersion "5.3.1" -Force
Import-Module -Name (Join-Path -Path $(if ([string]::IsNullOrEmpty($PSScriptRoot)) { "." } else { $PSScriptRoot }) -ChildPath "SteadyStateHypothesisAssertions" -AdditionalChildPath "SteadyStateHypothesisAssertions.psm1") -DisableNameChecking -Force
if (!(Test-Path $chaosExperimentsPath)) {
Write-Error -Message "Path to experiments not found." -ErrorAction Stop
}
if (!(Get-AzContext)) {
Connect-AzAccount -UseDeviceAuthentication -ErrorAction Stop
}
if ((Get-AzContext)) {
[System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor [System.Net.SecurityProtocolType]::Tls12
$container = New-PesterContainer -Path 'website.chaos.experiment.tests.ps1' -Data @{ ResourceGroupName = "<replace-with-your-resource-group-name>"; WebsiteUrl = "http://<replace-with-your-vm-ip>"; ExperimentName = "tvl-azcs-blog-experiment" }
$PesterConfig = New-PesterConfiguration -HashTable @{
Run = @{
Path = $ChaosExperimentsPath
Container = $container
# SkipRemainingOnFailure = 'Container' # 👈 Only skips tests, but will still execute Before* and After* blocks.
}
Output = @{
Verbosity = "Detailed" # Verbosity: The verbosity of output, options are None, Normal, Detailed and Diagnostic.
}
Should = @{
ErrorAction = 'Continue'
}
}
Invoke-Pester -Configuration $PesterConfig
}
Now for the chaos experiment/Pester test, which I will admit is a bit of a behemoth at this point and could probably do with just a tad more optimization.
This test expects you to have deployed the complete ARM template as it will use the Azure Studio Chaos Experiment in the experiment activities (method) block. It’s written in such a way that it will skip the test in case the “Microsoft.Chaos/experiments” resource was not found or if it is not available. By not “not available” what I really mean is that the resource is already in one of three states: “preprocessing queued”, “running” or “cancelling”. I’ve found that you cannot start another instance of this type of experiment if it is in one of those states.
#Requires -Version 7
[CmdletBinding()]
<#
.SYNOPSIS
Starts an chaos experiment and performs tests.
.DESCRIPTION
Tests the steady state hypothesis of a web resource during the first pass of the steady-state hypothesis.
If the first pass of the steady state hypothesis is valid:
- The experiment method runs and an Azure Chaos Studio experiment is started.
- The steady state hypothesis is validated in a second pass.
- The rollback section kicks off, canceling the Azure Chaos Studio experiment.
.PARAMETER ExperimentName
Provide the Azure Chaos Studio experiment name.
.PARAMETER ResourceGroupName
Provide the resource group name which holds the Azure Chaos Studio experiment.
.PARAMETER WebsiteUrl
Provide the address of a the chaos target resource.
.INPUTS
None.
.OUTPUTS
System.Management.Automation.PSObject
.EXAMPLE
PS C:\> .\website.chaos.experiment.tests.ps1 -ExperimentName "" -ResourceGroupName "tvl-azcs-rg" -WebsiteUrl "https://thomasvanlaere.com""
#>
param (
[Parameter(Mandatory = $true)]
[string]
$ExperimentName,
[Parameter(Mandatory = $true)]
[string]
$ResourceGroupName,
[Parameter(Mandatory = $true)]
[string]
$WebsiteUrl
)
BeforeDiscovery {
# Get the chaos experiment resource
$ChaosExperimentResource = Get-AzResource -ResourceType "Microsoft.Chaos/experiments" -Name $ExperimentName -ResourceGroupName $ResourceGroupName -ErrorAction SilentlyContinue
$isExperimentAvailableForStart = $false
if ($null -ne $ChaosExperimentResource) {
# Running Invoke-AzRestMethod with a relative URL will *sometimes* result in:
# "This operation is not supported for a relative URI".
# So we will prefix the url with "https://management.azure.com",
# turning it into an absolute URL, for now. 🤷♂️
$experimentStatusesUrl = '{0}{1}/statuses?api-version=2021-09-15-preview' -f "https://management.azure.com", $ChaosExperimentResource.Id
$experimentStatusesResponse = Invoke-AzRestMethod -Uri $experimentStatusesUrl -Method Get
if ($null -ne $experimentStatusesResponse) {
$experimentStatusesResult = $experimentStatusesResponse.Content | ConvertFrom-Json
$isExperimentAvailableForStart = ($experimentStatusesResult.Value | Where-Object { $_.properties.status -iin "preprocessingqueued", "running", "cancelling" }) -eq $null
# Going to assume that the frequencies of the runs is low enough so
# I do not need to worry about the "nextLink" property, fingers crossed.
# 👆 I want to skip this set of tests when a Chaos Studio experiment has already been queued,
# started its run or is in the process of cancelling.
}
}
# A function which contains the steady-state hypothesis test.
function Invoke-SteadyStateHypothesis {
Context "steady-state hypothesis" {
BeforeAll {
# Since this block is called twice we must check whether
# the first pass has failed, so we can skip the second pass.
if (!(Test-SteadyStateHypothesisFirstPass)) { return; }
$webResponse = try { Invoke-WebRequest -Uri $WebsiteUrl -MaximumRetryCount 1 -TimeoutSec 3 -SkipHttpErrorCheck } catch { $_ | Write-Verbose }
}
BeforeEach {
Skip-SteadyStateHypothesisWhenFirstPassFailed
}
It "should return status code 200" {
# Remember, the both passes are initially marked as OK (true)
# and will switch to NOT OK (false), when the custom should assertion operator
# detects a mistake.
$webResponse.StatusCode | Should -BeSteadyStateHypothesisValue 200
}
AfterAll {
# Once everything has been completed in this context block,
# tell our module to get ready for pass two.
Complete-SteadyStateHypothesisPass
}
}
}
# A function which contains the chaos experiment methods
function Invoke-ExperimentMethods {
Context "experiment activities" {
Context "Azure Chaos Studio" {
BeforeAll {
# If the first pass failed, skip the experiment setup phase.
if (!(Test-SteadyStateHypothesisFirstPass)) { return; }
# Request a start of the Azure Chaos Experiment
$startExperimentUrl = '{0}/start?api-version=2021-09-15-preview' -f $ChaosExperimentResource.Id
$startExperimentReponse = Invoke-AzRestMethod -Path $startExperimentUrl -Method POST
$experimentStartOperationResult = $startExperimentReponse.Content | ConvertFrom-Json
# Track the status of the start request
$experimentStatusResponse = Invoke-AzRestMethod -Uri $experimentStartOperationResult.statusUrl -Method Get
$experimentStatusResult = $experimentStatusResponse.Content | ConvertFrom-Json
# 👇 If we're fortunate enough, we can get the status we are looking for immediately.
while ($experimentStatusResponse.StatusCode -ne 404 -AND $experimentStatusResult.properties.Status -inotin "Running", "Failed", "Cancelled" ) {
Start-Sleep -Seconds 5
$experimentStatusResult = $null
$experimentStatusResponse = Invoke-AzRestMethod -Uri $experimentStartOperationResult.statusUrl -Method Get
$experimentStatusResult = $experimentStatusResponse.Content | ConvertFrom-Json
}
# Get the experiment's execution details, allowing us to track the status of individual "steps" in the experiment.
$executionDetailsUrl = '{0}{1}/executionDetails/{2}?api-version=2021-09-15-preview' -f "https://management.azure.com", $ChaosExperimentResource.Id, $experimentStatusResult.name
$executionDetailsResponse = Invoke-AzRestMethod -Uri $executionDetailsUrl -Method Get
$executionDetailsResult = $executionDetailsResponse.Content | ConvertFrom-Json
# And again if we're fortunate enough we might just get the result straight away.
$executionAction = $executionDetailsResult.properties.runInformation.steps | Where-Object { $_.branches.actions.actionName -ieq "urn:csci:microsoft:networkSecurityGroup:securityRule/1.0" }
# If not, we'll wait for it.
while ($executionDetailsResponse.StatusCode -ne 404 -AND $executionAction.Status -inotin "Running", "Failed", "Cancelled", "Completed" ) {
Start-Sleep -Seconds 5
$executionDetailsResult = $null
$executionDetailsResponse = Invoke-AzRestMethod -Uri $executionDetailsUrl.statusUrl -Method Get
$executionDetailsResult = $executionDetailsResponse.Content | ConvertFrom-Json
# Since my complete ARM template only has one step with this action name I'm going to assume the Where-Object will return one result.
$executionAction = $executionDetailsResult.properties.runInformation.steps | Where-Object { $_.branches.actions.actionName -ieq "urn:csci:microsoft:networkSecurityGroup:securityRule/1.0" }
}
}
BeforeEach {
# If the first pass has failed, set each it block as skipped.
Skip-SteadyStateHypothesisWhenFirstPassFailed
}
It "should have accepted the start the Azure Chaos Studio experiment" {
$startExperimentReponse.StatusCode | Should -Be 202
}
It "should have the Azure Chaos Studio experiment in running state" {
$experimentStatusResult.properties.Status | Should -Be "Running"
}
It "should have a NSG security rule fault action in running state, in the running Azure Chaos Studio experiment" {
if ($experimentStatusResult.properties.Status -ine "Running") {
Set-ItResult -Skipped "experiment is not in running state."
}
$executionDetailsResult.properties | Should -Not -BeNullOrEmpty
$executionDetailsResult.properties.runInformation | Should -Not -BeNullOrEmpty
$executionAction | Should -Not -BeNullOrEmpty
$executionAction | Should -HaveCount 1
$executionAction.Status | Should -Be "Running"
}
It "should pause <_> seconds for changes to take effect." -ForEach(60) {
Start-Sleep -Seconds $_
}
}
}
}
# A function which contains the rollback actions
function Invoke-RollbackActions {
Context "rollback actions" {
Context "Azure Chaos Studio" {
BeforeAll {
# If the first pass failed, skip the rollback setup phase.
if (!(Test-SteadyStateHypothesisFirstPass)) { return; }
# Request cancellation of the Azure Chaos Studio experiment
$cancelExperimentPath = '{0}/cancel?api-version=2021-09-15-preview' -f $ChaosExperimentResource.Id
$cancelExperimentReponse = Invoke-AzRestMethod -Path $cancelExperimentPath -Method POST
$experimentStartOperationResult = $cancelExperimentReponse.Content | ConvertFrom-Json
# Track the status of the cancellation request
$experimentStatusResponse = Invoke-AzRestMethod -Uri $experimentStartOperationResult.statusUrl -Method Get
$experimentStatusResult = $experimentStatusResponse.Content | ConvertFrom-Json
while ($experimentStatusResponse.StatusCode -ne 404 -AND $experimentStatusResult.properties.Status -inotin "Success", "Failed", "Cancelled" ) {
Start-Sleep -Seconds 5
$experimentStatusResult = $null
$experimentStatusResponse = Invoke-AzRestMethod -Uri $experimentStartOperationResult.statusUrl -Method Get
$experimentStatusResult = $experimentStatusResponse.Content | ConvertFrom-Json
}
}
BeforeEach {
# If the first pass has failed, set each it block as skipped.
Skip-SteadyStateHypothesisWhenFirstPassFailed
}
It "should accept the cancelation request of the Azure Chaos Studio experiment" {
$cancelExperimentReponse.StatusCode | Should -Be 202
}
It "should have rolled back the Azure Chaos Studio experiment" {
$experimentStatusResult.properties.Status | Should -BeIn "Success", "Cancelled"
}
}
}
}
}
BeforeAll {
# Ensure any previous state is cleaned up
# before any of these test start
Reset-SteadyStateHypothesis
}
# Only run the pester chaos experiment only if the Azure Chaos Studio Experiment
# exists and it is not performing anything else.
#
# By using the -ForEach param we can pass the ChaosExperimentResource variable
# from the discovery phase into the run phase.
Describe -Name "Chaos Experiment: $resourceGroupName\$ExperimentName - Testing endpoint $WebsiteUrl" `
-Tag "Chaos" `
-ForEach @( @{ChaosExperimentResource = $ChaosExperimentResource }) `
-Skip:($null -eq $ChaosExperimentResource -OR $isExperimentAvailableForStart -eq $false) `
-Fixture {
Invoke-SteadyStateHypothesis
Invoke-ExperimentMethods
Invoke-SteadyStateHypothesis
Invoke-RollbackActions
}
AfterAll {
# Ensure any previous state is cleaned up
# after all tests are done.
Reset-SteadyStateHypothesis
}
I placed those functions in the “BeforeDiscovery” block so I could reference them in the “Describe” block. Being able to call “Invoke-SteadyStateHypothesis” twice without having to repeat the same code is what I wanted to go for, but the trade-off here seems to be that I have to make sure that I manage the steady-state result properly. That means having to call “Complete-SteadyStateHypothesisPass”,"Reset-SteadyStateHypothesis", “Skip-SteadyStateHypothesisWhenFirstPassFailed” and “Test-SteadyStateHypothesisFirstPass” at the appropriate time, which seems OK to me.
So what is the result of a full test run?
Running tests from '/workspaces/blog-2021-12-azure-chaos-studio-pester/chaos-experiments/website.chaos.experiment.tests.ps1'
Describing Chaos Experiment: tvl-azsc-rg\tvl-azcs-blog-experiment - Testing endpoint http://40.74.13.15
Context steady-state hypothesis
[+] should return status code 200 14ms (3ms|11ms)
Context experiment activities
Context Azure Chaos Studio
[+] should have accepted the start the Azure Chaos Studio experiment 6ms (4ms|2ms)
[+] should have the Azure Chaos Studio experiment in running state 11ms (10ms|1ms)
[+] should have a NSG security rule fault action in running state, in the running Azure Chaos Studio experiment 15ms (14ms|1ms)
[+] should pause 60 seconds for changes to take effect. 60.01s (60s|4ms)
Context steady-state hypothesis
[-] should return status code 200 14ms (7ms|8ms)
Steady-state hypothesis failed: Expected 200, but got $null.
at $webResponse.StatusCode | Should -BeSteadyStateHypothesisValue 200, /workspaces/blog-2021-12-azure-chaos-studio-pester/chaos-experiments/website.chaos.experiment.tests.ps1:78
at <ScriptBlock>, /workspaces/blog-2021-12-azure-chaos-studio-pester/chaos-experiments/website.chaos.experiment.tests.ps1:78
Context rollback actions
Context Azure Chaos Studio
[+] should accept the cancelation request of the Azure Chaos Studio experiment 10ms (4ms|6ms)
[+] should have rolled back the Azure Chaos Studio experiment 6ms (5ms|1ms)
Tests completed in 141.86s
Tests Passed: 7, Failed: 1, Skipped: 0 NotRun: 0
That seems like it does pretty much what I would like it to do! The fact that the experiment’s second steady-state hypothesis pass has failed, indicates to me that there is a problem with the setup. The main culprit is easily identified though, as the website is running on a single virtual machine and blocking traffic over port 80 is just the thing it needs to become inaccessible.
What happens when I turn off the virtual machine and run the experiment afterwards?
Describing Chaos Experiment: tvl-azsc-rg\tvl-azcs-blog-experiment - Testing endpoint http://40.74.13.15
Context steady-state hypothesis
[-] should return status code 200 5ms (3ms|2ms)
Steady-state hypothesis failed: Expected 200, but got $null.
at $webResponse.StatusCode | Should -BeSteadyStateHypothesisValue 200, /workspaces/blog-2021-12-azure-chaos-studio-pester/chaos-experiments/website.chaos.experiment.tests.ps1:78
at <ScriptBlock>, /workspaces/blog-2021-12-azure-chaos-studio-pester/chaos-experiments/website.chaos.experiment.tests.ps1:78
Context experiment activities
Context Azure Chaos Studio
[!] should have accepted the start the Azure Chaos Studio experiment is skipped, because the first pass of the steady-state hypothesis did not complete succesfully. 13ms (4ms|9ms)
[!] should have the Azure Chaos Studio experiment in running state is skipped, because the first pass of the steady-state hypothesis did not complete succesfully. 13ms (3ms|10ms)
[!] should have a NSG security rule fault action in running state, in the running Azure Chaos Studio experiment is skipped, because the first pass of the steady-state hypothesis did not complete succesfully. 6ms (4ms|2ms)
[!] should pause <_> seconds for changes to take effect. is skipped, because the first pass of the steady-state hypothesis did not complete succesfully. 5ms (4ms|2ms)
Context steady-state hypothesis
[!] should return status code 200 is skipped, because the first pass of the steady-state hypothesis did not complete succesfully. 21ms (4ms|17ms)
Context rollback actions
Context Azure Chaos Studio
[!] should accept the cancelation request of the Azure Chaos Studio experiment is skipped, because the first pass of the steady-state hypothesis did not complete succesfully. 16ms (9ms|6ms)
[!] should have rolled back the Azure Chaos Studio experiment is skipped, because the first pass of the steady-state hypothesis did not complete succesfully. 13ms (11ms|3ms)
Tests completed in 5.74s
Tests Passed: 0, Failed: 1, Skipped: 7 NotRun: 0
The first pass fails so everything else is essentially skipped. I think that’s pretty… Neat.
Conclusion
This was fun, albeit a little complex. Though now this can serve as a jumping-off point for me to use Pester’s other cool features.. Perhaps I can combine these Chaos experiments with Pester’s capability to output NUnit test results, which I can display as part of an Azure DevOps pipeline.
If you happen to know a simpler way for achieving this sort of setup with Pester 5, feel free to let me know!
Related posts
- Azure Chaos Studio - Public Preview
- Azure Purview
- Azure Resource Locks
- Local OpenShift 4 with Azure App Services on Azure Arc
- Verifying Azure AD tenant availability
- Open Policy Agent
- Azure Red Hat OpenShift 4
- Chaos Engineering on Azure
- Azure Policies & Azure AD PIM breakdown - Part 2
- Azure Policies & Azure AD PIM breakdown - Part 1