Adding the PowerApps Solution Checker to your Azure DevOps Pipeline

Continuous Integration and Delivery is somewhat passé these days, but what is often missed is the need for good tests and analysis in your build pipeline. The PowerApps team has been working hard on the Solution Checker over the last year, and it's become an essential part of every PowerApps solution development process. If you have a solution that is going to be put into App Source, you'll need to make sure it passes a special set of rules specifically for App Source solutions.

This post shows you how to add the Solution Checker to your Build pipeline.

Step 1 - Application User

Before you can run the solution checker PowerShell module, you'll need to create an Application User in your Azure Active Directory Tenant. There is a great set of instructions in the PowerApps Solution Checker documentation - https://docs.microsoft.com/en-gb/powershell/powerapps/get-started-powerapps-checker?view=pa-ps-latest

Step 2- PowerShell Script

So that our Build Pipeline can run the Solution Checker, we add a PowerShell script to our repo.

Note that you'll need to:

Create a secured variable in your pipeline to store the client secret so it can be passed to the script as a parameter.
Update for your Tennant and Application ID
Update for the location of your solution.zip that you've built in the pipeline. Mine is

$env:BUILD_SOURCESDIRECTORY\DeploymentPackage\DeploymentPackage\bin\Release\PkgFolder\Solution.zip

Your Script should look something like:

param (
    [string]$clientsecret
 )
# Requires App User be set up https://docs.microsoft.com/en-gb/powershell/powerapps/get-started-powerapps-checker?view=pa-ps-latest
$env:TENANTID = "65483ec4-ac1c-4cba-91ca-83d5b0ba6d88"
$env:APPID = "2fa068dd-7b61-415b-b8b5-c4b5e3d28f61"

$ErrorActionPreference = "Stop"
install-module Microsoft.PowerApps.Checker.PowerShell -Force -Verbose -Scope CurrentUser

$rulesets = Get-PowerAppsCheckerRulesets
$rulesetToUse = $rulesets | where Name -NE 'AppSource Certification'

$analyzeResult = Invoke-PowerAppsChecker -Geography UnitedStates -ClientApplicationId "$env:APPID" -TenantId "$env:TENANTID" -Ruleset $rulesetToUse `
    -FileUnderAnalysis "$env:BUILD_SOURCESDIRECTORY\DeploymentPackage\DeploymentPackage\bin\Release\PkgFolder\Solution.zip" `
    -OutputDirectory "$env:BUILD_SOURCESDIRECTORY" `
    -ClientApplicationSecret (ConvertTo-SecureString -AsPlainText -Force -String $clientsecret)

# Unzip and results
Expand-Archive -LiteralPath "$($analyzeResult.DownloadedResultFiles.Get(0))" -DestinationPath "$env:BUILD_SOURCESDIRECTORY" 

#Rename
$extractedFile = $($analyzeResult.DownloadedResultFiles.Get(0))
$extractedFile = $extractedFile -replace ".zip", ".sarif"
Rename-Item -Path $extractedFile -NewName "PowerAppsCheckerResults.sarif"

If ($analyzeResult.IssueSummary.CriticalIssueCount -ne 0 -or $analyzeResult.IssueSummary.HighIssueCount -ne 0) {
Write-Error -Message "Critical or High issue in PowerApps Checker" -ErrorAction Stop
}

You can change the ruleset and add overrides as per https://docs.microsoft.com/en-gb/powershell/module/microsoft.powerapps.checker.powershell/Invoke-PowerAppsChecker?view=pa-ps-latest

Step 3 - Call and Collect Results in your build pipeline

I'm assuming that you are using AzureDevOps YAML pipelines. If not, I'd recommend you do it because it makes source control and versioning of your pipelines so much easier.

I have three tasks for the Solution Checker as follows:

# PowerAppsChecker
- task: PowerShell@2
  displayName: Solution Checker
  inputs:
    filePath: 'BuildTools\BuildScripts\SolutionChecker.ps1'
    arguments: '"$(ClientSecret)"'
    errorActionPreference: 'continue'

- task: CopyFiles@2
  displayName: Collect - Solution Checker Results
  inputs:
    Contents: '**/PowerAppsCheckerResults.sarif'
    TargetFolder: '$(Build.ArtifactStagingDirectory)'

- task: PublishBuildArtifacts@1
  displayName: Publish CodeAnalysisLogs
  inputs:
    PathtoPublish: '$(Build.ArtifactStagingDirectory)/PowerAppsCheckerResults.sarif'
    ArtifactName: 'CodeAnalysisLogs'
    publishLocation: 'Container'

The first task runs the PowerShell script, and the second and third collects the results so that we can report on them.

To ensure that the $(ClientSecret) parameter is provided, you need to add a pipeline variable for the same:

Step 4 - Reporting the results

The Solution Checker outputs the results in a 'Static Analysis Results Interchange Format' (SARIF) which is a standard format. There are various viewers you can use, but I find having the results directly in the pipeline very useful.

You will need to install the 'Sarif Viewer Build Tab' - https://marketplace.visualstudio.com/items?itemName=sariftools.sarif-viewer-build-tab

Once you've got this working, it'll scan your build artifacts for a sarif file and show the results!

So that's it! When you run your pipeline (which I recommend you do every time a new commit is made to the source branch), the solution will be automatically run through the solution checker, and if there are any critical issues, the build will fail.

If you do find that there are some critical issues that are false positives (which can happen), you can exclude those rules by modifying your script to something like:

$overrides = New-PowerAppsCheckerRuleLevelOverride -Id 'il-avoid-parallel-plugin' -OverrideLevel Informational

$analyzeResult = Invoke-PowerAppsChecker -RuleLevelOverrides $overrides `
...

Hope this helps!

@ScottDurow