Multi Feature Merge - Test Engine Authentication, Providers and Power…

… Fx extensions (#380)

* Adding namespace checks and support for Power FX #330

* Adding TestEngine.PlaywrightScript() #335

* Adding TestEngine.PlaywrightAction() #337

* Adding TestEngine.PlaywrightAction docs

* Adding TestEngine.PlaywrightScript docs

* Update TestEngine.PlaywrightScript.md

* Initial portal provider

* Partial PowerApps Portal provider implementation

* WIP Power Apps Portal provider implementation

* Adding TestEngine.GetConnections()

* Test update

* Adding CreateConnection

* Adding TestEngine.CheckConnectionExists

* TestEngine.CreateConnection update

* Review edits

* Add TestEngine.UpdateConnectionReferences()

* Solution update

* Connection list

* Connection and format

* Export connections

* Package update

* NuGet version update

* NuGet Updates

* Playright updates

* Provider update

* Review edits.

* Update TestEngine namespace

* Power Apps Portal update

* Format updates

* Add default certificate provider

* Adding Variables ans Collection support

* Remoding date tets

* Adding TestEngine.SelectSection()

* Adding tests

* Format update

* Adding MDA module and CoE custom page Sample

* Review edit

* Adding browser locale change

* Adding Experimental.SelectControl() for MDA custom pages

* Adding WIP Experimental.SimulateDataverse()

* WIP SimuateDataverse GET list

* Update for $batch and Query

* Basic query use cases

* Update .Net 8.0

* Update to .Net 8

* Additional Context error handling

* Remove legacy player

* Adding SimulateConnector Power Fx function

* Updates for Debug/Trace logging

* Networking monitoring update

* Minior edits

* Simulate update

* WIP record implementation for SimulateDataverse and SimulateConnector

* Sign only Release build record SimulateDataverse and SimulateConnector

* Adding Mouse recording

* Recorder update and Experimental functions

* Review edits

* Reviw edits for .Net 8

* Review edits

* Adding parameter for custom page

* Adding audio recording

* Add audio event tracking

* Asing storage auth provider

* Suppotr for negative test case

* Adding error checks for storage state

* Adding NotificationTitle error detection

* Storage state and permissions example

* Update variable state

* Refactor to common PowerPlatformLogin

* Adding missing common files

* Review changes

* Adding storage state for #389

* Update samples to storagestate

* Adding changes

* Log updates and Test Cases

* Format update and .Net 8.0 build

* Build update

* Pull request review edits

* Review edit

* Review changes

* Update dotnet-format.yml

* Adding docs

.github/workflows/build-test.yml

@@ -21,7 +21,7 @@ jobs:
     runs-on: windows-latest
     strategy:
       matrix:
-        dotnet-version: ['6.0.x']
+        dotnet-version: ['8.0.x']
 
     steps:
       - uses: actions/checkout@v2

.github/workflows/dotnet-format.yml

@@ -17,7 +17,7 @@ jobs:
       - name: Setup .NET Core
         uses: actions/setup-dotnet@v1
         with:
-          dotnet-version: '6.0.x'
+          dotnet-version: '8.0.x'
 
       - name: Install dotnet-format tool
         run: dotnet tool install -g dotnet-format

README.md

@@ -26,8 +26,8 @@ To get started, you will need to clone the Test Engine code from GitHub, locally
 
 ### Prerequisites for building Test Engine
 
-1. Install [.NET Core 6.0.x SDK](https://dotnet.microsoft.com/en-us/download/dotnet/6.0)
-1. Ensure that your `MSBuildSDKsPath` environment variable is pointing to [.NET Core 6.0.x SDK](https://dotnet.microsoft.com/en-us/download/dotnet/6.0).
+1. Install [.NET Core 8.0.x SDK](https://dotnet.microsoft.com/en-us/download/dotnet/8.0)
+1. Ensure that your `MSBuildSDKsPath` environment variable is pointing to [.NET Core 8.0.x SDK](https://dotnet.microsoft.com/en-us/download/dotnet/8.0).
 1. Make sure [PowerShell](https://docs.microsoft.com/en-us/powershell/scripting/install/installing-powershell?view=powershell-7.2) is installed.
 
 ### Build locally

azure-pipelines.yml

@@ -27,9 +27,9 @@ jobs:
 
   steps:
   - task: UseDotNet@2
-    displayName: 'Use dotnet sdk 6.0'
+    displayName: 'Use dotnet sdk 8.0'
     inputs:
-      version: 6.0.x
+      version: 8.0.x
       installationPath: '$(Agent.ToolsDirectory)/dotnet'
 
   - task: CodeQL3000Init@0

build-pipelines/scripts/yaml-integration-tests.sh

@@ -44,7 +44,7 @@ do
        fi                                                                
     done
   if [[ -n "${envId}" && -n "${tenantId}" && -n "${domain}" && -n "${testPlanFile}" && -n "${outputDir}" ]]; then     # null checks on args
-    dotnet run -f net6.0 -- -e ${envId} -t ${tenantId} -d ${domain} -i ${testPlanFile} -o ${outputDir} -q "&PAOverrideFGRollout.OnePlayerStandaloneWebPlayer=false";
-    dotnet run -f net6.0 -- -e ${envId} -t ${tenantId} -d ${domain} -i ${testPlanFile} -o ${outputDir} -q "&PAOverrideFGRollout.OnePlayerStandaloneWebPlayer=true";
+    dotnet run -f net8.0 -- -e ${envId} -t ${tenantId} -d ${domain} -i ${testPlanFile} -o ${outputDir} -q "&PAOverrideFGRollout.OnePlayerStandaloneWebPlayer=false";
+    dotnet run -f net8.0 -- -e ${envId} -t ${tenantId} -d ${domain} -i ${testPlanFile} -o ${outputDir} -q "&PAOverrideFGRollout.OnePlayerStandaloneWebPlayer=true";
   fi
 done

docs/Extensions/PowerAppsPortal.md

@@ -0,0 +1,3 @@
+# Power Apps Portal Provider
+
+The -p "powerapps.portal" provider allow automation of the Power Apps portal using Test Engine.

docs/PowerFX/Pause.md

@@ -1,9 +1,13 @@
 # Pause
 
-`Pause()`
+`Experimental.Pause()`
 
-This will open the interactive Playwright Inspector and wait for the user to resume execution.
+This will open the interactive [Playwright Inspector](https://playwright.dev/dotnet/docs/debug#playwright-inspector) and wait for the user to resume execution. 
+
+## Using with Simulation commands
+
+Using the playwright inspector you can access the [Browser Developer Tools](https://playwright.dev/dotnet/docs/debug#browser-developer-tools) to monitor network traffic between the page and the services
 
 ## Example
 
-`Pause()`
+`Experimental.Pause()`

docs/PowerFX/README.md

@@ -4,11 +4,21 @@ There are several specifically defined functions for the test framework.
 
 - [Assert](./Assert.md)
 - [Screenshot](./Screenshot.md)
-- [Pause](./Pause.md)
 - [Select](./Select.md)
 - [SetProperty](./SetProperty.md)
 - [Wait](./Wait.md)
 
+- [Experimental.Pause](./Pause.md)
+- [Experimental.PlaywrightAction](./PlaywrightAction.md)
+- [Experimental.PlaywrightScript](./PlaywrightAction.md)
+
+## Experimental Functions
+
+The following functions will be enabled in Debug build and when Experimental is enabled as a Namespace
+
+- [Experimental.SimulateConnector](./SimulateConnector.md)
+- [Experimental.SimulateDataverse](./SimulateDataverse.md)
+
 ## Naming
 
 When creating additional functions using [modules](../modules.md) for Power Fx in the Test Engine, it's important to follow naming standards to ensure consistency and readability.
@@ -23,6 +33,14 @@ Here are some guidelines for naming your functions in Power Fx:
 
 By following these naming standards, your Power Fx code will be easier to read and maintain, and other developers will be able to understand your code more easily.
 
+### Use Namespaces
+
+Namespaces should be used for Power Fx functions in the Power Apps Test Engine for several reasons. First, using namespaces ensures that there is no clash with built-in functions, which can cause confusion and errors. By using namespaces, Power Fx functions can be organized and grouped together in a clear and concise manner.
+
+Additionally, namespaces make it clear that these Power Fx functions belong to the Test Engine, and are not part of the larger Power Apps ecosystem. This helps to avoid confusion and ensures that the functions are used appropriately within the context of the Test Engine.
+
+Overall, using namespaces for Power Fx functions in the Power Apps Test Engine is a best practice that helps to ensure clarity, organization, and consistency in the testing process.
+
 ### Using Descriptive Names
 
 Using descriptive names is important because it makes it easier for others (and yourself) to understand what the function or service does. A good name should be concise but also convey the function's or service's purpose. For example, instead of naming a function "Calculate," you could name it "CalculateTotalCost" to make it clear what the function is doing.

docs/PowerFX/SimulateConnector.md

@@ -0,0 +1,41 @@
+# Simulate Connection
+
+The Experimental.SimluateConnection function allows you to simulate requests to Power Platform connector and provide responses without actually making live requests. This is particularly useful for testing and development purposes, as it enables you to create predictable and controlled responses for various scenarios.
+
+```powerfx
+Experimental.SimulateConnection({Name: "connectorname", Action: "actionname", Parameters: {}, Filter: "optionalfilter", Then: {Value: Table()}})
+```
+
+## Parameters
+
+| Name | Description |
+|------|-------------|
+| Name | The name of the connector from thr url of the [connector list](https://learn.microsoft.com/connectors/connector-reference/connector-reference-powerapps-connectors). For example the name of the [Office 365 Users](https://learn.microsoft.com/en-us/connectors/office365users/) is **office365users**
+| Action | The part of the url request that will match against the action
+| Parameters | A Power Fx Record that will be mapped to Query parameters required to me matched
+| Filter | A Power Fx expression that needs to be matched |
+
+## Recording Sample Values
+
+To obtain values for the `Experimental.SimulateConnection()` function you can use the network trace of the Browser Developer Tools when using [Experimental.Pause()](./Pause.md) where you can filter traffic by searching for **/invoke**
+
+## Examples
+
+1. Query user using Power 365 Users connector
+
+```powerfx
+Experimental.SimulateConnection({Name: "office365users", Action: "/v1.0/me", Then: {
+    displayName: "Sample User",
+    "id": "c12345678-1111-2222-3333-44445555666",
+    "jobTitle": null,
+    "mail": "[email protected]",
+    "userPrincipalName": "[email protected]",
+    "userType": "Member"
+}})
+```
+
+2. Query groups using Power 365 groups connector
+
+```powerfx
+Experimental.SimulateConnection({Name: "office365groups", Filter: "name = '[email protected]'", Then: Table()})
+```

docs/PowerFX/SimulateDataverse.md

@@ -0,0 +1,46 @@
+# Simulate Dataverse
+
+The Experimental.SimulateDataverse function allows you to simulate responses from the Dataverse without actually querying the live data. This is particularly useful for testing and development purposes, as it enables you to create predictable and controlled responses for various scenarios.
+
+```powerfx
+Experimental.SimulateDatarse({ Action: "query", Entity: "TableName", When: { Field: "value" }, Then: Table({Name: "Test"}) })
+```
+
+| Name | Description |
+|------|-------------|
+| Action | The dataverse action to simulate from Query, Create, Update, Delete
+| Entity | The name pluralized entity name from [metadata](https://learn.microsoft.com/power-apps/developer/data-platform/webapi/web-api-service-documents)
+| When | The optional query string to apply
+| Filter | A Power Fx expression that needs to be matched. This will automatically be mapped to odata $filter command |
+| When | The Power Fx table to return in the odata value response that will be returned to the Power App
+
+## Recording Sample Values
+
+To obtain values for the `Experimental.SimulateDataverse()` function you can use the network trace of the Browser Developer Tools when using [Experimental.Pause()](./Pause.md) where you can filter traffic by searching for **/api/data/v**
+
+## Example
+
+1. Simulate a Query Response with Sample Data
+
+When the Power App queries all accounts, respond with sample data:
+
+```powerfx
+Experimental.SimulateDataverse({Action:"query",Entity: "accounts", Then: Table({accountid: "a1234567-1111-2222-3333-44445555666", name: "Test"}) });
+```
+
+2. Simulate a Query with Specific Conditions
+
+When make request with account with query name of Other return no results
+
+```powerfx
+Experimental.SimulateDataverse({Action:"query",Entity: "accounts", When: {Name: "Other"}, Then: Table()});
+```
+
+## Why This Function is Useful
+The `Experimental.SimulateDataverse()` function is useful because it allows developers and makers to:
+
+1. **Test and Debug**: Simulate different scenarios and responses without affecting live data, making it easier to test and debug applications.
+1. **Predictable Results**: Create controlled and predictable responses, which is essential for automated testing and ensuring consistent behavior.
+1. **Development Efficiency**: Speed up the development process by allowing developers to work with simulated data instead of waiting for actual data to be available.
+
+By using this function, you can ensure that your Power Apps behave as expected in various scenarios, leading to more robust and reliable applications.

docs/PowerFX/TestEngine.PlaywrightAction.md

@@ -0,0 +1,36 @@
+# Experimental.PlaywrightAction
+
+` Experimental.PlaywrightAction(Locator, Action)`
+
+` Experimental.PlaywrightAction(Url, Action)`
+
+This use the locators or Url to apply an action to the current web page.
+
+## Locators
+
+When selecting actions that require a locator you can make use of [CSS Selectors](https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Selectors) or XPath queries.
+
+Locators for web pages are based on Playwright locators. More information on locators is available from [Playwright documentation](https://playwright.dev/docs/other-locators).
+
+Playwright also supports experimental React and vue base selectors that can be useful for selecting elements on code first extensions like PCF controls within a Power App.
+
+## Actions
+
+The following actions are supported
+
+| Action   | Description                            |
+|----------|----------------------------------------|
+| click    | Select matching locator items          |
+| exists   | Returns True or False is locator exist |
+| navigate | Navigate to the url                    |
+| wait     | Wait for locator items to exist        |
+
+## Examples
+
+` Experimental.PlaywrightAction("//button", "click")`
+
+` Assert(Experimental.PlaywrightAction("//button", "exists") = true)`
+
+` Experimental.PlaywrightAction("https://www.microsoft.com", "navigate")`
+
+` Experimental.PlaywrightAction("//button", "wait")`

docs/PowerFX/TestEngine.PlaywrightScript.md

@@ -0,0 +1,44 @@
+# TestEngine.PlaywrightScript
+
+`TestEngine.PlaywrightScript(csxFileName)`
+
+The PlaywrightScript function provides a "no cliffs" extensibility for Test Engine providing the ability to execute CSharp Scripts (*.csx) files inside a Test Engine web provider based test that uses Playwright as web page test framework.
+
+You can use the playwright inspector to record C# commands to build the C# script
+
+## C# Script
+
+This action takes advantage of [dotnet-script](https://github.com/dotnet-script/dotnet-script) and the underlying [Rosyln](https://github.com/dotnet/roslyn) compiler to allow projectless scripting of Playwright code. The Action assumes the following:
+
+1. Any required .Net Assemblies are globally available or in the current folder and can be loaded using #r compiler directive
+2. A public class named **PlaywrightScript** MUST exist
+3. A method with **public static void Run(IBrowserContext context, ILogger logger)** MUST exist
+
+## Sample Test
+
+A sample [testPlan.fx.yaml](../../samples/playwrightscript/testPlan.fx.yaml) and [sample.csx](../../samples/playwrightscript/sample.csx) provide a demonstration of how this action can be integrated into a Test Engine test.
+
+## Example
+
+` TestEngine.PlaywrightScript("sample.csx")
+
+Where sample could use template to include Playwright
+
+```csharp
+#r "Microsoft.Playwright.dll"
+#r "Microsoft.Extensions.Logging.dll"
+using Microsoft.Playwright;
+using Microsoft.Extensions.Logging;
+using System.Linq;
+using System.Threading.Tasks;
+
+public class PlaywrightScript {
+    public static void Run(IBrowserContext context, ILogger logger) {
+        Execute(context, logger).Wait();
+    }
+
+    public static async Task Execute(IBrowserContext context, ILogger logger) {
+        // Insert your code here
+    }
+}
+```

samples/.gitignore

@@ -0,0 +1 @@
+**\config.json

samples/basicgallery/README.md

@@ -0,0 +1,24 @@
+# Overview
+
+This Power Apps Test Engine sample demonstrates how to basic gallery of a canvas application
+
+## Usage
+
+2. Get the Environment Id and Tenant of the environment that the solution has been imported into
+
+3. Create config.json file using tenant, environment and user1Email
+
+```json
+{
+    "environmentId": "a0000000-1111-2222-3333-444455556666",
+    "tenantId": "ccccdddd-1111-2222-3333-444455556666",
+    "installPlaywright": false,
+    "user1Email": "[email protected]"
+}
+```
+
+4. Execute the test
+
+```pwsh
+.\RunTests.ps1
+```

samples/basicgallery/RunTests.ps1

@@ -0,0 +1,31 @@
+# Get current directory so we can reset back to it after running the tests
+$currentDirectory = Get-Location
+
+$config = Get-Content -Path .\config.json -Raw  | ConvertFrom-Json
+$tenantId = $config.tenantId
+$environmentId = $config.environmentId
+$user1Email = $config.user1Email
+
+if ([string]::IsNullOrEmpty($environmentId)) {
+    Write-Error "Environment not configured. Please update config.json"
+    return
+}
+
+# Build the latest debug version of Test Engine from source
+Set-Location ..\..\src
+dotnet build
+
+if ($config.installPlaywright) {
+    Start-Process -FilePath "pwsh" -ArgumentList "-Command `"..\bin\Debug\PowerAppsTestEngine\playwright.ps1 install`"" -Wait
+} else {
+    Write-Host "Skipped playwright install"
+}
+
+Set-Location ..\bin\Debug\PowerAppsTestEngine
+$env:user1Email = $user1Email
+# Run the tests for each user in the configuration file.
+dotnet PowerAppsTestEngine.dll -u "storagestate" -p "canvas" -a "none" -i "$currentDirectory\testPlan.fx.yaml" -t $tenantId -e $environmentId
+dotnet PowerAppsTestEngine.dll -u "storagestate" -p "canvas" -a "none" -i "$currentDirectory\testPlanForRegionUseSemicolonAsSeparator.fx.yaml" -t $tenantId -e $environmentId
+
+# Reset the location back to the original directory.
+Set-Location $currentDirectory

samples/basicgallery/testPlan.fx.yaml

@@ -1,3 +1,4 @@
+# yaml-embedded-languages: powerfx
 testSuite:
   testSuiteName: Basic Gallery
   testSuiteDescription: Verifies that you can interact with controls within a basic gallery
@@ -35,4 +36,4 @@ environmentVariables:
   users:
     - personaName: User1
       emailKey: user1Email
-      passwordKey: user1Password
+      passwordKey: NotNeeded

samples/basicgallery/testPlanForRegionUseSemicolonAsSeparator.fx.yaml

@@ -1,3 +1,4 @@
+# yaml-embedded-languages: powerfx
 testSuite:
   testSuiteName: Basic Gallery
   testSuiteDescription: Verifies that you can interact with controls within a basic gallery
@@ -32,4 +33,4 @@ environmentVariables:
   users:
     - personaName: User1
       emailKey: user1Email
-      passwordKey: user1Password
+      passwordKey: NotNeeded

samples/buttonclicker/README.md

@@ -4,17 +4,21 @@ This Power Apps Test Engine sample demonstrates how to clicking button of a canv
 
 ## Usage
 
-1. Build the Test Engine solution
+2. Get the Environment Id and Tenant of the environment that the solution has been imported into
 
-2. Get the Environment Id and Tenant of the environment that the samplication solution has been imported into
+3. Create config.json file using tenant, environment and user1Email
 
-3. Execute the test for custom page changing the example below to the url of your organization using browser persistant cookies
+```json
+{
+    "environmentId": "a0000000-1111-2222-3333-444455556666",
+    "tenantId": "ccccdddd-1111-2222-3333-444455556666",
+    "installPlaywright": false,
+    "user1Email": "[email protected]"
+}
+```
+
+4. Execute the test
 
 ```pwsh
-cd bin\Debug\PowerAppsEngine
-dotnet PowerAppsTestEngine.dll -i ..\..\..\samples\buttonclicker\testPlan.fx.yaml -e 00000000-0000-0000-0000-11112223333 -t 11112222-3333-4444-5555-666677778888 -u browser -p canvas
+.\RunTests.ps1
 ```
-
-NOTES:
-- If the BrowserCache folder does not exist with valid Persistent Session cookies an interactive login will be required
-- After an interactive login has been completed a headless test session can be run