Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implemented overriding crank behavior for ASP.NET Benchmarks #4065

Merged
merged 2 commits into from
Mar 20, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,53 @@ public static (string, string) Build(ASPNetBenchmarksConfiguration configuration
// Add the extra metrics by including the configuration.
commandStringBuilder.Append($" --config {Path.Combine("Commands", "RunCommand", "BaseSuite", "PercentileBasedMetricsConfiguration.yml")} ");

return (processName, commandStringBuilder.ToString());
string commandString = commandStringBuilder.ToString();

// Apply overrides.
if (!string.IsNullOrEmpty(configuration.benchmark_settings.override_arguments))
{
List<KeyValuePair<string, string>> overrideCommands = GetCrankArgsAsList(configuration.benchmark_settings.override_arguments);
if (overrideCommands.Count > 0)
{
// Take the current commands and first replace all the keys that match the override commands.
// Subsequently, add the new overrides and then convert the key-value pair list back to a string.
List<KeyValuePair<string, string>> currentCommands = GetCrankArgsAsList(commandString);
foreach (var item in overrideCommands)
{
var existing = currentCommands.Where(kv => kv.Key == item.Key).ToList();
foreach (var kv in existing)
{
if (kv.Key != null)
{
currentCommands.Remove(kv);
}
}

currentCommands.Add(item);
}

commandString = string.Join(" ", currentCommands.Select(c => $"--{c.Key} {c.Value}"));
}
}

return (processName, commandString);
}

internal static List<KeyValuePair<string, string>> GetCrankArgsAsList(string input)
{
var keyValuePairs = new List<KeyValuePair<string, string>>();
var splitStr = input.Split(new[] { "--" }, StringSplitOptions.RemoveEmptyEntries);

foreach (var item in splitStr)
{
var keyValue = item.Split(' ', StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries);
if (keyValue.Length == 2)
{
keyValuePairs.Add(new KeyValuePair<string, string>(keyValue[0], keyValue[1]));
}
}

return keyValuePairs;
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ public class BenchmarkSettings
{
public string? benchmark_file { get; set; }
public string additional_arguments { get; set; } = "";
public string? override_arguments { get; set; } = "";
public List<string> benchmarkFilters { get; set; } = new();
}

Expand Down
13 changes: 13 additions & 0 deletions src/benchmarks/gc/GC.Infrastructure/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -222,6 +222,19 @@ benchmark_settings:

If there is a match, these filters will run in the order specified in the yaml file.

###### How To Override Parameters

You can override parameters specified in the benchmark csv file by replacing all instances of the command arg with values in the `override_arguments` field.

```yaml
benchmark_settings:
benchmark_file: C:\InfraRuns\RunNew_All\Suites\ASPNETBenchmarks\ASPNetBenchmarks.csv
additional_arguments: --chart --chart-type hex
override_arguments: --profile aspnet-citrine-win
```

As an example based on the configuration immediately above, all `--profile` values will be replaces with `--profile aspnet-citrine-win`.

## All Commands

The infrastructure can be run in modular manner. What this means is that you can invoke a particular command that runs some part of the infrastructure. A list of all the commands can be found here:
Expand Down
Loading