Promitor exports metric with name Size to Prometheus that does not match the one in Azure Monitor

[esierrapena]

Report

I have deployed the promitor scraper in a k8s cluster but metric "Size" of Eventhub resource exported to Prometheus doesn't match the one i can see in Azure Portal.

Expected Behavior

Same metric in Azure portal as in Prometheus

Actual Behavior

Different metric value

Steps to Reproduce the Problem

1. Define configmap

apiVersion: v1
data:
  metrics-declaration.yaml: |-
    version: v1
    azureMetadata:
      tenantId: a5bXXXXXXXXXXXXXXXX4bf20
      subscriptionId: 8f17XXXXXXXXXXXXXX680e1e0
      resourceGroupName: test-northeurope
    metricDefaults:
      aggregation:
        interval: 00:01:00
      scraping:
        schedule: "* * * * *"
    metrics:
      - name: azure_eventhubs_size
        description: "Size of an EventHub in Bytes."
        resourceType: EventHubs
        azureMetricConfiguration:
          metricName: Size
          aggregation:
            type: Average
        resources:
          - namespace: test-eventhub-namespace-hgu

2. Check metrics in Grafana with Azure Monitor as datasource (Figure 1) and Prometheus as datasource (Figure 2). Figure 3 metrics in Azure portal
   Figure 1
   
   Figure 2
   
   Figure 3
   

Component

Scraper

Version

2.0.0

Configuration

Configuration:

apiVersion: v1
data:
  metrics-declaration.yaml: |-
    version: v1
    azureMetadata:
      tenantId: a5bXXXXXXXXXXXXXXXX4bf20
      subscriptionId: 8f17XXXXXXXXXXXXXX680e1e0
      resourceGroupName: test-northeurope
    metricDefaults:
      aggregation:
        interval: 00:01:00
      scraping:
        schedule: "* * * * *"
    metrics:
      - name: azure_eventhubs_size
        description: "Size of an EventHub in Bytes."
        resourceType: EventHubs
        azureMetricConfiguration:
          metricName: Size
          aggregation:
            type: Average
        resources:
          - namespace: test-eventhub-namespace-hgu

Logs

No response

Platform

Microsoft Azure

Contact Details

No response

[tomkerkhove]

Thanks for reporting, can you share the configuration of the graph in Azure Portal please? Does it also use the same aggregation as Promitor (00:01:00)?

[tomkerkhove]

Using 1 minute as an aggregation is also not ideal since Azure Monitor has a lag, so I'm not sure if this is related but better alignt the Azure Portal metrics with the Promitor configuration and verify.

Relates to #1290 & #1563

[esierrapena]

hi! seems to work with the recommended configuration, closing this issue..

[tomkerkhove]

Huh so it was just a difference in the aggregation then? Thanks for letting me know!

[esierrapena]

Now, seems to work well with both configuration as you can see below:

  metrics-declaration.yaml: |-
    version: v1
    azureMetadata:
      tenantId: xxxxx
      subscriptionId: xxxxxxxxxx
      resourceGroupName: xxxxxxx
    metricDefaults:
      aggregation:
        interval: 00:01:00
      scraping:
        schedule: "* * * * *"
    metrics:
      - name: azure_eventhubs_size
        description: "Size of an EventHub in Bytes."
        resourceType: EventHubs
        azureMetricConfiguration:
          metricName: Size
          aggregation:
            type: Average
          dimension:
            name: EntityName
        resources:
          - namespace: test-eventhub-namespace-hgu
  runtime.yaml: |-
    server:
      httpPort: 88
    metricSinks:
      prometheusScrapingEndpoint:
        metricUnavailableValue: "NaN"
        enableMetricTimestamps: "true"
        baseUriPath: "/metrics"
    metricsConfiguration:
      absolutePath: /config/metrics-declaration.yaml
    telemetry:
      containerLogs:
        isEnabled: "true"
      defaultVerbosity: "Error"

Metrics in grafana

Metrics in Azure Portal

[tomkerkhove]

Good to hear, thanks for sharing!