-
Notifications
You must be signed in to change notification settings - Fork 25
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bug: Metrics related to Outbound Traffic are not exported after enabling the appmesh metric extension re #488
Comments
Hi @ssun3, I confirmed that AppMesh public image v1.27.2.0 works for both inbound and outbound metrics. Can you check if there were requests send to upstream? E.g Envoy stats metrics with prefix There are known issues #484 and #485 related to Envoy metrics. If you met the same issue, Envoy logs can help identify them. The issues are fixed with the latest release of v1.27.3.0. |
Hi @liubnu. Thanks for replying. I have tried updating the appmesh public image to both v.1.27.2.0 as well as v.1.27.3.0. However I still do not see the expected metrics related to Outbound Traffic. When you confirmed that the image works for both inbound and outbound metrics, do you mean that you see metrics for: envoy_appmesh_TargetProcessedBytes when curling the http://localhost:9901/stats/prometheus endpoint? Using the latest envoy images : (v1.27.2.0 as well as v.1.27.3.0.) and execing into the container namespace:
I still only see InboundMetrics I can confirm that there were requests sent to upstream. Have pasted envoy_cluster_upstream metrics below.
Apologise if there is something I am doing incorrectly. Thank you for your help. |
Glad to help. As the metrics show, the egress traffic goes through |
Yes that was it! Link to relevant documentation in case anyone else encounters this:
Thank you so much @liubnu ! <3 |
Summary
Envoy sidecar container configured with APPMESH_METRIC_EXTENSION_VERSION=1 does not export metrics related to outbound traffic (e.g. envoy.appmesh.TargetProcessedBytes, envoy.appmesh.HTTPCode_Target_2XX_Count ...)
Steps to Reproduce
Sample anonymised task config:
Metrics related to Inbound Traffic are available:
But Metrics related to Outbound Traffic are not:
Are you currently working around this issue?
Have to use a metric-log-filter-alarm to catch all Exceptions on the envoy log stream
Additional context
Platform: ECS:
Envoy Image: ws-appmesh-envoy:v1.27.2.0-prod
Thank you :)
The text was updated successfully, but these errors were encountered: