Expose more circuit metrics

[rgarcia]

In order to calibrate our circuits and roll them out safely, we'd like to run circuits in a way that does not actually block requests, but just logs circuit metrics like request times, number of concurrent requests, and error rate (basically any metric that could trigger timeouts or the circuit opening). The idea would be to look at these metrics and set reasonable thresholds.

AFAICT, it's fairly difficult to get these metrics and log them. The two options I see are

• plugins, e.g. write/register something very similar to DefaultMetricCollector and log metrics like error rate. This isn't a complete solution, though, since the number of concurrent requests lives in the executorPool, which AFAICT is independent of the metric collectors.

• Serve up the SSE port, and in the same app set up a client to the SSE port that logs circuit metrics. This would provide all of the metrics that could trigger circuits opening, but seems needlessly complex...

Is there any other option? If not, is there any change to hystrix-go that you'd be open to accepting to accommodate the use case we have in mind? Thanks!