prombench: Remote Write receiver tester CLI

[bwplotka]

I propose we created dedicated testing CLI that will allow us:

• Test basic remote write expectation (shape of the data sent, or only if it's parsable)
• It will allow injecting different ingestion errors or behavior (different err codes, fake "delays", partial writes)
• It will be fast enough, so if needed we benchmark Prometheus sending not the performance of the receiver trying to catch up with the load.

One could use Prometheus with receive enabled, but it won't solve all the above goals.

[bwplotka]

I wrote some quick sink for PRW benchmarking, in personal repo for now, but contributors welcome (can give you access). The idea is that it will work against other protocols too, but we could consider moving it to Prometheus org if useful for Prom scope only.

https://github.com/bwplotka/sink

I am not saying we have to use for prombench, but something useful and to consider for some remote-write mode.

[bwplotka]

I assume https://github.com/bwplotka/sink fixes this issue, until we find otherwise (e.g. that we have to have it outside of my personal repo)

[kushalShukla-web]

@bwplotka, has this issue been closed?

[bwplotka]

Have you seen this comment? #766 (comment)

[kushalShukla-web]

Hi @bwplotka i have created a basic Deployment file to use Sink in prombench. Here #787 .