0.4.0

Introduce `http_gen` (#52)

* Implement http_gen

This commit introduces a new program into the project, http_gen. It's similar to
file_gen except it works over HTTP. I've pretty substantially reworked the
configuration of file_gen and extracted common code from file_gen / http_gen,
though the two are still quite similar. I haven't tested http_gen against vector
yet and the configuration documentation must be updated.

Signed-off-by: Brian L. Troutwine <[email protected]>

* Slightly trim depedency list

Signed-off-by: Brian L. Troutwine <[email protected]>

* Work on http_gen + vector bug

When I apply http_gen with DatadogLogs as variant vector will sometimes complain
about sheered payloads. I've added tests to confirm that lading_common generates
blocks that deserialize in this regard.

Signed-off-by: Brian L. Troutwine <[email protected]>

* No longer coordinate omission

As of this commit http_gen will no longer coordinate its throughput with the
target. Total successful throughput is still limited by target capacity but
request generation now is a combination of internal clock and maximum connection
saturation.

Signed-off-by: Brian L. Troutwine <[email protected]>

* correct test failure

Signed-off-by: Brian L. Troutwine <[email protected]>

* Also release http_gen binary

Signed-off-by: Brian L. Troutwine <[email protected]>

0.3.8

multi-artifact release

Signed-off-by: Brian L. Troutwine <[email protected]>

0.3.7

Correct FoundationDB building (#44)

This commit addresses important mistakes in the FoundationDB generator. Firstly,
the size of the serialized FoundationDB json lines will cause some block sizes
to exceed the maximum bytes per second, a problem. Second, the names of fields
were not correct. Numbers are now quoted as in the FDB logs.

Signed-off-by: Brian L. Troutwine <[email protected]>

0.3.6

Add the ability to generate limited FoundationDB logs

This commit adds a limited ability to generate FoundationDB logs. There's more
variants to be added and any fields in the existing variants that have an
internal complex structure have been dropped.

Signed-off-by: Brian L. Troutwine <[email protected]>

0.3.5

Crash when a worker crashes (#38)

* Crash when a worker crashes

We have discovered situations where file_gen runs out of disk space and then
quietly stop. This happens when testing vector once vector falls behind.

Signed-off-by: Brian L. Troutwine <[email protected]>

* Always truncate files on startup

Signed-off-by: Brian L. Troutwine <[email protected]>

0.3.4

Make generated data less blank (#37)

This commit ensures that the data we generate from Ascii and Json has a definite
size to it, primarily by selecting from a fixed array of sizes. This is... a
little less arbitrary than I would have liked but it's better than a file full
of blank lines.

Signed-off-by: Brian L. Troutwine <[email protected]>

0.3.3

Use less CPU for slow targets (#36)

This commit re-introduces buffered writing, using less CPU when we have slow
targets. Once we have a need for GB/s range writing we'll need to figure
something else out but for now this is good enough. I'm more interested in
addressing generated data quality.

Signed-off-by: Brian L. Troutwine <[email protected]>

0.3.2

Use rayon to push faster byte block builds (#35)

This commit breaks up the block building into iterators so that we can bring
rayon to bear in their construction. This significantly reduces startup time.

Signed-off-by: Brian L. Troutwine <[email protected]>

0.3.1

Fix file writing (#34)

I worked quite hard yesterday on the previous commit and convinced myself of a
non-thing. This resulted in there being no logs written to disk.

Signed-off-by: Brian L. Troutwine <[email protected]>

0.3.0

Pivot to the use of Arbitrary (#33)

This commit contains some interesting work. Firstly, I have replaced the
`buffer` code with the use of `Arbitrary`, reducing the project's dependence on
wonky custom generation code. I have also removed any `BufWriter` in the spin
loop, pivoting instead to the use of fixed MiB sized "blocks" that are
constructed ahead of time before entering the spin loop. This slows down
construction some -- hence the introduction of rayon -- but does significantly
improve on file_gen's throughput capability.

The config file has changed somewhat. We'll need to address the loss of line
length control but this work feels overlong as is.

Resolves #32
Resolves #31
REF #10

Signed-off-by: Brian L. Troutwine <[email protected]>