Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(openobserve transform): Add OpenObserve as an officially supported sink #21531

Open
wants to merge 18 commits into
base: master
Choose a base branch
from

Conversation

chaitanya-sistla
Copy link

@chaitanya-sistla chaitanya-sistla commented Oct 17, 2024

This PR adds OpenObserve as an officially supported sink in Vector's documentation. OpenObserve utilizes an HTTP-based API for data ingestion, and this PR proposes updating the documentation with an example configuration for users to follow.

No new functionality is required, as the current HTTP sink already serves the purpose effectively. The primary goal is to improve user experience by offering clear guidance for setting up Vector with OpenObserve, ensuring confidence in its usage as a supported observability solution.

type = "http"
inputs = [ "source_or_transform_id" ]
uri = "http://localhost:5080/api/default/default/_json"
method = "post"
auth.strategy = "basic"
auth.user = "[email protected]"
auth.password = "v9Ca7qHaMKELlDtU" (fake)
compression = "gzip"
encoding.codec = "json"
encoding.timestamp_format = "rfc3339"
healthcheck.enabled = false

Why This Matters?
By officially documenting OpenObserve as a supported sink, we provide clear setup instructions for users, allowing them to confidently configure Vector for OpenObserve.

Closes: #21514

@chaitanya-sistla chaitanya-sistla requested review from a team as code owners October 17, 2024 04:17
@bits-bot
Copy link

bits-bot commented Oct 17, 2024

CLA assistant check
All committers have signed the CLA.

@github-actions github-actions bot added the domain: external docs Anything related to Vector's external, public documentation label Oct 17, 2024
@chaitanya-sistla
Copy link
Author

@jszwedko could you help me review this? Is anything pending from my end?

Copy link
Member

@pront pront left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello, this PR doesn't require any code changes and it seems more like a user guide on how to configure an existing sink. It feels like https://github.com/vectordotdev/vector-demos might be the right place for this. Or it could live under a new Community section in https://vector.dev/guides/

Cargo.toml Show resolved Hide resolved
@chaitanya-sistla chaitanya-sistla force-pushed the feature/openobserve-sink branch 9 times, most recently from d9abffa to e9bd1fe Compare October 18, 2024 14:14
@github-actions github-actions bot added the domain: sinks Anything related to the Vector's sinks label Oct 20, 2024
@chaitanya-sistla chaitanya-sistla marked this pull request as draft October 20, 2024 18:42
@chaitanya-sistla chaitanya-sistla marked this pull request as ready for review October 21, 2024 01:30
@chaitanya-sistla chaitanya-sistla marked this pull request as draft October 21, 2024 01:30
@@ -104,6 +104,8 @@ on:
value: ${{ jobs.int_tests.outputs.nginx }}
opentelemetry:
value: ${{ jobs.int_tests.outputs.opentelemetry }}
openobserve:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

int_tests is a GitHub job that detects if a branch contains changes to integration tests. Since you didn't added any integration tests dedicated to openobserve you should revert all changes to .github/workflows/*,

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make sense

Copy link
Member

@pront pront left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👋 Left some nits. Since it affects the generated docs, you can run make generate-component-docs && make check-component-docs to regren and validate.

Makefile Outdated Show resolved Hide resolved
src/sinks/openobserve.rs Outdated Show resolved Hide resolved
src/sinks/openobserve.rs Show resolved Hide resolved
src/sinks/openobserve.rs Show resolved Hide resolved
prabhatsharma and others added 2 commits October 30, 2024 10:43
Apply suggestions

Co-authored-by: Pavlos Rontidis <[email protected]>
Copy link
Member

@pront pront left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, this looks almost ready to merge 🚀

One comment here.

removing the trailing space
@chaitanya-sistla
Copy link
Author

I have removed the space @pront

@pront pront enabled auto-merge October 31, 2024 20:09
@chaitanya-sistla chaitanya-sistla changed the title Add OpenObserve as an officially supported sink feat: Add OpenObserve as an officially supported sink Nov 5, 2024
auto-merge was automatically disabled November 5, 2024 02:42

Head branch was pushed to by a user without write access

@pront pront enabled auto-merge November 5, 2024 15:59
@pront
Copy link
Member

pront commented Nov 6, 2024

Please add a changelog entry. You can find details on this document.

auto-merge was automatically disabled November 7, 2024 02:47

Head branch was pushed to by a user without write access

@chaitanya-sistla
Copy link
Author

added @pront

@pront pront enabled auto-merge November 7, 2024 19:25
auto-merge was automatically disabled November 8, 2024 03:11

Head branch was pushed to by a user without write access

@pront pront enabled auto-merge November 8, 2024 16:18
@pront pront disabled auto-merge November 8, 2024 16:18
@pront
Copy link
Member

pront commented Nov 8, 2024

Let's add a new "openobserve transform" scope somewhere here: https://github.com/vectordotdev/vector/blob/master/.github/workflows/semantic.yml#L182

@pront pront changed the title feat: Add OpenObserve as an officially supported sink feat(openobserve transform): Add OpenObserve as an officially supported sink Nov 8, 2024
@pront
Copy link
Member

pront commented Nov 8, 2024

Also, please see

CI=true make check-docs

@chaitanya-sistla
Copy link
Author

chaitanya-sistla commented Nov 13, 2024

@pront I don't have the file in my PR, do you want me to open a new PR or just add the file in my existing PR? https://github.com/vectordotdev/vector/blob/master/.github/workflows/semantic.yml#L182

@pront
Copy link
Member

pront commented Nov 13, 2024

@pront I don't have the file in my PR, do you want me to open a new PR or just add the file in my existing PR? https://github.com/vectordotdev/vector/blob/master/.github/workflows/semantic.yml#L182

Feel free to ignore this, but you can sync your fork, git merge origin master and it should be there.

More importantly, the changelog format is off and is blocking this PR from merging.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
domain: external docs Anything related to Vector's external, public documentation domain: sinks Anything related to the Vector's sinks
Projects
None yet
Development

Successfully merging this pull request may close these issues.

OpenObserve as a Sink for Vector
7 participants