-## :wave: Introduction
+[](https://hub.docker.com/r/parseable/parseable)
+[](https://twitter.com/parseableio)
+[](https://launchpass.com/parseable)
+[](https://www.parseable.io/docs)
+[](https://github.com/parseablehq/parseable/actions)
-Parseable is a open source log observability platform. Written in Rust, it is designed for simplicity of deployment and use. It is compatible with standard logging agents via their HTTP output. Parseable also offers a builtin GUI for log query and analysis.
+
-We're focussed on
+Parseable is a lightweight, cloud native log observability engine. It can use either a local drive or S3 (and compatible stores) for backend data storage.
-* Simplicity - ease of deployment and use.
-* Efficiency - lesser CPU, Memory usage.
-* Extensibility - freedom to do more with event data.
-* Performance - lower latency, higher throughput.
+Parseable is written in Rust and uses Apache Arrow and Parquet as underlying data structures. Additionally, it uses a simple, index-free mechanism to organize and query data allowing low latency, and high throughput ingestion and query.
-## :dart: Motivation
-
-Given the analytical nature of log data, columnar formats like Parquet are the best way to store and analyze. Parquet offers compression and inherent analytical capabilities. However, indexing based text search engines are _still_ prevalent. We are building Parseable to take full advantage of advanced data formats like Apache Parquet and Arrow. This approach is simpler, efficient and much more scalable.
-
-Parseable is developer friendly, cloud native, logging platforms today that is simple to deploy and run - while offering a rich set of features.
+Parseable consumes up to **_~80% lower memory_** and **_~50% lower CPU_** than Elastic for similar ingestion throughput.
-## :question: How it works
+## :rocket: Features
-Parseable exposes REST API to ingest and query log data. Under the hood, it uses Apache Arrow and Parquet to handle and compress high volume log data. All data is stored in S3 (or compatible systems). Parseable also has a bundled web console to visualize and query log data.
+- Choose your own storage backend - local drive or S3 (or compatible) object store.
+- Ingestion API compatible with HTTP + JSON output of log agents - [Fluentbit ↗︎](https://fluentbit.io/), [Vector ↗︎](http://vector.dev/), [Logstash ↗︎](https://www.elastic.co/logstash/) and others.
+- Query log data with PostgreSQL compatible SQL.
+- [Grafana ↗︎](https://github.com/parseablehq/parseable-datasource) for visualization.
+- Auto schema inference (schema evolution [coming soon ↗︎](https://github.com/parseablehq/parseable/issues/195)).
+- [Send alerts ↗︎](https://www.parseable.io/docs/api/alerts) to webhook targets including Slack.
+- [Stats API ↗︎](https://www.postman.com/parseable/workspace/parseable/request/22353706-b32abe55-f0c4-4ed2-9add-110d265888c3) to track ingestion and compressed data.
+- Single binary includes all components - ingestion, store and query. Built-in UI.
-- Written in Rust. Low CPU & memory footprint, with low latency, high throughput.
-- Open data format (Parquet). Complete ownership of data. Wide range of possibilities for data analysis.
-- Single binary / container based deployment (including UI). Deploy in minutes if not seconds.
-- Indexing free design. Lower CPU and storage overhead. Similar levels of performance as indexing based systems.
-- Kubernetes and Cloud native design, build ground up for cloud native environments.
+## :white_check_mark: Getting Started
-## :white_check_mark: Installing
-
-Run the below command to deploy Parseable in demo mode with Docker.
+Run the below command to deploy Parseable in local storage mode with Docker.
```sh
-mkdir -p /tmp/data
-docker run \
- -p 8000:8000 \
- -v /tmp/data:/data \
+mkdir -p /tmp/parseable/data
+mkdir -p /tmp/parseable/staging
+
+docker run -p 8000:8000 \
+ -v /tmp/parseable/data:/parseable/data \
+ -v /tmp/parseable/staging:/parseable/staging \
+ -e P_FS_DIR=/parseable/data \
+ -e P_STAGING_DIR=/parseable/staging \
parseable/parseable:latest \
- parseable server --demo
+ parseable local-store
```
-Once this runs successfully, you'll see dashboard at [http://localhost:8000](http://localhost:8000). You can login to the dashboard with `parseable`, `parseable` as the credentials. Please make sure not to post any important data while in demo mode.
-
-Prefer other platforms? Check out installation options (Kubernetes, bare-metal), in the [documentation](https://www.parseable.io/docs/category/installation).
-
-#### Live demo
+Once this runs successfully, you'll see dashboard at [http://localhost:8000](http://localhost:8000). You can login to the dashboard default credentials `admin`, `admin`.
-Instead of installing locally, you can also try out Parseable on our [Demo instance](https://demo.parseable.io). Credentials to login to the dashboard are `parseable` / `parseable`.
-
-## :100: Usage
-
-If you've already deployed Parseable using the above Docker command, use below commands to create stream and post event(s) to the stream. Make sure to replace `` with the name of the stream you want to create and post events (e.g. `my-stream`).
-#### Create a stream
+### Create a stream
```sh
-curl --location --request PUT 'http://localhost:8000/api/v1/logstream/' \
---header 'Authorization: Basic cGFyc2VhYmxlOnBhcnNlYWJsZQ=='
+curl --location --request PUT 'http://localhost:8000/api/v1/logstream/demo' \
+--header 'Authorization: Basic YWRtaW46YWRtaW4='
```
-#### Send events to the stream
+### Send events to the stream
```sh
-curl --location --request POST 'http://localhost:8000/api/v1/logstream/' \
+curl --location --request POST 'http://localhost:8000/api/v1/logstream/demo' \
--header 'X-P-META-meta1: value1' \
--header 'X-P-TAG-tag1: value1' \
---header 'Authorization: Basic cGFyc2VhYmxlOnBhcnNlYWJsZQ==' \
+--header 'Authorization: Basic YWRtaW46YWRtaW4=' \
--header 'Content-Type: application/json' \
--data-raw '[
{
@@ -93,19 +82,44 @@ curl --location --request POST 'http://localhost:8000/api/v1/logstream/> /etc/default/parseable
-P_S3_BUCKET=
-P_S3_ACCESS_KEY=
-P_S3_SECRET_KEY=
-P_S3_REGION=
-P_S3_URL=
-EOT
-```
-
-## Systemctl
-
-Download `parseable.service` in `/etc/systemd/system/`
-```
-( cd /etc/systemd/system/; curl -O https://raw.githubusercontent.com/parseablehq/parseable/main/deploy/systemd/parseable.service )
-```
-
-Note: If you want to bind to a port < 1024 with the service running as a regular user, you will need to add bind capability via the AmbientCapabilities directive in the parseable.service file:
-
-```
-[Service]
-AmbientCapabilities=CAP_NET_BIND_SERVICE
-WorkingDirectory=/usr/local/
-```
-
-### Enable startup on boot
-```
-systemctl enable parseable.service
-```
-
-### Disable Parseable service
-```
-systemctl disable parseable.service
-```
diff --git a/helm-releases/collector-0.0.1.tgz b/helm-releases/collector-0.0.1.tgz
index d76d4a024..dfd9f8d7d 100644
Binary files a/helm-releases/collector-0.0.1.tgz and b/helm-releases/collector-0.0.1.tgz differ
diff --git a/helm-releases/parseable-0.1.0.tgz b/helm-releases/parseable-0.1.0.tgz
new file mode 100644
index 000000000..2d26ea814
Binary files /dev/null and b/helm-releases/parseable-0.1.0.tgz differ
diff --git a/helm/parseable/Chart.yaml b/helm/Chart.yaml
similarity index 72%
rename from helm/parseable/Chart.yaml
rename to helm/Chart.yaml
index 757c25964..017a2667f 100644
--- a/helm/parseable/Chart.yaml
+++ b/helm/Chart.yaml
@@ -2,5 +2,5 @@ apiVersion: v2
name: parseable
description: Helm chart for Parseable Server
type: application
-version: 0.0.8
-AppVersion: "v0.0.8"
+version: 0.1.0
+AppVersion: "v0.1.0"
diff --git a/helm/README.md b/helm/README.md
new file mode 100644
index 000000000..db133f357
--- /dev/null
+++ b/helm/README.md
@@ -0,0 +1,3 @@
+# Parseable Helm Chart
+
+Refer the Parseable Helm Chart [documentation ↗︎](https://www.parseable.io/docs/deployment/kubernetes)
diff --git a/helm/parseable/templates/_helpers.tpl b/helm/templates/_helpers.tpl
similarity index 100%
rename from helm/parseable/templates/_helpers.tpl
rename to helm/templates/_helpers.tpl
diff --git a/helm/parseable/templates/deployment.yaml b/helm/templates/deployment.yaml
similarity index 95%
rename from helm/parseable/templates/deployment.yaml
rename to helm/templates/deployment.yaml
index 9b2d08999..07f2f2ab2 100644
--- a/helm/parseable/templates/deployment.yaml
+++ b/helm/templates/deployment.yaml
@@ -33,10 +33,10 @@ spec:
imagePullPolicy: {{ .Values.parseable.image.pullPolicy }}
# Uncomment to debug
# command: [ "/bin/sh", "-c", "sleep 1000000" ]
- {{- if .Values.parseable.demo }}
- args: ["parseable", "server", "--demo"]
+ {{- if .Values.parseable.local }}
+ args: ["parseable", "local-store"]
{{- else }}
- args: ["parseable", "server"]
+ args: ["parseable", "s3-store"]
{{- end }}
env:
{{- range $key, $value := .Values.parseable.env }}
diff --git a/helm/parseable/templates/service.yaml b/helm/templates/service.yaml
similarity index 100%
rename from helm/parseable/templates/service.yaml
rename to helm/templates/service.yaml
diff --git a/helm/parseable/templates/serviceaccount.yaml b/helm/templates/serviceaccount.yaml
similarity index 100%
rename from helm/parseable/templates/serviceaccount.yaml
rename to helm/templates/serviceaccount.yaml
diff --git a/helm/parseable/values.yaml b/helm/values.yaml
similarity index 93%
rename from helm/parseable/values.yaml
rename to helm/values.yaml
index 77ef50722..900c16719 100644
--- a/helm/parseable/values.yaml
+++ b/helm/values.yaml
@@ -1,9 +1,9 @@
parseable:
image:
repository: parseable/parseable
- tag: v0.0.8
+ tag: v0.1.0
pullPolicy: Always
- demo: false
+ local: false
secrets:
- type: env
name: parseable-env-secret
@@ -14,14 +14,15 @@ parseable:
# - tls.key.path
# - storage.upload.interval
- addr
+ - username
+ - password
+ - staging.dir
- s3.url
- s3.access.key
- s3.secret.key
- s3.bucket
- s3.region
- - local.storage
- - username
- - password
+ - fs.dir
replicaCount: 1
env:
RUST_LOG: "parseable=info"
diff --git a/index.yaml b/index.yaml
index 98e1e3a0f..47f588746 100644
--- a/index.yaml
+++ b/index.yaml
@@ -3,18 +3,28 @@ entries:
collector:
- apiVersion: v2
appVersion: 4769fbf
- created: "2022-12-13T14:27:14.386887+05:30"
+ created: "2023-01-05T19:46:15.999688+05:30"
description: Helm chart for Parseable Collector
- digest: ab31cdecafecbafd4f6788cbfc1e83b76c7395654cd0efc87a5a874a8cb11cfc
+ digest: 297c0b6ec05e8868c99639ad5d80e974af20a376af2f984e661d5dc98d9f03e1
name: collector
type: application
urls:
- https://charts.parseable.io/helm-releases/collector-0.0.1.tgz
version: 0.0.1
parseable:
+ - apiVersion: v2
+ appVersion: v0.1.0
+ created: "2023-01-05T19:46:16.007425+05:30"
+ description: Helm chart for Parseable Server
+ digest: 1ca2985befbc2500a7faeae8702238ad9afe05b00368f5c5b7cfbdea3d6f3a7b
+ name: parseable
+ type: application
+ urls:
+ - https://charts.parseable.io/helm-releases/parseable-0.1.0.tgz
+ version: 0.1.0
- apiVersion: v2
appVersion: v0.0.8
- created: "2022-12-13T14:27:14.393538+05:30"
+ created: "2023-01-05T19:46:16.006931+05:30"
description: Helm chart for Parseable Server
digest: c805254ffa634f96ecec448bcfff9973339aa9487dd8199b21b17b79a4de9345
name: parseable
@@ -24,7 +34,7 @@ entries:
version: 0.0.8
- apiVersion: v2
appVersion: v0.0.7
- created: "2022-12-13T14:27:14.392796+05:30"
+ created: "2023-01-05T19:46:16.005811+05:30"
description: Helm chart for Parseable Server
digest: c591f617ed1fe820bb2c72a4c976a78126f1d1095d552daa07c4700f46c4708a
name: parseable
@@ -34,7 +44,7 @@ entries:
version: 0.0.7
- apiVersion: v2
appVersion: v0.0.6
- created: "2022-12-13T14:27:14.391679+05:30"
+ created: "2023-01-05T19:46:16.004658+05:30"
description: Helm chart for Parseable Server
digest: f9ae56a6fcd6a59e7bee0436200ddbedeb74ade6073deb435b8fcbaf08dda795
name: parseable
@@ -44,7 +54,7 @@ entries:
version: 0.0.6
- apiVersion: v2
appVersion: v0.0.5
- created: "2022-12-13T14:27:14.390665+05:30"
+ created: "2023-01-05T19:46:16.003581+05:30"
description: Helm chart for Parseable Server
digest: 4d6b08a064fba36e16feeb820b77e1e8e60fb6de48dbf7ec8410d03d10c26ad0
name: parseable
@@ -54,7 +64,7 @@ entries:
version: 0.0.5
- apiVersion: v2
appVersion: v0.0.2
- created: "2022-12-13T14:27:14.389309+05:30"
+ created: "2023-01-05T19:46:16.002418+05:30"
description: Helm chart for Parseable Server
digest: 38a0a3e4c498afbbcc76ebfcb9cb598fa2ca843a53cc93b3cb4f135b85c10844
name: parseable
@@ -64,7 +74,7 @@ entries:
version: 0.0.2
- apiVersion: v2
appVersion: v0.0.1
- created: "2022-12-13T14:27:14.388158+05:30"
+ created: "2023-01-05T19:46:16.001072+05:30"
description: Helm chart for Parseable Server
digest: 1f1142db092b9620ee38bb2294ccbb1c17f807b33bf56da43816af7fe89f301e
name: parseable
@@ -72,4 +82,4 @@ entries:
urls:
- https://charts.parseable.io/helm-releases/parseable-0.0.1.tgz
version: 0.0.1
-generated: "2022-12-13T14:27:14.385392+05:30"
+generated: "2023-01-05T19:46:15.998828+05:30"
diff --git a/server/Cargo.toml b/server/Cargo.toml
index 295cbb8dd..fd67ecfde 100644
--- a/server/Cargo.toml
+++ b/server/Cargo.toml
@@ -1,6 +1,6 @@
[package]
name = "parseable"
-version = "0.0.8"
+version = "0.1.0"
authors = ["Parseable Team "]
edition = "2021"
categories = ["olap", "logging", "analytics-store"]
diff --git a/server/src/banner.rs b/server/src/banner.rs
index c1762ac4f..be637e8df 100644
--- a/server/src/banner.rs
+++ b/server/src/banner.rs
@@ -129,7 +129,7 @@ pub mod about {
eprintln!(
"
Commit: \"{}\"
- Docs: \"https://www.parseable.io/docs/introduction\"",
+ Docs: \"https://www.parseable.io/docs\"",
commit_hash
);
}
diff --git a/systemd/README.md b/systemd/README.md
new file mode 100644
index 000000000..3b029afc8
--- /dev/null
+++ b/systemd/README.md
@@ -0,0 +1,3 @@
+# Linux Systemd Service
+
+Refer the Systemd setup [documentation ↗︎](https://www.parseable.io/docs/installation/systemd)
diff --git a/systemd/parseable.local.service b/systemd/parseable.local.service
new file mode 100644
index 000000000..ba16091c9
--- /dev/null
+++ b/systemd/parseable.local.service
@@ -0,0 +1,33 @@
+[Unit]
+Description=Parseable
+Wants=network-online.target
+After=network-online.target
+AssertFileIsExecutable=/usr/local/bin/parseable
+
+[Service]
+WorkingDirectory=/usr/local/
+
+User=parseable-user
+Group=parseable-user
+ProtectProc=invisible
+
+EnvironmentFile=/etc/default/parseable
+ExecStart=/usr/local/bin/parseable local-store
+
+# Let systemd restart this service always
+Restart=always
+
+# Specifies the maximum file descriptor number that can be opened by this process
+LimitNOFILE=1048576
+
+# Specifies the maximum number of threads this process can create
+TasksMax=infinity
+
+# Disable timeout logic and wait until process is stopped
+TimeoutStopSec=infinity
+SendSIGKILL=no
+
+[Install]
+WantedBy=multi-user.target
+
+# Built for ${project.name}-${project.version} (${project.name})
diff --git a/deploy/linux-systemd/parseable.service b/systemd/parseable.s3.service
similarity index 94%
rename from deploy/linux-systemd/parseable.service
rename to systemd/parseable.s3.service
index 8656c9320..891786204 100644
--- a/deploy/linux-systemd/parseable.service
+++ b/systemd/parseable.s3.service
@@ -12,7 +12,7 @@ Group=parseable-user
ProtectProc=invisible
EnvironmentFile=/etc/default/parseable
-ExecStart=/usr/local/bin/parseable
+ExecStart=/usr/local/bin/parseable s3-store
# Let systemd restart this service always
Restart=always
- 
-
-
+ 
- Quick Start |
- Documentation |
- Community |
- Live Demo
-
+