v1.0.4 is there!

[jotak]

Key highlights

This release brings a bunch of optional features implemented in our eBPF Agent and our OpenShift Console plugin: ability to monitor packet drops (with their drop reason), TCP latency (RTT) and DNS traffic (including error codes).

Also, Loki isn't a hard requirement anymore: NetObserv can now be deployed without storing the flows. This makes sense if you configure a custom exporter (e.g. via Kafka or IPFIX) and have your own downstream pipeline for processing generated flows. Note that our Console plugin won't work without Loki, so it will be disabled in that case. Metrics generated for Prometheus are, however, still available.

In the Console plugin, the notion of "Reporter" has gone: you don't have anymore to make the difficult choice of choosing Source, Destination or both reporters (and getting duplicates): flows reported from different nodes are now merged and deduped automatically.

There are also a bunch of other UI enhancements that you will hopefully find improving the user experience, such as the "Not" filter, a "Back and forth" option to get return traffic, a Scope slider in the topology view, etc.

The eBPF Agent now supports SR-IOV devices.

IBM-Z was added as a supported architecture.

And last but not least, some performance improvements especially on the eBPF Agent side, and some bug fixes...

You can see the full changelog below.

What's Changed

Breaking changes 🛠

• Remove ConfigOrSecret API, replaced with the newer FileReference by @jotak in Remove ConfigOrSecret API, replaced with the newer FileReference #411

Non-breaking changes

• NETOBSERV-1110: Enable support for Flow RTT by @dushyantbehl in NETOBSERV-1110: Enable support for Flow RTT #394
• CRD doc updates by @jotak in CRD doc updates #364
• NETOBSERV-1050 & NETOBSERV-1065 Transport field in IPFIX exporter not working by @jpinsonneau in NETOBSERV-1050 & NETOBSERV-1065 Transport field in IPFIX exporter not working #351
• NETOBSERV-1070: Fix reconcile from DISABLED to AUTO configuration by @OlivierCazade in NETOBSERV-1070: Fix reconcile from DISABLED to AUTO configuration #365
• NETOBSERV-915 add conversation direction by @jpinsonneau in NETOBSERV-915 add conversation direction #313
• Fix outdated upstream doc on user access by @jotak in Fix outdated upstream doc on user access #373
• NETOBSERV-1076 Fix CR stuck in Updating state by @jotak in NETOBSERV-1076 Fix CR stuck in Updating state #374
• Add IBM Z (s390x) by @jotak in Add IBM Z (s390x) #381
• NETOBSERV-974: Add SASL support for KAFKA (unsupported feature) by @jotak in NETOBSERV-974: Add SASL support for KAFKA (unsupported feature) #377
• Doc updates: by @jotak in Doc updates: #370
• (trivial) makefile: avoid kubectl calls in global scope by @jotak in (trivial) makefile: avoid kubectl calls in global scope #383
• Refactor bundling for upstream operatorhub releases by @jotak in Refactor bundling for upstream operatorhub releases #363
• NETOBSERV-1053 optionnal loki / console plugin by @jpinsonneau in NETOBSERV-1053 optionnal loki / console plugin #354
• Add nil-check for loki.enable by @jotak in Add nil-check for loki.enable #385
• NETOBSERV-1131: Filter for Duplicate=false in metrics by @jotak in NETOBSERV-1131: Filter for Duplicate=false in metrics #387
• github ok-to-test: add more info in comment by @jotak in github ok-to-test: add more info in comment #390
• Upgrade linter - fix found issues - use go1.20 by @jotak in Upgrade linter - fix found issues - use go1.20 #393
• NETOBSERV-1107: Add GoMemLimit env setting to prevent ebpf agent OOM conditions by @msherif1234 in NETOBSERV-1107: Add GoMemLimit env setting to prevent ebpf agent OOM conditions #382
• NETOBSERV-1190: change DNS to use latency instead of TS by @msherif1234 in NETOBSERV-1190: change DNS to use latency instead of TS #395
• NETOBSERV-1197: Generalize TCPDrop to include other prototocl drops by @msherif1234 in NETOBSERV-1197: Generalize TCPDrop to include other prototocl drops #396
• Update downstream CSV description to link to official doc by @jotak in Update downstream CSV description to link to official doc #397
• NETOBSERV-1201: Make empty CR work out of the box by @jotak in NETOBSERV-1201: Make empty CR work out of the box #398
• NETOBSERV-1075 Dashboard feedback: split infra / applications by @jpinsonneau in NETOBSERV-1075 Dashboard feedback: split infra / applications #388
• NETOBSERV-1222: Pull request template by @jotak in NETOBSERV-1222: Pull request template #400
• NETOBSERV-1182 added cluster name to flp configuration by @KalmanMeth in NETOBSERV-1182 added cluster name to flp configuration #386
• NETOBSERV-1233 conntrack reportMissing + DNS response code by @jpinsonneau in NETOBSERV-1233 conntrack reportMissing + DNS response code #401
• NETOBSERV-1232 "netobserv" NS is not excluded from Applications for Flow dashboards by @jpinsonneau in NETOBSERV-1232 "netobserv" NS is not excluded from Applications for Flow dashboards #404
• NETOBSERV-1104: remove italic text in CRD doc by @jotak in NETOBSERV-1104: remove italic text in CRD doc #407
• NETOBSERV-1224 Flowcollector does not report status != Ready in OCP Console by @jpinsonneau in NETOBSERV-1224 Flowcollector does not report status != Ready in OCP Console #403
• fix DNStracking and packet drop features name in the API by @msherif1234 in fix DNStracking and packet drop features name in the API #410
• NETOBSERV-1283: Enable netns mount for privileged pods by @msherif1234 in NETOBSERV-1283: Enable netns mount for privileged pods #406
• NETOBSERV-1087: Added fields for ca certificate configuration by @OlivierCazade in NETOBSERV-1087: Added fields for ca certificate configuration #379

New Contributors

• @dushyantbehl made their first contribution in NETOBSERV-1110: Enable support for Flow RTT #394

Full Changelog: 1.0.3...1.0.4

Related components

eBPF Agent

What's Changed

Breaking changes 🛠

Non-breaking changes

• NETOBSERV-1061: Add TCP drop and DNS tracking hooks by @msherif1234 in NETOBSERV-1061: Add TCP drop and DNS tracking hooks netobserv-ebpf-agent#115
• update agent spec for profiling docs by @memodi in update agent spec for profiling docs netobserv-ebpf-agent#132
• NETOBSERV-1091: remove CO-RE file and extensions as that causes douple allocations by @msherif1234 in NETOBSERV-1091: remove CO-RE file and extensions as that causes douple allocations netobserv-ebpf-agent#133
• Fix makefile image-build with podman by @jotak in Fix makefile image-build with podman netobserv-ebpf-agent#136
• Fix make docker-generate issue Fix issues with doc/code/manifest generation #137 by @msherif1234 in Fix make docker-generate issue #137 netobserv-ebpf-agent#138
• Add IBM Z (s390x) by @jotak in Add IBM Z (s390x) netobserv-ebpf-agent#139
• NETOBSERV-1107: optimize ebpf agent map memory and cpu usage by @msherif1234 in NETOBSERV-1107: optimize ebpf agent map memory and cpu usage netobserv-ebpf-agent#140
• NETOBSERV-1112: Add TCP RTT calculation to ebpf-agent and userspace. by @dushyantbehl in NETOBSERV-1112: Add TCP RTT calculation to ebpf-agent and userspace. netobserv-ebpf-agent#117
• Bump google.golang.org/grpc from 1.45.0 to 1.53.0 by @dependabot in Bump google.golang.org/grpc from 1.45.0 to 1.53.0 netobserv-ebpf-agent#145
• NETOBSERV-1129: Add kernel version chk to disable hooks not supported on older version by @msherif1234 in NETOBSERV-1129: Add kernel version chk to disable hooks not supported on older version netobserv-ebpf-agent#144
• NETOBSERV-1107: Update cilium to v0.11.0 by @msherif1234 in NETOBSERV-1107: Update cilium to v0.11.0 netobserv-ebpf-agent#147
• Upgrade linter - fix found issues - use go1.20 by @jotak in Upgrade linter - fix found issues - use go1.20 netobserv-ebpf-agent#148
• NETOBSERV-1190: update dns tracker to calc latency instead of TS by @msherif1234 in NETOBSERV-1190: update dns tracker to calc latency instead of TS netobserv-ebpf-agent#149
• github ok-to-test: add more info in comment by @jotak in github ok-to-test: add more info in comment netobserv-ebpf-agent#150
• fix build for s390x and ppc64le by @msherif1234 in fix build for s390x and ppc64le netobserv-ebpf-agent#151
• NETOBSERV-1199: Socket doesn't always have valid if_index so we can't use it as key by @msherif1234 in NETOBSERV-1199: Socket doesn't always have valid if_index so we can't use it as key netobserv-ebpf-agent#153
• NETOBSERV-1197: Generalize TCPDrop to include other protocols by @msherif1234 in NETOBSERV-1197: Generalize TCPDrop to include other protocols netobserv-ebpf-agent#152
• Avoid using LRU hashmap because they consume lots of memory by @msherif1234 in Avoid using LRU hashmap because they consume lots of memory netobserv-ebpf-agent#162
• NETOBSERV-1198: drop_cause enum in vmlinux are based off old kernel by @msherif1234 in NETOBSERV-1198: drop_cause enum in vmlinux are based off old kernel netobserv-ebpf-agent#161
• Add support to evict DNS and RTT stale entries by @msherif1234 in Add support to evict DNS and RTT stale entries netobserv-ebpf-agent#163
• Shrink DNSMap if DNS not enabled to save memory by @msherif1234 in Shrink DNSMap if DNS not enabled to save memory netobserv-ebpf-agent#165
• NETOBSERV-1223: check Maps to make sure not nil b4 iterating by @msherif1234 in NETOBSERV-1223: check Maps to make sure not nil b4 iterating netobserv-ebpf-agent#167
• NETOBSERV-1112: This patch fixes a bug where RTT was not visible for flow logs at times. by @dushyantbehl in NETOBSERV-1112: This patch fixes a bug where RTT was not visible for flow logs at times. netobserv-ebpf-agent#159
• NETOBSERV-1283: Make ifaces watcher netns aware by @msherif1234 in NETOBSERV-1283: Make ifaces watcher netns aware netobserv-ebpf-agent#171
• NETOBSERV-1268: handle concurrency issues between kernel and userspace by @msherif1234 in NETOBSERV-1268: handle concurrency issues between kernel and userspace netobserv-ebpf-agent#172

New Contributors

• @memodi made their first contribution in update agent spec for profiling docs netobserv-ebpf-agent#132
• @dushyantbehl made their first contribution in NETOBSERV-1112: Add TCP RTT calculation to ebpf-agent and userspace. netobserv-ebpf-agent#117

Full Changelog: netobserv/netobserv-ebpf-agent@v0.3.1...v0.3.2

Flowlogs-Pipeline

What's Changed

Breaking changes 🛠

• NETOBSERV-1017 extract timebased indexKeys by @jpinsonneau in NETOBSERV-1017 extract timebased indexKeys flowlogs-pipeline#450
• move add_if and add_regex_if to transform_filter from transform_network by @KalmanMeth in move add_if and add_regex_if to transform_filter from transform_network flowlogs-pipeline#453

Non-breaking changes

• NETOBSERV-1063: Add TCP drop & DNS tracking to flowlogs-pipeline by @msherif1234 in NETOBSERV-1063: Add TCP drop & DNS tracking to flowlogs-pipeline flowlogs-pipeline#429
• NETOBSERV-1111: Add support for RoundTripTime in FLP. by @dushyantbehl in NETOBSERV-1111: Add support for RoundTripTime in FLP. flowlogs-pipeline#446
• NETOBSERV-915 skip reinterpret direction for conversations by @jpinsonneau in NETOBSERV-915 skip reinterpret direction for conversations flowlogs-pipeline#430
• Add encode s3 to pipeline builder by @jpinsonneau in Add encode s3 to pipeline builder flowlogs-pipeline#442
• Fix makefile imgae-build with podman by @jotak in Fix makefile imgae-build with podman flowlogs-pipeline#444
• fixed generation of flow logs for more connections by @KalmanMeth in fixed generation of flow logs for more connections flowlogs-pipeline#432
• Add IBM Z (s390x) by @jotak in Add IBM Z (s390x) flowlogs-pipeline#445
• NETOBSERV-1131: need to allow setting multiple filters by @jotak in NETOBSERV-1131: need to allow setting multiple filters flowlogs-pipeline#448
• Bump google.golang.org/grpc from 1.45.0 to 1.53.0 by @dependabot in Bump google.golang.org/grpc from 1.45.0 to 1.53.0 flowlogs-pipeline#449
• changed error to warning by @KalmanMeth in changed error to warning flowlogs-pipeline#443
• NETOBSERV-1186: extract DNS flags rcode field and stringify it by @msherif1234 in NETOBSERV-1186: extract DNS flags rcode field and stringify it flowlogs-pipeline#451
• Add msherif1234 by @jotak in Add msherif1234 flowlogs-pipeline#454
• NETOBSERV-1190: update DNS tracking to report latency instead of TS by @msherif1234 in NETOBSERV-1190: update DNS tracking to report latency instead of TS flowlogs-pipeline#455
• Upgrade linter - fix found issues - use go1.20 by @jotak in Upgrade linter - fix found issues - use go1.20 flowlogs-pipeline#452
• fix ppc64le and s390x targets build by @msherif1234 in fix ppc64le and s390x targets build flowlogs-pipeline#458
• NETOBSERV-1197: Generalize TCPDrop to support more protocols by @msherif1234 in NETOBSERV-1197: Generalize TCPDrop to support more protocols flowlogs-pipeline#457
• github ok-to-test: add more info in comment by @jotak in github ok-to-test: add more info in comment flowlogs-pipeline#456
• NETOBSERV-1111 Change RTT output to ns by @dushyantbehl in NETOBSERV-1111 Change RTT output to ns flowlogs-pipeline#468
• Add test for missing labels by @jotak in Add test for missing labels flowlogs-pipeline#463
• NETOBSERV-1208 & NETOBSERV-1233 Aggregators skip missing fields by @jpinsonneau in NETOBSERV-1208 & NETOBSERV-1233 Aggregators skip missing fields flowlogs-pipeline#470
• NETOBSERV-1135: add ovs subsystem drop causes by @msherif1234 in NETOBSERV-1135: add ovs subsystem drop causes flowlogs-pipeline#472
• NETOBSERV-1233: Remove L3 fields for ARP ethtype packets and update unit-test by @msherif1234 in NETOBSERV-1233: Remove L3 fields for ARP ethtype packets and update unit-test flowlogs-pipeline#471
• Update README and config by @ronensc in Update README and config flowlogs-pipeline#475
• Fix loki writer config in README.md by @ronensc in Fix loki writer config in README.md flowlogs-pipeline#479
• NETOBSERV-1275: Introduce new "INNER" direction for inner-node traffic by @jotak in NETOBSERV-1275: Introduce new "INNER" direction for inner-node traffic flowlogs-pipeline#483

New Contributors

• @dushyantbehl made their first contribution in NETOBSERV-1111: Add support for RoundTripTime in FLP. flowlogs-pipeline#446

Full Changelog: netobserv/flowlogs-pipeline@v0.1.9...v0.1.10

Console Plugin

What's Changed

Non-breaking changes

• Add standalone image creation by @eranra in Add standalone image creation network-observability-console-plugin#325
• NETOBSERV-915 add conversation direction by @jpinsonneau in NETOBSERV-915 add conversation direction network-observability-console-plugin#317
• Add length to metrics by @Amoghrd in Add length to metrics network-observability-console-plugin#338
• Fix makefile image-build with podman by @jotak in Fix makefile image-build with podman network-observability-console-plugin#345
• Fix missing results in merged streams by @jotak in Fix missing results in merged streams network-observability-console-plugin#342
• NETOBSERV-1090: initialize empty slice for returning json by @jotak in NETOBSERV-1090: initialize empty slice for returning json network-observability-console-plugin#340
• Add IBM Z (s390x) by @jotak in Add IBM Z (s390x) network-observability-console-plugin#346
• NETOBSERV-997 Cypress test console errors by @jpinsonneau in NETOBSERV-997 Cypress test console errors network-observability-console-plugin#349
• Allow faster frontend builds by @jotak in Allow faster frontend builds network-observability-console-plugin#351
• NETOBSERV-387 UI: filter "NOT" by @jpinsonneau in NETOBSERV-387 UI: filter "NOT" network-observability-console-plugin#333
• github ok-to-test: add more info in comment by @jotak in github ok-to-test: add more info in comment network-observability-console-plugin#353
• Upgrade linter - fix found issues - use go1.20 by @jotak in Upgrade linter - fix found issues - use go1.20 network-observability-console-plugin#354
• NETOBSERV-1097 react-router-dom upgrade with dynamic loader by @jpinsonneau in NETOBSERV-1097 react-router-dom upgrade with dynamic loader network-observability-console-plugin#350
• NETOBSERV-1190: use DNS latency instead of timestamps by @msherif1234 in NETOBSERV-1190: use DNS latency instead of timestamps network-observability-console-plugin#356
• NETOBSERV-1197: Generalize TCPDrop to include more protocols by @msherif1234 in NETOBSERV-1197: Generalize TCPDrop to include more protocols network-observability-console-plugin#357
• Bump word-wrap from 1.2.3 to 1.2.4 in /web by @dependabot in Bump word-wrap from 1.2.3 to 1.2.4 in /web network-observability-console-plugin#359
• NETOBSERV-1025: include back and forth traffic by @jotak in NETOBSERV-1025: include back and forth traffic network-observability-console-plugin#343
• Mention 'dropped by the kernel' by @jotak in Mention 'dropped by the kernel' network-observability-console-plugin#358
• NETOBSERV-1200: Show "< 1ms" for 0 latency instead of n/a by @jotak in NETOBSERV-1200: Show "< 1ms" for 0 latency instead of n/a network-observability-console-plugin#360
• NETOBSERV-1098: Show scope as slider by @jotak in NETOBSERV-1098: Show scope as slider network-observability-console-plugin#344
• Fix filters comparison by @jotak in Fix filters comparison network-observability-console-plugin#363
• Bump libs by @jotak in Bump libs network-observability-console-plugin#364
• NETOBSERV-1200 UI: DNSTracking feedback improvments by @jpinsonneau in NETOBSERV-1200 UI: DNSTracking feedback improvments network-observability-console-plugin#361
• NETOBSERV-1099: remove reporter option by @jotak in NETOBSERV-1099: remove reporter option network-observability-console-plugin#311
• NETOBSERV-1216 PacketDrop enhancements by @jpinsonneau in NETOBSERV-1216 PacketDrop enhancements network-observability-console-plugin#362
• NETOBSERV-1216 pktDrop cause filter by @jpinsonneau in NETOBSERV-1216 pktDrop cause filter network-observability-console-plugin#367
• NETOBSERV-970 Typo in Query Summary panel by @jpinsonneau in NETOBSERV-970 Typo in Query Summary panel network-observability-console-plugin#368
• NETOBSERV-1135: add ovs subsystem drop causes by @msherif1234 in NETOBSERV-1135: add ovs subsystem drop causes network-observability-console-plugin#370
• NETOBSERV-1241 Back and forth show the opposite by @jpinsonneau in NETOBSERV-1241 Back and forth show the opposite network-observability-console-plugin#369
• NETOBSERV-1241 Followup - Back and forth show the opposite by @jpinsonneau in NETOBSERV-1241 Followup - Back and forth show the opposite network-observability-console-plugin#372
• NETOBSERV-1235 Reporter option issues by @jpinsonneau in NETOBSERV-1235 Reporter option issues network-observability-console-plugin#366
• NETOBSERV-1264 fix custom time range + reset button by @jpinsonneau in NETOBSERV-1264 fix custom time range + reset button network-observability-console-plugin#371
• NETOBSERV-1265 Overview / Topology fully drop metrics doesn't show by @jpinsonneau in NETOBSERV-1265 Overview / Topology fully drop metrics doesn't show network-observability-console-plugin#373
• NETOBSERV-1235 NETOBSERV-1241 UI Polishing by @jpinsonneau in NETOBSERV-1235 NETOBSERV-1241 UI Polishing network-observability-console-plugin#374
• NETOBSERV-1242 Console plugin build infos by @jpinsonneau in NETOBSERV-1242 Console plugin build infos network-observability-console-plugin#376
• NETOBSERV-1266 Netflow traffic tab crash by @jpinsonneau in NETOBSERV-1266 Netflow traffic tab crash network-observability-console-plugin#377
• NETOBSERV-1113 implements RTT option in Console Plugin by @jpinsonneau in NETOBSERV-1113 implements RTT option in Console Plugin network-observability-console-plugin#365
• NETOBSERV-967 skip ticks when popups are open by @jpinsonneau in NETOBSERV-967 skip ticks when popups are open network-observability-console-plugin#375
• NETOBSERV-1113 fix pktDrop & FlowRTT merge by @jpinsonneau in NETOBSERV-1113 fix pktDrop & FlowRTT merge network-observability-console-plugin#381
• update ovs_drop_last_action doc by @msherif1234 in update ovs_drop_last_action doc network-observability-console-plugin#382
• NETOBSERV-1275: Introduce new "INNER" direction for inner-node traffic by @jotak in NETOBSERV-1275: Introduce new "INNER" direction for inner-node traffic network-observability-console-plugin#378
• NETOBSERV-1269 refactor overlapping detection for BNF by @jotak in NETOBSERV-1269 refactor overlapping detection for BNF network-observability-console-plugin#379

New Contributors

• @eranra made their first contribution in Add standalone image creation network-observability-console-plugin#325

Full Changelog: netobserv/network-observability-console-plugin@v0.1.10...v0.1.11

---

This discussion was created from the release 1.0.4.