Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

in_forward: Recreate connection when resumed #9605

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

cosmo0920
Copy link
Contributor

@cosmo0920 cosmo0920 commented Nov 18, 2024

When in_forward is paused, the remaining connections are discarded and disposed.
This causes SEGV when resuming from the pause status of in_forward.
In this case, we need to recreate the connections for downstreams.

Plus, the connection pool should be protested with mutex lock.
I also added to prevent thread switching during the checking status of pausing/resuming.

Closes #9443.
Closes #9288.


Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

  • Example configuration file for the change
[INPUT]
    Name                forward
    Listen              127.0.0.1
    Port                24224
    Buffer_Chunk_Size   512KB
    Buffer_Max_Size     2MB
    Mem_Buf_Limit       10MB

[OUTPUT]
    Name null

If you have a capability to use Fluentd for causing flood of messages constantly exceeding the limit of max size for chunks,
you can use the following configuration:

<system>
  workers 7
</system>
<source>
  @type sample
  tag test
  sample "{\"message\":\"#{'hello' * 1024}\"}"
  rate 370
</source>

<match test>
  @type forward

  <server>
    # first server
    host 127.0.0.1
    port 24224
  </server>

  flush_interval 0
  send_timeout 60
  heartbeat_type udp
  heartbeat_interval 1
  recover_wait 10
  hard_timeout 60
  expire_dns_cache nil
  phi_threshold 16
  phi_failure_detector true
  require_ack_response true
</match>
  • Debug log output from testing the change

With this patch, we can resume properly from the paused status:

<snip>
[2024/11/19 15:27:34] [ info] [output:null:null.0] worker #0 started
[2024/11/19 15:27:38] [debug] [task] created task=0x7e65a001d310 id=0 OK
[2024/11/19 15:27:38] [debug] [task] created task=0x7e65a001d3e0 id=1 OK
[2024/11/19 15:27:38] [debug] [task] created task=0x7e65a001d510 id=2 OK
[2024/11/19 15:27:38] [debug] [output:null:null.0] task_id=0 assigned to thread #0
[2024/11/19 15:27:38] [debug] [output:null:null.0] task_id=1 assigned to thread #0
[2024/11/19 15:27:38] [debug] [output:null:null.0] task_id=2 assigned to thread #0
[2024/11/19 15:27:38] [debug] [output:null:null.0] discarding 2854365 bytes
[2024/11/19 15:27:38] [debug] [output:null:null.0] discarding 2854365 bytes
[2024/11/19 15:27:38] [debug] [out flush] cb_destroy coro_id=0
[2024/11/19 15:27:38] [debug] [out flush] cb_destroy coro_id=1
[2024/11/19 15:27:38] [debug] [output:null:null.0] discarding 951455 bytes
[2024/11/19 15:27:38] [debug] [out flush] cb_destroy coro_id=2
[2024/11/19 15:27:38] [debug] [task] destroy task=0x7e65a001d310 (task_id=0)
[2024/11/19 15:27:38] [debug] [task] destroy task=0x7e65a001d3e0 (task_id=1)
[2024/11/19 15:27:38] [debug] [task] destroy task=0x7e65a001d510 (task_id=2)
[2024/11/19 15:27:39] [ warn] [input] forward.0 paused (mem buf overlimit)
[2024/11/19 15:27:39] [ info] [input] pausing forward.0
[2024/11/19 15:27:39] [debug] [input chunk] forward.0 is paused, cannot append records
[2024/11/19 15:27:39] [error] [input:forward:forward.0] could not append logs. ret=-1
[2024/11/19 15:27:39] [debug] [task] created task=0x7e65a001c550 id=0 OK
[2024/11/19 15:27:39] [debug] [task] created task=0x7e65a001c620 id=1 OK
[2024/11/19 15:27:39] [debug] [task] created task=0x7e65a001d720 id=2 OK
[2024/11/19 15:27:39] [debug] [output:null:null.0] task_id=0 assigned to thread #0
[2024/11/19 15:27:39] [debug] [output:null:null.0] discarding 3805820 bytes
[2024/11/19 15:27:39] [debug] [output:null:null.0] task_id=1 assigned to thread #0
[2024/11/19 15:27:39] [debug] [output:null:null.0] task_id=2 assigned to thread #0
[2024/11/19 15:27:39] [debug] [output:null:null.0] discarding 3805820 bytes
[2024/11/19 15:27:39] [debug] [out flush] cb_destroy coro_id=3
[2024/11/19 15:27:39] [debug] [out flush] cb_destroy coro_id=4
[2024/11/19 15:27:39] [debug] [output:null:null.0] discarding 3805820 bytes
[2024/11/19 15:27:39] [debug] [out flush] cb_destroy coro_id=5
[2024/11/19 15:27:39] [debug] [task] destroy task=0x7e65a001c550 (task_id=0)
[2024/11/19 15:27:39] [ info] [input] resume forward.0
[2024/11/19 15:27:39] [ info] [input] forward.0 resume (mem buf overlimit)
[2024/11/19 15:27:39] [debug] [task] destroy task=0x7e65a001c620 (task_id=1)
[2024/11/19 15:27:39] [debug] [task] destroy task=0x7e65a001d720 (task_id=2)
[2024/11/19 15:27:40] [ warn] [input] forward.0 paused (mem buf overlimit)
[2024/11/19 15:27:40] [ info] [input] pausing forward.0
[2024/11/19 15:27:40] [debug] [input chunk] forward.0 is paused, cannot append records
[2024/11/19 15:27:40] [error] [input:forward:forward.0] could not append logs. ret=-1
<snip>

Full log:

Fluent Bit v3.2.1
* Copyright (C) 2015-2024 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

______ _                  _    ______ _ _           _____  _____ 
|  ___| |                | |   | ___ (_) |         |____ |/ __  \
| |_  | |_   _  ___ _ __ | |_  | |_/ /_| |_  __   __   / /`' / /'
|  _| | | | | |/ _ \ '_ \| __| | ___ \ | __| \ \ / /   \ \  / /  
| |   | | |_| |  __/ | | | |_  | |_/ / | |_   \ V /.___/ /./ /___
\_|   |_|\__,_|\___|_| |_|\__| \____/|_|\__|   \_/ \____(_)_____/


[2024/11/19 15:27:34] [ info] Configuration:
[2024/11/19 15:27:34] [ info]  flush time     | 1.000000 seconds
[2024/11/19 15:27:34] [ info]  grace          | 5 seconds
[2024/11/19 15:27:34] [ info]  daemon         | 0
[2024/11/19 15:27:34] [ info] ___________
[2024/11/19 15:27:34] [ info]  inputs:
[2024/11/19 15:27:34] [ info]      forward
[2024/11/19 15:27:34] [ info] ___________
[2024/11/19 15:27:34] [ info]  filters:
[2024/11/19 15:27:34] [ info] ___________
[2024/11/19 15:27:34] [ info]  outputs:
[2024/11/19 15:27:34] [ info]      null.0
[2024/11/19 15:27:34] [ info] ___________
[2024/11/19 15:27:34] [ info]  collectors:
[2024/11/19 15:27:34] [ info] [fluent bit] version=3.2.1, commit=600b5a955b, pid=128174
[2024/11/19 15:27:34] [debug] [engine] coroutine stack size: 24576 bytes (24.0K)
[2024/11/19 15:27:34] [ info] [storage] ver=1.1.6, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2024/11/19 15:27:34] [ info] [simd    ] SSE2
[2024/11/19 15:27:34] [ info] [cmetrics] version=0.9.9
[2024/11/19 15:27:34] [ info] [ctraces ] version=0.5.7
[2024/11/19 15:27:34] [ info] [input:forward:forward.0] initializing
[2024/11/19 15:27:34] [ info] [input:forward:forward.0] storage_strategy='memory' (memory only)
[2024/11/19 15:27:34] [debug] [forward:forward.0] created event channels: read=25 write=26
[2024/11/19 15:27:34] [debug] [in_fw] Listen='127.0.0.1' TCP_Port=24224
[2024/11/19 15:27:34] [debug] [downstream] listening on 127.0.0.1:24224
[2024/11/19 15:27:34] [ info] [input:forward:forward.0] listening on 127.0.0.1:24224
[2024/11/19 15:27:34] [debug] [null:null.0] created event channels: read=28 write=29
[2024/11/19 15:27:34] [ info] [sp] stream processor started
[2024/11/19 15:27:34] [ info] [output:null:null.0] worker #0 started
[2024/11/19 15:27:38] [debug] [task] created task=0x7e65a001d310 id=0 OK
[2024/11/19 15:27:38] [debug] [task] created task=0x7e65a001d3e0 id=1 OK
[2024/11/19 15:27:38] [debug] [task] created task=0x7e65a001d510 id=2 OK
[2024/11/19 15:27:38] [debug] [output:null:null.0] task_id=0 assigned to thread #0
[2024/11/19 15:27:38] [debug] [output:null:null.0] task_id=1 assigned to thread #0
[2024/11/19 15:27:38] [debug] [output:null:null.0] task_id=2 assigned to thread #0
[2024/11/19 15:27:38] [debug] [output:null:null.0] discarding 2854365 bytes
[2024/11/19 15:27:38] [debug] [output:null:null.0] discarding 2854365 bytes
[2024/11/19 15:27:38] [debug] [out flush] cb_destroy coro_id=0
[2024/11/19 15:27:38] [debug] [out flush] cb_destroy coro_id=1
[2024/11/19 15:27:38] [debug] [output:null:null.0] discarding 951455 bytes
[2024/11/19 15:27:38] [debug] [out flush] cb_destroy coro_id=2
[2024/11/19 15:27:38] [debug] [task] destroy task=0x7e65a001d310 (task_id=0)
[2024/11/19 15:27:38] [debug] [task] destroy task=0x7e65a001d3e0 (task_id=1)
[2024/11/19 15:27:38] [debug] [task] destroy task=0x7e65a001d510 (task_id=2)
[2024/11/19 15:27:39] [ warn] [input] forward.0 paused (mem buf overlimit)
[2024/11/19 15:27:39] [ info] [input] pausing forward.0
[2024/11/19 15:27:39] [debug] [input chunk] forward.0 is paused, cannot append records
[2024/11/19 15:27:39] [error] [input:forward:forward.0] could not append logs. ret=-1
[2024/11/19 15:27:39] [debug] [task] created task=0x7e65a001c550 id=0 OK
[2024/11/19 15:27:39] [debug] [task] created task=0x7e65a001c620 id=1 OK
[2024/11/19 15:27:39] [debug] [task] created task=0x7e65a001d720 id=2 OK
[2024/11/19 15:27:39] [debug] [output:null:null.0] task_id=0 assigned to thread #0
[2024/11/19 15:27:39] [debug] [output:null:null.0] discarding 3805820 bytes
[2024/11/19 15:27:39] [debug] [output:null:null.0] task_id=1 assigned to thread #0
[2024/11/19 15:27:39] [debug] [output:null:null.0] task_id=2 assigned to thread #0
[2024/11/19 15:27:39] [debug] [output:null:null.0] discarding 3805820 bytes
[2024/11/19 15:27:39] [debug] [out flush] cb_destroy coro_id=3
[2024/11/19 15:27:39] [debug] [out flush] cb_destroy coro_id=4
[2024/11/19 15:27:39] [debug] [output:null:null.0] discarding 3805820 bytes
[2024/11/19 15:27:39] [debug] [out flush] cb_destroy coro_id=5
[2024/11/19 15:27:39] [debug] [task] destroy task=0x7e65a001c550 (task_id=0)
[2024/11/19 15:27:39] [ info] [input] resume forward.0
[2024/11/19 15:27:39] [ info] [input] forward.0 resume (mem buf overlimit)
[2024/11/19 15:27:39] [debug] [task] destroy task=0x7e65a001c620 (task_id=1)
[2024/11/19 15:27:39] [debug] [task] destroy task=0x7e65a001d720 (task_id=2)
[2024/11/19 15:27:40] [ warn] [input] forward.0 paused (mem buf overlimit)
[2024/11/19 15:27:40] [ info] [input] pausing forward.0
[2024/11/19 15:27:40] [debug] [input chunk] forward.0 is paused, cannot append records
[2024/11/19 15:27:40] [error] [input:forward:forward.0] could not append logs. ret=-1
[2024/11/19 15:27:40] [debug] [task] created task=0x7e65a001d030 id=0 OK
[2024/11/19 15:27:40] [debug] [task] created task=0x7e65a001d9f0 id=1 OK
[2024/11/19 15:27:40] [debug] [output:null:null.0] discarding 3805820 bytes
[2024/11/19 15:27:40] [debug] [task] created task=0x7e65a001dac0 id=2 OK
[2024/11/19 15:27:40] [debug] [output:null:null.0] discarding 3805820 bytes
[2024/11/19 15:27:40] [debug] [output:null:null.0] task_id=0 assigned to thread #0
[2024/11/19 15:27:40] [debug] [out flush] cb_destroy coro_id=6
[2024/11/19 15:27:40] [debug] [output:null:null.0] task_id=1 assigned to thread #0
[2024/11/19 15:27:40] [debug] [out flush] cb_destroy coro_id=7
[2024/11/19 15:27:40] [debug] [output:null:null.0] task_id=2 assigned to thread #0
[2024/11/19 15:27:40] [debug] [output:null:null.0] discarding 3805820 bytes
[2024/11/19 15:27:40] [debug] [out flush] cb_destroy coro_id=8
[2024/11/19 15:27:40] [debug] [task] destroy task=0x7e65a001d030 (task_id=0)
[2024/11/19 15:27:40] [ info] [input] resume forward.0
[2024/11/19 15:27:40] [ info] [input] forward.0 resume (mem buf overlimit)
[2024/11/19 15:27:40] [debug] [task] destroy task=0x7e65a001d9f0 (task_id=1)
[2024/11/19 15:27:40] [debug] [task] destroy task=0x7e65a001dac0 (task_id=2)
[2024/11/19 15:27:41] [ warn] [input] forward.0 paused (mem buf overlimit)
[2024/11/19 15:27:41] [ info] [input] pausing forward.0
[2024/11/19 15:27:41] [debug] [input chunk] forward.0 is paused, cannot append records
[2024/11/19 15:27:41] [error] [input:forward:forward.0] could not append logs. ret=-1
[2024/11/19 15:27:41] [debug] [task] created task=0x7e65a001d9f0 id=0 OK
[2024/11/19 15:27:41] [debug] [task] created task=0x7e65a001dac0 id=1 OK
[2024/11/19 15:27:41] [debug] [task] created task=0x7e65a001db90 id=2 OK
[2024/11/19 15:27:41] [debug] [output:null:null.0] discarding 3805820 bytes
[2024/11/19 15:27:41] [debug] [output:null:null.0] task_id=0 assigned to thread #0
[2024/11/19 15:27:41] [debug] [output:null:null.0] discarding 3805820 bytes
[2024/11/19 15:27:41] [debug] [output:null:null.0] task_id=1 assigned to thread #0
[2024/11/19 15:27:41] [debug] [out flush] cb_destroy coro_id=9
[2024/11/19 15:27:41] [debug] [output:null:null.0] task_id=2 assigned to thread #0
[2024/11/19 15:27:41] [debug] [out flush] cb_destroy coro_id=10
[2024/11/19 15:27:41] [debug] [output:null:null.0] discarding 3805820 bytes
[2024/11/19 15:27:41] [debug] [out flush] cb_destroy coro_id=11
[2024/11/19 15:27:41] [debug] [task] destroy task=0x7e65a001d9f0 (task_id=0)
[2024/11/19 15:27:41] [ info] [input] resume forward.0
[2024/11/19 15:27:41] [ info] [input] forward.0 resume (mem buf overlimit)
[2024/11/19 15:27:41] [debug] [task] destroy task=0x7e65a001dac0 (task_id=1)
[2024/11/19 15:27:41] [debug] [task] destroy task=0x7e65a001db90 (task_id=2)
[2024/11/19 15:27:42] [ warn] [input] forward.0 paused (mem buf overlimit)
[2024/11/19 15:27:42] [ info] [input] pausing forward.0
[2024/11/19 15:27:42] [debug] [input chunk] forward.0 is paused, cannot append records
[2024/11/19 15:27:42] [error] [input:forward:forward.0] could not append logs. ret=-1
[2024/11/19 15:27:42] [debug] [task] created task=0x7e65a001dac0 id=0 OK
[2024/11/19 15:27:42] [debug] [output:null:null.0] discarding 3805820 bytes
[2024/11/19 15:27:42] [debug] [task] created task=0x7e65a001db60 id=1 OK
[2024/11/19 15:27:42] [debug] [output:null:null.0] discarding 3805820 bytes
[2024/11/19 15:27:42] [debug] [task] created task=0x7e65a001c780 id=2 OK
[2024/11/19 15:27:42] [debug] [out flush] cb_destroy coro_id=12
[2024/11/19 15:27:42] [debug] [output:null:null.0] task_id=0 assigned to thread #0
[2024/11/19 15:27:42] [debug] [out flush] cb_destroy coro_id=13
[2024/11/19 15:27:42] [debug] [output:null:null.0] task_id=1 assigned to thread #0
[2024/11/19 15:27:42] [debug] [output:null:null.0] discarding 3805820 bytes
[2024/11/19 15:27:42] [debug] [output:null:null.0] task_id=2 assigned to thread #0
[2024/11/19 15:27:42] [debug] [out flush] cb_destroy coro_id=14
[2024/11/19 15:27:42] [debug] [task] destroy task=0x7e65a001dac0 (task_id=0)
[2024/11/19 15:27:42] [ info] [input] resume forward.0
[2024/11/19 15:27:42] [ info] [input] forward.0 resume (mem buf overlimit)
[2024/11/19 15:27:42] [debug] [task] destroy task=0x7e65a001db60 (task_id=1)
[2024/11/19 15:27:42] [debug] [task] destroy task=0x7e65a001c780 (task_id=2)
[2024/11/19 15:27:43] [ warn] [input] forward.0 paused (mem buf overlimit)
[2024/11/19 15:27:43] [ info] [input] pausing forward.0
[2024/11/19 15:27:43] [debug] [input chunk] forward.0 is paused, cannot append records
[2024/11/19 15:27:43] [error] [input:forward:forward.0] could not append logs. ret=-1
[2024/11/19 15:27:43] [debug] [task] created task=0x7e65a001d9f0 id=0 OK
[2024/11/19 15:27:43] [debug] [output:null:null.0] discarding 3805820 bytes
[2024/11/19 15:27:43] [debug] [task] created task=0x7e65a001da90 id=1 OK
[2024/11/19 15:27:43] [debug] [output:null:null.0] discarding 3805820 bytes
[2024/11/19 15:27:43] [debug] [task] created task=0x7e65a001c780 id=2 OK
[2024/11/19 15:27:43] [debug] [out flush] cb_destroy coro_id=15
[2024/11/19 15:27:43] [debug] [output:null:null.0] task_id=0 assigned to thread #0
[2024/11/19 15:27:43] [debug] [out flush] cb_destroy coro_id=16
[2024/11/19 15:27:43] [debug] [output:null:null.0] task_id=1 assigned to thread #0
[2024/11/19 15:27:43] [debug] [output:null:null.0] discarding 3805820 bytes
[2024/11/19 15:27:43] [debug] [output:null:null.0] task_id=2 assigned to thread #0
[2024/11/19 15:27:43] [debug] [out flush] cb_destroy coro_id=17
[2024/11/19 15:27:43] [debug] [task] destroy task=0x7e65a001d9f0 (task_id=0)
[2024/11/19 15:27:43] [ info] [input] resume forward.0
[2024/11/19 15:27:43] [ info] [input] forward.0 resume (mem buf overlimit)
[2024/11/19 15:27:43] [debug] [task] destroy task=0x7e65a001da90 (task_id=1)
[2024/11/19 15:27:43] [debug] [task] destroy task=0x7e65a001c780 (task_id=2)
[2024/11/19 15:27:44] [ warn] [input] forward.0 paused (mem buf overlimit)
[2024/11/19 15:27:44] [ info] [input] pausing forward.0
[2024/11/19 15:27:44] [debug] [input chunk] forward.0 is paused, cannot append records
[2024/11/19 15:27:44] [error] [input:forward:forward.0] could not append logs. ret=-1
[2024/11/19 15:27:44] [debug] [task] created task=0x7e65a001db60 id=0 OK
[2024/11/19 15:27:44] [debug] [output:null:null.0] discarding 3805820 bytes
[2024/11/19 15:27:44] [debug] [task] created task=0x7e65a001dc30 id=1 OK
[2024/11/19 15:27:44] [debug] [output:null:null.0] discarding 3805820 bytes
[2024/11/19 15:27:44] [debug] [task] created task=0x7e65a001c780 id=2 OK
[2024/11/19 15:27:44] [debug] [out flush] cb_destroy coro_id=18
[2024/11/19 15:27:44] [debug] [output:null:null.0] task_id=0 assigned to thread #0
[2024/11/19 15:27:44] [debug] [out flush] cb_destroy coro_id=19
[2024/11/19 15:27:44] [debug] [output:null:null.0] task_id=1 assigned to thread #0
[2024/11/19 15:27:44] [debug] [output:null:null.0] discarding 3805820 bytes
[2024/11/19 15:27:44] [debug] [output:null:null.0] task_id=2 assigned to thread #0
[2024/11/19 15:27:44] [debug] [out flush] cb_destroy coro_id=20
[2024/11/19 15:27:44] [debug] [task] destroy task=0x7e65a001db60 (task_id=0)
[2024/11/19 15:27:44] [ info] [input] resume forward.0
[2024/11/19 15:27:44] [ info] [input] forward.0 resume (mem buf overlimit)
[2024/11/19 15:27:44] [debug] [task] destroy task=0x7e65a001dc30 (task_id=1)
[2024/11/19 15:27:44] [debug] [task] destroy task=0x7e65a001c780 (task_id=2)
[2024/11/19 15:27:45] [ warn] [input] forward.0 paused (mem buf overlimit)
[2024/11/19 15:27:45] [ info] [input] pausing forward.0
[2024/11/19 15:27:45] [debug] [input chunk] forward.0 is paused, cannot append records
[2024/11/19 15:27:45] [error] [input:forward:forward.0] could not append logs. ret=-1
[2024/11/19 15:27:45] [debug] [task] created task=0x7e65a001c780 id=0 OK
[2024/11/19 15:27:45] [debug] [output:null:null.0] discarding 3805820 bytes
[2024/11/19 15:27:45] [debug] [task] created task=0x7e65a001c820 id=1 OK
[2024/11/19 15:27:45] [debug] [output:null:null.0] discarding 3805820 bytes
[2024/11/19 15:27:45] [debug] [task] created task=0x7e65a001c9e0 id=2 OK
[2024/11/19 15:27:45] [debug] [out flush] cb_destroy coro_id=21
[2024/11/19 15:27:45] [debug] [output:null:null.0] task_id=0 assigned to thread #0
[2024/11/19 15:27:45] [debug] [out flush] cb_destroy coro_id=22
[2024/11/19 15:27:45] [debug] [output:null:null.0] task_id=1 assigned to thread #0
[2024/11/19 15:27:45] [debug] [output:null:null.0] discarding 3805820 bytes
[2024/11/19 15:27:45] [debug] [output:null:null.0] task_id=2 assigned to thread #0
[2024/11/19 15:27:45] [debug] [out flush] cb_destroy coro_id=23
[2024/11/19 15:27:45] [debug] [task] destroy task=0x7e65a001c780 (task_id=0)
[2024/11/19 15:27:45] [ info] [input] resume forward.0
[2024/11/19 15:27:45] [ info] [input] forward.0 resume (mem buf overlimit)
[2024/11/19 15:27:45] [debug] [task] destroy task=0x7e65a001c820 (task_id=1)
[2024/11/19 15:27:45] [debug] [task] destroy task=0x7e65a001c9e0 (task_id=2)
[2024/11/19 15:27:46] [ warn] [input] forward.0 paused (mem buf overlimit)
[2024/11/19 15:27:46] [ info] [input] pausing forward.0
[2024/11/19 15:27:46] [debug] [input chunk] forward.0 is paused, cannot append records
[2024/11/19 15:27:46] [error] [input:forward:forward.0] could not append logs. ret=-1
[2024/11/19 15:27:46] [debug] [task] created task=0x7e65a001c780 id=0 OK
[2024/11/19 15:27:46] [debug] [task] created task=0x7e65a001c850 id=1 OK
[2024/11/19 15:27:46] [debug] [task] created task=0x7e65a001c920 id=2 OK
[2024/11/19 15:27:46] [debug] [output:null:null.0] discarding 3805820 bytes
[2024/11/19 15:27:46] [debug] [output:null:null.0] task_id=0 assigned to thread #0
[2024/11/19 15:27:46] [debug] [output:null:null.0] discarding 3805820 bytes
[2024/11/19 15:27:46] [debug] [output:null:null.0] task_id=1 assigned to thread #0
[2024/11/19 15:27:46] [debug] [out flush] cb_destroy coro_id=24
[2024/11/19 15:27:46] [debug] [output:null:null.0] task_id=2 assigned to thread #0
[2024/11/19 15:27:46] [debug] [out flush] cb_destroy coro_id=25
[2024/11/19 15:27:46] [debug] [output:null:null.0] discarding 3805820 bytes
[2024/11/19 15:27:46] [debug] [out flush] cb_destroy coro_id=26
[2024/11/19 15:27:46] [debug] [task] destroy task=0x7e65a001c780 (task_id=0)
[2024/11/19 15:27:46] [ info] [input] resume forward.0
[2024/11/19 15:27:46] [ info] [input] forward.0 resume (mem buf overlimit)
[2024/11/19 15:27:46] [debug] [task] destroy task=0x7e65a001c850 (task_id=1)
[2024/11/19 15:27:46] [debug] [task] destroy task=0x7e65a001c920 (task_id=2)
[2024/11/19 15:27:46] [ warn] [input] forward.0 paused (mem buf overlimit)
[2024/11/19 15:27:46] [ info] [input] pausing forward.0
[2024/11/19 15:27:47] [debug] [task] created task=0x7e65a001c8e0 id=0 OK
[2024/11/19 15:27:47] [debug] [output:null:null.0] discarding 3044656 bytes
[2024/11/19 15:27:47] [debug] [task] created task=0x7e65a001c9b0 id=1 OK
[2024/11/19 15:27:47] [debug] [out flush] cb_destroy coro_id=27
[2024/11/19 15:27:47] [debug] [task] created task=0x7e65a001ca80 id=2 OK
[2024/11/19 15:27:47] [debug] [output:null:null.0] discarding 3044656 bytes
[2024/11/19 15:27:47] [debug] [task] created task=0x7e65a001cb50 id=3 OK
[2024/11/19 15:27:47] [debug] [output:null:null.0] discarding 3044656 bytes
[2024/11/19 15:27:47] [debug] [output:null:null.0] task_id=0 assigned to thread #0
[2024/11/19 15:27:47] [debug] [out flush] cb_destroy coro_id=28
[2024/11/19 15:27:47] [debug] [output:null:null.0] task_id=1 assigned to thread #0
[2024/11/19 15:27:47] [debug] [out flush] cb_destroy coro_id=29
[2024/11/19 15:27:47] [debug] [output:null:null.0] task_id=2 assigned to thread #0
[2024/11/19 15:27:47] [debug] [output:null:null.0] discarding 1712619 bytes
[2024/11/19 15:27:47] [debug] [output:null:null.0] task_id=3 assigned to thread #0
[2024/11/19 15:27:47] [debug] [out flush] cb_destroy coro_id=30
[2024/11/19 15:27:47] [debug] [task] destroy task=0x7e65a001c8e0 (task_id=0)
[2024/11/19 15:27:47] [ info] [input] resume forward.0
[2024/11/19 15:27:47] [ info] [input] forward.0 resume (mem buf overlimit)
[2024/11/19 15:27:47] [debug] [task] destroy task=0x7e65a001c9b0 (task_id=1)
[2024/11/19 15:27:47] [debug] [task] destroy task=0x7e65a001ca80 (task_id=2)
[2024/11/19 15:27:47] [debug] [task] destroy task=0x7e65a001cb50 (task_id=3)
^C[2024/11/19 15:28:46] [engine] caught signal (SIGINT)
[2024/11/19 15:28:46] [ warn] [engine] service will shutdown in max 5 seconds
[2024/11/19 15:28:46] [ info] [input] pausing forward.0
[2024/11/19 15:28:46] [ info] [engine] service has stopped (0 pending tasks)
[2024/11/19 15:28:46] [ info] [input] pausing forward.0
[2024/11/19 15:28:46] [ info] [output:null:null.0] thread worker #0 stopping...
[2024/11/19 15:28:46] [ info] [output:null:null.0] thread worker #0 stopped
  • Attached Valgrind output that shows no leaks or memory corruption was found
==128639== 
==128639== HEAP SUMMARY:
==128639==     in use at exit: 0 bytes in 0 blocks
==128639==   total heap usage: 6,259 allocs, 6,259 frees, 551,594,087 bytes allocated
==128639== 
==128639== All heap blocks were freed -- no leaks are possible
==128639== 
==128639== For lists of detected and suppressed errors, rerun with: -s
==128639== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

  • Run local packaging test showing all targets (including any new ones) build.
  • Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

  • Documentation required for this feature

Backporting

  • Backport to latest stable release.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
1 participant