Thread main panicked - ARM64

[michaelschapira]

Describe the bug

Initial block indexing crash

[2024-04-05T05:17:12.533Z INFO  electrs::index] indexing 2000 blocks: [617051..619050]
thread 'main' panicked at 'core returned invalid block: Needed(4294967295)', src/index.rs:294:49
stack backtrace:
   0: rust_begin_unwind
             at /usr/src/rustc-1.63.0/library/std/src/panicking.rs:584:5
   1: core::panicking::panic_fmt
             at /usr/src/rustc-1.63.0/library/core/src/panicking.rs:142:14
   2: core::result::unwrap_failed
             at /usr/src/rustc-1.63.0/library/core/src/result.rs:1805:5
   3: electrs::index::index_single_block
   4: prometheus::histogram::Histogram::observe_closure_duration
   5: electrs::metrics::metrics_impl::Histogram::observe_duration
   6: prometheus::histogram::Histogram::observe_closure_duration
   7: electrs::metrics::metrics_impl::Histogram::observe_duration
   8: prometheus::histogram::Histogram::observe_closure_duration
   9: electrs::metrics::metrics_impl::Histogram::observe_duration
  10: electrs::daemon::Daemon::for_blocks
  11: electrs::index::Index::sync
  12: prometheus::histogram::Histogram::observe_closure_duration
  13: electrs::server::run
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

Electrs version

To Reproduce
Steps to reproduce the behavior:

1. Configure and start electrs

Expected behavior

Configuration

electrs.toml

auth=<auth.py token>
daemon_rpc_addr = <remote_ip:port>
daemon_p2p_addr =<remote_ip:port>
db_dir = "/electrs"
network = "bitcoin"
electrum_rpc_addr = "0.0.0.0:50001"
log_filters = "INFO"

System running electrs

• self build v0.10.4 using this guide
• debian 12.5 ARM64 (Raspberry CM4 with NVMe SSD)

[michaelschapira]

adding full stack as it crashed again:

2024-04-05T12:42:00.957Z INFO  electrs::index] indexing 2000 blocks: [645171..647170]
thread 'main' panicked at 'core returned invalid block: Needed(4294967295)', src/index.rs:294:49
stack backtrace:
   0:       0x5561d0dd28 - std::backtrace_rs::backtrace::libunwind::trace::h137073a3a1f593a6
                               at /usr/src/rustc-1.63.0/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5
   1:       0x5561d0dd28 - std::backtrace_rs::backtrace::trace_unsynchronized::hab992635d0edd37e
                               at /usr/src/rustc-1.63.0/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:       0x5561d0dd28 - std::sys_common::backtrace::_print_fmt::h9c097270d950e8bc
                               at /usr/src/rustc-1.63.0/library/std/src/sys_common/backtrace.rs:66:5
   3:       0x5561d0dd28 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h757f44ca309a3bb5
                               at /usr/src/rustc-1.63.0/library/std/src/sys_common/backtrace.rs:45:22
   4:       0x5561d1e7dc - core::fmt::write::h7d2f41d8cce9c61d
                               at /usr/src/rustc-1.63.0/library/core/src/fmt/mod.rs:1197:17
   5:       0x5561ce9b68 - std::io::Write::write_fmt::hc0adc688406af2c8
                               at /usr/src/rustc-1.63.0/library/std/src/io/mod.rs:1672:15
   6:       0x5561cf41fc - std::sys_common::backtrace::_print::hd3c4acaacf992fc3
                               at /usr/src/rustc-1.63.0/library/std/src/sys_common/backtrace.rs:48:5
   7:       0x5561cf41fc - std::sys_common::backtrace::print::h9508c30a026e937f
                               at /usr/src/rustc-1.63.0/library/std/src/sys_common/backtrace.rs:35:9
   8:       0x5561cf41fc - std::panicking::default_hook::{{closure}}::h70abbef5ea3f155c
                               at /usr/src/rustc-1.63.0/library/std/src/panicking.rs:295:22
   9:       0x5561cf3e78 - std::panicking::default_hook::h03ed0ae708f93f49
                               at /usr/src/rustc-1.63.0/library/std/src/panicking.rs:314:9
  10:       0x5561cf46e0 - std::panicking::rust_panic_with_hook::h01dae9edaa4a0d99
                               at /usr/src/rustc-1.63.0/library/std/src/panicking.rs:698:17
  11:       0x5561d0e33c - std::panicking::begin_panic_handler::{{closure}}::hfd4b396d9cf692c0
                               at /usr/src/rustc-1.63.0/library/std/src/panicking.rs:588:13
  12:       0x5561d0de30 - std::sys_common::backtrace::__rust_end_short_backtrace::h40fb34855cb856bd
                               at /usr/src/rustc-1.63.0/library/std/src/sys_common/backtrace.rs:138:18
  13:       0x5561cf43d0 - rust_begin_unwind
                               at /usr/src/rustc-1.63.0/library/std/src/panicking.rs:584:5
  14:       0x55615846a0 - core::panicking::panic_fmt::h4e2410bf58d1d606
                               at /usr/src/rustc-1.63.0/library/core/src/panicking.rs:142:14
  15:       0x556158473c - core::result::unwrap_failed::h31648b284bec598a
                               at /usr/src/rustc-1.63.0/library/core/src/result.rs:1805:5
  16:       0x55616387d8 - electrs::index::index_single_block::h8b14eb2f3fc45367
  17:       0x55615eba78 - prometheus::histogram::Histogram::observe_closure_duration::h117273c0ce80808c
  18:       0x5561646378 - electrs::metrics::metrics_impl::Histogram::observe_duration::h408656516cca8cd9
  19:       0x55615ebc88 - prometheus::histogram::Histogram::observe_closure_duration::h1e4434ba08e5efb4
  20:       0x556164669c - electrs::metrics::metrics_impl::Histogram::observe_duration::h739d21c806274bad
  21:       0x55615ebef8 - prometheus::histogram::Histogram::observe_closure_duration::h30a52b796881d91c
  22:       0x5561646178 - electrs::metrics::metrics_impl::Histogram::observe_duration::h1426d297bc513475
  23:       0x55615e4b48 - electrs::daemon::Daemon::for_blocks::h9192ab691d1ae26d
  24:       0x5561637e18 - electrs::index::Index::sync::he73a330622a00534
  25:       0x55615ee0ac - prometheus::histogram::Histogram::observe_closure_duration::h9a2d127abaa0fef6
  26:       0x55615beedc - electrs::server::run::h743dfe68ae13ca4b
  27:       0x55615b0cdc - std::sys_common::backtrace::__rust_begin_short_backtrace::h13cc9af3ac2f01e3
  28:       0x55615b0e20 - std::rt::lang_start::{{closure}}::h9417d98068b0b10f
  29:       0x5561ce9520 - core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once::h203583f1558b0503
                               at /usr/src/rustc-1.63.0/library/core/src/ops/function.rs:280:13
  30:       0x5561ce9520 - std::panicking::try::do_call::h95dcde817035ba94
                               at /usr/src/rustc-1.63.0/library/std/src/panicking.rs:492:40
  31:       0x5561ce9520 - std::panicking::try::h750bd3b0fb992042
                               at /usr/src/rustc-1.63.0/library/std/src/panicking.rs:456:19
  32:       0x5561ce9520 - std::panic::catch_unwind::h5709f0e020b3fa09
                               at /usr/src/rustc-1.63.0/library/std/src/panic.rs:137:14
  33:       0x5561ce9520 - std::rt::lang_start_internal::{{closure}}::h862a83ee600c9edc
                               at /usr/src/rustc-1.63.0/library/std/src/rt.rs:128:48
  34:       0x5561ce9520 - std::panicking::try::do_call::h45c796c04930b780
                               at /usr/src/rustc-1.63.0/library/std/src/panicking.rs:492:40
  35:       0x5561ce9520 - std::panicking::try::h723d369a4a2dba66
                               at /usr/src/rustc-1.63.0/library/std/src/panicking.rs:456:19
  36:       0x5561ce9520 - std::panic::catch_unwind::hc06495a5a23e5647
                               at /usr/src/rustc-1.63.0/library/std/src/panic.rs:137:14
  37:       0x5561ce9520 - std::rt::lang_start_internal::hb66ca75d64827996
                               at /usr/src/rustc-1.63.0/library/std/src/rt.rs:128:20
  38:       0x55615b0e78 - main
  39:       0x7f8e377780 - __libc_start_call_main
                               at ./csu/../sysdeps/nptl/libc_start_call_main.h:58:16
  40:       0x7f8e377858 - __libc_start_main_impl
                               at ./csu/../csu/libc-start.c:360:3
  41:       0x55615b0bf0 - _start
  42:                0x0 - <unknown>
[2024-04-05T12:43:39.385Z INFO  electrs::db] closing DB at /electrs/bitcoin

[romanz]

Is it possible that the block data is corrupted?

[michaelschapira]

I don't think that was the issue. When restarting the process, it proceeded for a while until if crashed again, but it was moving forward.
I compiled and ran on x86, against the same bitcoind rpc server and it worked fine.