Skip to content

v0.17.4.6 - SQLite-based trie and index storage

Pre-release
Pre-release
Compare
Choose a tag to compare
@BrannonKing BrannonKing released this 24 Jun 21:13
v0.17.4.6
c9637f8

This release includes a major refactor to store the claim trie and other indexes in Sqlite instead of LevelDB. The build does not include any code to read LevelDB. Hence, on its first run it will require you to run with the -reindex parameter. The reindex process takes 1-2 hours.

The wallet storage remains in Berkeley DB. The new backing storage brings an advantage of allowing other tools to monitor the storage in real-time. Moreover, the trie structure that was stored in RAM previously is now fully contained in the Sqlite tables. This repairs a long-standing issue: the RAM requirement no longer increases daily. It's tuned to run at about 2GB out of the box and less if you're already caught up to the current block — half that of the current build.

A clean exit is required to fully flush the disk buffers. This will be fast when lbrycrd is caught up to the current block height as it flushes on every new block, but it could be slow if exiting during the initial block download (IBD). Be patient and wait for it.

The schema for the data can be explored by opening these files via the sqlite3 shell: ~/.lbrycrd/*.sqlite

Notes

Breaking change: The blocksdir parameter, when utilized, now specifies the blocks directory fully rather than getting a "blocks" folder appended onto whatever was specified in the parameter. It now matches the behavior specified in the documentation for that parameter.

Wallet RPC sendtoaddress times are still unreasonable for large wallets (aka, wallets that have grown over time through numerous sends and receives). For frequent sends we recommend the use of the sendmany RPC call. Another workaround is to create a new wallet and forward your funds there. This works if you don't need your transaction history.

It is still expected that the v0.17.4+ builds will be marginally slower than the v0.17.3 builds because they exchange some performance for less (and more deterministic) RAM usage. Hence, this and all other v0.17.4 builds are not recommended for use on a spinning disk (an HDD).

Changelog Summary

  1. The RPC calls dealing with claim sequence now use the original block creation height rather than the last update height for their sequence comparisons.
  2. Much of the LBRY-specific code has been isolated into its own library.
  3. Socket selection was changed to use the poll mechanism on Linux.
  4. The CLI help for getclaimproofbybid was repaired.
  5. lsn_reset is no longer triggered as part of every wallet flush. Run the wallet backup RPC command to trigger it.
  6. The wallet auto-flush period was changed from half a second to a full second.
  7. Regtest block generation performance was improved.
  8. Linux builds are no longer a mix of Clang and GCC. GCC 7 is now used instead of GCC 5.
  9. Reindex will be continued on restart (after a clean shutdown), and the corruption that was happening with that has been repaired.
  10. A maxblocksize parameter was added to aid in testing.
  11. Block generation RPC calls are no longer allowed during initial block download (or reindexing).
  12. Database flushes will now trigger before the ZeroMQ events for the new block trigger.
  13. We now only attempt to disk sync for 4 seconds instead of 20.
  14. The size estimate from RPC gettxoutsetinfo is now much more reasonable.
  15. The testnet chain was reset

SHA256 Sums

aacd79497f41c14c8177eefcdabca068392f375f7707a8f5c8bf17cc3d2a2297  lbrycrd-darwin-1746.zip
9c902ae1b82ee52a9e3ad109c693553bfaff6c6ab18a225e9f1c327c3a395085  lbrycrd-linux-1746.zip
8e149b6eb7feaadef850744a6e19d20a4f88367f6b1a4110f4d6c839ee69890e  lbrycrd-windows-1746.zip
Assets 5
Loading