1

I have been fiddling with this error for a little bit now and wondering what's going wrong. I'm running a testnet instance of Bitcoin Core on a relatively low-spec EC2 instance (2MB ram, 40gb disk)

It is unable to sync correctly. No matter the size of the dbcache and mempool, it will always crash before emptying it. There are no error messages in the debug.log, just the latest UpdateTip.

example with DBcache=4, mempool=5: 2022-08-15T13:25:48Z UpdateTip: new best=00000000781c1335e275cb33c0f16be3eb60c803482995f6fdf88fc1cc9cce8b height=21799 version=0x00000002 log2_work=48.609020 tx=33303 date='2012-08-30T16:51:17Z' progress=0.000525 cache=4.1MiB(22817txo) Killed Increasing the dbcache/mempool will allow it to run longer but it will inevitably be killed in the same way.

More debug.log:

bitcoin@localhost:~$ bitcoind -testnet -rpcuser="user" -rpcpassword="password" -dbcache=20 -maxmempool=5 
2022-08-15T16:04:02Z Bitcoin Core version v23.0.0 (release build)
2022-08-15T16:04:02Z Assuming ancestors of block 00000000000163cfb1f97c4e4098a3692c8053ad9cab5ad9c86b338b5c00b8b7 have valid signatures.
2022-08-15T16:04:02Z Setting nMinimumChainWork=00000000000000000000000000000000000000000000064728c7be6fe4b2f961
2022-08-15T16:04:02Z Using the 'x86_shani(1way,2way)' SHA256 implementation
2022-08-15T16:04:02Z Using RdSeed as additional entropy source
2022-08-15T16:04:02Z Using RdRand as an additional entropy source
2022-08-15T16:04:02Z Default data directory /home/bitcoin/.bitcoin
2022-08-15T16:04:02Z Using data directory /home/bitcoin/.bitcoin/testnet3
2022-08-15T16:04:02Z Config file: /home/bitcoin/.bitcoin/bitcoin.conf (not found, skipping)
2022-08-15T16:04:02Z Command-line arg: dbcache="20"
2022-08-15T16:04:02Z Command-line arg: maxmempool="5"
2022-08-15T16:04:02Z Command-line arg: rpcpassword=****
2022-08-15T16:04:02Z Command-line arg: rpcuser=****
2022-08-15T16:04:02Z Command-line arg: testnet=""
2022-08-15T16:04:02Z Using at most 125 automatic connections (1024 file descriptors available)
2022-08-15T16:04:02Z Using 16 MiB out of 32/2 requested for signature cache, able to store 524288 elements
2022-08-15T16:04:02Z Using 16 MiB out of 32/2 requested for script execution cache, able to store 524288 elements
2022-08-15T16:04:02Z Script verification uses 0 additional threads
2022-08-15T16:04:02Z scheduler thread start
2022-08-15T16:04:02Z HTTP: creating work queue of depth 16
2022-08-15T16:04:02Z Config options rpcuser and rpcpassword will soon be deprecated. Locally-run instances may remove rpcuser to use cookie-based auth, or may be replaced with rpcauth. Please see share/rpcauth for rpcauth auth generation.
2022-08-15T16:04:02Z HTTP: starting 4 worker threads
2022-08-15T16:04:02Z Using wallet directory /home/bitcoin/.bitcoin/testnet3/wallets
2022-08-15T16:04:02Z init message: Verifying wallet(s)…
2022-08-15T16:04:02Z Using /16 prefix for IP bucketing
2022-08-15T16:04:02Z init message: Loading P2P addresses…
2022-08-15T16:04:02Z Loaded 0 addresses from peers.dat  6ms
2022-08-15T16:04:02Z init message: Loading banlist…
2022-08-15T16:04:02Z SetNetworkActive: true
2022-08-15T16:04:02Z Failed to read fee estimates from /home/bitcoin/.bitcoin/testnet3/fee_estimates.dat. Continue anyway.
2022-08-15T16:04:02Z Cache configuration:
2022-08-15T16:04:02Z * Using 2.0 MiB for block index database
2022-08-15T16:04:02Z * Using 8.0 MiB for chain state database
2022-08-15T16:04:02Z * Using 10.0 MiB for in-memory UTXO set (plus up to 4.8 MiB of unused mempool space)
2022-08-15T16:04:02Z init message: Loading block index…
2022-08-15T16:04:02Z Switching active chainstate to Chainstate [ibd] @ height -1 (null)
2022-08-15T16:04:02Z Opening LevelDB in /home/bitcoin/.bitcoin/testnet3/blocks/index
2022-08-15T16:04:02Z Opened LevelDB successfully
2022-08-15T16:04:02Z Using obfuscation key for /home/bitcoin/.bitcoin/testnet3/blocks/index: 0000000000000000
2022-08-15T16:04:02Z LoadBlockIndexDB: last block file = 0
2022-08-15T16:04:02Z LoadBlockIndexDB: last block file info: CBlockFileInfo(blocks=0, size=0, heights=0...0, time=1970-01-01...1970-01-01)
2022-08-15T16:04:02Z Checking all blk files are present...
2022-08-15T16:04:02Z Initializing databases...
2022-08-15T16:04:02Z Opening LevelDB in /home/bitcoin/.bitcoin/testnet3/chainstate
2022-08-15T16:04:02Z Opened LevelDB successfully
2022-08-15T16:04:02Z Using obfuscation key for /home/bitcoin/.bitcoin/testnet3/chainstate: fb97b88490e524df
2022-08-15T16:04:02Z init message: Verifying blocks…
2022-08-15T16:04:02Z  block index              49ms
2022-08-15T16:04:02Z loadblk thread start
2022-08-15T16:04:02Z UpdateTip: new best=000000000933ea01ad0ee984209779baaec3ced90fa3f408719526f8d77f4943 height=0 version=0x00000001 log2_work=32.000022 tx=1 date='2011-02-02T23:16:42Z' progress=0.000000 cache=0.0MiB(0txo)
2022-08-15T16:04:02Z block tree size = 1
2022-08-15T16:04:02Z nBestHeight = 0
2022-08-15T16:04:02Z AddLocal(-:18333,1)
2022-08-15T16:04:02Z Discover: IPv4 - eth0: 
2022-08-15T16:04:02Z Bound to 127.0.0.1:18334
2022-08-15T16:04:02Z Bound to [::]:18333
2022-08-15T16:04:02Z Bound to 0.0.0.0:18333
2022-08-15T16:04:02Z Failed to open mempool file from disk. Continuing anyway.
2022-08-15T16:04:02Z loadblk thread exit
2022-08-15T16:04:02Z torcontrol thread start
2022-08-15T16:04:02Z 0 block-relay-only anchors will be tried for connections.
2022-08-15T16:04:02Z init message: Starting network threads…
2022-08-15T16:04:02Z init message: Done loading
2022-08-15T16:04:02Z addcon thread start
2022-08-15T16:04:02Z opencon thread start
2022-08-15T16:04:02Z msghand thread start
2022-08-15T16:04:02Z dnsseed thread start
2022-08-15T16:04:02Z Loading addresses from DNS seed testnet-seed.bluematt.me.
2022-08-15T16:04:02Z net thread start
2022-08-15T16:04:03Z Loading addresses from DNS seed testnet-seed.bitcoin.jonasschnelli.ch.
2022-08-15T16:04:03Z Loading addresses from DNS seed seed.testnet.bitcoin.sprovoost.nl.
2022-08-15T16:04:03Z Loading addresses from DNS seed seed.tbtc.petertodd.org.
2022-08-15T16:04:03Z 62 addresses found from DNS seeds
2022-08-15T16:04:03Z dnsseed thread exit

2022-08-15T16:09:21Z UpdateTip: new best=000000000e74e1d0d23ec2ad5ea79b01df8a7f423c0f1676cdef2d21ec9f43cf height=56054 version=0x00000002 log2_work=52.424615 tx=93808 date='2013-03-08T16:20:21Z' progress=0.001480 cache=13.3MiB(85444txo)
Killed
Pieter Wuille
  • 98,249
  • 9
  • 183
  • 287
Rutger Versteegden
  • 1,071
  • 7
  • 11
  • 3
    2 GB, not MB, yes? Either way, the machine does not have enough memory to run Bitcoin Core and that's why it keeps getting killed. You could disable the mempool by running in blocks only mode. That would allow you to free up some memory for the dbcache. – Andrew Chow Aug 15 '22 at 16:18
  • Correct. This was my suspicion also but I've read numerous accounts of people sucessfully running core with 1gb (or sometimes even less!?), and here there did not seem to be any obvious relation to the size of the dbcache/mempool. Thanks. – Rutger Versteegden Aug 15 '22 at 16:32
  • 1
    What else is running on the machine that may be consuming RAM? You probably want a very lightweight OS, with little GUI/desktop things running on it. Also, testnet uses inherently more memory than mainnet as it has far more headers (probably ~300 MB just in headers alone). – Pieter Wuille Aug 15 '22 at 17:10
  • OS indeed uses a decent chunk of the 2GB for reasons unknown. I resolved it using a swap file and core works fine now. – Rutger Versteegden Aug 15 '22 at 18:11
  • If you wish, you could either tidy up by deleting your question or (better?) add an answer describing how you solved the problem in sufficient detail to help others with the same problem. – RedGrittyBrick Aug 16 '22 at 09:46
  • @RedGrittyBrick Done. Will accept my own answer tomorrow. – Rutger Versteegden Aug 16 '22 at 10:21
  • I wonder if you have been able to sync the blockchain with such a few disk space (even for testnet, is that enough?). For mainnet I use an Amazon EC2 instance type of "tx2.large" with 750Gb of SSB disk and 16Gb or RAM. And I start bitcoind with -dbcache=8192 and -banscore=10 – Invest Data Systems FR Sep 15 '22 at 15:29

1 Answers1

2

If Bitcoin Core continually gets killed before writing anything to your disk independently of your dbcache and mempool configurations and with no error logs in the debug.log, you are most likely lacking sufficient memory.

You can check if this is the case by monitoring your ram more closely sudo free -h

If your system allows for it you can create a swap file, which resolved the issue for me, along with the commands found here.

Rutger Versteegden
  • 1,071
  • 7
  • 11