1

I compiled v0.5.3 of the core client from source, in order to compare IDB (Initial Blockchain Download) performance between versions.

I created a fresh Ubuntu Xenial 16.04 machine on Amazon EC2. I then ran the following commands:

sudo apt-get update
sudo apt-get upgrade -y
sudo apt-get install build-essential autoconf libboost-all-dev \
                     libssl-dev libtool pkg-config libevent-dev \
                     libdb++-dev libminiupnpc-dev

I then checked out the source:

git clone https://github.com/bitcoin/bitcoin.git
cd bitcoin
git checkout v0.5.3
cd src

In order to deal with Strict DER issue, I replaced the function bool Verify in src/key.h with this.

I then compile:

make -f makefile.unix

To deal with the database issue introduced in 0.8, I added a file ~/.bitcoin/DB_CONFIG as per these instructions:

set_lg_dir database
set_lk_max_locks 50000

I then start bitcoind:

./bitcoind -dbcache=8000 -daemon

It happily syncs up to block 274,011, where it remained stuck for at least a day. I only tried this once, so I'm not sure if it's reproducible.

Here's a sample of the logs around that point; I can provide more if needed:

received block 00000000000000032b4e
SetBestChain: new best=00000000000000032b4e  height=274011  work=29398500860863841218972
ProcessBlock: ACCEPTED
askfor tx 2b909ba77c89c324a669   1499456128000000
sending getdata: tx b187cf6896ac46ef99b7
sending getdata: tx ecc1cef110e97830171d
sending getdata: tx f2cf74028a74602e8197
sending getdata: tx 4ed765e52248503c18a8
sending getdata: tx cf9cd92853c50e156366
sending getdata: tx 10e5e68f221e4451df73
sending getdata: tx 585900b4e015ed2ac1ef
askfor tx 221fbf80861ce7a6d584   0
sending getdata: tx 144e4167f4bef1b7ed42
sending getdata: tx 0ff657e09ac2bec18ffe
sending getdata: tx 6fdd97713fcb14c7521f
sending getdata: tx fdf5d59441c5b8a0c60d
sending getdata: tx 28d85b339859cb995328
sending getdata: tx 4df8f62f7328bca3b940
sending getdata: tx 15bf18055f55320481e5
sending getdata: tx 9a0a74186e9787b65cfe
sending getdata: tx 221fbf80861ce7a6d584
askfor tx e92fff32bc853b9aaa74   0
askfor tx 894d1cbfb82b61160bec   1499455908000000
askfor tx 48f22fcab4174c22ca0b   1499455909000000
askfor tx db4145ed6b4d366e429e   0
askfor tx 221fbf80861ce7a6d584   1499455912000000
askfor tx eb604d6581ab29aac596   0
sending getdata: tx aba92ed0b3ad91cff821
sending getdata: tx d0175f40dc118a5c8aef
sending getdata: tx 8dcc57d538a3391a40a6
sending getdata: tx 699d9a01db321519373c
sending getdata: tx 6dec27e77519e9968909
sending getdata: tx aab87f6f03f9e9a13c8c
sending getdata: tx d415b3535065b0f7f45a
sending getdata: tx 389e7c4af3b8c6eb7a23
sending getdata: tx 224f97a4e9dc09976302
sending getdata: tx da54c3bd141b537e2a32
sending getdata: tx 5c7b7bae2d3fccdc61e9
sending getdata: tx e92fff32bc853b9aaa74
...
ERROR: nonstandard txout: OP_HASH160 432a41db83cc1f7e5cc9c48a0808b00ff2992a3a OP_EQUAL
ERROR: AcceptToMemoryPool() : nonstandard transaction type
ERROR: ConnectInputs() : cf9cd92853 mapTransactions prev not found 4c77e274a8
ERROR: AcceptToMemoryPool() : ConnectInputs failed cf9cd92853
ERROR: ConnectInputs() : 10e5e68f22 mapTransactions prev not found d5d9ba282e
ERROR: AcceptToMemoryPool() : ConnectInputs failed 10e5e68f22
ERROR: ConnectInputs() : aba92ed0b3 mapTransactions prev not found b5646cc672
ERROR: AcceptToMemoryPool() : ConnectInputs failed aba92ed0b3
ERROR: nonstandard txin: 0 3044022075066e21b2d705e77bfed0d4fbdd235c8993b91cae47daeac7438881e35e83dd022059b4e62df6e4dafc1ccf01b7f8dedf82be3c5fc1e804e0885a885274905fc65801 3044022024c5b0cfdf748245a4ad8654710412ae5430b052ebb693e941131786a3fd369b02203f53c18e76676f3280e5b3198466f9348d26769166e43de27f997da4dac9299401 52210204f0cf6188b88806fa626dec13a8969d7d157fbe8963d08cf7046e07ef4900d02102a3d1066654c906f66f4778ef70aa3b97ee31518325a97b2bb571e2fc57c6fb6e52ae
ERROR: AcceptToMemoryPool() : nonstandard transaction type
ERROR: ConnectInputs() : 8dcc57d538 mapTransactions prev not found f055f633bf
...
askfor block 000000000000000000d1   0
askfor tx cb7c501051cb8a14797d   0
sending getdata: block 000000000000000000d1
sending getdata: tx cb7c501051cb8a14797d
received block 000000000000000000d1
ProcessBlock: ORPHAN BLOCK, prev=0000000000000000008f
ERROR: ConnectInputs() : cb7c501051 mapTransactions prev not found b1fd3405d1
ERROR: AcceptToMemoryPool() : ConnectInputs failed cb7c501051
sending getdata: tx eff6ce1819d75352b6a4
...
Sjors Provoost
  • 858
  • 6
  • 16
  • Can you try again, perhaps with an even larger lock limit? – Pieter Wuille Jul 09 '17 at 22:14
  • 2
    The number suggested here is much larger: https://bitcoin.org/en/alert/2013-03-15-upgrade-deadline – Pieter Wuille Jul 09 '17 at 22:15
  • That did the trick! – Sjors Provoost Jul 10 '17 at 15:13
  • Similarly 0.6 got stuck around 299,927 and 0.7 at 319,884. Both resumed once I bumped this setting. I wonder why they don't all stop at the exact same block. I restarted these nodes; based on performance of more modern versions, it should take them at least a week to sync, maybe even two weeks. – Sjors Provoost Jul 10 '17 at 15:31
  • 1
    Note that the synchronization mechanism used before 0.10 (which added headers-first sync) was prone to randomly stop, and download blocks multiple times (especially 0.9 was bad). You can avoid most of these issues by using -connect=IP to a known good peer. – Pieter Wuille Jul 10 '17 at 16:10
  • I set `connect` to my own `v0.14.2` node whitelisted IP. `0.14` log says `2017-09-05 18:04:12 receive version message: : version 50300, blocks=273076, us=X.X.X.X:8333, peer=15, peeraddr=35.158.4.220:54134` over and over again. The `v0.5` log says `trying connection X.X.XX:8333 lastseen=-390175.9hrs lasttry=-417953.7hrs connected X.X.XX:8333 Added time data, samples 2, offset +0 (+0 minutes) version message: version 70015, blocks=483687 accepted alert 2147483647, AppliesToMe()=1 getheaders 221534 to 00000000000000000000 limit 264153 socket recv error 104 disconnecting node X.X.XX:8333` – Sjors Provoost Sep 05 '17 at 18:40
  • Anyway, just stopping and restarting the 0.5 node seems to be enough to keep it moving. This time I'm only trying to sync it to 1-1-2014. It went reasonably well until roughly the last month where it just stops hours at a time. There's also no `peers.dat` file yet, is that correct? – Sjors Provoost Sep 05 '17 at 18:43
  • Correction, it was running v0.14.1, not v0.14.2. – Sjors Provoost Sep 05 '17 at 19:26
  • 1
    "why they don't all stop at the exact same block." because prior to 0.8 the consensus behavior was effectively non-deterministic-- it depends on how records in the database were aligned in pages. – G. Maxwell Nov 30 '18 at 10:52
  • @PieterWuille I actually used `connect=`, but it's still happily connecting to other peers (like `addnode=` would do). Did the semantics change? – Sjors Provoost Nov 30 '18 at 13:07

1 Answers1

4

Increase set_lk_max_locks from 50000 to 537000 as per these instructions. Thanks Pieter Wuille for pointing that out.

Sjors Provoost
  • 858
  • 6
  • 16
  • Followup: on [a more recent attempt](https://bitcoin.stackexchange.com/questions/81485/bitcoind-0-5-3-stuck-at-block-364-670-what-to-do) the above change got me to block 364670, and bumping `set_lk_max_locks` no longer seems to help. – Sjors Provoost Nov 30 '18 at 13:09