Crash after migrating instance from one machine to another

[cdecker]

This was reported by raucao on IRC. He migrated an existing node from
one host to another, including a switch from Intel Core i7 to AMD
Ryzen (not sure if the arch switch has anything to do with it, just
mentioning it for context).

The following backtrace was provided:

lightning_connectd: FATAL SIGNAL 6 (version v0.9.2)
0x562b3d3b5304 send_backtrace
        common/daemon.c:38
0x562b3d3b539e crashdump
        common/daemon.c:51
0x7f139c1fa20f ???
        ???:0
0x7f139c1fa18b ???
        ???:0
0x7f139c1d9858 ???
        ???:0
0x562b3d3e2e3d call_error
        ccan/ccan/tal/tal.c:93
0x562b3d3e2ed8 check_bounds
        ccan/ccan/tal/tal.c:165
0x562b3d3e2f00 to_tal_hdr
        ccan/ccan/tal/tal.c:174
0x562b3d3e3dfe tal_resize_
        ccan/ccan/tal/tal.c:694
0x562b3d3e2379 do_vfmt
        ccan/ccan/tal/str/str.c:60
0x562b3d3e2613 tal_append_vfmt
        ccan/ccan/tal/str/str.c:102
0x562b3d3e26b6 tal_append_fmt
        ccan/ccan/tal/str/str.c:111
0x562b3d3afade add_errors_to_error_list
        connectd/connectd.c:680
0x562b3d3afb8d destroy_io_conn
        connectd/connectd.c:702
0x562b3d3da84e destroy_conn
        ccan/ccan/io/poll.c:244
0x562b3d3dabb0 cleanup_conn_without_close
        ccan/ccan/io/poll.c:264
0x562b3d3d9559 io_close_taken_fd
        ccan/ccan/io/io.c:463
0x562b3d3af9d3 peer_connected
        connectd/connectd.c:511
0x562b3d3aff16 peer_init_received
        connectd/peer_exchange_initmsg.c:94
0x562b3d3d8fc0 next_plan
        ccan/ccan/io/io.c:59
0x562b3d3d946b do_plan
        ccan/ccan/io/io.c:407
0x562b3d3d9508 io_ready
        ccan/ccan/io/io.c:417
0x562b3d3dae4c io_loop
        ccan/ccan/io/poll.c:445
0x562b3d3afcb0 main
        connectd/connectd.c:1703
0x7f139c1db0b2 ???
        ???:0
0x562b3d3ab59d ???
        ???:0
0xffffffffffffffff ???
        ???:0

From this it appears that &connect->errors is not tal allocated
which should not happen since it is initialized in try_connect_peer:

lightning/connectd/connectd.c

Lines 1526 to 1537 in abad494

	connect = tal(daemon, struct connecting);
	connect->daemon = daemon;
	connect->id = *id;
	connect->addrs = tal_steal(connect, addrs);
	connect->addrnum = 0;
	/* connstate is supposed to be updated as we go, to give context for
	* errors which occur. We miss it in a few places; would be nice to
	* fix! */
	connect->connstate = "Connection establishment";
	connect->seconds_waited = seconds_waited;
	connect->addrhint = tal_steal(connect, addrhint);
	connect->errors = tal_strdup(connect, "");

[ZmnSCPxj]

Very strange. I migrated a node (0.9.1) from AMD64 to a RaspPi successfully. Maybe not related to the migration?

[raucao]

@cdecker Thanks for creating the issue! As requested on IRC, here's the full stacktrace including valgrind output:

http://pastie.org/p/2npR0dBcNvrYSWaQOyRwRx/raw

[cdecker]

Ok, this looks like a use-after-free, caused by freeing the struct instance representing the outgoing connection, and then later calling destroy_io_conn with that struct as a parameter. This is strange since just before tal_freeing the struct we remove the destructor:

lightning/connectd/connectd.c

Lines 292 to 300 in 0a01111

	outgoing = find_connecting(daemon, id);
	if (outgoing) {
	/* Don't call destroy_io_conn, since we're done. */
	if (outgoing->conn)
	io_set_finish(outgoing->conn, NULL, NULL);
	/* Now free the 'connecting' struct. */
	tal_free(outgoing);
	}

The destructor is only set once, before calling io_connect (in two places, one for proxied connections and one for direct connections), so we should be removing the destroying the instance, so I don't really see what's going wrong here.

[raucao]

I'm wondering if there's a different migration path for me to use in order to be able to shut down the old machine (it's a rented metal box in a data center, and c-lightning is now the last thing running on it, due to me not being able to migrate the node).

I saw that there's a backup plugin for example. Perhaps backup-and-restore would be an alternative that doesn't trigger the bug?

[raucao]

Just in case it helps, I just updated c-lightning to 0.9.3 on both the old and new machine, and tried to do the migration again in the same way as before (shut down, rsync lightning dir, slightly adjust RPC config, start daemon with synced data). The error is still the same AFAICS. Here's the full output from valgrind:

Stacktrace

satoshi@bitcoin-2:/var/opt$ valgrind --trace-children=yes -q lightningd --lightning-dir=/var/opt/lightning
2021-01-26T15:49:16.838Z INFO    plugin-bcli: bitcoin-cli initialized and connected to bitcoind.
2021-01-26T15:49:24.102Z INFO    lightningd: --------------------------------------------------
2021-01-26T15:49:24.105Z INFO    lightningd: Server started with public key 03e26a98ee4d1320d5775fab291580969180592db3679e6d32e360dbd147066ca3, alias ln1.kosmos.org (color #03e26a) and lightningd v0.9.3
2021-01-26T15:49:24.303Z INFO    plugin-fetchinvoice: Killing plugin: offers not enabled in config
2021-01-26T15:49:24.314Z INFO    plugin-offers: Killing plugin: offers not enabled in config
==2271620== Invalid read of size 8
==2271620==    at 0x113011: destroy_io_conn (connectd.c:703)
==2271620==    by 0x144BF3: destroy_conn (poll.c:244)
==2271620==    by 0x144F55: cleanup_conn_without_close (poll.c:264)
==2271620==    by 0x1438FE: io_close_taken_fd (io.c:463)
==2271620==    by 0x112EB6: peer_connected (connectd.c:511)
==2271620==    by 0x1133F9: peer_init_received (peer_exchange_initmsg.c:94)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==    by 0x143810: do_plan (io.c:407)
==2271620==    by 0x1438AD: io_ready (io.c:417)
==2271620==    by 0x1451F1: io_loop (poll.c:445)
==2271620==    by 0x113193: main (connectd.c:1703)
==2271620==  Address 0x4d3e238 is 136 bytes inside a block of size 160 free'd
==2271620==    at 0x483CA3F: free (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==2271620==    by 0x14D81D: del_tree (tal.c:421)
==2271620==    by 0x14DD17: tal_free (tal.c:486)
==2271620==    by 0x1101D3: connected_to_peer (connectd.c:299)
==2271620==    by 0x112DB4: peer_connected (connectd.c:481)
==2271620==    by 0x1133F9: peer_init_received (peer_exchange_initmsg.c:94)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==    by 0x143810: do_plan (io.c:407)
==2271620==    by 0x1438AD: io_ready (io.c:417)
==2271620==    by 0x1451F1: io_loop (poll.c:445)
==2271620==    by 0x113193: main (connectd.c:1703)
==2271620==  Block was alloc'd at
==2271620==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==2271620==    by 0x14D4EE: allocate (tal.c:250)
==2271620==    by 0x14DBC3: tal_alloc_ (tal.c:428)
==2271620==    by 0x1119D5: try_connect_peer (connectd.c:1526)
==2271620==    by 0x111BFA: connect_to_peer (connectd.c:1558)
==2271620==    by 0x1129D8: recv_req (connectd.c:1627)
==2271620==    by 0x118FCB: handle_read (daemon_conn.c:31)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==    by 0x143810: do_plan (io.c:407)
==2271620==    by 0x1438AD: io_ready (io.c:417)
==2271620==    by 0x1451F1: io_loop (poll.c:445)
==2271620==    by 0x113193: main (connectd.c:1703)
==2271620==
==2271620== Invalid read of size 8
==2271620==    at 0x113015: destroy_io_conn (connectd.c:703)
==2271620==    by 0x144BF3: destroy_conn (poll.c:244)
==2271620==    by 0x144F55: cleanup_conn_without_close (poll.c:264)
==2271620==    by 0x1438FE: io_close_taken_fd (io.c:463)
==2271620==    by 0x112EB6: peer_connected (connectd.c:511)
==2271620==    by 0x1133F9: peer_init_received (peer_exchange_initmsg.c:94)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==    by 0x143810: do_plan (io.c:407)
==2271620==    by 0x1438AD: io_ready (io.c:417)
==2271620==    by 0x1451F1: io_loop (poll.c:445)
==2271620==    by 0x113193: main (connectd.c:1703)
==2271620==  Address 0x4d3e220 is 112 bytes inside a block of size 160 free'd
==2271620==    at 0x483CA3F: free (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==2271620==    by 0x14D81D: del_tree (tal.c:421)
==2271620==    by 0x14DD17: tal_free (tal.c:486)
==2271620==    by 0x1101D3: connected_to_peer (connectd.c:299)
==2271620==    by 0x112DB4: peer_connected (connectd.c:481)
==2271620==    by 0x1133F9: peer_init_received (peer_exchange_initmsg.c:94)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==    by 0x143810: do_plan (io.c:407)
==2271620==    by 0x1438AD: io_ready (io.c:417)
==2271620==    by 0x1451F1: io_loop (poll.c:445)
==2271620==    by 0x113193: main (connectd.c:1703)
==2271620==  Block was alloc'd at
==2271620==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==2271620==    by 0x14D4EE: allocate (tal.c:250)
==2271620==    by 0x14DBC3: tal_alloc_ (tal.c:428)
==2271620==    by 0x1119D5: try_connect_peer (connectd.c:1526)
==2271620==    by 0x111BFA: connect_to_peer (connectd.c:1558)
==2271620==    by 0x1129D8: recv_req (connectd.c:1627)
==2271620==    by 0x118FCB: handle_read (daemon_conn.c:31)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==    by 0x143810: do_plan (io.c:407)
==2271620==    by 0x1438AD: io_ready (io.c:417)
==2271620==    by 0x1451F1: io_loop (poll.c:445)
==2271620==    by 0x113193: main (connectd.c:1703)
==2271620==
==2271620== Invalid read of size 8
==2271620==    at 0x113027: destroy_io_conn (connectd.c:703)
==2271620==    by 0x144BF3: destroy_conn (poll.c:244)
==2271620==    by 0x144F55: cleanup_conn_without_close (poll.c:264)
==2271620==    by 0x1438FE: io_close_taken_fd (io.c:463)
==2271620==    by 0x112EB6: peer_connected (connectd.c:511)
==2271620==    by 0x1133F9: peer_init_received (peer_exchange_initmsg.c:94)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==    by 0x143810: do_plan (io.c:407)
==2271620==    by 0x1438AD: io_ready (io.c:417)
==2271620==    by 0x1451F1: io_loop (poll.c:445)
==2271620==    by 0x113193: main (connectd.c:1703)
==2271620==  Address 0x4d3e228 is 120 bytes inside a block of size 160 free'd
==2271620==    at 0x483CA3F: free (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==2271620==    by 0x14D81D: del_tree (tal.c:421)
==2271620==    by 0x14DD17: tal_free (tal.c:486)
==2271620==    by 0x1101D3: connected_to_peer (connectd.c:299)
==2271620==    by 0x112DB4: peer_connected (connectd.c:481)
==2271620==    by 0x1133F9: peer_init_received (peer_exchange_initmsg.c:94)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==    by 0x143810: do_plan (io.c:407)
==2271620==    by 0x1438AD: io_ready (io.c:417)
==2271620==    by 0x1451F1: io_loop (poll.c:445)
==2271620==    by 0x113193: main (connectd.c:1703)
==2271620==  Block was alloc'd at
==2271620==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==2271620==    by 0x14D4EE: allocate (tal.c:250)
==2271620==    by 0x14DBC3: tal_alloc_ (tal.c:428)
==2271620==    by 0x1119D5: try_connect_peer (connectd.c:1526)
==2271620==    by 0x111BFA: connect_to_peer (connectd.c:1558)
==2271620==    by 0x1129D8: recv_req (connectd.c:1627)
==2271620==    by 0x118FCB: handle_read (daemon_conn.c:31)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==    by 0x143810: do_plan (io.c:407)
==2271620==    by 0x1438AD: io_ready (io.c:417)
==2271620==    by 0x1451F1: io_loop (poll.c:445)
==2271620==    by 0x113193: main (connectd.c:1703)
==2271620==
==2271620== Invalid read of size 4
==2271620==    at 0x12004D: fmt_wireaddr_internal (wireaddr.c:212)
==2271620==    by 0x120152: fmt_wireaddr_internal_ (wireaddr.c:231)
==2271620==    by 0x11EFE4: type_to_string_ (type_to_string.c:35)
==2271620==    by 0x11303D: destroy_io_conn (connectd.c:703)
==2271620==    by 0x144BF3: destroy_conn (poll.c:244)
==2271620==    by 0x144F55: cleanup_conn_without_close (poll.c:264)
==2271620==    by 0x1438FE: io_close_taken_fd (io.c:463)
==2271620==    by 0x112EB6: peer_connected (connectd.c:511)
==2271620==    by 0x1133F9: peer_init_received (peer_exchange_initmsg.c:94)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==    by 0x143810: do_plan (io.c:407)
==2271620==    by 0x1438AD: io_ready (io.c:417)
==2271620==  Address 0x4d3e060 is 304 bytes inside a block of size 568 free'd
==2271620==    at 0x483CA3F: free (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==2271620==    by 0x14D81D: del_tree (tal.c:421)
==2271620==    by 0x14D7DA: del_tree (tal.c:412)
==2271620==    by 0x14DD17: tal_free (tal.c:486)
==2271620==    by 0x1101D3: connected_to_peer (connectd.c:299)
==2271620==    by 0x112DB4: peer_connected (connectd.c:481)
==2271620==    by 0x1133F9: peer_init_received (peer_exchange_initmsg.c:94)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==    by 0x143810: do_plan (io.c:407)
==2271620==    by 0x1438AD: io_ready (io.c:417)
==2271620==    by 0x1451F1: io_loop (poll.c:445)
==2271620==    by 0x113193: main (connectd.c:1703)
==2271620==  Block was alloc'd at
==2271620==    at 0x483DFAF: realloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==2271620==    by 0x14E23A: tal_resize_ (tal.c:699)
==2271620==    by 0x110FBD: add_gossip_addrs (connectd.c:1453)
==2271620==    by 0x11197E: try_connect_peer (connectd.c:1494)
==2271620==    by 0x111BFA: connect_to_peer (connectd.c:1558)
==2271620==    by 0x1129D8: recv_req (connectd.c:1627)
==2271620==    by 0x118FCB: handle_read (daemon_conn.c:31)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==    by 0x143810: do_plan (io.c:407)
==2271620==    by 0x1438AD: io_ready (io.c:417)
==2271620==    by 0x1451F1: io_loop (poll.c:445)
==2271620==    by 0x113193: main (connectd.c:1703)
==2271620==
==2271620== Invalid read of size 4
==2271620==    at 0x11FEB6: fmt_wireaddr_without_port (wireaddr.c:238)
==2271620==    by 0x12001D: fmt_wireaddr (wireaddr.c:261)
==2271620==    by 0x1200B2: fmt_wireaddr_internal (wireaddr.c:218)
==2271620==    by 0x120152: fmt_wireaddr_internal_ (wireaddr.c:231)
==2271620==    by 0x11EFE4: type_to_string_ (type_to_string.c:35)
==2271620==    by 0x11303D: destroy_io_conn (connectd.c:703)
==2271620==    by 0x144BF3: destroy_conn (poll.c:244)
==2271620==    by 0x144F55: cleanup_conn_without_close (poll.c:264)
==2271620==    by 0x1438FE: io_close_taken_fd (io.c:463)
==2271620==    by 0x112EB6: peer_connected (connectd.c:511)
==2271620==    by 0x1133F9: peer_init_received (peer_exchange_initmsg.c:94)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==  Address 0x4d3e064 is 308 bytes inside a block of size 568 free'd
==2271620==    at 0x483CA3F: free (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==2271620==    by 0x14D81D: del_tree (tal.c:421)
==2271620==    by 0x14D7DA: del_tree (tal.c:412)
==2271620==    by 0x14DD17: tal_free (tal.c:486)
==2271620==    by 0x1101D3: connected_to_peer (connectd.c:299)
==2271620==    by 0x112DB4: peer_connected (connectd.c:481)
==2271620==    by 0x1133F9: peer_init_received (peer_exchange_initmsg.c:94)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==    by 0x143810: do_plan (io.c:407)
==2271620==    by 0x1438AD: io_ready (io.c:417)
==2271620==    by 0x1451F1: io_loop (poll.c:445)
==2271620==    by 0x113193: main (connectd.c:1703)
==2271620==  Block was alloc'd at
==2271620==    at 0x483DFAF: realloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==2271620==    by 0x14E23A: tal_resize_ (tal.c:699)
==2271620==    by 0x110FBD: add_gossip_addrs (connectd.c:1453)
==2271620==    by 0x11197E: try_connect_peer (connectd.c:1494)
==2271620==    by 0x111BFA: connect_to_peer (connectd.c:1558)
==2271620==    by 0x1129D8: recv_req (connectd.c:1627)
==2271620==    by 0x118FCB: handle_read (daemon_conn.c:31)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==    by 0x143810: do_plan (io.c:407)
==2271620==    by 0x1438AD: io_ready (io.c:417)
==2271620==    by 0x1451F1: io_loop (poll.c:445)
==2271620==    by 0x113193: main (connectd.c:1703)
==2271620==
==2271620== Invalid read of size 1
==2271620==    at 0x4B572F0: inet_ntop4 (inet_ntop.c:85)
==2271620==    by 0x4B572F0: inet_ntop (inet_ntop.c:57)
==2271620==    by 0x11FEE1: fmt_wireaddr_without_port (wireaddr.c:240)
==2271620==    by 0x12001D: fmt_wireaddr (wireaddr.c:261)
==2271620==    by 0x1200B2: fmt_wireaddr_internal (wireaddr.c:218)
==2271620==    by 0x120152: fmt_wireaddr_internal_ (wireaddr.c:231)
==2271620==    by 0x11EFE4: type_to_string_ (type_to_string.c:35)
==2271620==    by 0x11303D: destroy_io_conn (connectd.c:703)
==2271620==    by 0x144BF3: destroy_conn (poll.c:244)
==2271620==    by 0x144F55: cleanup_conn_without_close (poll.c:264)
==2271620==    by 0x1438FE: io_close_taken_fd (io.c:463)
==2271620==    by 0x112EB6: peer_connected (connectd.c:511)
==2271620==    by 0x1133F9: peer_init_received (peer_exchange_initmsg.c:94)
==2271620==  Address 0x4d3e06a is 314 bytes inside a block of size 568 free'd
==2271620==    at 0x483CA3F: free (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==2271620==    by 0x14D81D: del_tree (tal.c:421)
==2271620==    by 0x14D7DA: del_tree (tal.c:412)
==2271620==    by 0x14DD17: tal_free (tal.c:486)
==2271620==    by 0x1101D3: connected_to_peer (connectd.c:299)
==2271620==    by 0x112DB4: peer_connected (connectd.c:481)
==2271620==    by 0x1133F9: peer_init_received (peer_exchange_initmsg.c:94)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==    by 0x143810: do_plan (io.c:407)
==2271620==    by 0x1438AD: io_ready (io.c:417)
==2271620==    by 0x1451F1: io_loop (poll.c:445)
==2271620==    by 0x113193: main (connectd.c:1703)
==2271620==  Block was alloc'd at
==2271620==    at 0x483DFAF: realloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==2271620==    by 0x14E23A: tal_resize_ (tal.c:699)
==2271620==    by 0x110FBD: add_gossip_addrs (connectd.c:1453)
==2271620==    by 0x11197E: try_connect_peer (connectd.c:1494)
==2271620==    by 0x111BFA: connect_to_peer (connectd.c:1558)
==2271620==    by 0x1129D8: recv_req (connectd.c:1627)
==2271620==    by 0x118FCB: handle_read (daemon_conn.c:31)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==    by 0x143810: do_plan (io.c:407)
==2271620==    by 0x1438AD: io_ready (io.c:417)
==2271620==    by 0x1451F1: io_loop (poll.c:445)
==2271620==    by 0x113193: main (connectd.c:1703)
==2271620==
==2271620== Invalid read of size 1
==2271620==    at 0x4B572F4: inet_ntop4 (inet_ntop.c:85)
==2271620==    by 0x4B572F4: inet_ntop (inet_ntop.c:57)
==2271620==    by 0x11FEE1: fmt_wireaddr_without_port (wireaddr.c:240)
==2271620==    by 0x12001D: fmt_wireaddr (wireaddr.c:261)
==2271620==    by 0x1200B2: fmt_wireaddr_internal (wireaddr.c:218)
==2271620==    by 0x120152: fmt_wireaddr_internal_ (wireaddr.c:231)
==2271620==    by 0x11EFE4: type_to_string_ (type_to_string.c:35)
==2271620==    by 0x11303D: destroy_io_conn (connectd.c:703)
==2271620==    by 0x144BF3: destroy_conn (poll.c:244)
==2271620==    by 0x144F55: cleanup_conn_without_close (poll.c:264)
==2271620==    by 0x1438FE: io_close_taken_fd (io.c:463)
==2271620==    by 0x112EB6: peer_connected (connectd.c:511)
==2271620==    by 0x1133F9: peer_init_received (peer_exchange_initmsg.c:94)
==2271620==  Address 0x4d3e069 is 313 bytes inside a block of size 568 free'd
==2271620==    at 0x483CA3F: free (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==2271620==    by 0x14D81D: del_tree (tal.c:421)
==2271620==    by 0x14D7DA: del_tree (tal.c:412)
==2271620==    by 0x14DD17: tal_free (tal.c:486)
==2271620==    by 0x1101D3: connected_to_peer (connectd.c:299)
==2271620==    by 0x112DB4: peer_connected (connectd.c:481)
==2271620==    by 0x1133F9: peer_init_received (peer_exchange_initmsg.c:94)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==    by 0x143810: do_plan (io.c:407)
==2271620==    by 0x1438AD: io_ready (io.c:417)
==2271620==    by 0x1451F1: io_loop (poll.c:445)
==2271620==    by 0x113193: main (connectd.c:1703)
==2271620==  Block was alloc'd at
==2271620==    at 0x483DFAF: realloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==2271620==    by 0x14E23A: tal_resize_ (tal.c:699)
==2271620==    by 0x110FBD: add_gossip_addrs (connectd.c:1453)
==2271620==    by 0x11197E: try_connect_peer (connectd.c:1494)
==2271620==    by 0x111BFA: connect_to_peer (connectd.c:1558)
==2271620==    by 0x1129D8: recv_req (connectd.c:1627)
==2271620==    by 0x118FCB: handle_read (daemon_conn.c:31)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==    by 0x143810: do_plan (io.c:407)
==2271620==    by 0x1438AD: io_ready (io.c:417)
==2271620==    by 0x1451F1: io_loop (poll.c:445)
==2271620==    by 0x113193: main (connectd.c:1703)
==2271620==
==2271620== Invalid read of size 1
==2271620==    at 0x4B572FE: inet_ntop4 (inet_ntop.c:85)
==2271620==    by 0x4B572FE: inet_ntop (inet_ntop.c:57)
==2271620==    by 0x11FEE1: fmt_wireaddr_without_port (wireaddr.c:240)
==2271620==    by 0x12001D: fmt_wireaddr (wireaddr.c:261)
==2271620==    by 0x1200B2: fmt_wireaddr_internal (wireaddr.c:218)
==2271620==    by 0x120152: fmt_wireaddr_internal_ (wireaddr.c:231)
==2271620==    by 0x11EFE4: type_to_string_ (type_to_string.c:35)
==2271620==    by 0x11303D: destroy_io_conn (connectd.c:703)
==2271620==    by 0x144BF3: destroy_conn (poll.c:244)
==2271620==    by 0x144F55: cleanup_conn_without_close (poll.c:264)
==2271620==    by 0x1438FE: io_close_taken_fd (io.c:463)
==2271620==    by 0x112EB6: peer_connected (connectd.c:511)
==2271620==    by 0x1133F9: peer_init_received (peer_exchange_initmsg.c:94)
==2271620==  Address 0x4d3e06c is 316 bytes inside a block of size 568 free'd
==2271620==    at 0x483CA3F: free (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==2271620==    by 0x14D81D: del_tree (tal.c:421)
==2271620==    by 0x14D7DA: del_tree (tal.c:412)
==2271620==    by 0x14DD17: tal_free (tal.c:486)
==2271620==    by 0x1101D3: connected_to_peer (connectd.c:299)
==2271620==    by 0x112DB4: peer_connected (connectd.c:481)
==2271620==    by 0x1133F9: peer_init_received (peer_exchange_initmsg.c:94)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==    by 0x143810: do_plan (io.c:407)
==2271620==    by 0x1438AD: io_ready (io.c:417)
==2271620==    by 0x1451F1: io_loop (poll.c:445)
==2271620==    by 0x113193: main (connectd.c:1703)
==2271620==  Block was alloc'd at
==2271620==    at 0x483DFAF: realloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==2271620==    by 0x14E23A: tal_resize_ (tal.c:699)
==2271620==    by 0x110FBD: add_gossip_addrs (connectd.c:1453)
==2271620==    by 0x11197E: try_connect_peer (connectd.c:1494)
==2271620==    by 0x111BFA: connect_to_peer (connectd.c:1558)
==2271620==    by 0x1129D8: recv_req (connectd.c:1627)
==2271620==    by 0x118FCB: handle_read (daemon_conn.c:31)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==    by 0x143810: do_plan (io.c:407)
==2271620==    by 0x1438AD: io_ready (io.c:417)
==2271620==    by 0x1451F1: io_loop (poll.c:445)
==2271620==    by 0x113193: main (connectd.c:1703)
==2271620==
==2271620== Invalid read of size 1
==2271620==    at 0x4B57303: inet_ntop4 (inet_ntop.c:85)
==2271620==    by 0x4B57303: inet_ntop (inet_ntop.c:57)
==2271620==    by 0x11FEE1: fmt_wireaddr_without_port (wireaddr.c:240)
==2271620==    by 0x12001D: fmt_wireaddr (wireaddr.c:261)
==2271620==    by 0x1200B2: fmt_wireaddr_internal (wireaddr.c:218)
==2271620==    by 0x120152: fmt_wireaddr_internal_ (wireaddr.c:231)
==2271620==    by 0x11EFE4: type_to_string_ (type_to_string.c:35)
==2271620==    by 0x11303D: destroy_io_conn (connectd.c:703)
==2271620==    by 0x144BF3: destroy_conn (poll.c:244)
==2271620==    by 0x144F55: cleanup_conn_without_close (poll.c:264)
==2271620==    by 0x1438FE: io_close_taken_fd (io.c:463)
==2271620==    by 0x112EB6: peer_connected (connectd.c:511)
==2271620==    by 0x1133F9: peer_init_received (peer_exchange_initmsg.c:94)
==2271620==  Address 0x4d3e06b is 315 bytes inside a block of size 568 free'd
==2271620==    at 0x483CA3F: free (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==2271620==    by 0x14D81D: del_tree (tal.c:421)
==2271620==    by 0x14D7DA: del_tree (tal.c:412)
==2271620==    by 0x14DD17: tal_free (tal.c:486)
==2271620==    by 0x1101D3: connected_to_peer (connectd.c:299)
==2271620==    by 0x112DB4: peer_connected (connectd.c:481)
==2271620==    by 0x1133F9: peer_init_received (peer_exchange_initmsg.c:94)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==    by 0x143810: do_plan (io.c:407)
==2271620==    by 0x1438AD: io_ready (io.c:417)
==2271620==    by 0x1451F1: io_loop (poll.c:445)
==2271620==    by 0x113193: main (connectd.c:1703)
==2271620==  Block was alloc'd at
==2271620==    at 0x483DFAF: realloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==2271620==    by 0x14E23A: tal_resize_ (tal.c:699)
==2271620==    by 0x110FBD: add_gossip_addrs (connectd.c:1453)
==2271620==    by 0x11197E: try_connect_peer (connectd.c:1494)
==2271620==    by 0x111BFA: connect_to_peer (connectd.c:1558)
==2271620==    by 0x1129D8: recv_req (connectd.c:1627)
==2271620==    by 0x118FCB: handle_read (daemon_conn.c:31)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==    by 0x143810: do_plan (io.c:407)
==2271620==    by 0x1438AD: io_ready (io.c:417)
==2271620==    by 0x1451F1: io_loop (poll.c:445)
==2271620==    by 0x113193: main (connectd.c:1703)
==2271620==
==2271620== Invalid read of size 2
==2271620==    at 0x120023: fmt_wireaddr (wireaddr.c:262)
==2271620==    by 0x1200B2: fmt_wireaddr_internal (wireaddr.c:218)
==2271620==    by 0x120152: fmt_wireaddr_internal_ (wireaddr.c:231)
==2271620==    by 0x11EFE4: type_to_string_ (type_to_string.c:35)
==2271620==    by 0x11303D: destroy_io_conn (connectd.c:703)
==2271620==    by 0x144BF3: destroy_conn (poll.c:244)
==2271620==    by 0x144F55: cleanup_conn_without_close (poll.c:264)
==2271620==    by 0x1438FE: io_close_taken_fd (io.c:463)
==2271620==    by 0x112EB6: peer_connected (connectd.c:511)
==2271620==    by 0x1133F9: peer_init_received (peer_exchange_initmsg.c:94)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==    by 0x143810: do_plan (io.c:407)
==2271620==  Address 0x4d3e08c is 348 bytes inside a block of size 568 free'd
==2271620==    at 0x483CA3F: free (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==2271620==    by 0x14D81D: del_tree (tal.c:421)
==2271620==    by 0x14D7DA: del_tree (tal.c:412)
==2271620==    by 0x14DD17: tal_free (tal.c:486)
==2271620==    by 0x1101D3: connected_to_peer (connectd.c:299)
==2271620==    by 0x112DB4: peer_connected (connectd.c:481)
==2271620==    by 0x1133F9: peer_init_received (peer_exchange_initmsg.c:94)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==    by 0x143810: do_plan (io.c:407)
==2271620==    by 0x1438AD: io_ready (io.c:417)
==2271620==    by 0x1451F1: io_loop (poll.c:445)
==2271620==    by 0x113193: main (connectd.c:1703)
==2271620==  Block was alloc'd at
==2271620==    at 0x483DFAF: realloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==2271620==    by 0x14E23A: tal_resize_ (tal.c:699)
==2271620==    by 0x110FBD: add_gossip_addrs (connectd.c:1453)
==2271620==    by 0x11197E: try_connect_peer (connectd.c:1494)
==2271620==    by 0x111BFA: connect_to_peer (connectd.c:1558)
==2271620==    by 0x1129D8: recv_req (connectd.c:1627)
==2271620==    by 0x118FCB: handle_read (daemon_conn.c:31)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==    by 0x143810: do_plan (io.c:407)
==2271620==    by 0x1438AD: io_ready (io.c:417)
==2271620==    by 0x1451F1: io_loop (poll.c:445)
==2271620==    by 0x113193: main (connectd.c:1703)
==2271620==
==2271620== Invalid read of size 8
==2271620==    at 0x14C9F5: tal_append_vfmt (str.c:102)
==2271620==    by 0x14CABE: tal_append_fmt (str.c:111)
==2271620==    by 0x112FC1: add_errors_to_error_list (connectd.c:680)
==2271620==    by 0x113070: destroy_io_conn (connectd.c:702)
==2271620==    by 0x144BF3: destroy_conn (poll.c:244)
==2271620==    by 0x144F55: cleanup_conn_without_close (poll.c:264)
==2271620==    by 0x1438FE: io_close_taken_fd (io.c:463)
==2271620==    by 0x112EB6: peer_connected (connectd.c:511)
==2271620==    by 0x1133F9: peer_init_received (peer_exchange_initmsg.c:94)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==    by 0x143810: do_plan (io.c:407)
==2271620==    by 0x1438AD: io_ready (io.c:417)
==2271620==  Address 0x4d3e240 is 144 bytes inside a block of size 160 free'd
==2271620==    at 0x483CA3F: free (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==2271620==    by 0x14D81D: del_tree (tal.c:421)
==2271620==    by 0x14DD17: tal_free (tal.c:486)
==2271620==    by 0x1101D3: connected_to_peer (connectd.c:299)
==2271620==    by 0x112DB4: peer_connected (connectd.c:481)
==2271620==    by 0x1133F9: peer_init_received (peer_exchange_initmsg.c:94)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==    by 0x143810: do_plan (io.c:407)
==2271620==    by 0x1438AD: io_ready (io.c:417)
==2271620==    by 0x1451F1: io_loop (poll.c:445)
==2271620==    by 0x113193: main (connectd.c:1703)
==2271620==  Block was alloc'd at
==2271620==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==2271620==    by 0x14D4EE: allocate (tal.c:250)
==2271620==    by 0x14DBC3: tal_alloc_ (tal.c:428)
==2271620==    by 0x1119D5: try_connect_peer (connectd.c:1526)
==2271620==    by 0x111BFA: connect_to_peer (connectd.c:1558)
==2271620==    by 0x1129D8: recv_req (connectd.c:1627)
==2271620==    by 0x118FCB: handle_read (daemon_conn.c:31)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==    by 0x143810: do_plan (io.c:407)
==2271620==    by 0x1438AD: io_ready (io.c:417)
==2271620==    by 0x1451F1: io_loop (poll.c:445)
==2271620==    by 0x113193: main (connectd.c:1703)
==2271620==
==2271620== Invalid read of size 1
==2271620==    at 0x14CA05: tal_append_vfmt (str.c:102)
==2271620==    by 0x14CABE: tal_append_fmt (str.c:111)
==2271620==    by 0x112FC1: add_errors_to_error_list (connectd.c:680)
==2271620==    by 0x113070: destroy_io_conn (connectd.c:702)
==2271620==    by 0x144BF3: destroy_conn (poll.c:244)
==2271620==    by 0x144F55: cleanup_conn_without_close (poll.c:264)
==2271620==    by 0x1438FE: io_close_taken_fd (io.c:463)
==2271620==    by 0x112EB6: peer_connected (connectd.c:511)
==2271620==    by 0x1133F9: peer_init_received (peer_exchange_initmsg.c:94)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==    by 0x143810: do_plan (io.c:407)
==2271620==    by 0x1438AD: io_ready (io.c:417)
==2271620==  Address 0x4d3f208 is 40 bytes inside a block of size 128 free'd
==2271620==    at 0x483CA3F: free (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==2271620==    by 0x14D81D: del_tree (tal.c:421)
==2271620==    by 0x14D7DA: del_tree (tal.c:412)
==2271620==    by 0x14DD17: tal_free (tal.c:486)
==2271620==    by 0x1101D3: connected_to_peer (connectd.c:299)
==2271620==    by 0x112DB4: peer_connected (connectd.c:481)
==2271620==    by 0x1133F9: peer_init_received (peer_exchange_initmsg.c:94)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==    by 0x143810: do_plan (io.c:407)
==2271620==    by 0x1438AD: io_ready (io.c:417)
==2271620==    by 0x1451F1: io_loop (poll.c:445)
==2271620==    by 0x113193: main (connectd.c:1703)
==2271620==  Block was alloc'd at
==2271620==    at 0x483DFAF: realloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==2271620==    by 0x14E23A: tal_resize_ (tal.c:699)
==2271620==    by 0x14C7E0: do_vfmt (str.c:72)
==2271620==    by 0x14CA1B: tal_append_vfmt (str.c:102)
==2271620==    by 0x14CABE: tal_append_fmt (str.c:111)
==2271620==    by 0x112FC1: add_errors_to_error_list (connectd.c:680)
==2271620==    by 0x113070: destroy_io_conn (connectd.c:702)
==2271620==    by 0x144BF3: destroy_conn (poll.c:244)
==2271620==    by 0x144C19: destroy_conn_close_fd (poll.c:250)
==2271620==    by 0x14D6B6: notify (tal.c:240)
==2271620==    by 0x14D788: del_tree (tal.c:402)
==2271620==    by 0x14DD17: tal_free (tal.c:486)
==2271620==
==2271620== Invalid read of size 8
==2271620==    at 0x14E1FF: tal_resize_ (tal.c:694)
==2271620==    by 0x14C781: do_vfmt (str.c:60)
==2271620==    by 0x14CA1B: tal_append_vfmt (str.c:102)
==2271620==    by 0x14CABE: tal_append_fmt (str.c:111)
==2271620==    by 0x112FC1: add_errors_to_error_list (connectd.c:680)
==2271620==    by 0x113070: destroy_io_conn (connectd.c:702)
==2271620==    by 0x144BF3: destroy_conn (poll.c:244)
==2271620==    by 0x144F55: cleanup_conn_without_close (poll.c:264)
==2271620==    by 0x1438FE: io_close_taken_fd (io.c:463)
==2271620==    by 0x112EB6: peer_connected (connectd.c:511)
==2271620==    by 0x1133F9: peer_init_received (peer_exchange_initmsg.c:94)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==  Address 0x4d3e240 is 144 bytes inside a block of size 160 free'd
==2271620==    at 0x483CA3F: free (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==2271620==    by 0x14D81D: del_tree (tal.c:421)
==2271620==    by 0x14DD17: tal_free (tal.c:486)
==2271620==    by 0x1101D3: connected_to_peer (connectd.c:299)
==2271620==    by 0x112DB4: peer_connected (connectd.c:481)
==2271620==    by 0x1133F9: peer_init_received (peer_exchange_initmsg.c:94)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==    by 0x143810: do_plan (io.c:407)
==2271620==    by 0x1438AD: io_ready (io.c:417)
==2271620==    by 0x1451F1: io_loop (poll.c:445)
==2271620==    by 0x113193: main (connectd.c:1703)
==2271620==  Block was alloc'd at
==2271620==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==2271620==    by 0x14D4EE: allocate (tal.c:250)
==2271620==    by 0x14DBC3: tal_alloc_ (tal.c:428)
==2271620==    by 0x1119D5: try_connect_peer (connectd.c:1526)
==2271620==    by 0x111BFA: connect_to_peer (connectd.c:1558)
==2271620==    by 0x1129D8: recv_req (connectd.c:1627)
==2271620==    by 0x118FCB: handle_read (daemon_conn.c:31)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==    by 0x143810: do_plan (io.c:407)
==2271620==    by 0x1438AD: io_ready (io.c:417)
==2271620==    by 0x1451F1: io_loop (poll.c:445)
==2271620==    by 0x113193: main (connectd.c:1703)
==2271620==
==2271620== Invalid read of size 8
==2271620==    at 0x14D2F8: to_tal_hdr (tal.c:174)
==2271620==    by 0x14E206: tal_resize_ (tal.c:694)
==2271620==    by 0x14C781: do_vfmt (str.c:60)
==2271620==    by 0x14CA1B: tal_append_vfmt (str.c:102)
==2271620==    by 0x14CABE: tal_append_fmt (str.c:111)
==2271620==    by 0x112FC1: add_errors_to_error_list (connectd.c:680)
==2271620==    by 0x113070: destroy_io_conn (connectd.c:702)
==2271620==    by 0x144BF3: destroy_conn (poll.c:244)
==2271620==    by 0x144F55: cleanup_conn_without_close (poll.c:264)
==2271620==    by 0x1438FE: io_close_taken_fd (io.c:463)
==2271620==    by 0x112EB6: peer_connected (connectd.c:511)
==2271620==    by 0x1133F9: peer_init_received (peer_exchange_initmsg.c:94)
==2271620==  Address 0x4d3f1f8 is 24 bytes inside a block of size 128 free'd
==2271620==    at 0x483CA3F: free (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==2271620==    by 0x14D81D: del_tree (tal.c:421)
==2271620==    by 0x14D7DA: del_tree (tal.c:412)
==2271620==    by 0x14DD17: tal_free (tal.c:486)
==2271620==    by 0x1101D3: connected_to_peer (connectd.c:299)
==2271620==    by 0x112DB4: peer_connected (connectd.c:481)
==2271620==    by 0x1133F9: peer_init_received (peer_exchange_initmsg.c:94)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==    by 0x143810: do_plan (io.c:407)
==2271620==    by 0x1438AD: io_ready (io.c:417)
==2271620==    by 0x1451F1: io_loop (poll.c:445)
==2271620==    by 0x113193: main (connectd.c:1703)
==2271620==  Block was alloc'd at
==2271620==    at 0x483DFAF: realloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==2271620==    by 0x14E23A: tal_resize_ (tal.c:699)
==2271620==    by 0x14C7E0: do_vfmt (str.c:72)
==2271620==    by 0x14CA1B: tal_append_vfmt (str.c:102)
==2271620==    by 0x14CABE: tal_append_fmt (str.c:111)
==2271620==    by 0x112FC1: add_errors_to_error_list (connectd.c:680)
==2271620==    by 0x113070: destroy_io_conn (connectd.c:702)
==2271620==    by 0x144BF3: destroy_conn (poll.c:244)
==2271620==    by 0x144C19: destroy_conn_close_fd (poll.c:250)
==2271620==    by 0x14D6B6: notify (tal.c:240)
==2271620==    by 0x14D788: del_tree (tal.c:402)
==2271620==    by 0x14DD17: tal_free (tal.c:486)
==2271620==
==2271620== Invalid read of size 8
==2271620==    at 0x14D309: to_tal_hdr (tal.c:175)
==2271620==    by 0x14E206: tal_resize_ (tal.c:694)
==2271620==    by 0x14C781: do_vfmt (str.c:60)
==2271620==    by 0x14CA1B: tal_append_vfmt (str.c:102)
==2271620==    by 0x14CABE: tal_append_fmt (str.c:111)
==2271620==    by 0x112FC1: add_errors_to_error_list (connectd.c:680)
==2271620==    by 0x113070: destroy_io_conn (connectd.c:702)
==2271620==    by 0x144BF3: destroy_conn (poll.c:244)
==2271620==    by 0x144F55: cleanup_conn_without_close (poll.c:264)
==2271620==    by 0x1438FE: io_close_taken_fd (io.c:463)
==2271620==    by 0x112EB6: peer_connected (connectd.c:511)
==2271620==    by 0x1133F9: peer_init_received (peer_exchange_initmsg.c:94)
==2271620==  Address 0x4d3f1e0 is 0 bytes inside a block of size 128 free'd
==2271620==    at 0x483CA3F: free (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==2271620==    by 0x14D81D: del_tree (tal.c:421)
==2271620==    by 0x14D7DA: del_tree (tal.c:412)
==2271620==    by 0x14DD17: tal_free (tal.c:486)
==2271620==    by 0x1101D3: connected_to_peer (connectd.c:299)
==2271620==    by 0x112DB4: peer_connected (connectd.c:481)
==2271620==    by 0x1133F9: peer_init_received (peer_exchange_initmsg.c:94)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==    by 0x143810: do_plan (io.c:407)
==2271620==    by 0x1438AD: io_ready (io.c:417)
==2271620==    by 0x1451F1: io_loop (poll.c:445)
==2271620==    by 0x113193: main (connectd.c:1703)
==2271620==  Block was alloc'd at
==2271620==    at 0x483DFAF: realloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==2271620==    by 0x14E23A: tal_resize_ (tal.c:699)
==2271620==    by 0x14C7E0: do_vfmt (str.c:72)
==2271620==    by 0x14CA1B: tal_append_vfmt (str.c:102)
==2271620==    by 0x14CABE: tal_append_fmt (str.c:111)
==2271620==    by 0x112FC1: add_errors_to_error_list (connectd.c:680)
==2271620==    by 0x113070: destroy_io_conn (connectd.c:702)
==2271620==    by 0x144BF3: destroy_conn (poll.c:244)
==2271620==    by 0x144C19: destroy_conn_close_fd (poll.c:250)
==2271620==    by 0x14D6B6: notify (tal.c:240)
==2271620==    by 0x14D788: del_tree (tal.c:402)
==2271620==    by 0x14DD17: tal_free (tal.c:486)
==2271620==
==2271620== Invalid read of size 8
==2271620==    at 0x14D312: to_tal_hdr (tal.c:176)
==2271620==    by 0x14E206: tal_resize_ (tal.c:694)
==2271620==    by 0x14C781: do_vfmt (str.c:60)
==2271620==    by 0x14CA1B: tal_append_vfmt (str.c:102)
==2271620==    by 0x14CABE: tal_append_fmt (str.c:111)
==2271620==    by 0x112FC1: add_errors_to_error_list (connectd.c:680)
==2271620==    by 0x113070: destroy_io_conn (connectd.c:702)
==2271620==    by 0x144BF3: destroy_conn (poll.c:244)
==2271620==    by 0x144F55: cleanup_conn_without_close (poll.c:264)
==2271620==    by 0x1438FE: io_close_taken_fd (io.c:463)
==2271620==    by 0x112EB6: peer_connected (connectd.c:511)
==2271620==    by 0x1133F9: peer_init_received (peer_exchange_initmsg.c:94)
==2271620==  Address 0x4d3f1e8 is 8 bytes inside a block of size 128 free'd
==2271620==    at 0x483CA3F: free (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==2271620==    by 0x14D81D: del_tree (tal.c:421)
==2271620==    by 0x14D7DA: del_tree (tal.c:412)
==2271620==    by 0x14DD17: tal_free (tal.c:486)
==2271620==    by 0x1101D3: connected_to_peer (connectd.c:299)
==2271620==    by 0x112DB4: peer_connected (connectd.c:481)
==2271620==    by 0x1133F9: peer_init_received (peer_exchange_initmsg.c:94)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==    by 0x143810: do_plan (io.c:407)
==2271620==    by 0x1438AD: io_ready (io.c:417)
==2271620==    by 0x1451F1: io_loop (poll.c:445)
==2271620==    by 0x113193: main (connectd.c:1703)
==2271620==  Block was alloc'd at
==2271620==    at 0x483DFAF: realloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==2271620==    by 0x14E23A: tal_resize_ (tal.c:699)
==2271620==    by 0x14C7E0: do_vfmt (str.c:72)
==2271620==    by 0x14CA1B: tal_append_vfmt (str.c:102)
==2271620==    by 0x14CABE: tal_append_fmt (str.c:111)
==2271620==    by 0x112FC1: add_errors_to_error_list (connectd.c:680)
==2271620==    by 0x113070: destroy_io_conn (connectd.c:702)
==2271620==    by 0x144BF3: destroy_conn (poll.c:244)
==2271620==    by 0x144C19: destroy_conn_close_fd (poll.c:250)
==2271620==    by 0x14D6B6: notify (tal.c:240)
==2271620==    by 0x14D788: del_tree (tal.c:402)
==2271620==    by 0x14DD17: tal_free (tal.c:486)
==2271620==
==2271620== Invalid read of size 8
==2271620==    at 0x14D31B: to_tal_hdr (tal.c:177)
==2271620==    by 0x14E206: tal_resize_ (tal.c:694)
==2271620==    by 0x14C781: do_vfmt (str.c:60)
==2271620==    by 0x14CA1B: tal_append_vfmt (str.c:102)
==2271620==    by 0x14CABE: tal_append_fmt (str.c:111)
==2271620==    by 0x112FC1: add_errors_to_error_list (connectd.c:680)
==2271620==    by 0x113070: destroy_io_conn (connectd.c:702)
==2271620==    by 0x144BF3: destroy_conn (poll.c:244)
==2271620==    by 0x144F55: cleanup_conn_without_close (poll.c:264)
==2271620==    by 0x1438FE: io_close_taken_fd (io.c:463)
==2271620==    by 0x112EB6: peer_connected (connectd.c:511)
==2271620==    by 0x1133F9: peer_init_received (peer_exchange_initmsg.c:94)
==2271620==  Address 0x4d3f1f0 is 16 bytes inside a block of size 128 free'd
==2271620==    at 0x483CA3F: free (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==2271620==    by 0x14D81D: del_tree (tal.c:421)
==2271620==    by 0x14D7DA: del_tree (tal.c:412)
==2271620==    by 0x14DD17: tal_free (tal.c:486)
==2271620==    by 0x1101D3: connected_to_peer (connectd.c:299)
==2271620==    by 0x112DB4: peer_connected (connectd.c:481)
==2271620==    by 0x1133F9: peer_init_received (peer_exchange_initmsg.c:94)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==    by 0x143810: do_plan (io.c:407)
==2271620==    by 0x1438AD: io_ready (io.c:417)
==2271620==    by 0x1451F1: io_loop (poll.c:445)
==2271620==    by 0x113193: main (connectd.c:1703)
==2271620==  Block was alloc'd at
==2271620==    at 0x483DFAF: realloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==2271620==    by 0x14E23A: tal_resize_ (tal.c:699)
==2271620==    by 0x14C7E0: do_vfmt (str.c:72)
==2271620==    by 0x14CA1B: tal_append_vfmt (str.c:102)
==2271620==    by 0x14CABE: tal_append_fmt (str.c:111)
==2271620==    by 0x112FC1: add_errors_to_error_list (connectd.c:680)
==2271620==    by 0x113070: destroy_io_conn (connectd.c:702)
==2271620==    by 0x144BF3: destroy_conn (poll.c:244)
==2271620==    by 0x144C19: destroy_conn_close_fd (poll.c:250)
==2271620==    by 0x14D6B6: notify (tal.c:240)
==2271620==    by 0x14D788: del_tree (tal.c:402)
==2271620==    by 0x14DD17: tal_free (tal.c:486)
==2271620==
==2271620== Invalid free() / delete / delete[] / realloc()
==2271620==    at 0x483DFAF: realloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==2271620==    by 0x14E23A: tal_resize_ (tal.c:699)
==2271620==    by 0x14C781: do_vfmt (str.c:60)
==2271620==    by 0x14CA1B: tal_append_vfmt (str.c:102)
==2271620==    by 0x14CABE: tal_append_fmt (str.c:111)
==2271620==    by 0x112FC1: add_errors_to_error_list (connectd.c:680)
==2271620==    by 0x113070: destroy_io_conn (connectd.c:702)
==2271620==    by 0x144BF3: destroy_conn (poll.c:244)
==2271620==    by 0x144F55: cleanup_conn_without_close (poll.c:264)
==2271620==    by 0x1438FE: io_close_taken_fd (io.c:463)
==2271620==    by 0x112EB6: peer_connected (connectd.c:511)
==2271620==    by 0x1133F9: peer_init_received (peer_exchange_initmsg.c:94)
==2271620==  Address 0x4d3f1e0 is 0 bytes inside a block of size 128 free'd
==2271620==    at 0x483CA3F: free (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==2271620==    by 0x14D81D: del_tree (tal.c:421)
==2271620==    by 0x14D7DA: del_tree (tal.c:412)
==2271620==    by 0x14DD17: tal_free (tal.c:486)
==2271620==    by 0x1101D3: connected_to_peer (connectd.c:299)
==2271620==    by 0x112DB4: peer_connected (connectd.c:481)
==2271620==    by 0x1133F9: peer_init_received (peer_exchange_initmsg.c:94)
==2271620==    by 0x143365: next_plan (io.c:59)
==2271620==    by 0x143810: do_plan (io.c:407)
==2271620==    by 0x1438AD: io_ready (io.c:417)
==2271620==    by 0x1451F1: io_loop (poll.c:445)
==2271620==    by 0x113193: main (connectd.c:1703)
==2271620==  Block was alloc'd at
==2271620==    at 0x483DFAF: realloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==2271620==    by 0x14E23A: tal_resize_ (tal.c:699)
==2271620==    by 0x14C7E0: do_vfmt (str.c:72)
==2271620==    by 0x14CA1B: tal_append_vfmt (str.c:102)
==2271620==    by 0x14CABE: tal_append_fmt (str.c:111)
==2271620==    by 0x112FC1: add_errors_to_error_list (connectd.c:680)
==2271620==    by 0x113070: destroy_io_conn (connectd.c:702)
==2271620==    by 0x144BF3: destroy_conn (poll.c:244)
==2271620==    by 0x144C19: destroy_conn_close_fd (poll.c:250)
==2271620==    by 0x14D6B6: notify (tal.c:240)
==2271620==    by 0x14D788: del_tree (tal.c:402)
==2271620==    by 0x14DD17: tal_free (tal.c:486)
==2271620==
lightning_connectd: FATAL SIGNAL 6 (version v0.9.3)
0x118e70 send_backtrace
        common/daemon.c:38
0x118f0a crashdump
        common/daemon.c:51
0x4a5c20f ???
        ???:0
0x4a5c18b ???
        ???:0
0x4a3b858 ???
        ???:0
0x14d245 call_error
        ccan/ccan/tal/tal.c:93
0x14e30b tal_resize_
        ccan/ccan/tal/tal.c:701
0x14c781 do_vfmt
        ccan/ccan/tal/str/str.c:60
0x14ca1b tal_append_vfmt
        ccan/ccan/tal/str/str.c:102
0x14cabe tal_append_fmt
        ccan/ccan/tal/str/str.c:111
0x112fc1 add_errors_to_error_list
        connectd/connectd.c:680
0x113070 destroy_io_conn
        connectd/connectd.c:702
0x144bf3 destroy_conn
        ccan/ccan/io/poll.c:244
0x144f55 cleanup_conn_without_close
        ccan/ccan/io/poll.c:264
0x1438fe io_close_taken_fd
        ccan/ccan/io/io.c:463
0x112eb6 peer_connected
        connectd/connectd.c:511
0x1133f9 peer_init_received
        connectd/peer_exchange_initmsg.c:94
0x143365 next_plan
        ccan/ccan/io/io.c:59
0x143810 do_plan
        ccan/ccan/io/io.c:407
0x1438ad io_ready
        ccan/ccan/io/io.c:417
0x1451f1 io_loop
        ccan/ccan/io/poll.c:445
0x113193 main
        connectd/connectd.c:1703
0x4a3d0b2 ???
        ???:0
0x10e59d ???
        ???:0
0xffffffffffffffff ???
        ???:0
lightning_connectd: FATAL SIGNAL (version v0.9.3)
0x118e70 send_backtrace
        common/daemon.c:38
0x11ebcc status_failed
        common/status.c:206
0x11ecde status_backtrace_exit
        common/subdaemon.c:25
0x118f10 crashdump
        common/daemon.c:54
0x4a5c20f ???
        ???:0
0x4a5c18b ???
        ???:0
0x4a3b858 ???
        ???:0
0x14d245 call_error
        ccan/ccan/tal/tal.c:93
0x14e30b tal_resize_
        ccan/ccan/tal/tal.c:701
0x14c781 do_vfmt
        ccan/ccan/tal/str/str.c:60
0x14ca1b tal_append_vfmt
        ccan/ccan/tal/str/str.c:102
0x14cabe tal_append_fmt
        ccan/ccan/tal/str/str.c:111
0x112fc1 add_errors_to_error_list
        connectd/connectd.c:680
0x113070 destroy_io_conn
        connectd/connectd.c:702
0x144bf3 destroy_conn
        ccan/ccan/io/poll.c:244
0x144f55 cleanup_conn_without_close
        ccan/ccan/io/poll.c:264
0x1438fe io_close_taken_fd
        ccan/ccan/io/io.c:463
0x112eb6 peer_connected
        connectd/connectd.c:511
0x1133f9 peer_init_received
        connectd/peer_exchange_initmsg.c:94
0x143365 next_plan
        ccan/ccan/io/io.c:59
0x143810 do_plan
        ccan/ccan/io/io.c:407
0x1438ad io_ready
        ccan/ccan/io/io.c:417
0x1451f1 io_loop
        ccan/ccan/io/poll.c:445
0x113193 main
        connectd/connectd.c:1703
0x4a3d0b2 ???
        ???:0
0x10e59d ???
        ???:0
0xffffffffffffffff ???
        ???:0

[rustyrussell]

Are you using Tor (or another socks proxy?). I'm having no luck reproducing this here...

[raucao]

No Tor on that one. After having tried migrating via the backup plugin, and that also failed, I have closed all channels and shut down the node for good now.

Thanks for trying!

[sangbida]

This issue is being closed as part of issue cleanup.

Reason: Very old issue (5.1 years old) no recent activity

If you believe this issue should remain open, please comment and we can reopen it.