docs/UX: two separate ways to silently lose funds — non-environment-aware wallet.dat backups, and non-deterministic key generation (both are solved problems in modern Bitcoin forks)

[bogdan984]

Following up on my earlier crash reports (uncaught DbException / STATUS_STACK_BUFFER_OVERRUN,
and the later Db::put crash on the #38 build). While debugging that, I found two separate,
serious ways an ordinary user can silently lose funds with this wallet — neither is really a
crash bug, they're design issues, and I think they're worth documenting (or fixing) prominently.

Issue 1: wallet.dat alone is not a safe backup

I backed up wallet.dat alone (just that one file, copied out of %AppData%\Bitflash\) before
upgrading versions. After upgrading and replacing wallet.dat with my backup, the wallet would
not load — this is what I reported earlier (DbException, both Db::open and Db::put depending on
the exact test).

I traced it to: %AppData%\Bitflash\ also contains a database/ subfolder (BDB transactional
environment — log files, region files, etc.) alongside wallet.dat, blk0001.dat,
blkindex.dat. Berkeley DB in transactional mode ties the wallet file to this environment
(LSNs recorded in the file's pages). Copying wallet.dat alone, without its matching
database/ folder, produces a file that a different (freshly created) environment can't
open/write to correctly — hence Db::open/Db::put: Invalid argument.

Confirmed experimentally: copying the entire %AppData%\Bitflash\ folder (wallet.dat +
database/ + blk*.dat together) into a fresh install fixed it immediately. Copying wallet.dat
alone reproduced the failure every single time, across multiple versions (v1.2.1, v1.2.3, and
the #38 build), which is why it initially looked like a version-to-version regression but
wasn't.

By the time I understood this, I had already cleared the folder that had the working
environment (in a sandboxing tool), leaving me with only the standalone wallet.dat — which,
per the above, isn't enough on its own to reopen the wallet. Recovery attempts since then
(file recovery tools, Sandboxie's own snapshot features) haven't turned up a usable copy, and
my disk is an SSD with TRIM, which likely means the original data is physically gone. This may
end up being a real, permanent loss of funds for me, caused entirely by backing up the
"obviously important" file and not knowing the rest of the folder mattered too.

Issue 2: mined coins vanish if you restore a wallet.dat backup made before mining them

Separately, and worse: I created a new wallet and immediately backed up the entire data
directory (the full folder, correctly this time — not just wallet.dat). I then mined several
blocks with that wallet. On a whim I switched to a different wallet, then a few minutes later
restored my original backup (the one saved before I'd mined those blocks). After restoring
it, the mined coins were gone — not visible in the wallet at all.

As far as I can tell, this is because the wallet generates receiving keys/addresses on demand
rather than deterministically from a single seed. My backup only contained the keys that
existed at backup time; the address(es) that received the mined block rewards were generated
after that backup, so restoring the earlier wallet.dat means the keys for those coins simply
aren't present in the file — even though the coins are presumably still sitting on-chain at
those addresses. This is the exact "keypool exhaustion" / non-deterministic wallet backup
problem that affected early Bitcoin Core (pre-BIP32/HD wallets), well before HD wallets
(deterministic key derivation from a single seed) became the standard fix industry-wide around
2013.

Question for the maintainers

I want to ask this directly and I hope it comes across as constructive rather than just
frustrated, because I am frustrated: why is a project released in 2026 reproducing data-loss
bugs that the Bitcoin ecosystem identified and fixed roughly a decade ago (non-deterministic
wallets requiring constant re-backup, an unhandled-exception crash path, a wallet file that
silently depends on an untracked second folder)? These aren't obscure edge cases — they're
exactly the kinds of bugs that cost real users real money in Bitcoin's own early years, which
is why HD wallets, seed phrases, and defensive exception handling became standard practice
industry-wide. Is there a reason this project intentionally started from the 0.1.0 codebase
without carrying forward those fixes, or is this just development priority / time constraint,
and HD wallets / safer backups are on the roadmap?

Suggestions, roughly in order of impact vs. effort

1. Immediate, cheap: add a line to the README and any in-app "backup your wallet" messaging
   stating explicitly that (a) a safe backup means the entire data directory, not just
   wallet.dat, and (b) any backup is a point-in-time snapshot — coins received/mined after
   that backup will not be recoverable from it, so backups need to be repeated regularly or
   after any new address is used.

2. Medium effort: add a "Backup Wallet" menu action that zips up the whole data directory
   to a location the user picks, instead of relying on users to know which files matter.

3. The real fix for issue 2: move to deterministic (HD/BIP32-style) key derivation from a
   single seed phrase, the way virtually every modern wallet does. This turns "backup" into
   "write down 12/24 words once," and makes both of the issues above structurally impossible —
   there's no environment-coupling problem and no stale-backup problem if the keys are all
   derivable from one seed instead of stored as a growing, mutable list in a database file.

4. For issue 1 specifically, short of full HD wallets: a "Dump Private Keys" / export
   function that writes out just the raw keys + labels in a plain, environment-independent
   format would let a user salvage access even from an isolated wallet.dat.

Not asking for anything specific here

I don't want to prescribe exactly which of these you implement — just flagging that the
current setup makes it very easy for a well-intentioned user (me, twice now) to think they've
backed up correctly and still end up unable to recover funds, with no warning anywhere that
either of these could happen.

Happy to help test any of the above if useful.

[Bitflash-sh]

Your diagnosis is correct in every particular, and there is something in here you should read before anything else.

Your wallet.dat is probably not lost

If you still have that standalone file, the keys are almost certainly still in it and reachable. What you hit is not corruption — it is a refusal. Berkeley DB stamps every page with a log sequence number belonging to the environment that wrote it, and carried into a new environment those numbers point at logs that do not exist, so BDB declines to write. Same bytes, same keys, no damage.

I reproduced your exact failure, then confirmed the file survives the repair intact. Key records before and after are byte-identical — defaultkey, key, name, setting, version, same digest — and the node then opens it with no exceptions.

#46 makes the node do this for you: a wallet.dat sitting in a directory with no database/ beside it is adopted on open and its sequence numbers reset. Point a fixed build at a directory containing only your wallet.dat and it should come up.

Do not overwrite that file while trying things. Copy it somewhere safe first and work on the copy. If it would help, say the word and I will put a build of the branch somewhere you can fetch it rather than making you compile it.

What was actually wrong

You worked out the environment coupling, which is the hard part. What sat underneath it:

_endthread() is pthread_exit() on POSIX, and glibc implements that by throwing abi::__forced_unwind through the stack. Catching that without rethrowing is a fatal error by contract. The thread wrapper ended in a bare catch(...), so every Linux shutdown ended:

StopNode()
UNKNOWN EXCEPTION  bitflash-node in ThreadSocketHandler()
FATAL: exception not rethrown          ← abort

An abort partway through shutdown, so nothing after StopNode() ran. And what should have run there is DBFlush(true), which calls lsn_reset — the exact operation that releases a wallet from its environment. It has been declared in this tree since 2009 and called from nowhere. Even had it been called, the abort stood in front of it.

So the wallet was never released from its directory, and the one file anybody would copy was the one file that could not travel. Your report is what made me look; I would not have found it from the symptom alone.

You were also right that #38 did not finish the job. I closed #34 on the strength of a load-time fix and your Db::put crash was on the write path, which that fix never touched. That one is mine.

Answering your question

You asked it directly and it deserves the same. Starting from 0.1.0 was deliberate — the point of the project is a faithful restart from that codebase rather than a modern fork with the history filed off. What that buys is a real 2009 Bitcoin; what it costs is 2009's wallet design, and you just paid part of that bill. That is a trade, and it was made without warning anyone, which is the part I would defend least.

The specific things you hit were not intentional, though. A missing BackupWallet and a shutdown that aborts are not heritage, they are damage — both were lost in this project's own refactoring, not inherited from Satoshi. 0.1.0 had a backup command; this tree has shipped without one and without saying so.

What #46 does and does not do

Does: shutdown completes, so new wallets end up portable by construction; already-stranded files are adopted on open; /backupwallet=FILE writes a copy that stands on its own, with your suggestion 1 printed on every run.

Does not: your issue 2. Keys are generated as they are needed rather than derived from a seed, so any backup remains a snapshot and coins paid to an address created after it are not in it. That is the deeper problem and it is a wallet format change — deterministic derivation, your suggestion 3. I am opening a separate issue for it rather than quietly folding it into a bug fix, because it deserves to be argued about in the open and it is not my call alone to make.

Your suggestion 4 — a plain-text key export — is the pragmatic middle and I think it is worth doing regardless of what happens with HD. It is the thing that makes a wallet salvageable no matter what state the database is in.

Thank you for writing this up as carefully as you did while being out of pocket for it. Please do check whether that file still opens.

[Bitflash-sh]

Shipped: v1.2.6. No need to build anything — binaries are on the release for Windows and Linux.

Try your stranded wallet.dat. Put it alone in an empty directory and point this version at it:

Bitflash.exe -datadir=C:\somewhere\empty

./bitflash-node-1.2.6-x86_64 -nogui -debug -datadir=/somewhere/empty

If it works you will see this in debug.log, and the wallet will simply open:

CDB() : adopted wallet.dat from another environment (log sequence numbers reset)

Copy the file first and work on the copy. I am fairly confident but you should not spend your only artifact on my confidence.

Once it opens, take a real backup straight away:

bitflash -backupwallet=/path/to/somewhere/safe.dat

That writes a copy that stands on its own — which is what you were trying to make in the first place, and what the software gave you no way to do.

---

What your report turned out to be worth, since you were owed a real answer and not just a fix:

The wallet was the symptom. Underneath it, _endthread() is pthread_exit() on POSIX and glibc implements that by throwing abi::__forced_unwind through the stack; catching it without rethrowing is fatal by contract, and the thread wrapper ended in a bare catch(...). Every Linux shutdown aborted partway through, so nothing after StopNode() ran — including DBFlush(true), the call that releases the wallet from its directory, which has been declared in this codebase since 2009 and invoked from nowhere. The node had never once been able to put the wallet down cleanly.

That was not going to be found from the crash. It was found because you sat down and worked out that database/ mattered, then wrote it up precisely enough to follow.

#47 is open for the other half — the snapshot problem — and I would genuinely like your view there, particularly on migration, since you are the one person here who has hit the failure rather than reasoned about it.

Please do report back either way. If it does not open I want to know that too.

[Bitflash-sh]

v1.2.7 adds the plain-text key export you suggested, which is the fourth item on your list:

bitflash -dumpwallet=keys.txt
bitflash -importwallet=keys.txt    # into any wallet, on any machine

One key per line with its address and label — no Berkeley DB, no environment, no version coupling. It exists precisely for the case where the wallet file itself will not open, which is where you were. Written owner-only on Linux and macOS; on Windows it inherits the folder's permissions, so pick the folder deliberately and delete the copy once it is somewhere safe.

Still hoping to hear whether the recovery in v1.2.6 got you back into that file. If it did not, I would like to know that too — the fix is based on reproducing your symptom exactly, but yours is the only real instance of it, and "it worked here" is not the same as "it worked for you".