The ideal Bitcoin wallet

I'm writing this down mostly to be used as a discussion basis / scratchpad by the tech minds in TMSR. It is publicly accessible for historical reasons, which doesn't automatically mean you have something to contribute to the discussion.ⁱ

A. Separation of cache and datastore functions. While it is true that without caching relevant transactions "simple"ⁱⁱ requirements such as "tell me how much I have" become O(N) (per address!), this is nevertheless no excuse to keep the two in the same file (and see below).

B. Textual representation. With some general reference to the broader topic of literacy (as opposed to symbology) amply discussed in the logsⁱⁱⁱ, there is no acceptable reason for the wallet data to be stored in any format that is not readily comprehensible, and editable. This means ASCII^iv, strictly.

C. Proper password space. Currently wallet passphrases are both weak and remarkable. The extant wallet encryption scheme^v has certain supposed strengths^vi and various weaknesses, both known^vii and unknown. The whole thing should be reviewed in all seriousness ; the password issue must be fixed in any case. A stand-alone wallet encrypter/decrypter/verifier is required so that people can check wallets with a tool that doesn't have the capability of emptying them at the same time.

D. Better support code and integration generally. The accounting functions as provided by Bitcoin code are a notorious laughingstock the shame therewhich will burn through the generations undulled. The current input selection process is idiotic to say the least. Absent a proper computed solution, the correct approach is the direct : inputs are selected in the order they are found in the wallet (which the user can modify to his heart's content, as per B above) up to the sum of the required outputs. Various other items^viii.

It's about high time already, if the sorry state of PRB with its inept "we replaced BDB with LevelDB only to a) create bugs, unintentional forks and generally an unmaintainable mess ; and b) to still keep the BDB code for the wallet anyway" wasn't enough to make that point.

1. Historical means what people will say long after you - our anonymous and therefore by then forgotten contemporary - will have finally disappeared without remainder. [↩]
2. There could, and obviously there should, for which reason there won't be written entire books on the meaning of "simple" in extended worlds.

   What is a "simple" expectation such as "we can resolve systems of equations" supposed mean in a properly multivariate world ? The "lesswrong" crowd of idiots may well proceed in their John Smith-esque manner, much like the country bumpkin harbors the "simple" expectation that he may slay dragons and figure out how to use the subway on the basis of his prior experience back home with trapping rabbits and finding his way to the outhouse. You however have no such excuse, not here at any rate.

   Do you expect "simple" rules to hold forever, as some sort of token of a wholly imagined "deep connection" between you and "the universe" ? Are you an idiot ? [↩]

3. 22nd of July 2016 ; 24th of February 2016 ; 3rd of February 2016 ; 11th of January 2016 ; 29th of October 2015. You are expected to read these. [↩]
4. And if it were stored as such, A wouldn't be much of a problem! Such is the power of correct design - that it protects you from "problems" which can't even exist in your space. Such is the curse of idiotic feelings - that they create problems entirely out of thin air.

   Stop being women. It's unseemly, shameful and wicked in the eyes of the lordship. [↩]

5. Create master key, encrypt it via AES-256-CBC on the basis of a key derived from a sha-2 (a version known-weak re preimage resistance for 57 rounds or fewer) and a number of rounds "derived from the speed of the machine" - often not over 60 for this reason! The individual privkeys are then encrypted to this master key through a dubious process. Various metadata is not itself encrypted. The code itself is not lengthy :

   #include "crypter.h"
   #include "script.h"
   #include <string>
   #include <vector>
   #include <boost/foreach.hpp>
   #include <openssl/aes.h>
   #include <openssl/evp.h>
   bool CCrypter::SetKeyFromPassphrase(const SecureString& strKeyData, const std::vector<unsigned char>& chSalt, const unsigned int nRounds, const unsigned int nDerivationMethod)
   {
   if (nRounds < 1 || chSalt.size() != WALLET_CRYPTO_SALT_SIZE)
   return false;
   int i = 0;
   if (nDerivationMethod == 0)
   i = EVP_BytesToKey(EVP_aes_256_cbc(), EVP_sha512(), &chSalt[0],
   (unsigned char *)&strKeyData[0], strKeyData.size(), nRounds, chKey, chIV);
   if (i != (int)WALLET_CRYPTO_KEY_SIZE)
   {
   OPENSSL_cleanse(chKey, sizeof(chKey) );
   OPENSSL_cleanse(chIV, sizeof(chIV) );
   return false;
   }
   fKeySet = true;
   return true;
   }
   bool CCrypter::SetKey(const CKeyingMaterial& chNewKey, const std::vector<unsigned char>& chNewIV)
   {
   if (chNewKey.size() != WALLET_CRYPTO_KEY_SIZE || chNewIV.size() != WALLET_CRYPTO_KEY_SIZE)
   return false;
   memcpy(&chKey[0], &chNewKey[0], sizeof chKey);
   memcpy(&chIV[0], &chNewIV[0], sizeof chIV);
   fKeySet = true;
   return true;
   }
   bool CCrypter::Encrypt(const CKeyingMaterial& vchPlaintext, std::vector<unsigned char> &vchCiphertext)
   {
   if (!fKeySet)
   return false;
   // max ciphertext len for a n bytes of plaintext is
   // n + AES_BLOCK_SIZE - 1 bytes
   int nLen = vchPlaintext.size();
   int nCLen = nLen + AES_BLOCK_SIZE, nFLen = 0;
   vchCiphertext = std::vector<unsigned char> (nCLen);
   EVP_CIPHER_CTX ctx;
   bool fOk = true;
   EVP_CIPHER_CTX_init(&ctx);
   if (fOk) fOk = EVP_EncryptInit_ex(&ctx, EVP_aes_256_cbc(), NULL, chKey, chIV);
   if (fOk) fOk = EVP_EncryptUpdate(&ctx, &vchCiphertext[0], &nCLen, &vchPlaintext[0], nLen);
   if (fOk) fOk = EVP_EncryptFinal_ex(&ctx, (&vchCiphertext[0])+nCLen, &nFLen);
   EVP_CIPHER_CTX_cleanup(&ctx);
   if (!fOk) return false;
   vchCiphertext.resize(nCLen + nFLen);
   return true;
   }
   bool CCrypter::Decrypt(const std::vector<unsigned char>& vchCiphertext, CKeyingMaterial& vchPlaintext)
   {
   if (!fKeySet)
   return false;
   // plaintext will always be equal to or lesser than length of ciphertext
   int nLen = vchCiphertext.size();
   int nPLen = nLen, nFLen = 0;
   vchPlaintext = CKeyingMaterial(nPLen);
   EVP_CIPHER_CTX ctx;
   bool fOk = true;
   EVP_CIPHER_CTX_init(&ctx);
   if (fOk) fOk = EVP_DecryptInit_ex(&ctx, EVP_aes_256_cbc(), NULL, chKey, chIV);
   if (fOk) fOk = EVP_DecryptUpdate(&ctx, &vchPlaintext[0], &nPLen, &vchCiphertext[0], nLen);
   if (fOk) fOk = EVP_DecryptFinal_ex(&ctx, (&vchPlaintext[0])+nPLen, &nFLen);
   EVP_CIPHER_CTX_cleanup(&ctx);
   if (!fOk) return false;
   vchPlaintext.resize(nPLen + nFLen);
   return true;
   }
   bool EncryptSecret(const CKeyingMaterial& vMasterKey, const CKeyingMaterial &vchPlaintext, const uint256& nIV, std::vector<unsigned char> &vchCiphertext)
   {
   CCrypter cKeyCrypter;
   std::vector<unsigned char> chIV(WALLET_CRYPTO_KEY_SIZE);
   memcpy(&chIV[0], &nIV, WALLET_CRYPTO_KEY_SIZE);
   if(!cKeyCrypter.SetKey(vMasterKey, chIV) )
   return false;
   return cKeyCrypter.Encrypt(*( (const CKeyingMaterial*)&vchPlaintext), vchCiphertext);
   }
   bool DecryptSecret(const CKeyingMaterial& vMasterKey, const std::vector<unsigned char>& vchCiphertext, const uint256& nIV, CKeyingMaterial& vchPlaintext)
   {
   CCrypter cKeyCrypter;
   std::vector<unsigned char> chIV(WALLET_CRYPTO_KEY_SIZE);
   memcpy(&chIV[0], &nIV, WALLET_CRYPTO_KEY_SIZE);
   if(!cKeyCrypter.SetKey(vMasterKey, chIV) )
   return false;
   return cKeyCrypter.Decrypt(vchCiphertext, *( (CKeyingMaterial*)&vchPlaintext) );
   }
   bool CCryptoKeyStore::SetCrypted()
   {
   LOCK(cs_KeyStore);
   if (fUseCrypto)
   return true;
   if (!mapKeys.empty() )
   return false;
   fUseCrypto = true;
   return true;
   }
   bool CCryptoKeyStore::Lock()
   {
   if (!SetCrypted() )
   return false;
   {
   LOCK(cs_KeyStore);
   vMasterKey.clear();
   }
   NotifyStatusChanged(this);
   return true;
   }
   bool CCryptoKeyStore::Unlock(const CKeyingMaterial& vMasterKeyIn)
   {
   {
   LOCK(cs_KeyStore);
   if (!SetCrypted() )
   return false;
   CryptedKeyMap::const_iterator mi = mapCryptedKeys.begin();
   for (; mi != mapCryptedKeys.end(); ++mi)
   {
   const CPubKey &vchPubKey = (*mi).second.first;
   const std::vector<unsigned char> &vchCryptedSecret = (*mi).second.second;
   CKeyingMaterial vchSecret;
   if(!DecryptSecret(vMasterKeyIn, vchCryptedSecret, vchPubKey.GetHash(), vchSecret) )
   return false;
   if (vchSecret.size() != 32)
   return false;
   CKey key;
   key.Set(vchSecret.begin(), vchSecret.end(), vchPubKey.IsCompressed() );
   if (key.GetPubKey() == vchPubKey)
   break;
   return false;
   }
   vMasterKey = vMasterKeyIn;
   }
   NotifyStatusChanged(this);
   return true;
   }
   bool CCryptoKeyStore::AddKeyPubKey(const CKey& key, const CPubKey &pubkey)
   {
   {
   LOCK(cs_KeyStore);
   if (!IsCrypted() )
   return CBasicKeyStore::AddKeyPubKey(key, pubkey);
   if (IsLocked() )
   return false;
   std::vector<unsigned char> vchCryptedSecret;
   CKeyingMaterial vchSecret(key.begin(), key.end() );
   if (!EncryptSecret(vMasterKey, vchSecret, pubkey.GetHash(), vchCryptedSecret) )
   return false;
   if (!AddCryptedKey(pubkey, vchCryptedSecret) )
   return false;
   }
   return true;
   }
   bool CCryptoKeyStore::AddCryptedKey(const CPubKey &vchPubKey, const std::vector<unsigned char> &vchCryptedSecret)
   {
   {
   LOCK(cs_KeyStore);
   if (!SetCrypted() )
   return false;
   mapCryptedKeys[vchPubKey.GetID()] = make_pair(vchPubKey, vchCryptedSecret);
   }
   return true;
   }
   bool CCryptoKeyStore::GetKey(const CKeyID &address, CKey& keyOut) const
   {
   {
   LOCK(cs_KeyStore);
   if (!IsCrypted() )
   return CBasicKeyStore::GetKey(address, keyOut);
   CryptedKeyMap::const_iterator mi = mapCryptedKeys.find(address);
   if (mi != mapCryptedKeys.end() )
   {
   const CPubKey &vchPubKey = (*mi).second.first;
   const std::vector<unsigned char> &vchCryptedSecret = (*mi).second.second;
   CKeyingMaterial vchSecret;
   if (!DecryptSecret(vMasterKey, vchCryptedSecret, vchPubKey.GetHash(), vchSecret) )
   return false;
   if (vchSecret.size() != 32)
   return false;
   keyOut.Set(vchSecret.begin(), vchSecret.end(), vchPubKey.IsCompressed() );
   return true;
   }
   }
   return false;
   }
   bool CCryptoKeyStore::GetPubKey(const CKeyID &address, CPubKey& vchPubKeyOut) const
   {
   {
   LOCK(cs_KeyStore);
   if (!IsCrypted() )
   return CKeyStore::GetPubKey(address, vchPubKeyOut);
   CryptedKeyMap::const_iterator mi = mapCryptedKeys.find(address);
   if (mi != mapCryptedKeys.end() )
   {
   vchPubKeyOut = (*mi).second.first;
   return true;
   }
   }
   return false;
   }
   bool CCryptoKeyStore::EncryptKeys(CKeyingMaterial& vMasterKeyIn)
   {
   {
   LOCK(cs_KeyStore);
   if (!mapCryptedKeys.empty() || IsCrypted() )
   return false;
   fUseCrypto = true;
   BOOST_FOREACH(KeyMap::value_type& mKey, mapKeys)
   {
   const CKey &key = mKey.second;
   CPubKey vchPubKey = key.GetPubKey();
   CKeyingMaterial vchSecret(key.begin(), key.end() );
   std::vector<unsigned char> vchCryptedSecret;
   if (!EncryptSecret(vMasterKeyIn, vchSecret, vchPubKey.GetHash(), vchCryptedSecret) )
   return false;
   if (!AddCryptedKey(vchPubKey, vchCryptedSecret) )
   return false;
   }
   mapKeys.clear();
   }
   return true;
   }

   Many eyes, right ? [↩]

6. There is no way currently known to bruteforce an encrypted wallet.dat file. [↩]
7. Metadata not encrypted ; rounds may be insufficient ; individual private key to master key encryption may be weak ; AES is suspicious by its NIST-nature ; passphrases are weaker than their size naively promises, and readily recognized in a string dump. [↩]
8. Modularity for instance might be a good idea, there's no reason for a third party to know what exact encryption scheme any end user employs for his wallet. [↩]