MPEx tech stuff

I see I've never discussed this, so let's do it.

If you'll pardon the crude illustration aboveⁱ MPEx works as a (webserver) - (trade engine) system.

This means that the physical machine which hosts the trading engine is not accessible directlyⁱⁱ from the Internet. In order to talk to the trade engine you have to talk to the webservers it delegates for this purpose.

This particular design has its advantages and its disadvantages. The main advantage is that since the trade engine does not connect to the Internet, it does not need a whole lot of software. Apache, for instance, is something that it doesn't need to have installed. This lack of software makes it a lot easier to secure, and a lot more difficult to mess with. In general I have found so far that the best results in computer security are achieved not by those who are most competent at securing a system, but by those who are backed by management most inclined to simplify their task. Your mileage may, obviously, vary.

The disadvantages are many and varied. For instance the webserver receiving a signed order from you has no way of knowing whether that order will be executed immediately or put on the book. This means that the most it may tell you is "I swear I've received your order", which is inconvenient for many people. It doesn't currently even know what id that order will receive in the trade engine, which is even more inconvenient for brokers that handle multiple customer accounts (but we're working on a way to significantly mitigate this, which will be the next major update barring exigent necessity).

In case you are curious, the reason this design is not really used by anyone else is the problem of securing content. HTTP is a stateless protocol, which practically for our interests is to say there's no native "logged in" state as different from a baseline "not logged in" state. This fundamental problem is being addressed by most everyone else with the bandaid of a sort of point-to-point encryption (which is what HTTPS is, entirely). The flaws of HTTPS are many and varied, including the fact that the "root authority", some self-appointed equivalent of the dns system, can be and has been hacked in the past, resulting in a complete gutting of your imagined "security". The mess is further compounded by security dongles (the Yubikey etc), but none really address the fundamental point : as far as computers are concerned, anything and everything stuffed into their 80 port is in the same state.ⁱⁱⁱ

Because everyone else's security (such as it is) depends on that entire abomination, a hole in the webserver equals total compromise. Because a hole in the server is a total compromise already, the extra overhead of having the trade engine separate is not justifiable. And so it goes.

Contrary to that, MPEx uses PGP signed orders. This has the notable disadvantage of forcing customers to use PGP^iv, but the even more notable advantage that MPEx doesn't need connections to be stateful. Nobody can "log into" MPEx, for the very simple reason that MPEx - much like the Internet itself - doesn't even know what "logged in" means. In fact it means nothing, in spite of a lot of pretense to the contrary.^v

The endless advantages of this particular correct design choice appear readily when, for instance, someone tries to DDOS MPEx. A Distributed Denial of Service attack is something nobody on the Internet is safe from. Absolutely any website, including google.com, facebook.com, paypal.com, anyone and everyone is on their knees when it comes to DDOS. In fact, the only (absolutely only!) reason any one of those sites is currently online and answering to clients is simply that nobody is bothering to DDOS them right now. If someone were bothering, they wouldn't be here, and for as long as someone will be bothering in the future, they won't be here. This is not a point of speculative thought, it's simple fact, as Anon's LOIC has recently proven to the clueless members of the general public.

The reason DDOS works is simple : since everyone is using point to point encryption to hack around the stateless nature of http, an endpoint has to be defined.^vi If an endpoint is defined, an endpoint can be flooded, and there's (by design) nothing anyone can do about that. End of story.

The reason DDOS doesn't work on MPEx is also simple : you don't have to send your order to any particular website, as long as MPEx eventually hears of it. Take the classical way phishing works : you intend to use paypal.com. Unbeknownst to you, evil phisher is feeding your browser pages from paupa1.com. You type in your password. paupa1.com sends it along to paypal.com, discovers it's valid, feeds you back the page and now evil phisher has your password, and your account, and all your jools.

On the other hand, imagine you intend to talk to MPEx. Unbeknownst to you, you're not talking to MPEx, but to ButtPlox. You naively paste your hard-earned PGP code into ButtPlox. Well... ButtPlox doesn't have MPEx' private key, and consequently has no idea what your order says. If ButtPlox passes this black box mystery along to MPEx, it will receive a reply, which is of course encrypted with your private key, which ButtPlox also doesn't have. If it passes it along to you then you decode it and that's that, transaction successful. If it doesn't pass it along to you then you know it's ButtPlox and just move on.

There's no more phishing, or in other words one simple design choice has eliminated - not "virtually", not "practically", not "in theory" but absolutely and definitively - the main problem financial services contend with these days.^vii Furthermore, because the very most a phisher can do to MPEx is the harmless office of a proxy, DDOS mitigation becomes really simple : just open a ton of proxies all over the web, on cheap / free hosting. What are the DDOSers going to do, take down the entire Interwebs ? It doesn't work that way.

This is what we mean when we say things like "Bitcoin will revolutionize the banking industry". It's not something along the lines of Amir "Retard" Taaki going "rabble rabble maffs rabble rabble down with the banks". It's something along the lines of, "here's this complete solution to the one problem you people don't know how to solve". From there on the future is pretty clear : either they move to the light side or fall by the wayside. Nobody can continue to offer flawed services once somebody opens up shop doing the Right Thing. That's revolution for you, and it's certainly enabled by Bitcoin : if it wasn't for Bitcoin I wouldn't have bothered to get involved, and if I wouldn't have bothered to get involved practical deployment of PGP for financial services would have been still waiting for whatever it was awaiting up to this point. It's 1990s technology after all.

Getting back to the many and varied disadvantages : MPEx webservers will have to keep a copy of the database. This is practically unavoidable (I think) because of two things : you'd like to be able to issue account statements for customers when they want, rather than when you get around to it^viii and you'd like to pipe trade data into other services (like twitter, assbot on #bitcoin-assets etc).

Since webservers are keeping copies of the database, you're in for a world of hurt : you have to keep everything synced. This isn't exactly as trivial as it may sound, which isn't of course to say it can not be done. Just, in doing it, you'll always have to sacrifice something. Up until last night, the way this worked in practice was that desyncing locked trading updates completely. People would put orders in, trade engine would execute, webservers would not report anything. Panic. In spite of me having multiple times tried to both explain how it works and so forth, there's something very viscerally unsettling about talking into a silent void.

More problematicaly, the old mechanisms weren't really coping too well with the exponential increase in trade^ix. The lock-outs were sporadic, maybe weekly two minute events back in Autumn of last year. They became almost daily occurences this year, and finally yesterday there was a massive hour-and-a-half mess, panic and disaster.

No orders were actually affected, no data was lost, no grevious stuff like that. Just, everyone was driven up the wall by the uncanny silence. And then when news of the actual positions finally did make it to the market some brokers had the pleasant experience of encountering an "Oh, you've sent us these 500 orders in the interval ? Well this is how it all ended up. O, you'd like to know which customers did what to whom ? So figure it out, all trades are ordered in the order we received them. Oh, you actually have some cases with same-second orders and you thread to prevent your customers from having to wait on one another ? Awww!"

Fortunately everything was unwound correctly, due to a lot of hard work (at least as far as I know). However this can't continue as a going concern, and we've made significant changes which should, in principle, make trade reporting lock-outs of that nature a thing of the past. At least until trade increases a few more degrees of magnitude we should be okay.

Of course the new arrangements mean sometimes order reporting in the public venues (assbot, mostly) has an echo.^x Working to resolve this as well.

That's about it for now, but in case all this hasn't bored you completely senseless, there may be further installments in the future.

1. Our many and talented graphical artists are hard at work refining the abyssal beauty that is MPEx.co, and I dare not interrupt them. Why just two days ago they switched the colors of the little dots on the little graphs from full red/full green (FFxxxx) to half red/half green (80xxxx). It takes skill. [↩]
2. And this is the cause for such ominous quotes as

   The machine handling encryption and decryption on the MPEx side never connects to the Internet. There is no cable. As such, it does not even do updates of keys. Should your key become expired or revoked you will have to create a new account. We will not transfer your assets for you if it cannot be established beyond the shadow of a doubt that they are in fact yours. Please handle key expiration and revocation responsibly.

   in the FAQ.
   [↩]

3. The concept of "state" is important for this discussion, and if you're not familiar with the notion of a Turing machine's internal state you're not following. [↩]
4. Which is, I will point out, really a sum of very many notable advantages. [↩]
5. And I will personally count as "web 2.0" that point where people finally give up the pretense of "logged in". Certanly "rounded corners" does not qualify anything to a 2.0 status. [↩]
6. That's what the noise around "phishing" is all about : phishing is when someone tries to replace your intended endpoint with something else of their own. [↩]
7. A 2007 Gartner survey places the total loss for that year over 3 bn USD. Data has since ceased to be available, pretty much for the simple reason that it's been growing, and nobody inside the financial sector wants to share with the public the grim reality of how phishing generates more revenue than the GDP of most countries these days. [↩]
8. In theory at least nothing would stop MPEx from working as a pre-Internet stock exchange, be open every day for 4 hours, then close down, do the bookkeeping, send everyone their account statements, open again the next day. [↩]
9. As of the time of this writing we've processed 5`659`416 trade orders - this figure includes buys and sells, but excludes anything else - and received close to a billion signed items - this figure includes everything, STATs, CANCELs, the works. MPEx is doing Terrabytes' worth of begin pgp signed message end pgp signed message. It's hard work, but someone's gotta do it. [↩]
10. The MPEx RSS feed does not, for the record. [↩]