A little bit of DDOS

Monday, 14 January, Year 5 d.Tr. | Author: Mircea Popescu

Earlier today MPEx was hit by a moderate to respectable DDOS. We measured it peaking around 20-30 Gb/s, but on average it was probably under 10. Still, that's a lot.

One of the advantages of being well popular (other than being DDOS'ed, of course) is you find out about it immediately :

[12:30] {jurov} mircea_popescu, mpex.co is not even DNS resolving ?!
[12:30] {mircea_popescu} hm
[12:30] {mircea_popescu} looking into this.
[12:38] {mircea_popescu} looks like a little bit of ddos.

So after the initial moment of surprised confusion (wait, what ?!) the offending traffic was nulled and the eternal story of filtering began. The servers per se were not affected, in fact they were happily churning along at 0.01 loads and the like, muchly surprised that suddenly the world is quiet. A router upstream however was cooking eggs at about the same time.

The attack did cause a lot of trouble for a lot of people : those who became (through no fault of their own) unable to resolve mpex.co - the fix was both trivial to implement and obvious, except nobody on the outside would have had any reason to intuit it ; the ISP people stuck with a blown router and having to filter traffic for their other customers and so forth.

I suppose this is where you expect me to go into details of how I've made arrangments for much larger pipes, DDOS mitigation, protection and whatnot. I will disappoint. I'm not getting a larger pipe at this time, I already have a large one and it's just not economically sound to compete in that field and on these terms. Any two bit neckbeard can gather da jiggabits, it'd be much like trying to win a war by having more children.

I'm not getting DDOS protection, either. I have a much simpler solution : mpex.5gbfree.com ; mpex.host56.com ; that sort of stuff. They run a very simple php proxy for MPEx, something like

$url = 'http://xx.xx.xx.xx/';
if ($_GET["mpsic"]) $url.= "?mpsic=".$_GET["mpsic"];
if ($_POST["msg"]) {

$params = array ('http' => array ('method' => 'POST', 'content' => http_build_query (array ("msg" => $_POST["msg"]))));
$ctx = stream_context_create($params);
$fp = @fopen("http://xx.xx.xx.xx/", 'rb', false, $ctx);
$resp = @stream_get_contents($fp);
fclose($fp);
echo $resp;

}
else echo str_replace ( "http://mpex.co/","http://mpex.host56.com/" , file_get_contents ($url) );

?>

They're small and inexpensive, that little bit of passing back and forth poses no problems to their limited resources but they do hide the IPs of the back-up webservers, and this makes DDOS-ing MPEx impractical : if I have one webserver exposed to the Internet directly and a couple more exposed indirectly through a few dozen such little skirmishers the job of someone trying to take MPEx out suddenly becomes impossible. Sure, they can go after the directly visible server, possibly getting lucky again and causing it to become unavailable or at any rate very slow to respond for a few hours. Trade can nevertheless continue. They could go after all the free hosts and blow the lot off the internet, but it is probably cheaper for me to add more than it is to attack them.

All this, it is worth the mention, at no security cost whatever, what's passed back and forth is PGP encoded and thus safe. Now you tell me which other Bitcoin service can boast DDOS immunity through this fight fire with fire approach, or for that matter which service at all ? Pretty much every major anything was taken out at least once in its history, and the reason they're not out right now is simply that nobody feels inclined to do it, a point amply proven by anon already. Turns out there is a lot of value in designing around PGP. Whodda thunk it ?!

I will obviously be adding more, and I will obviously not publicize all I add. To quote from the very FAQ,

No government will ever be able to stop the Internet, in general. We're prepared to show this in the particular.

Turns out it's not really da gobbemint trying to cause trouble. Whodda thunk it ?!

But anyway, this coupla hours of spotty service has been, other than an excellent test for MPEx, a good opportunity to expose some growing lassitude in the coding practices of dependant systems. One bot, for instance, asked no less than 720 times for the same order to be cancelled, because it wasn't getting the one response it was expecting. To cut a long story short : dear everyone, MPEx has been serving all requests, there's no api keys, no throttling, no rate limiting, no anything. The idea is that you implement reasonable rate limiting yourself, so that I don't find myself forced to implement one-shoe-fits-all, can't possibly be reasonable rate limiting. Makes sense, neh ?

Category: MPEx
Comments feed : RSS 2.0. Leave your own comment below, or send a trackback.

5 Responses

  1. Does it mean that, for example pyMPEx will come bundled with fall back list of proxies to send requests to in case of mpex.co is unreachable? Or you will handle it automatically somehow?

    BTW, CoinBr was NOT related whatsoever to the rogue bot mentioned and itself was not affected outside of impossibility to place/cancel orders.

  2. Mircea Popescu`s avatar
    2
    Mircea Popescu 
    Monday, 14 January 2013

    Indeed not.

    The very nature of the beast is that I can't very well handle it automatically - if I could I wouldn't need them as it were. There's probably value in having a community maintained list, and it being kept together with pympex makes perfect sense, I'll chat up Azelphur about it.

  1. [...] way of illustration of this concept, the one and only time MPEx was DDOSed the response was swift and withering. Read that to understand why it isn't really feasible to DDOS MPEx anymore, and to marvel at the [...]

  2. [...] endless advantages of this particular correct design choice appear readily when, for instance, someone tries to DDOS MPEx. A Distributed Denial of Service attack is something nobody on the Internet is safe from. [...]

  3. [...] major services have been hit with DDoS attacks in the past week or two : MPEx (and once before in January but that didn't do much at all), blockchain.info as pretty good examples. Bitcointalk was also down [...]

Add your cents! »
    If this is your first comment, it will wait to be approved. This usually takes a few hours. Subsequent comments are not delayed.