Phuctor checkpoint

Because reasons, here's a snapshot of the interesting stuff in Phuctor today :

Submissions: 8`223`355 ; Known Moduli: 5`782`106 ; Moduli Waiting for Test: 1`921 ; Submissions with One or More Duplicate Modulus: 239`328 ; Moduli with Obvious Brain-Damage: 822`426<sup>i</sup>; Moduli Broken: 1`409.

And now for the details : phuctor_snapshot.html

Note : all the IPs listed as users come from the same 13-14 June 2016 extraction of ssh keys by Framedragger.

PS. If anyone's curious, the solution to the problem is

sed -r 's/((1?[0-9][0-9]?|2[0-4][0-9]|25[0-5])\.){3}(1?[0-9][0-9]?|2[0-4][0-9]|25[0-5])//2g'

-r is extended syntax, s/<pattern>//2g wipes out the pattern starting with the 2nd occurence, and the lengthy gnarl within paranthesises is how you match IPs.

PPS. Of no particular interest -- I managed to witness my first MySql crash while attempting to publish this articleⁱⁱ before settling on the more reasonable solution here displayed. I managed to hose the mysql server in the following manner : first I told it to update every entry in the article table with the 2.7 MB pile of html in question ; then I got bored while it was hosing my database and killed the process, which corrupted the table.

I ended up spending hours trying to repair it, which yielded an elephantine monster consisting of 15 GB worth of null bytes and a further 25 GB worth of verbatim repeats of the same phuctor snapshot. Lulzy, huh ?

Anyway, here's some things I learned from the experience :

• If using MySql, always use MyIsam, it's practically unsinkable. If it does somehow get corrupted (which is almost always because you killed the process while it was writing), do not panic.
• Check and make sure the partition MySql stores data on has enough space (often - it doesn't, which may make you panic).
• Run myisamchk --key_buffer_size=64M --myisam_sort_buffer_size=64M --read_buffer_size=1M --write_buffer_size=1M table_name.MYI ; expect about a minute per GB ; good idea to also make copies. If the index is divorced from the datafile, re-run the above with switch -r. This will fix it.
• This is also a good time to optimize the table, sort the indices etcetera.

In conclusion, MySql is an extremely rugged set-up which only the most boneheaded cases of PEBKAC can sink, and even then recovery is quite likely in skilled hands.

PPPS. Most coreutils handle long (10M+) null byte lines extremely poorly. Head, dd, wc etc are all included. Sed simply crashes. Etcetera. I suppose nobody could have predicted.

1. TEN percent! [↩]
2. Originally I intended to have the above-linked html file published as regular part of the article. This is not particularily a smart move, seeing how that's large (2.7 MB is above the customary 2MB limit built in all sorts of webserver interfaces and by virtue of unchallenged history into all sorts of mental expectations) and a very poor candidate for dynamicism. [↩]