Fetlife, the meat market

Friday, 13 February, Year 7 d.Tr. | Author: Mircea Popescu

Fetlife is a facebook clone on a niche. This rarely succeeds, but in this case it did - a decade ago collarme ruled the BSDM seasi, meanwhile Fetlife apparently took over.

This, I will propose, is mostly "luck", in the sense that it is based on anything and everything but "coding skills". Just like Spolsky's success (stackexchange). Such projects need a great strategist (and I do mean great, I mean better than what fiat sovereigns can affordii) and connectionsiii, that's the story of success, being the complete story of economics. Technological questions are entirely secondary considerations.iv

Its success brought out the whiny idiots en force, of course. Here's a quotev :

In my usual style, I gave a prepared talk and presented an accompanying slideshow. My talk was called “FetLife Considered Harmful: The Risks of Sex Ghettoization,” and I discussed what I see as a deeply dangerous, insular, growing monoculture within sexuality communities, epitomized by FetLife.com. This monoculture whitewashes the effects of privilege hierarchies while simultaneously reifying them in

That nonsense asidevi, Fetlife does try something or the other in the general vein of "being nice". For instance, it doesn't have a search function implementedvii because "it doesn't want to turn into a meat market". Which is silly, seeing how the first driver for people getting together in the first place, and therefore the foremost underlier of all society, is exactly trade. Not to mention that the prototype of all trade is the trade of women for that purpose. Whether done by the women themselves or whatever way it's organised, that is still the most important function of the marketplaceviii, which is the most important underpining of society. So... tough.

Let's turn fetlife into a meat market together.

for i in {1..3000000}; do curl --cookie "FL=00050ee2-a893-1536-9272-047f87682456;_fl_sessionid=9c69a3c9bb86f4b1f6ff74064e788824" https://fetlife.com/users/$i | egrep -A 64 "[1-2][0-9]F " | sed 's/<[^>]*>//g' | tr -d "\n" | tr "\t" " " | tr -s " " >> fetlife.txt && printf "======$i\n" >> fetlife.txt; done

What's this then ? Well, first of all, Fetlife never heard of salted hashing (like other experts) therefore their userbase is trivially enumerated at the uri levelix. So that's what the for is doing, iterating over their database.x

Then it calls curl, which passes along the relevant cookiesxi and queries pages.xii They're then put through a match to throw out everything but the good part of the profiles made by women under the age of 30xiii. The results are then further processed (removing spurious line feeds, tabs and whitespace) and dumped into a file. From whence you can peruse it at your leisure.

If you run this on a home connection you should go through about one page a second (per instance) so the whole shebang shouldn't really take you a whole month. If you fire up a server cluster it can probably be done in an afternoon.

So there you have it, you've now turned Fetlife into a meat market. What now ?

PS. If you're on Windows, get off, seriously. Or at least install cygwin. No, firefox plugins are not an equivalent solution for this problem, God love you.

  1. It imploded recently, yielding some lulzy drama. Apparently it was run by a coufple which split up, the chick ran away with the domain, put up a lengthy if inconsistent sob story about how she had been mistreated, the guy ran off with the db/site, recreated it under a new name. []
  2. This is possible because of that old "how much does your enemy have to pay you" dilemma.

    Specifically, inasmuch as fiat sovereigns are by definition the natural enemies of great strategists (and great people generally, and thinking people even more generally), anyone that's not a fiat sovereign starts with a major head start.

    This, obviously, has not been known before. But Bitcoin came around and disrupted the shit out of it. []

  3. Ie, membership of the relevant WoT. Obviously most niches are still run on some sort of feudal or fiat remnant, so you will rarely encounter a neat and clean implementation of a WoT that'll readily be recognizable on formal grounds. Nevertheless, in some manner and to some (likely pitiful) degree, that's what it'll be. []
  4. Think : the world existed and functioned before any arbitrarily chosen technological advance. Women rode the cock, people were happy, soups were had and bottles broken. If your idea of success is anything but "at my terminal surrounded by fetid odors" then this should be sufficient proof that technology is a second rate consideration. []
  5. I'm not going to link to him, because he didn't publish my comment. It read "Eh, just embrace it. This derpy “feminism” thing is dead and buried, a minor niche of less interest than foreign market shut-ins." which is no excuse not to publish it.

    Instead, I'm going to share a pic, so he gets to feel violated even moar. Here :

    The Derpful Hipster

    The Derpful Hipster

    Nice manboobs, Meitar! []

  6. By the way, did you know there's a "feminist antiporn" movement thingee ? O yeah, they're totally happening. []
  7. I suspect this is because the codebase sucks and the stack underlying the site is so rotten they simply couldn't economically run a search, but hey, who am I and what do I know about such matters, right ? []
  8. As a complete tangent : Dreptul la constiinta

    De alt exemplu, in Cairo, care-i in Egipt, care nu-i neaparat perceput ca un varf de lance in ce priveste sofisticarea sau inteligenta organizarii societatii toate magazinele, inclusiv farmaciile sunt pline de ciurde de fete de maritat, 16-20 de ani, pentru ca ei asa fac pe post de discoteci, feisbook scl : trimit fetele sa stea in magazin. Rationamentul nu-i neaparat defect, dat fiind ca tipii care cumpara chestii au, logic vorbind, bani, si daca tot au bani poate doresc sa-si cumpere si-o vacuta bipeda si frumos bronzata. Nu ?

    which comes to

    As another example, in Cairo, which is in Egypt, which is not necessarily perceived as the bleeding edge of sophistication or the organisation of society, all the shops, pharmacies included, are packed full of gaggles of marriageable girlies, 16-20 years of age. Because that's what they do instead of Discos, facebook and so on : they send the girlies to sit in the store. The reasoning's not necessarily broken, seeing how people who buy things necessarily have money, and since they have money perhaps they also wish to buy a bipedal cowsy with a nice tan ? No ?


  9. This, other than horrible coding, is also very bad for business. []
  10. They claim "over 3mn users", which is why the upper bound. Feel free to tinker. []
  11. You obtain these by opening a session in a browser and authenticating. Fetlife doesn't allow you to see any content unless you're logged in, which is great because it keeps the search engines out, and something they should be commended for. It also has the dubious effect of creating an illusion of security among the more herbivore of its users, but I suspect that's unavoidable. []
  12. You might pass along a benign looking user agent, with -A. I didn't bother deliberately, and to no ill effects so far. []
  13. Which in my experience are the more educable representatives of the gender of interest.

    Much older than that and you run into a lot of "you don't teach old mares new tricks" plus all the attendant baggage (you might not be interested in your women having other people's children, for instance). Nevertheless, feel free to alter the "[1-2][0-9]F " part to suit your tastes. Notice that the final space has to be there. []

Category: Meta psihoza
Comments feed : RSS 2.0. Leave your own comment below, or send a trackback.

14 Responses

  1. What is salted hashing?

  2. Mircea Popescu`s avatar
    Mircea Popescu 
    Saturday, 14 February 2015

    Suppose you actually have 3 mn users. What you are currently doing is, you're referencing them by a count, /user/1 to /user/3000000.

    The correct way to do this is : take a user variable (say, their name, "Derpy McHerperson"), add a salt to it (this will be fixed and should be long and random, say "jUlXk2RvBB695XfkyG/7/DOX5mdkwJ6VD26f+iZ"). Now, hash them together :

    sha512sum Derpy McHerpersonjUlXk2RvBB695XfkyG/7/DOX5mdkwJ6VD26f+iZ
    9d436eb782767a186c4604ab6886bfa69664c14c81ee1e6009fbecb10849a1 -

    You can now reference the user as /user/3b9ff5d44b403df3bd9efb396

    If such a long url is inconvenient for whatever reason (say you wish for users to be able to jot down another user's profile link even if they don't have a computer available - how this works is anyone's guess, but maybe they have internet connected paper) you still have to remember that if your url is made out of numbers your space is merely ten digits, whereas if you use low and uppercase plus digits your space is 64 "digits". So 5 characters which may be [A-Za-z0-9] are roughly equivalent to 9 characters which may be [0-9] (by the following formula : 645 = 26 * 5 ~= 10 9 - because 1024 is 210 and also is ~= 1000 so then 210 is ~= 103).

    So you base64 your string, and take say the fifth to tenth characters :

    $ base64 3b9ff5d44b403df3bd9efb3967644facf50e


    And so Derpy McHerperson becomes /user/mY1ZD

    Why go to all this trouble ? Simply because a 3rd party (in the article above, me) is not now able to go through your entire database and enumerate all your users : they're not ordered but distributed over the space, and their relative density is low - if you have 3mn users, and the 5 character space can keep about one billion users, then their density is about 0.3% which means that if I try a random string I have about one chance in 333 to hit on an actual user. This means that I will have to make one billion requests rather than 3 million requests to enumerate your database, or in other words you've put a 333x factor on the effort I need. And if you use six instead of five characters, that's a further 64x, taking it to over 20k tries per result.

    This is your first step into the wonderful world of cryptography. Make more, because survival in the future depends strictlier on familiarity with these topics than on paying your taxes.

  3. lol

  4. Seems like right now FetLife auto-bans user accounts for making too many requests to the server in a given period of time. Is this something they implemented after your scrape? Or did you get around it somehow? Just curious

  5. Mircea Popescu`s avatar
    Mircea Popescu 
    Monday, 4 May 2015

    I believe it must be novel, yes.

  6. Cant you just use the username ?

  7. Mircea Popescu`s avatar
    Mircea Popescu 
    Thursday, 7 July 2016

    I'm sure you can.

  1. [...] if I want to search by different criteria ? Just run your own damned bash script, you can set it up any way you [...]

  2. [...] Baku What is salted hashing? [...]

  3. [...] ———For the scholarly inclined, see Fetlife, the meat market, Sooo... FetLife is butthurt and On how I ended up suing Phoenix NAP, LLC ; on how Ira R. Cadwell's [...]

  4. [...] Of course, there’s way more interesting stuff we can do than a simple database crawl like this. For starters, what if we combined this with another, different data set? By cross-referencing the FetLife Creeplist with the database of rape accusations made against FetLife members compiled in the Predator Alert Tool for FetLife, we can begin to ask and answer questions like, “Which American city likely has the highest rate of sadomasochistic rape happening inside the so-called ‘safe, sane, and consensual’ BDSM Scene?” And then why stop there? Using other tools I wrote like FetLife Maltego, we can automate the process of cross-referencing FetLife user profiles with Facebook data. But don’t take my word for it. Here’s how “Why the FetLife Meatlist is Just the Tip of the Iceberg” puts it, a posting in which I am quite flattered to see my works discussed at such length: In the past couple of days folks on FetLife have been getting (justifiably) upset about the now infamous misogynistic cluster fuck FetLife Meatlist. [...]

  5. [...] Sam Seems like right now FetLife auto-bans user accounts for making too many requests... [...]

  6. [...] bankrupt concern : skimping on all the things one can't afford to skimp on. [↩]Recall some other UGC WWW item that also had 3mn users ? [↩]Literotica, like all vBulletin systems, allows users to opt out of the PM system. But it [...]

  7. [...] made conserves any degree of relevancy. No, there isn't going to be a tindr of the future. There's barely any today. No, Steve Jobs wasn't important universally. He might have been important to you, which is fine, [...]

Add your cents! »
    If this is your first comment, it will wait to be approved. This usually takes a few hours. Subsequent comments are not delayed.