FurAffinity Crawler by blueotter

I spent the afternoon writing a FurAffinity crawler/spider to point at my favorite artists and copy all their work to my local hard disk. Who knows if FA will be around in a year. On the chance it won't be, at least I have a local copy of ~200 artists I really dig.

If anyone else wants the source/binary for my little .NET program, let me know - happy to share.

Report

FurAffinity Crawler

blueotter

March 23, 2015 at 00:03:47 UTC

Journal Information

Views:: 323
Comments:: 8
Favorites:: 0
Rating:: General

Tags

Comments

Link

nikolaiswiss on March 23, 2015 at 01:23:32 UTC

Share :3
- Link
  
  blueotter on March 24, 2015 at 02:47:14 UTC
  
  https://www.weasyl.com/journal/80820/furaffinity-crawler-source-and-binary-instructions
Link

HelixVexium on March 23, 2015 at 03:54:40 UTC

I'd be interested!
- Link
  
  blueotter on March 24, 2015 at 02:47:18 UTC
  
  https://www.weasyl.com/journal/80820/furaffinity-crawler-source-and-binary-instructions
Link

Axlegear on March 23, 2015 at 07:28:02 UTC

Careful, Yak and Dragoneer have banned people in the past for using crawlers, on the grounds that 'it slows down the site' and 'there's no technical reason[to use a crawler]'

Go figure.
- Link
  
  blueotter on March 24, 2015 at 00:30:13 UTC
  
  No, it's a really good point. I'm going to add some random jitter to it to help make it indistinguishable from regular browser traffic and will post replies here tonight with the link.
  - Link
    
    Axlegear on March 24, 2015 at 02:42:35 UTC
    
    Cool, keep us updated, and thanks!
    - Link
      
      blueotter on March 24, 2015 at 02:47:21 UTC
      
      https://www.weasyl.com/journal/80820/furaffinity-crawler-source-and-binary-instructions