Cans has source? I’ve been hacking on a per-instance community scraper (that pulls a list of communities from the community directory page) for !lemmy411!lemmy411@lemmy.ca, in an attempt to have a regularly published community list, but I’m a bit bogged.
I have links to the source for my crawler (written in nodejs with redis task queuing) on https://lemmyverse.net (GitHub link top right) 🤗
It’s a bit messy at the moment but has all the community crawling stuff.
Cool!
Cans has source? I’ve been hacking on a per-instance community scraper (that pulls a list of communities from the community directory page) for !lemmy411!lemmy411@lemmy.ca, in an attempt to have a regularly published community list, but I’m a bit bogged.
I have links to the source for my crawler (written in nodejs with redis task queuing) on https://lemmyverse.net (GitHub link top right) 🤗 It’s a bit messy at the moment but has all the community crawling stuff.