https://github.com/lflare/lemmy-subscriber-bot
Your instance only has 45 communities. Those are rookie numbers. You got to pump them up :D
https://github.com/lflare/lemmy-subscriber-bot
Your instance only has 45 communities. Those are rookie numbers. You got to pump them up :D
@jgrim@discuss.online
Do you have any metrics to share regarding resource consumption? It does feel like you over planned for it, and ended up paying for much more resources than you’d be using.
It’s actually a tool I wrote that makes use of a bot account on your own instance to add and/or subscribe to communities throughout the fediverse. I defined “best” communities as simply communities above a certain level of activity threshold (omitting ghost/unpopular communities). It doesn’t subscribe all users on your instance to the communities, but it does make such that your All feed would be vastly more populated with content.
Very useful. I can federate with instances that don’t federate with each other. I can federate with both lemmy.ml and lemmygrad.ml, even if I don’t necessarily have to agree with either of their views.
Unfortunately, you will soon realize that due to bad design by the Lemmy developers, that the pictures pulled are actually on the smaller side, and a particular activity
table in your database will be a ever-growing tumor on your disk space.
First rule of fight club, you don’t talk about fight club. Nothing wrong with having an instance that federates with everyone if you want to have control over your own freedom of speech, but you really don’t want to be inviting others to do the same and test just how free your speech can go.
I personally also run an instance that federates with everyone, but I sure as hell don’t advertise it.
Unfortunately, depending on jurisdiction, remediation may range from just having a take down notice form, to having the cops at your door, it’s definitely something to weigh out before starting your own instance.
Hey there! Welcome to the joy of running your own instance, I built a tool to automatically discover and add communities to your local instance :)
https://github.com/lflare/lemmy-subscriber-bot
For support requests, I’ve created !lsbsupport@lemmy.world as well.
I’m not entirely sure whether deleting a user would remove the subscriptions either, but in my case, I added a reset functionality to my tool.
2023-07-11 11:02:27.636 | ERROR | main:retrieve_jwt:87 - {“error”:“couldnt_find_that_username_or_email”} Traceback (most recent call last): File “//bot.py”, line 248, in <module> main() File “//bot.py”, line 244, in main bot.start() File “//bot.py”, line 49, in start self.retrieve_jwt() File “//bot.py”, line 88, in retrieve_jwt raise e File “//bot.py”, line 84, in retrieve_jwt self.jwt = r.json()[“jwt”]
Yup, you will have to create a user first :)
For micro/personal server runners, I built a tool to automatically discover and add communities to your local instance :)
https://github.com/lflare/lemmy-subscriber-bot
EDIT: For support requests, I’ve created https://lemmy.world/c/lsbsupport as well.
It looks correct, are there any logs you can provide?
docker logs lemmy-subscriber-bot
EDIT: For support requests, I’ve created https://lemmy.world/c/lsbsupport as well.
For micro/personal server runners, I built a tool to automatically discover and add communities to your local instance :)
https://github.com/lflare/lemmy-subscriber-bot
Please let me know if it helps with overloaded or unavailable instances :)
EDIT: For support requests, I’ve created https://lemmy.world/c/lsbsupport as well.
For micro/personal server runners, I built a tool to automatically discover and add communities to your local instance :)
https://github.com/lflare/lemmy-subscriber-bot
Just for context, subscribing to about ~800 communities, of which have about more than 50 users/month, my metrics have indicated a disk space usage of about 2GiB/day, 20% of a single CPU core, and about 8~10GiB/traffic a day.
Unfortunately, it seems that most of the disk space used is by a single database table used primarily for debugging, thus once that issue is fixed, the disk space usage should drop dramatically into maybe just a hundred MiB a day or less.
EDIT: For support requests, I’ve created https://lemmy.world/c/lsbsupport as well.
I understand the goal of the tool, the defaults are a really bad approach at achieving it and the docs are really bad at identifying the pitfalls. A tool that subscribes to a list of communities provided in a text file would be great. Subscribing to the entire lemmyverse is a solution that creates problems that are worse than the discovery problem.
I agree that there’s probably no reason to subscribe to the whole lemmyverse if the majority of them are barren and void of activity. Subscribing to a list of communities provided in a text file… would still require user intervention to actually source said list of communities, and if that’s the case, copy-pasting into a text file, is not any easier than copying into the Lemmy search bar and hitting enter.
All of this content seems fairly clearly to me to fall into the category “content that can cause legal liability for the hoster depending on their jurisdiction”. Is that a controversial point of debate?
No, I hold no opinion on that stance, you make a valid point.
This all sounds eminently reasonable. 800 subs is a lot, but it’s much more reasonable than the 7k subs this tool leaves you with in it’s default config, and if you further curate it manually and that’s what it takes for your feed to feel lively… then go for it.
While that is the case for my own tool, I believe the author of Lemmony has already patched the code to only subscribe to the top instances, which shouldn’t leave anyone with 7k subscriptions. For me, while I don’t de-federate from right-wing subs on my instance (since I give the freedom for anyone on my instance to follow anything they want), I also configured my bot to not subscribe to any either.
Maybe consider releasing it? I totally agree that community discovery is rough all over, and moreso on tiny instances. A tool to help folks bootstrap 50-200 communities and that did a good job documenting the tradeoffs of oversubscription and helped folks identify/avoid legal risk would be a huge step up from the “subscribe all” approach.
It’s released, it’s in my profile, but given recent debate over Lemmony on OP’s multiple threads, I’m second-guessing whether or not my solution is the correct one. Undecided as of now I suppose.
Content is NOT served from the original instance. […]
I stand corrected, I guess I was referring to the full-sized images. You definitely make a valid point that the content of the post itself, in some jurisdictions, may already cause legal complications. I guess this is something Lemmy as a software, and as a community, would need to find a way to adapt to.
This is also not true. In the US, you have to register a copyright agent to receive the kinds of protection typically associated with commercial hosts. If you fail to do so, I believe that you run the risk of just getting sued out of the gate for copyright issues. There are also almost certainly jurisdictions where hosting gay porn or certain political speech is a “straight to jail” kind of maneuver.
Again, fair. I was shortsighted when I made my previous statements.
Of course, I have no evidence that OP is in a particularly dangerous jurisdiction. But my broader point is that new users of single-user instance often don’t consider that they may be signing up to host legally risky content that they themselves didn’t create, view, or want. If one curates their list of subs, they can gauge for themselves what communities they consider to be risky. If they “subscribe all”, they WILL be serving to the unauthenticated public internet the worst of the lemmyverse without realizing it… which is an entirely avoidable situation.
It would definitely be better for Lemmy in the long run, if there was a way for personal/small Lemmy instances to view another instances communities in its entirety (posts & comments) without storing anything on the database, a-la proxy but for Lemmy. Otherwise, discovery of content would always remain as an entry barrier for the average Lemmy user, further complicating the centralisation of the Fediverse as it is.
Well, yes, accessibility is what most Lemmy users are interested in. The majority of users don’t want to be told to manually crawl through lemmyverse.net, to find the communities that they might be interested in. It is for that reason that I wrote my own tool as well for my own instance, and the users on it.
Unfortunately, because of that reason, most people would rather register on larger instances like lemmy.ml, or lemmy.world, simply because it is easier to find communities on the site itself of larger instances, than smaller instances that do not run this tool, thus exacerbating the problem of centralisation in current Lemmy era.
Until Lemmy authors, or Lemmy admins start taking steps to reduce centralisation, be it by blocking registrations, or by somehow redirecting new users to newer or smaller instances, this will only lead to a feedback loop, where inevitably, an instance may grow so large to the detriment of the fediverse, be it sudden collapse of the instance because of funding (see vlemmy), or by hostile takeovers of the protocol (see Threads).
You shouldn’t run this script, at least not in its default config. It’s more work, but a much better approach is hitting lemmyverse.net and manually subscribing to a bunch of communities you’re interested in.
Unfortunately, given the current state of Lemmy, viewing posts by All gives a very limited view of what’s out there, and uncomparable to viewing All posts on R*ddit. The tool mentioned, while definitely could use some adjusting, alleviates that issue a lot more. Asking people to manually hit lemmyverse.net, and grab the communities they want, is definitely workable for maybe 10, 20 communities, but that’s just insufficient in seeing what’s popular on the regular basis.
Definitely amusing to see you gauge piracy on the same level as hate-speech or porn/loli. Not that I have any opinions about the matter, but amusing regardless.
It also increases the federation load your server generates by 50x or more compared to a “normal” single-user instance that subs to 100 communities or so
I run my own tool, written by myself, subs to about ~800 communities of a certain defined activity threshold, of which have about more than 50 users/month, my metrics have indicated a disk space usage of about 2GiB/day, 20% of a single CPU core, and about 8~10GiB/traffic a day. Is this workable for a tiny instance on a Pi? Probably not, but it is what it is, and while I think my fediverse activity is not agreeable, I try to take steps to alleviate that by manually unsubscribing from the communities that I absolutely have no interest in.
I definitely agree with the part about jurisdiction, but content serving is still done from the original instance, and while I’m not a lawyer, I think the most severe legal threat might be just a takedown.
Yes, that is definitely a drawback and the main motivation behind my tool. Once Lemmy has a easy way to preseed communities, then tools like mine would be made obsolete. Till then, it’s pretty much the only way for personal/small instances to get any good number of communities preseeded on their instances.
Developer of LSB here, no you do not have to use apps like these. They just make it easier to pre-seed communities and even subscribe to them. You could also manually add and subscribe to communities yourself by looking them up on aggregator websites like https://lemmyverse.net. If your personal server is not beefy enough, I certainly can’t recommend using the mentioned tools to subscribe to hundreds of communities out there.
haha