Lemmy.World is looking for 4 new Systems operators to help with our growing community.
Volunteers will assist our existing systems team with monitoring and maintenance.
We’re ideally looking for chill folks that want to give back to their community and work on our back-end infrastructure. Must have 4+ years of professional experience working in systems administration. We are not looking for junior admins at this time. Please keep in mind that, while this is a volunteer gig, we would ask you to be able to help at least 5-10 hours a week. We also understand this is a hobby and that family and work comes first.
Applicants must be okay with providing their CV and/or LinkedIn profile AND sitting for a video interview. This is due to the sensitivity of the infrastructure you will have access to.
We are an international team that works from both North America EST time (-4) and Europe CEST (+2) so we would ask that candidates be flexible with their availability.
If you are in AEST (+10) or JST (+9) please let us know, as we are looking for at least one Sysadmin to help out during our overnight.
You may be asked to participate in an on-call pool. Please keep in mind that this is a round-robin style pool, so it’s alright if you’re busy as it will just move along the chain.
If you’re interested and want to apply, click here.
I’m qualified, but 5-10 hours can mean a lot of different things.
Are you looking just for oncall/incident response, or are there more active reliability projects that you need help on?
I totally hope you’ll get enough admins to help out. Alas, I’m way to short on time to commit to this now. After 28y sysadmin and developing on Unix and Linux, I could use some project that is useful, but alas, no time to spare now. (Maybe later)
I’m curious what the backend looks like based on your requirements here… Ansible is always a red flag for me that your servers are pets not cattle. Just maintain a golden image, especially since you mention kubernetes. And if you’re using self managed kubernetes USE REMOTE ETCD. Trust me, it will save you so much time and drama.
I wouldn’t be interested unless it’s paid so I am just throwing that out there for y’all to consider.
This made me laugh. Configuration management systems like ansible, chef, salt, and puppet only exist because people wanted to manage a large numbers of systems and keep them consistent and replaceable, i.e treat them like cattle instead of pets. They were born out of the pets vs cattle analogy.
I realize containerization has taken that a step further but it’s funny to hear someone talk about these tools like they’re something archaic.
Probably more accurate to say “mutable” vs “immutable”.
Harder to have pets in immutable if it’s actually immutable.
If you want to replace them why would you not use a golden image? The same thing goes for VMs, not only containerized. You can sit there and wait for ansible to run, or just have your image come up immediately with what you need. It makes it take longer with 0 benefit.
And where do you think the golden images come from? The steps must be reproducible, versioned, audited, scripted and in the third hands of outsourced minions with 8 or 12h time lapse.
The golden rule of CICD does it survive if you’re visited by truck-kun?
Or just run everything in containers and use stock ubuntu or aws Linux or whatever.
I’d love to help out one day but right now my experience is just hosting random things for fun that I find on github
You may be asked to participate in an on-call pool
Unfortunately, this is where I noped out. But I ditch most paid positions where I can’t avoid standby-time.
deleted by creator
deleted by creator
Threw my hat in the ring, I’m a senior devops engineer.
Don’t have any Lemmy experience though. I have no desire to self host it, but I wouldn’t mind being part of the team to maintain a large instance.
Out of curiosity, will you be able to weed out bad faith volunteers? I am sure there are a variety of interests that would be more than willing to pay a junior admin to be a Lemmy Sysop and it’s not like the candidate will volunteer that information.
deleted by creator
In one word: Fraud. Deception.
Some one who operates in bad faith seeks to gain advantage by lying to get in and cause turmoil afterward.
Here is an alternative Piped link(s): https://piped.video/watch?v=cegalOZqmEo
Piped is a privacy-respecting open-source alternative frontend to YouTube.
I’m open-source, check me out at GitHub.
Someone acting in bad faith is someone doing something with ulterior, negative intentions.
This could mean joining an admin team with the goal of breaking the site. Joining a sports team with the goal of sabotaging their matches.
Without going into details to avoid giving out ideas, think the concept of script kiddies for example. Doing harm for no reason. Or with more experience doing harm for personal gain
We are not looking for junior admins at this time.
I think they’ll be fine, the form asks for a CV + video call + lemmy user name and optional github profile.
I’d be surprised if a bad faith candidate got through that. A probation period could work here, where their access is a bit restricted at first
Hum, I hate to be that guy, but the good (experienced) bad actors have multiple profiles curated exactly for this reason. There’s a reason why companies require proof of identity and a background security check.
A CTO of 5 years with many more years of experience here. I would be really glad to help, but not in scenario where I have to prepare a CV for international readers and have no pay at all as this looks to me like a job application with no job.
Considering you are running on Digitalocean infrastructure, I am completely unsure why you would ever need Ansible and Terraform as it just adds complexity without certain benefit, especially if you mention Kubernetes which DO already provides with two clicks.
I’d personally suggest trying out ArgoCD for declarative clusters. With this thing, I’ve seen 2 companies maintained by a single DevOps engineer with no problems. Huge timesaver and makes everything transparent.
In case this process changes and becomes less corporate-y and more transparent, I’ll be ready to apply. Hope you’re going to find the right people! Long live Lemmy World!
I’m a big fan of Kubernetes, and for larger projects the flexibility and power it brings is unrivalled. But for smaller projects, assuming equal levels of competence, delivery teams using managed Kubernetes are almost universally later and have more issues than teams that use simpler solutions. Container-as-a-service solutions like GCP CloudRun or AWS FarGate help somewhat, but are not cheap for a given amount of compute time.
Terraform (or IaC in general) absolutely has a place, because even if you use Kubernetes, most projects have more infrastructure to manage than just the cluster - at the very least, lemmy.world has a CloudFlare proxy to manage - and clicking buttons in a management portal is not a repeatable way of deploying that, or deploying the Kubernetes clusters themselves.
Ansible also has a place, particularly if you’re deploying onto bare metal. I wouldn’t use it for new deployments unless I had bare metal to configure and maintain, but lemmy.world is deployed onto a bare metal server as I understand it. Plus, the most effective tooling is generally the one your team understands.
IaC is the right move. It transfers complexity, it necessarilt doesn’t add it. It makes your deployments reproduceable and automated.
Which is a baseline to having highly available infrastructure. Not everyone will be familiar, or have the right mindset for that sort of DevOps.
Ansible/Teraform are portable. I don’t see it’s usage as a failing, rather as avoiding DO lock in.
Agreed with the rest though. This is quite the ask.
wow nice, hope you get some qualified folks! regrettably I dont have time or I’d toss my hat in the ring
In my opinion Lemmy.world should start selling a bit of merchandising (t-shirt and so on), just to add a little on the donation side.
BTW. the donation links are in the group info.
deleted by creator
If you need a janitor I can help but back end stuff is over my head.
I’m like a junior in experience but would love to help and especially learn how to help. I also have a lot of free time
I read it as Lemmy World PsyOp at first and thought there’s some conspiracy bullshit happening on the instance. lol. Good luck on your search!
Lemmy World psychological operations, the secret communist agenda of the administration team!!! leaked!
i knew it!
Glad to know I wasn’t the only one, lol.
I did as well! After I read your post I went back up to review the post headline
removed by mod
Lol don’t you look silly. Here’s who’s running this server. They’re already fully set up as a non-profit.
removed by mod
Unpaid moderators for a for-profit company.
As someone that fits this bill perfectly, and considered tossing my hat in the ring, I kind of agree with this person.
While I may not agree with the phrasing and hostility, the point still stand that they are effectively looking for Senior Level + DevOps.
And 5 hours a week will turn into 30+ as the person or persons takes over ownership of the required pieces.
My main holdup is just having enough time personally.
I agree I’m pretty pissy and numb to the former reddit and now lemmy crowd. It’s all just bitching that everything should be free without looking into the reality of costs (among others off topic topics). I want a decentralized network to happen but I am 100% expecting the community to fuck it up.
This is like year of the Linux desktop for social media just playing out in duplicate with no understanding why Linus desktop never took off.
As long as they don’t lament your existence when they disagree with you it’s a false dichotomy. It’s OK to ask for volunteers from the community, that’s kinda what makes it a community. It’s not OK to shit on your volunteers.
You can do whatever the heck you want. Successful communities are rarely successful at scale without funding. Debian, RHEL, C, Rust, Go, Apache, Nginx, Ruby, JS, etc all have a considerable commercial backing. Nothing tells me anyone had a viable long term plan for ensuring privacy and stability.
Yeah I think non profit corps are the only way any non-corpo social media effort will offer a viable alternative to the corporate bastards.
If one wants decent uptime that takes knowledge, skills, talent, as well as sound processes and supporting technology, and decent infosec. It is hard enough for corporations to find that (see: Reddit and its abysmal uptime, or any company with a major breach in the last decade) let alone volunteers.
I hope they can find the folks. I do.
I fear that going cheap on IT isn’t usually a winning strategy for the long term.
Perhaps someone out there will decide to do as you suggest to stand up their own instance and have a small paid staff. I think with the right tools and processes as force multipliers you can maximize uptime, while minimizing sysadmin drama and burnout.
I threw my hat in the ring. Hope you all get some good people!
