Site-Related > News

Another Server Outage

(1/2) > >>

Spectere:
Remember how I was planning to migrate to a different distro? Yeah, well, I got motivated to do that again. Why? Because systemd is literally one of the worst things to happen to Linux and I want it out of my life.

So the site went down some time today. Dunno when. Why don't I know? I'll get to that in a sec, don't you worry. What's important is what happened: several services died spontaneously. I tried to restart them. This happened:



What? Uh. Okay. Couldn't get any insight from the logs or anything, so I did what I do every time systemd packs a mental: I rebooted the server (because everyone loved it when Windows 95 required a reboot to do...well, anything...right?).

Then...nothing. No httpd and, more importantly, no sshd. Fortunately, Linode is awesome and has a little thing called Lish that lets me console into the server via SSH. I connect to that, log in, get root, etc etc.

Nothing is started. sshd, httpd, mysqld are all stopped. I don't even have a network interface.

What.

I check the logs. Nothing. The binary logs that systemd insists on using are corrupt and unreadable, so I have no idea what happened. Why does it have binary logs? Because whichever idiot "designed" it has never had to deal with a system failure before. Because they apparently think that the Windows Event Viewer system is super-convenient and shit. Because it's apparently not designed for bloody servers.

So I quickly restart everything. Fortunately, it all comes up. Hopefully "systemctl enable" actually starts working so that I can, y'know, reboot the server without having it (and myself) having a complete panic attack).

Way more excitement than I wanted on a Monday evening, but fortunately everything seems to be back in action. But yeah, I'm sick of this bullshit. I've had nothing but problems like these every time I've tried to use or administer a systemd-based distro, so fuck it. It's getting dumped in the trash where it belongs. sysvinit ain't perfect (though stuff like OpenRC helps) but at least it's a proven solution, and when something gets fucked up it's pretty transparent.

More updates to come as I continue with this project.

Bobbias:
That sounds like one massive headache.

Zephlar:
Christ man. Thanks for working hard to keep the handful of us left afloat!

Spectere:
No problemo. :)

And hey, regardless, I still use this server for quite a few things. I definitely need it to be up. When it's not, Ian no happy. And when Ian no happy, ain't nobody happy.

Edit: I've also noticed that the site has been getting slower and less responsive over time. There is literally no explanation for it, as nothing untoward is happening, judging from the Apache and system logs. I'm sure that'll be resolved when I migrate away from shitstormd and Arch.

Spectere:
So httpd randomly died last night. Something sent it SIGHUP and it went down like an anvil. Site was down from around midnight (EST) to a few minutes ago.

This has got to be the least stable server platform I've ever used.

Navigation

[0] Message Index

[#] Next page