Update: Actual spookiness

Okay, I finally found why I couldn't use my SSH.

It is a, in my opinion, really dumb problem. My $HOME-dir had the mode 775 (drwxrwxr-x) instead of 755 (drwxr-xr-x). That is apparently a silent problem. No errors/logs said anything more than something along the lines of Pubkey authentication failed: Permission denied. It should, in my opinion, say something along the lines of Authentication failed: improper permissions. (See https://link.to/solution/or/problem). And the link would tell you what permission shit needs to be.

Anyways, I solved it with a simple chmod 755 /home/$USERNAME.

Well, good night.

// BurnyLlama