Public-key login is a negotiation. The client offers public keys one at a time; the server checks each against the authorized_keys of the user you are logging in as, and either accepts one or replies with the methods that can still continue. When nothing is accepted, the client prints Permission denied, and the word in brackets is that list of methods — here, only publickey.
So the message says nothing about why. Reproduced against a throwaway OpenSSH 10.3 server: a key missing from authorized_keys, a user that does not exist, an authorized_keys file that was group-writable, and a group-writable directory above it all produced exactly the same client output. The server's log separated every one of them — Failed publickey, Invalid user, and Authentication refused: bad ownership or modes for the file or for the directory.
Two causes are entirely on the client. A private key file that other users can read is refused: WARNING: UNPROTECTED PRIVATE KEY FILE! and This private key will be ignored. The refusal can come late. When ssh can get the public key without the private file, from a .pub beside it or from a key in OpenSSH's own format, it offers that key first, the server accepts it, and only the signing step fails. Reproduced, ssh only says so when it reads the file itself — a key the ssh-agent already held was used from the agent regardless. And the right key may never be offered: with seven unrelated keys in the agent ahead of it and no -i, the server disconnected with Too many authentication failures rather than Permission denied, and naming the key with -i and IdentitiesOnly=yes made it work.
StrictModes is why permissions matter on the server. OpenSSH's manual says sshd checks the modes and ownership of the user's files and home directory before accepting a login, because people accidentally leave them writable, and it is on by default. A key that is in the file is still refused if someone other than its owner could have put it there.