It doesn’t have real shame. But it also doesn’t have, like, the concept of emulating shame to evoke empathy from the human, right? It is just a fine tuned prompt continuer.

Why wouldn't it? That's downright in-distribution. Plenty of it in the pretrain corpus.

Shame is a feeling. There’s no real reason to suspect it has feelings.

I mean, maybe everything has feelings, I don’t have any strong opinions against animism. But it has feelings in the same way a graphics card or a rock does.

I agree, but also we don't have a definition of what real shame is. Or how we would tell when we crossed the line from emulated shame to real shame.

I don’t think emulating shame (in the sense of a computer printing statement that look like shame) and real shame have a cross-over line, they are just totally different types of thing.

Feeling shame requires feeling. I can’t prove that an LLM isn’t feeling in the same way that I can’t prove that a rock or a graphics card isn’t feeling.

We do, you might find legal sentencing guidelines to be informative, they’ve already been dealing with this for a very long time. (E.g. It’s why a first offence and repeat offence are never considered in the same light.)