At first I was curious if OP tried arguing with it or telling it the correct answer. If he just asked the question without any further interaction, it would still have no data that it generated the wrong answer.

But as another comment says, many people have tried similar things and posted about it online, that information is siphoned by now and the LLMs weights get updated.

These types of conversations probably get automatically flagged and sent to labelers.

There's probably a robust detector for riddles and logic puzzles.