How well, really, do you know the back of your hand? Make sure you can't see it, then try to describe anything about it that wouldn't be precisely the same description as anyone else of your gender and approximate size. Can you say any more about the proportions, the distribution of hair, the location of veins and arteries and tendons, the particular lines and creases around the knuckles, or anything else that distinguishes the back of your own hand from anyone else's?
The phrase really doesn't make much sense if you take it at its word, even more than the one about taking candy from a baby, or talking behind someone's back. But I wonder if that's just because of what we do nowadays. Maybe way back when, when more people spent more time doing physical labor with their hands, labor that was often stultifying (and not even eased by the distraction of Walkman headphones), people tended to get to know the backs of their hands very well. Perhaps the phrase originates in a time when it made sense, and we just lost the sense, but didn't lose the phrase. Is it really just a fossil phrase, not nonsense?