What you say is true, but I still think there is an element of snobbery in there somewhere. For some collectors, just knowing that a gun has been refinished is enough to make it undesirable in their eyes, even if the refinishing is visually perfect and can only be detected by forensic examination. This is exactly the attitude of Dinky car and porcelain collectors, but not how collectors of vintage cars and paintings feel.
I sometimes think that the aversion to quality, sympathetic refinishing is Freudian. Maybe it is a subconscious dislike of another collector having previously dabbled with the object of their desires. Bit like they might feel about the woman in their life. This would explain why a factory-refinished gun is deemed OK, as it is a more impersonal intervention. Bit like letting your wife see a gynaecologist.