Sometimes filler will just break away in time, I managed to get old stuff off a Humber once by tapping it gently with a hammer, otherwise its just a case of sand carefully and watch the dust.
Value wise its all dependant on what you choose to do, if it matches your hand a loved one might like it later down the line but if you're selling it on then yes putting it back to standard is likely a must.