If you are using commercially cast bullets you will find that they are cast really hard. What is happening is that there is not enough pressure in your loads to "upset" the bullet into the rifling. As a result you get gas blowing through the gap which erodes metal off the bullet which then is deposited on the inside of the barrel, which is what you found. Subsequent bullets just run over the lead and make little engagement with the rifling. Starting with a clean barrel and chamber throats load shoot some ammo loaded near the top end or, better yet, buy a box of factory ammo and see what that does. If it shoots factory ammo ok you know its not the pistol.

This problem is made worse if the chamber throats, the bit between the front of the chamber and front of the cylinder, is smaller than the groove diameter of the barrel. You can check this by slugging barrel and chamber throats, there are plenty of tutorials on how to do this on-line.