By all means try 5.0gr of Unique, or even lower by a grain. See what the accuracy is like. As you drop below reasonable velocities, the groups get very spread out, so you'll know when you've dropped too far.

A velocity of 600-650fps is fine for out to 50m; no need for more, as long as it accurate enough for your needs. I've developed loads around 600fps for use in my Marlin, which give 3.9cm (edge to edge) 10 shot groups @25m. That's X-ring accuracy.

If you can scrounge the use of a chronograph sometime, you can establish precisely where your load lies on the velocity spectrum. Ask your club if they have one -- they really should if they have velocity/energy limits they want their shooters to adher to.