Not sure how you've got it modded (I take it it's a Crosman 1322?), but the screw which secures the barrel goes through the rear sight. Barrell is also secured at the front with the front sight assembly. Check to see the rear sight screw is tight, and the front site assembly should have no play. You need to use the smallest scope you can get away with on the 1322. You might also need to replace the plastic spacer between barrel and main body.
Hope this helps!