Barrel might be bent or misaligned.

But you could well be pushing the trigger to one side as you fire. Most commonly that would see a right handed shooter throw his or her shots to the right.

Which pistol?