The larger calibre only delivers if it hits the vital organ. Accuracy important? Accuracy is everything.

I've had a squirrel take x5 .22 pellets and then jump for my face from the tree. All because of poor shot placement. The .22 alone wasn't enough and all pellets were found the far side under the skin. It was dead but didn't know it.
It used to be normal practice to heart shoot rabbits with a .22. Problem was the rabbits had enough time before expiring to get to their burrows.
A .22 can feather plug. Generally pigeon flies 200m and then keels out of the sky. Seen it, been there.
I do like the .22 on rats though but at least distance is never great.

Modern air rifle vermin shooting asks for precision kills, not the solid hits of old. If you can do it with a .22 then you are a better shot than I; a good few people are. Certainly John Darling could make the .22 work for him though I'm unsure he always used a 12ft/lbs rifle??
Your call, for me its .177.