Dave according to John Knibbs's excellent book on the pre-WW1 BSAs, there were at least three types of tap retaining methods employed on the LJ "H" rifles. The most common was this one, with a keeper screw.

Then in the third batch (serial nos. 4830 to 5829) an adjustable one, like Caddy's, was fitted (pic 1, pic 2), which had a knurled finger nut and a compensating spring, supposed to take up any wear when the nut was tightened. But tightening it also had the effect of moving the tap out of alignment with the bore so the design was abandoned.

There was also a "quick release breech plug catch" but it was only fitted to a small number of rifles.