There should be a screw with a hole the size of the pin on the swivels, this goes through the hole furthest from the band.
Like this - https://www.flickr.com/photos/leaky5/13622125593/
Be careful the allen headscrew nearest the band is very easy to round off.