If you make your pannier frame out of square section tubing ,you can weld some aluminium angle to your panniers that will fit over the frame .
Drill through both the frame and aluminium angle and fit a long bolt through the pair . If you have a thin strip of steel welded to the pannier frame that will hold the bolt head "captive" [ or in other words prevent it from rotating ] you can then attach the nut inside the pannier [use a wingnut if you wish].
When you lock the lid you will have prevented any access to the nut so it cannot be undone by a light fingered thief .
The beauty about this system is that you are spreading the load over a wide area and the bolts do not take any strain except of course to hold the mountings together and prevent the pannier bouncing off .
Blessed are the cracked, for they let in the light. - Spike Milligan
"When you come to a fork in the road ,take it ! When you come to a spoon in the road ,take that also ."