On old war ships it was necessary to keep a good supply of cannon balls near the cannon .
But preventing them from rolling about the deck was a problem.
A storage method was devised to stack them pyramid fashion on a square base plate with 16 symetical holes. This meant a ball ball on top, resting on four, resting on nine, which rested on sixteen allowed a supply of 30 cannon balls could be stacked in a small area right next to the cannon.
The metal base plate was called a Monkey, for reasons unknown, was made of brass so the iron, the iron balls would not rust to it. - hence, it bcame knwn as a Brass Monkeys.
As you know brass contracts much faster than iron when chilled. On early designs of monkey plates were not thick enough, so consequently it was found that when the temperature dropped too much, the brass and holes would shrink so much the iron cannon balls would just come right off the monkey.
Thus, it was quite literally, cold enough to freeze the balls off the brass monkey.
And all this time, you thought it was a about some poor defensless animal used as "painful eyes watering" ,male anaotmy metaphor, to describe the power of a cutting chill that can dismember even the cherished member from a brass statue if this poor creature, ... didn't you eh?
Now isn't this just a fabulous bit of historical knowledge to at least a few your uneducated friends.