If someone gives us something to watch for them, then comes to reclaim it, we are forbidden to deny having it; this includes denying a loan. Passive though this is, it’s still stealing. (According to the Talmud in Baba Metzia 5b, violation of this mitzvah is enough to disqualify a person as a witness!)
The reason is what we have said about theft leading to the breakdown of society. A person who would never break into another’s house might find it easier to justify keeping something left in his care; it doesn’t matter. The end result is that both are theft.
This mitzvah applies to both men and women in all times and places. It is discussed in the Talmud in several places in tractate Shevuos (see, for example, pages 37a-b). It is codified in the Shulchan Aruch in Choshen Mishpat 294. This prohibition is #248 of the 365 negative mitzvos in the Rambam’s Sefer HaMitzvos and #36 of the 194 negative mitzvos that can be fulfilled today as listed in the Sefer HaMitzvos HaKatzar of the Chofetz Chaim.