The verse continues, “You’ll end up worshipping their idols and it will be a trap for you.” So it’s evident that the reason the Canaanites could not be permitted to reside in the land is that their idols would be a snare for the Jews and inevitably lead many of them astray into idolatry.
The prohibition here forbids selling real estate in Israel to idolators. While it was not permitted to sell or rent them a residence (or land to be used for a residence), it might be permitted under certain circumstances to lease them a piece of business property. The laws were more lenient in conquered territories that were added to Israel in the time of King David and this mitzvah does not apply at all outside of Israel.
The Talmud in Avodah Zarah explains that a Canaanite who was willing to give up idolatry was permitted to reside as a “ger toshav,” a “resident alien.” The ger toshav had to affirm in front of a beis din that he was turning his back on his idols and willing to live according to the seven universal (“Noachide”) laws.
This mitzvah applies to both men and women, in Israel, at all times. It is discussed in the Talmudic tractates of Avodah Zarah (20-21 and 64-65) and Gittin (45b) and is codified in the Shuchan Aruch in Yoreh Deah 151. It is #51 of the 365 negative mitzvos in the Rambam’s Sefer HaMitzvos and #26 of the 26 mitzvos that can only be performed in Israel according to the Steipler Gaon.
The prohibition against settling idolators in Israel is the last of the 53 (!) mitzvos in parshas Mishpatim.