The point about a desert being "hefker" is that you can do what you like in a desert: dig a ditch, build a wall or a motel, or whatever. If you try to do the same thing in my backyard, even if it you think it is an improvement, I can stop you.
Something which prevents people from being receptive to Torah values is the trait of being a "kapdan" (which means, roughly, fussy, rigid, unaccommodating). People with this trait cannot study, for instance, unless they are seated at their own desk, their room is air-conditioned, they had a good breakfast, and happen to be in the right mood. But in order to absorb the Torah it is necessary to be the opposite: adaptable and accommodating, like the desert.
In Pirke Avos (6:4) it is written: "This is the way of the Torah: eat bread and salt, drink measured amounts water, sleep on the bare ground, and live a life of hardship, while you toil in Torah study. If you do this ... you will be happy in this world, and it will be well with you in the world to come."
There seem to be two problems with this advice: first, it does not seem a recipe for happiness, at least in this world, and secondly, most Torah scholars we know do not sleep on the bare ground (for example).
To help solve these problems, let me tell you a true story about Rav Gifter, the great Rosh Yeshiva of the Telzer Yeshiva in Cleveland. A student of his was having a running battle with his wife about who should take out the garbage. The student asserted that according to halacha, it was beneath his dignity (he being a Torah scholar) to take out the garbage. This did not satisfy the wife (understandably), and so they decided to take their problem to Rav Gifter. After listening to them, he said: "Look, I cannot decide this, go home and sort it out yourselves." This was on Wednesday afternoon. The following Friday morning, the couple, with their conflict still unresolved, were preparing for Shabbos. Suddenly there was a knock at the door. It was Rav Gifter, who announced to the startled couple: "I've come to take out your garbage!"
I was recently reading the great book of letters by the Steipler Rav, "Kiryana D'Igrasa". In one of these, he was responding to a correspondent who had listed some of his problems. It should be emphasized that the Steipler was not one to make light of other people's problems, but at the end of his three-page response, in which he considered each problem in detail, he wrote: "But do you want to know what your real problem is? In your letter, you use the word `I' six times." The Steipler was pointing out that the real hindrance to this correspondent's happiness was his "anochius" (egotism).
The mishna quoted in Pirke Avos is not saying that we should sleep on the floor (etc.). It is saying that we should not be wedded inflexibly to our creature comforts. If we have them, fine, we can enjoy them -- there's nothing wrong with that. But if we should have to give these things up, we should be sufficiently flexible, sufficiently accommodating, to do so with equanimity, like a desert. In this way, we can adapt our lives to Torah values, and find happiness in this world.