On a behavioral level, Maimonides writes that teshuvah involves "returning" to a situation in which one had previously failed, and not making the same mistake again. (Laws of Repentance, 2:1) No wonder we feel joy on Yom Kippur. The joy of being given a second chance.
A chassid once asked his rebbe, "why pray on Yom Kippur, after all we'll inevitably sin again?" "Look out the window," the rebbe said--"what do you see?" "A child standing and falling as he tries to learn how to walk," answered the chassid. Day after day the chassid returned to witness the same scene. At the week's end the child stood without falling. "So with us," said the rebbe, "we may fall again and again, but in the end, God gives us the opportunity we need to succeed."
The mystics understand teshuvah differently. For them, teshuvah means "returning" to being righteous. But suppose one has never been righteous, what does one return to then? Says the Sfat Emet, the soul of every person is fundamentally pure. Teshuvah means to return to the inner kernel of goodness we all posses. And so, we celebrate on Yom Kippur the opportunity to discover our true selves.
A Midrash: After Elisha ben Avuya left his faith he became known as Acher, which means stranger--as if his soul had been overtaken by an intruder. Rav Meir begged him to repent. Elisha replied, "I heard a heavenly voice proclaim, 'return, return, all can return except Acher.' There is no hope." Rav Ahron Soloveichik the contemporary Torah scholar commented, the voice was right, Acher, the intruding stranger cannot repent, but Elisha ben Avuya can return.
A third approach. Rabbi Abraham Joshua Heschel understands teshuvah to mean "answer." Teshuvah is a dialogue. On Yom Kippur we stand before a loving God who asks the questions, we offer the answers. A God of love seeks us out. As much as we are in search of Him, He is in search of us. Another reason for joy on Yom Kippur.
A chassidic legend. A young girl came crying to the Baal Shem Tov."Why do you cry? " asked the rebbe. "I was playing hide and seek," she said, "but no one came looking for me." "So too is it with God," reflected the Baal Shem Tov. "He too is crying. For as much as He is looking for us, we rarely look for Him."
It was left for Rav Kook to relate teshuvah to Israel. Teshuvah, he suggested, ought be understood eschatologically. It quite literally means, "go home," to our homeland--to establish a land that is different from others. A land that is a light to the nations; a land that marks the dawn of redemption; a land at peace. On this Yom Kippur let it be, let it be.