The word "Tashlich" means "You will cast away." In this context, it refers to a custom dating from at least as early as the fourteenth century, but probably much earlier, based on the last verses of the Book of Michah (Micah) 7:18-20, shown below:
"Who is a G-d like You?
You forgive sins and overlook transgressions
(Note: The Prophet has switched, grammatically, from addressing G-d in the "second person" (using "You") to addressing Him in the "third person" (using "He"). This may have to do with the difficulty finite man has when addressing his infinite Creator, and comes up again in connection with the topic of Blessings. Anyway, back to Michah:)
And You will cast into the
depths of the sea
Consequently, Jewish communities have for many generations gathered on the First Day of Rosh HaShanah at bodies of water and recited the Tashlich Prayer, which consists of certain chapters of Tehillim (Psalms) and the verses shown above, to symbolize our wish to get rid of our sins, and to be forgiven by G-d.
Some have the custom to throw crumbs, but some authorities say that one shouldn’t; each person should act according to his/her minhag.
Historically, Jews would sometimes be instructed by their own community leaders to seek out bodies of water not near the center of town, so as not to be seen during Tashlich by their non-Jewish neighbors, for fear of a "poisoning the wells" accusation. This was certainly true if actual food was thrown into the water, but even if not, they would be accused of mumbling curses and poisoning the wells by witchcraft.
If the first day of Rosh HaShanah falls on Shabbat , Tashlich is said on Sunday, the second day. If it cannot be recited at all on Rosh HaShanah , due to inclement weather or "whatever," it can be recited untilHoshana
Tashlich is preferably recited alongside a body of water containing fish, to remind us that just as fish are protected by the water in which they live, we pray to be protected by G-d. Also, just as fish swim freely and can suddenly be caught in a net, so too we can just as helplessly fall into the net of sin. And even as the eyes of fish are always open, so do we pray that G-d too will keep vigilant watch over his people.