עַל נַהֲרוֹת בָּבֶל שָׁם יָשַׁבְנוּ גַּם בָּכִינוּ בְּזָכְרֵנוּ אֶת-צִיּוֹן
By the rivers of Babylon, there we sat down; we also wept, when we remembered Zion.
The various commentators (Rashi, Radak, Metzudas David, ibn Ezra, etc.) all agree that David is speaking prophetically in the voice of Levites exiled in the days of Nebuchadnezzar. Their job was to sing and play music in the Temple; upon exile, they hid their instruments (verse 2), refused to sing (verses 3-4) and prayed that they lose their ability to play and sing if they failed to place Jerusalem foremost in their thoughts (verses 5-6).