While writing and recording sessions continued through the late 1980s, his debut album, on which the song was the lead single, would not be released until 1991. After visiting the church where former soul singer Al Green was preaching, Elvis Presley's former home of Graceland, and a small nightclub in nearby Mississippi, as well as wandering various streets of downtown Memphis, he returned to New York and began composing the song.

The song's lyrics are autobiographical, chronicling a 1985 trip that Cohn, then a struggling songwriter and singer, took to Memphis to overcome a bout of writer's block.

It received a Song of the Year nomination at the 34th Grammy Awards in 1992, the same year that the 31-year-old Cohn won the Grammy for Best New Artist. ' Walking in Memphis' is a song composed and originally recorded by American singer-songwriter Marc Cohn, for whom it remains his signature song.