Early in the 1980s, a pop legend in a mid-life lull reached back into ancient history for inspiration. The song did not come easy. Banging his head in frustration on a hotel-room floor, Leonard Cohenground out about 80 stanzas before finally achieving the perfect anthem that is “Hallelujah”.

And no one got it.

CBS Records rejected the album. After a 1984 indie release, “Hallelujah” hung in limbo for a decade until Jeff Buckley, sighing deeply over a steel guitar, gave a soft, introspective reinterpretation. Buckley’s death by drowning in 1997 added a tragic aura to the song. The producers of Shrek called in Rufus Wainwright to record it for the soundtrack.

Cohen, having been fleeced by a felonious manager, went back on the road, singing “Hallelujah” in a trademark brown hat. X Factor hopefuls heard it and one belted it out to victory. Suddenly, “Hallelujah” was being downloaded 100,000 times a day and turning into the most covered pop song of the 21st century.

Amid the resurrective clamour, few grasped the leap that Cohen had made into the past. In the depths of despair, he had sought the “secret chord / That David played, and it pleased the Lord” across three millennia of human creation, appealing as one lost Jew to an ancestor for the primal gift of music.

I think I know where he was coming from. Growing up in a devout and learned north London home, I became aware of the taboos and tensions that prevailed between Jews and music. I learned, for instance, that Jews, mourning the destruction of their temple in 70AD, were forbidden by rabbis to sing or play music, all the way down to Moses Maimonides in the 12th century.

I knew, too, that a woman’s voice was proscribed by the Talmud as “nakedness” and that hearing a woman sing was equivalent to having an illicit sexual liaison. Thrilling as that may have seemed to my boyish mind, women’s singing really was taboo. As was listening to music for seven and a half dark weeks of the year and at times of personal loss. In sorrow, music was the first thing to be switched off.

Yet, amid these constraints, music was everywhere. At any solemnity or celebration, someone would start a tune. There would be singing at all Sabbath meals. Since my father was tone deaf, it was my grownup sisters who floated the melodies that I, at three or four years old, learned to harmonise by ear. Music was our means of togetherness. Without music, I remember thinking, would we even be Jewish?

So when Radio 3 commissioned me to make a three-part series about music and the Jews, I made the decision to avoid popular cliches of “Jewish music” – klezmer bands, cantorial wails, Ladino lullabies – and focus on some of the bigger questions. How, for instance, has music shaped the character and history of the Jews? How did Jews influence music? Biggest of all, can music define personal and collective identity?

I started where Cohen did, in search of the elusive King David: poet, musician, warrior, sexual malefactor and author of a book of psalms that forms the basis of worship for Jews and Christians alike. Though there isn’t  much evidence that David wrote all or any of the 71 psalms that bear his name, we cannot read them today without becoming aware of this musician’s private world, his inner ear.

How could David, having sent a man to his death so he could steal his wife, sit down and write “Hallelujah”? “With great difficulty,” explains Yehoshua. “The Talmud tells us that David was punished for his sin.”

“How’s that?”

“He was deprived of his music for 10 years.”

Time stops still on the wall of David’s city. Yehoshua’s reading of Psalm 51 is that David was rendered musically, and perhaps sexually, impotent by guilt, an idea that does not exist until Freud adduces it in 20th-century Vienna. Could Jewish guilt be rooted in Jewish music?

American composer Steve Reich came to Jerusalem in the 1970s in search of his Jewish roots. His epiphany arrived while listening to the way Yemenites enunciate the Psalms. “I just had to chant a verse [with them],” he recalls, “and a melody popped into my head. What is that? It was an unconscious dredging up of Bulgarian rhythms from Béla Bartók, changing rhythms in The Rite of Spring, all unbidden. But it introduced a new kind of rhythmic writing for me, a specific idea of combining twos and threes into five/eights, seven/eights; something I hadn’t done before.” Reich considers his psalmic score, Tehillim, to be his towering masterpiece.

Tehillim were the songs of the temple. The search for their lost music is a bimillennial obsession. In 1905, a cantor called Abraham Zvi Idelsohn arrived in Jerusalem from South Africa and, like Bartók in the Balkans, began recording old men’s songs on wire machines. Applying new techniques of academic musicology, he surmised that the Jews of Yemen came closest to temple music. At the National Sound Archive in Jerusalem, I played Idelsohn’s cylinders and consider his boldest conclusion – that Yemenite-Jewish microtones lie at the root of Gregorian chant, and hence of all Christian music.

In a Tel Aviv apartment, I meet the anthropologist Tova Gamliel, an authority on mourning, and ask her to demonstrate the oldest known Jewish sound – the keening of Yemenite women. Gamliel stands, composes herself and sings a visceral, chilling trope that freezes my fingers to the chair. “The role,” she explains, “is to make people cry, to express sorrow in a very aesthetic performance. But the song has a text – the life of the departed – and the singer can vary that according to what the person deserves, good or bad. She is telling the others: when you die, I may not be so generous.”

The power of life after death was vested in a woman. “She was the only one who had this right. People were very afraid of her, very respectful,” says Gamliel. When the keening ends, the woman recomposes herself, then tells a joke. Life must go on.

Myriam Fuks from Brussels is an eighth-generation Yiddish singer whose repertoire has passed from mother to daughter for two centuries. Myriam’s mother, Frania, sang in Warsaw theatres, survived the Warsaw ghetto. Myriam wakes in the morning with fragments of Frania’s hundreds of songs. Unable to remember the refrain, she asks the pianist Martha Argerich to improvise for her on a new recording. The need to keep memory alive by song, I discover, a driving Jewish motivation.

It was the late 1820s before Jews were allowed into western music. There had been isolated intrusions – Salomone Rossi in Monteverdi’s Mantua, Lorenzo da Ponte in Mozart’s Vienna – but it took a pair of bankers’ sons from Berlin, Felix Mendelssohn and Giacomo Meyerbeer, to change the culture. Mendelssohn, aside from his own concert works, restored Bach’s oratorios to public performance – “giving classical music its Old Testament”, according to one of my contributors. Meyerbeer blew out the walls of existing opera houses with gargantuan music dramas, paving the way for Richard Wagnerand the romantic imagination.

Around the same time, on the front stoops of New York brownstones, the sons of Jewish refugees from Russian pogroms and of former African-Caribbean slaves from the deep south found an unsuspected common taste for busy rhythms, minor keys and blue notes. Their conversation signalled the birth of pop music.

How Jewish was that? George Gershwin, the most restless and creative of the early writers, never concealed his Jewish roots. When he sang “It Ain’t Necessarily So“, he not only challenged Scripture with Talmudic argument, he actually sang it in the traditional mode of Talmudic study. Visiting the Yiddish theatre star grandparents of the conductor Michael Tilson Thomas, Gershwin talked of the freygish mode as the key to America’s popular music. Freygish is Yiddish for questioning. What Jews added to pop music was a quizzical note.

Michael Grade, heir to an entertainment dynasty and ex-chair of the BBC, explains why Jews were so big in showbiz. “There’s something in the DNA of the Jews that makes us adept at assimilating,” he explains. “There’s a great openness to what’s going on. We are watching the audience, trying to keep in touch with what the audience wants. The best of the impresarios – I’d include my uncles and my late father – would be just ahead, not too far ahead, of public taste. And ready to take a chance on talent. Things are never the same again after the great talent has spoken.”

Jews became tastemakers, Grade believes, because they had learned to listen out for any change in the wind. A key to survival became a tool in identifying and managing public taste without sacrificing a hardwon identity.

That perspective makes a generic concept of “Jewish music” uninteresting and largely irrelevant beside the transformations that Jews brought to music wherever they lived, and the changes that music wrought in the matter of being Jewish. Could anyone, I have always wondered, be Jewish without music? “It doesn’t matter which you heard,” sings Leonard Cohen, “the holy, or the broken.” Hallelujah!

• Music and the Jews begins on Radio 3 on 9 March.