In New York City, at the turn of the century, Cantor Rabinowitz is determined that his thirteen-year-old son Jakie become the next in a long family line of cantors. On Yom Kippur, the Jewish Day of Atonement, he looks forward to the time when Jakie will take his place in the temple, but his loving wife Sara is concerned that their son wants to do something else. Meanwhile, Jakie is seen singing in a saloon by neighborhood kibbutzer Moishe Yudelson, who then rushes to inform the cantor. Jakie is then dragged home and given a whipping by his father. Later, Jakie tells his heart-broken mother that he is going to be on the stage, then runs away. Years later, in San Francisco, Jakie has become a singer performing at Coffee Dan's restaurant. When he sings the poignant song "Dirty Hands, Dirty Face" for the audience, followed by the jazz tune "Toot, Toot, Tootsie," vaudeville dancer Mary Dale, who is in the audience, is intrigued. She tells him that he has what other jazz singers do not, a tear in his voice, and helps him to get a job with her troupe. Some time later, while performing in Chicago, Jakie, who has changed his name to Jack Robin, goes to a concert of sacred songs given by famed Cantor Josef Rosenblatt, and is deeply moved. Through the years, Jack has sent letters home boasting of his success but has never reconciled with his father. Jack has grown to love Mary and is saddened when she leaves the troupe for a chance to appear in a Broadway show. A short time later, Jack is told by his agent that he, too, has been offered a part in a Broadway show, and he looks forward to a return home to New York and his mother. In the autumn of 1927, on Cantor Rabinowitz's sixtieth birthday, Jack pays a surprise visit home. Although Mrs. Rabinowitz is over-joyed to see her son, who promises to move them to a new house in the Bronx and buy her a new pink dress, Cantor Rabinowitz is furious to hear his son singing jazz music in the house. They have a violent argument over Jack's preference for show business over the family tradition of being a cantor, and Jack leaves after his father bitterly calls him a "jazz singer." On Yom Kippur, Cantor Rabinowitz is too ill to sing the Kol Nidre in the temple and dreams that his son will sing in his place. Yudelson goes to see Jack at the theater where April Follies , the show in which he is co-starring with Mary, is about to open, and asks him to sing in the temple. Although Jack is torn, he refuses. Just before Jack is to go on stage and perform his role in the dress rehearsal, Yudelson returns with Mrs. Rabinowitz, who begs her son to reconsider. Although Jack's heart is pulling at him, Mary reminds him of what he had just told her, that his career means everything to him. Jack refuses to leave the dress rehearsal and, seeing Jack on stage, Mrs. Rabinowitz realizes that her son no longer belongs to her and leaves. When his number is over, Jack is told by Mary that his mother realizes that his life is now show business, but Jack cannot deny what is in his heart, and rushes to see his father. Jack then goes to the temple and, after Cantor Rabinowitz hears his son singing the Kol Nidre, he dies in peace. Although the show's opening had to be canceled because of Jack, he is soon a Broadway star and sings "Mammy" as his mother and Yudelson proudly sit in the front row, and Mary happily watches from the wings.