The novel is divided into three parts. In the first part, "Sensei and I," the narrator, a guileless university student, befriends an older man, known only as "Sensei." Sensei lives as a recluse, interacting only with his wife and the narrator, and occasional unseen visitors, but still maintaining a distance between himself and them. He regularly visits the grave of a friend, but for the moment refuses to tell the narrator any details of his earlier life.

In the second part, "My Parents and I," the narrator graduates and returns to his home in the country to await his father's death. As his father lies dying, the narrator receives a letter from Sensei which is recounted in the third part of the novel, "Sensei and His Testament." Sensei reveals that in his own university days he was cheated out of most of his fortune by his uncle. As a result he moved to Tokyo and began living with a widow and her daughter, with whom he fell in love. Later he convinced his childhood friend (known only as K), who was in dire straits, to move in with him. Gradually K recovered, but also fell in love with the landlady's daughter. K confessed this love to Sensei, who was shocked, and later full of jealousy. Sensei then proposed marriage, and shortly after, K committed suicide. Sensei, who had lost his faith in humanity after being cheated by his uncle, was horrified to find the same dark impulses lurking in his own heart, and felt a heavy guilt for the death of his friend. In the present, 1912, Sensei is prompted by the suicide (junshi) of General Nogi Maresuke (following the death of the Meiji Emperor) to take his own life, writing the letter to his only friend to explain his decision.