Kemp was born in St Osyth, Essex. She was a cunning woman who was frequently called upon by her neighbours to heal ailments and sicknesses. She was later blamed for intentionally causing illness and death, eventually being tried for witchcraft in Chelmsford in February 1582. At her trial, several of her neighbours testified against her, making statements to Justice Brian Darcy. Along with her friend, Alice Newman, she was accused of causing the deaths of Edna Stratton and two children, Joan Thurlow and Elizabeth Letherdale.

Neighbour and former friend Grace Thurlow testified that when her son Davy was sick, she asked for Kemp's help. Davy temporarily recovered from his illness and Thurlow believed that Kemp had cured him. Some time later, Thurlow and Kemp argued over the care of Thurlow's baby daughter Joan. At a few months old, Joan fell from her cradle and died of a broken neck. When Thurlow became lame, she again asked for Kemp's help. Kemp agreed to heal her for 12 pence. Thurlow got better but then refused to pay Kemp her fee, saying she could not afford it. The two women argued again and Kemp threatened to get even with Thurlow, who became lame again. Thurlow testified that since that quarrel, either she or her son had suffered. She blamed Kemp for her son's illness, her own lameness, and the death of her baby. Thurlow complained to the magistrate and an investigation followed.

Alice Letherdale testified that Kemp had asked her for some scouring sand (an abrasive cleaner) and that she had refused her, knowing Kemp to be a "naughty beast". Letherdale's daughter Elizabeth later saw Kemp, who "murmured" at her. When Elizabeth fell ill and died, Letherdale blamed Kemp for bewitching the girl to death. Kemp's eight-year-old son Thomas testified that his mother kept four spirits, or familiars. He described them as a grey cat called Tyffin, a white lamb called Tyttey, a black toad called Pygine and a black cat called Jacke. He said that he had seen his mother give her familiars beer and cake, and let them suck blood from her body. Thomas said that he had been present when Alice Newman had visited his mother. He said that his mother had given Newman an earthenware pot, which he believed to contain the familiars. Days later, he saw Newman return telling Kemp that she had sent spirits to kill a local man and his wife.

Justice Brian Darcy said that Kemp made a full confession to him in private. Kemp told him that approximately ten years previously, she had experienced a "lameness in her bones". She had gone to a local cunning woman who had told Kemp that she had been bewitched and that she should "unwitch" herself. She recommended a ritual to Kemp using hog's dung, charnell, sage and St John's wort. Kemp performed the ritual and recovered. Two women who she knew requested her help for lameness. She helped them in the same way that she had helped herself, and they apparently recovered. Since then she had performed healing services for her neighbours. She admitted to the four familiars her son had mentioned. She said that they were two male spirits, that killed people, and two female spirits, who brought sickness to people, and destroyed cattle. Kemp went on to confess to sending her familiars to make Grace Thurlow lame and to kill Joan Thurlow, Elizabeth Letherdale and Kemp's sister-in-law. She named twelve other women as witches, six of whom were hanged, including Kemp, in 1582. Many of the accused freely confessed to witchcraft despite knowing they faced death as a result.

Ursula Kemp was hanged in Chelmsford in 1582. In 1921, the skeletons of two women were found in a St Osyth garden by Charles Brooker, one of which was believed to be that of Kemp. The 'witches skeletons' became a local tourist attraction with an admission charge to view them. In 2007, historian Alison Rowlands said that according to her research, the skeletons could belong to any of ten women who were executed for witchcraft in the 16th and 17th centuries. A more recent forensic study suggests that the bones are of Roman-era provenance.