In early June of 1862, the Confederate cavalry moved through campus rousing the students. Eight students left the next day to join the confederacy, although only two enlisted. By June 26th, 12 boys remained.
During the Confederate retreat from Gettysburg, Saint James was surrounded by both armies and overwhelmed. The spring house was robbed of 100 pounds of butter by Confederate troops. The main basement became a makeshift bunker, filled with food, supplies and valuables. Kerfoot departed for several days, returning to discover a "Sad, harrowing sight..."
He wrote: "Outer and inner doors burst opened; closets, wardrobes, trunks, broken open; kitchen and dining room filthy; evidently much stolen. But no furniture broken for mere wantonness. But so, too, in the College and boys' rooms. Dormitory, wardrobes, trunks, rifled; dining room, store room...robbed utterly. Food wasted and scattered. All this was done despite a general's orders, who could not prevent it...The loss has been very serious...."
Despite this heavy blow, Kerfoot and Bishop Wittingham kept the school running. On October 7, 1863, Saint James opened with an enrollment of 23 students for its 22nd and final session. By the end of 1863, 14 of Kerfoot's former students died due to the war or disease.
During the following summer, violent and destructive robberies erupted around the College. Kerfoot had students move their provisions to the attic for safekeeping, and students were placed on guard in preparation for raids. He wrote: "We are in the midst of outlaw soldiers and robbers." But, despite the chaos, mid-summer examinations continued.
The last Saint James Commencement was to be held at 4 pm on July 12, 1864. However, with news of 40,000 Confederate soldiers marching from Boonsboro to Hagerstown, the ceremony was held earlier, at 11 am. Four students received BA degrees, and another four students, including Kerfoot's son Abel, received MA degrees during the small, private ceremony. Following the ceremony, 14 students were able to leave for Chambersburg by stage coach, but 12 students from Baltimore were trapped at the College by the surrounding forces until July 17th when the roads were re-opened for travel.