As the political differences between Northern and Southern factions escalated dramatically entering the second half of the 19th century, Mahone grew to favor the secession of the Southern states. As the War for Southern Independence loomed ever closer, he was took an active role in the conflict even before he became an officer in the Confederate Army. Early in the war, in 1861, his Norfolk and Petersburg Railroad was especially valuable to the Confederacy and transported ordnance to the Norfolk area where it was used during the Confederate occupation. By the end of the war, most of what was left of the railroad was in Federal hands.
Virginia seceded from the Union in April 1861 - along with North Carolina, Tennessee, and Arkansas - in response to Lincoln’s call for States to supply troops to force the seven already seceded states back into the Union. Additionally, they stated that Federal troops would not be allowed to cross through Virginia to accomplish that purpose, feeling that states that had entered the Union voluntarily had the right, and the obligation, to also end that association.
Mahone helped bluff the Federal troops into abandoning the US Navy Gosport Shipyard in Portsmouth by running a single passenger train into Norfolk with great noise and whistle-blowing, then much more quietly, sending it back westward, and then returning the same train again, creating the illusion of large numbers of arriving troops to the Federals listening in Portsmouth across the Elizabeth River (and just barely out of sight). The ruse worked, and not a single Confederate soldier was lost as the Federal authorities abandoned the area, retreating to Fort Monroe across Hampton Roads under cover of darkness, after firing the Gosport Yard and its supplies. It was at this point that Mahone accepted a commission as Lieutenant Colonel, and soon to Colonel, of the 6th Virginia Infantry Regiment in the Confederate States Army. He was promoted to Brigadier General in November 1861, and commanded the Confederate Norfolk District until its evacuation in 1862.
In May, 1862, after the evacuation of Norfolk by Southern forces during the Peninsula Campaign, he aided in the construction of the defenses of Richmond on the James River around Drewry's Bluff. A short time later, he led his brigade at the Battle of Seven Pines and the Battle of Malvern Hill. He also fought at Second Bull Run, Fredericksburg, Chancellorsville, Gettysburg, the Wilderness and Spotsylvania Court House.
William Mahone was widely regarded as the "Hero of the Battle of the Crater" on July 30, 1864, during the Siege of Petersburg of 1864-65. Former Pennsylvania coal miners in the Union army tunneled under the Confederate line and blew it up in a massive explosion, killing and wounding many Confederates and breaching a key point in the Confederate defensive line around Petersburg. However, they lost their initial advantage and Mahone rallied the remaining Confederate forces nearby, repelling the attack. After beginning as an innovative initiative, the Crater scheme turned into a terrible loss for the Federal leaders. As a result of his actions, William Mahone was promoted to a Major General and was with General Robert E. Lee and the Army of Northern Virginia until the surrender at Appomattox Court House on April 9, 1865
Small of stature, 5 foot 5 or 6 inches, and weighing only 100 pounds, Mahone was nicknamed "Little Billy". As one of his soldiers put it, "He was every inch a soldier, though there were not many inches of him". Otelia Mahone was working in Richmond as a nurse when Virginia Governor John Letcher sent word that Mahone had been injured at the battle of Second Bull Run, but had only received a "flesh wound". Otelia is said to have replied; "Now I know it is serious for William has no flesh whatsoever". Otelia and their children moved to Petersburg to be near William during the final campaign of the War in 1864-65. Mahone suffered from acute dyspepsia all of his life. During the war, a cow and chickens always accompanied him to the field in order to provide dairy products.