The current Langleybury House was built by Sir Robert Raymond, a lawyer and friend of Sir Robert Walpole, who purchased the Langleybury Estate in 1711 and replaced the Manor House which previously stood there. But, after a number of changes of ownership, the Georgian building was drastically altered by William Jones Loyd, a London banker who bought the estate in 1856. Loyd created a new front entrance in what had previously been the rear of the building, also adding bay windows and grand porches to the exterior and substantially refurbishing the interior. He later provided the community with St Paul's School for infants and girls (now offices), and St Paul's Church.
Langleybury House passed to Loyd's son and grandson, the latter leasing the house to the Equity and Law Insurance Company during the Second World War, but in 1947 the estate was sold to Hertfordshire County Council who converted the house and grounds into a secondary school, which opened in 1949 but closed in the late 1990s. The school farm remains open to the public.