Ampthill Castle, of which there are no standing remains, was built in the 1420s by Sir John Cornwall, spouse of Elizabeth, sister of Henry IV. The castle came into the hands of Henry VIII in 1524 and Katherine of Aragon lived there during the divorce proceedings of 1533. By Elizabeth I's accession the buildings had already fallen into decay and at the time of the 1567 survey its partial demolition was planned. In 1605, the idea of rebuilding was revived and plans were made to convert the buildings to a hunting lodge for James I, however nothing came of them. In a Parliamentary report of 1649 it was recorded that the buildings were by then totally demolished. In 1771-2 the second Earl of Upper Ossory employed Lancelot Brown to landscape Ampthill Park.
The approximate site of the palace is marked by Katherine's Cross, erected in the 1770s to commemorate Katherine of Aragon who lived for a time at the palace. Although there are no upstanding walls, there exists a detailed survey of the palace precinct drawn up in 1534 which enables the plan and extent of the ruins to be ascertained. The main precinct comprised four wings ranged about a rectangular inner court, with a gatehouse on the east wing and kitchens and a great hall on the north. Other chambers were the state rooms and private quarters. The external dimensions of this precinct were about 65m east-west by 55m north-south. Outside the north wing was a smaller court, which contained a well-house. To the east and south the palace was surrounded by an outer, or 'Base', Court which had a range of buildings at the perimeter which contained stables, workshops and other rooms which housed the ancillary functions of the palace.