The Manor of Eltham was presented to The  Prince of Wales in 1305, and from that point it increasingly saw royal use. It was Edward III who really turned it into a royal palace, acquiring it in 1344, much building work was done from around 1350 and the manorial lands were augmented.

The future Queen Consort Philippa stayed at Eltham on her arrival in England, en-route to her marriage to Edward III. The Court was here quite often when an additional 700 people would have to be accommodated in the area, John of Gaunt seems to have been there in July 1390 when there was a discussion over the gift of Aquitaine to him by Richard II - legally the Duchy of Aquitaine could only be held by the heir to the throne, so the gift was controversial. In 1396, there was an exchange of gifts between John, The Duke of Lancaster and Queen Consort Isabella, not long married to Richard II. Given her role as England's first lady until Richard II married Isabella, it is likely that Katherine was at Eltham at this time since she was still second in the line of female precedence.

The only significant structure to remain from this period is the bridge over the moat, which Richard II had built. The Great Hall was built at the behest of Edward IV, and is magnificent. It had fallen on hard times, being used as a barn for a time up to the first part of the twentieth century (famously being the subject of a Turner painting portraying its use as such), then it was restored when Stephen and Virginia Courtauld had their beautiful Art Deco house built.