King William used the existing Roman city wall as the eastern and southern defences for his new castle, the southern line being preserved in the existing inner line of defences on the river front.
His great work, however, was the building, just within the Roman defences, of the great keep or White Tower, his overseer being Gundulf, Bishop of Rochester, who is recorded to have stayed at the house of Eadmer Anhaende, citizen of London, while the building was in progress.
The original tower was begun in 1078 and though its completion date is uncertain, the work was certainly accomplished by the end of the eleventh century. The great tower, if not actually the largest, is one of the loftiest of Norman keeps, and although has suffered some mutilation and alteration, stands very largely intact.
It shares with the castle of Colchester the distinction of possessing an apsidal chapel in its structure, and at the Tower this building with its aisles, ambulatory, and unribbed vault, is of the most valuable examples of early Norman architecture which we possess.
In 1091 the "Saxon Chronicle" mentions building of a stone wall round the Tower, but whether this implies the enclosing of an outer ward on the site of Tower Green is doubtful.
We have no further definite information about the buildings until days of Richard I This King and his masterful Chancellor, William Longchamp, appear to have begun to transform the fortress into something like the structure that we see today.