The oldest demonstrable building structures of the castle complex date back to the late Romanesque period - the second half of the 12th century. These are the lower floors of the castle tower and the 'well room' created on the first floor of the tower. In addition, it is probable that besides a moat and an embankment, a wall also contributed to the fortification. Remains of the wall, which could testify to this, were discovered during excavations in February 1986 in the first transverse cellar of the north cellar (today the salt mine exhibition). Final clarifying investigations to assign this wall to the Romanesque period are still pending.

A major reconstruction of the castle dates back to the turn of the 15th century. One floor was added to the castle tower and the former gable roof was replaced by a battlement. At this time, at the latest, further buildings were erected, which extended over large parts of the area of the present castle grounds (north, west and east sides). This is also indicated by the extent of the newly built defensive wall. The wall circumscribes an approximately square area and was equipped with four round defense towers. The course of the wall and the position of the towers are clearly visible in the ground plan of today's complex (see below).

Evidence of further reconstruction dates back to the 16th century. These processes can be read on inscription plates on the 'Kornhaus', 'Torhaus' and in the uppermost tower room. The work on the 'Kornhaus', which is located on the west side of the castle, was completed in 1541, as the inscription indicates. The other two dated building works were commissioned by Andreas von Meyendorff (1522-1583). In the process, the castle tower was raised by an octagonal floor (1576) and the roof was executed in the form of an onion-shaped dome (today a pointed roof), as a drawing from the middle of the 17th century shows. Even today, the 'gatehouse' spans the northern entrance to the area. This was completed in 1581. The lower part is made of sandstone, followed by a half-timbered structure. Judging by the drawing, the easternmost part of the building on the north side was also built in this way.

After the Ummendorf line of the von Meyendorffs died out in 1667, this circumstance was taken as an opportunity to transform the Ummendorf office into a domain. From then on, until 1912, the castle and its lands as well as the villages of Ummendorf, Eilsleben and Ovelgünne, which belonged to the office, were administered by bourgeois tenants. Again and again, reconstructions and new buildings were carried out, which contributed to the present appearance of the castle complex. Worth mentioning is an extensive measure in the first half of the 18th century. The staircase in the castle tower and the wooden floor in the small meeting room bear witness to this period. The wood of the staircase could be dated dendrochronologically to the year 1720. At this point we would like to mention the so-called 'Lusthaus', a baroque garden pavilion, which stands on the foundations of a former defense tower. The "Garten-Häusgen im Rondel" is described once in the inventory of the domain in 1722 and can be found on the ground plan of Sprengler from 1795 at this place.