Though referred to as a castle, Margam is actually a large Grade I listed Victorian mansion built for Christopher Rice Mansel Talbot for his family between 1830 and 1840. The house passed from the Talbot family when it was sold to Sir David Evans Bevan of the Vale of Neath Brewery, in 1941. For a time during World War II it was occupied by allied forces but due to the costs of running such a huge property Margam fell into disrepair. Then in 1977 it was gutted in a fire and left as an empty shell It is now restored, open to the public and under the care of the local authority.
The Gothic architecture of Margam Castle certainly gives it a spooky look which matches its haunted reputation. Robert Scott has been identified as possibly being one of the ghosts that haunt Margam. Scott was a gamekeeper that worked on the estate and was killed by a poacher. He is supposed to haunt the grounds of Margam and has been reputedly seen on the main staircase leading to the house. Some of the estate workers are also supposed to have seen the ghost of a blacksmith in the grounds and the sound of somebody running has been heard.
Within the house itself the sounds of giggling children have been heard in the corridors and some of the rooms. Apparitions of children in Victorian clothing have supposedly been seen going in and out of some of the room doorways.