The hall is the oldest building in Whitby built in 1516. It is now a hotel said to be haunted by Browne Bushell, a former owner who was executed for piracy. He has been seen walking up the staircase, and has also been heard in the same place.

There has been other strange phenomena associated with the hall over the years, including poltergeist activity.