Onigajo Castle consists of tuff rock, formed by volcanic eruptions over 15 million years ago. The stone itself is very soft so many caves have formed across the surface as a result of erosion from the waves and wind. There is a legend that there was a demon lord who lived in these caves called Tagamaru who was killed by the general Sakanoue no Tamuramaro. These caves must have made for a good dwelling for demons.

Onigajo Castle was originally called “Oni-no-iwaya,” or “grotto of demons,” but in the Warring States Period, a castle was built on the top of the mountain and it became known as “Onigajo Castle.”