I grew up in a part of the country rife with stories of the occult. It was near Pendle Hill that witches were famously “discovered” in 1612. Across the Ribble in Saint Leonard’s churchyard, notorious occultists Messrs John Dee and Edward Kelly were said to have raised the spirit of a newly buried man in order to learn the location of his hidden wealth. From my childhood home it was only a short drive to Chingle Hall, reputedly one of England’s most haunted houses. As a result, I was always fascinated by the otherworldly and could often be found engrossed in my Usborne Guide To The Supernatural World or Mystery Tales For Boys And Girls. In fact, I’ve been drawn to ghost stories all my life and I’ve read a great many over the years, but there are a handful that continue to unsettle me as a reader and inspire me as a writer.

Mr MR James’ reputation as the master of the English ghost story is well-deserved and I could fill this list with his work alone, but it’s his 1904 story, “Oh, Whistle And I’ll Come To You My Lad”, that has given me the most sleepless nights. It concerns the avowed sceptic, Professor Parkins, who takes himself off to a lonely part of the Suffolk coast to play golf and dabble in a little amateur archaeology. Grubbing about in an ancient abandoned burial site, he unearths an old whistle, which, when blown, seems to summon something malevolent to him. What’s unnerving is the insidiousness of this entity. At first, it taunts Parkins, flinging open the window of his hotel room and twisting his mind towards nightmares as he sleeps. He dreams of a terrified man being pursued along a beach by “a figure in pale, fluttering draperies, ill-defined”. When whatever “it” is finally appears to Parkins as a “face of crumpled linen”, the professor’s faith in his own rationality disintegrates in a moment of physical and intellectual paralysis.

Such humbling epiphanies are common in ghost stories, most notably in A Christmas Carol, of course. We’re reminded how little we can “know” for certain, especially when it comes to ourselves. “The inner nature,” of consciousness, Mr Sigmund Freud tells us, “is just as unknown to us as the reality of the external world.” Thus, the ghost story, says Mr Robert Aickman – the author of my next choice – “makes contact with the submerged nine tenths” of the mind.

As with Mr James, there are many of Mr Aickman’s stories about which I could rhapsodise, but one of his most disturbing tales by far is “The Cicerones” (1968). It centres on John Trant, who is alone somewhere in Belgium visiting the cathedral of Saint Bavon. He arrives near to closing time, finding the place silent and apparently empty. Yet as he tours the building, he fancies that he sees someone in the pulpit and meets a succession of characters who lead him into the hidden corners of the church. The last of these cicerones (an old term for a guide) is a small boy who takes Trant down to the crypt, an impossibly vast space, where “columns seemed to stretch away like trees into the distance”. With a mixture of pride and glee, the child shows off various significant holy relics and eventually brings Trant to a gruesome painting of three martyrs who died by “roasting on a very elaborate gridiron; by disembowelling”. As he tries to leave, Trant is trapped by the men he met earlier and taken away – where to, we’re never told.

Writing a good ghost story is difficult enough. Writing from the ghost’s point of view is more challenging still. Which is what makes Ms Jenn Ashworth’s novel, Fell (2016), such an achievement. It is narrated by the spirits of Netty and Jack Clifford, who are awakened from their eternal rest when their daughter, Annette, returns to her childhood home. Discovering that they can revisit their own history, the husband and wife return to the summer of 1963 when they met the enigmatic Tim Richardson, a young man blessed (or perhaps cursed) with a mystical gift of healing. He revives dead rabbits and starlings and, his abilities thus proved, is finally persuaded to work the same magic on Netty, who is riddled with cancer.

Although the novel is set in the rather genteel Lancashire seaside resort of Grange-over-Sands, Ms Ashworth draws out the menace of the English coastline in a manner reminiscent of Mr James’ stories or Ms Susan Hill’s The Woman In Black. There are dangers in this mercurial landscape, as the tides sweep in and “turn the sea-washed turf into a treacherous maze of unmapped islands, slippery knolls and sucking mudflats”. But the elemental power of this place manifests in more uncanny, supernatural episodes, too. In a kind of exorcism, Tim causes Netty to vomit gallons of sea water “thickened with sand and silt, some tiny grey pebbles… a long muscular string of dark, bubbled seaweed”. So much of what happens in Fell – not least the experience of being dead – is left strange and unexplained, which is what makes the novel so disquieting and unique.

Doing something different with the ghost story form is always a considerable challenge for any writer, something which Mr Stephen King acknowledges in his introduction to The Shining (1977). He calls it his “crossroads novel”, one in which he pushed himself as a writer and created something which transcends the genre. Because of Mr Stanley Kubrick’s famous 1980 film, the story of the Torrances and the Overlook Hotel is so firmly entrenched in popular culture that it’s sometimes easy to forget what an inventive and original novel The Shining is. It’s more than just a “haunted house” story, it’s a study of the power of guilt. As Mr King says, “Aren’t memories the true ghosts of our lives?” Perhaps it is Jack’s self-loathing that brings the Overlook to life with its grotesque and horrifying apparitions.

The idea of a house being a living entity is at the heart of my final story on the list, Ms Shirley Jackson’s The Haunting Of Hill House (1959), in which Eleanor Vance is one of a select group invited by a Doctor Montague to help research the eponymous building’s preternatural phenomena. They experience the usual things – footsteps, banging, cold spots, disembodied voices – but none of these occurrences are as unsettling as the house itself. “Stepping into Hill House,” says Mr King, “is like stepping into the mind of a madman.” It’s true. In a way that the recent Netflix adaptation couldn’t quite capture, reading the novel is like being in a strange, disorientating dream. Everything is out of kilter, the house designed in such a way that “left it chillingly wrong in all its dimensions”. It’s a place that cannot be understood by logic alone. Doctor Montague laments the fact that “people are always so anxious to get things out into the open where they can put a name to them, even a meaningless name, so long as it has something of a scientific ring”. It’s a rebuff to the arrogant thought that we have educated ourselves out of primitive superstition.

The prediction of anthropologist Mr Anthony FC Wallace, in 1966 that, “Belief in supernatural powers is doomed to die out all over the world as a result of the increasing adequacy and diffusion of scientific knowledge,” was an almost laughable miscalculation. Via the internet, “scientific knowledge” is available to more people today than it ever has been in the history of our species and yet our belief in the paranormal persists. Why? Because our brains have never left the primeval forest. As the physicist Mr Michio Kaku says, “There’s a gene for superstition, but no gene for science.” It’s in our nature to be wary of what we think is lurking in the shadows and so the ghost story will continue to haunt us for a long while to come.

Mr Andrew Michael Hurley’s previous books include the award-winning The Loney_. His latest book,_ Starve Acre (John Murray), is out now