More than 1,700 people phoned a hotline in just three weeks to make allegations of child sex abuse within football.  This caused shock – but it shouldn’t have.  Those who were surprised had failed to recognise the scale of the child abuse crisis.

According to the Children’s Commissioner, 50,000 children have been identified as having been sexually abused in England and Wales in the two years to March 2014.  She believes a further 400,000 unidentified children were sexually abused in that period.

Just pause a moment and let those statistics sink in.  Almost half a million children in England and Wales were probably sexually abused in two years.  Either those figures are substantially wrong, or else we have an absolute crisis in child protection that we are simply not dealing with adequately.

If so, then why?  For one thing, those initiatives that have been launched – such as Childline – are insufficient to deal with the scale of the problem.  They also do not address the psychological difficulties in reporting the crime.

For survivors, abuse is traumatic to report – not least because part of the abuse can be to make the victim feel responsible for the crime.  Many survivors would prefer to forget or repress past events: they are also likely to fear that they will not be believed.  These factors mean the scale of the crime has been persistently underestimated.

A culture exists in which survivors have not been taken seriously.  Survivors’ motives and their credibility have been challenged in the media and in the courts.  Police officers have been sceptical and institutions in denial.  Insurers have cynically protected their financial interests, not even allowing institutions to apologise for past failings.

Lawyers and some journalists have been apologists for the perpetrators, arguing sometimes viciously in defence of well known people.  Parts of the tabloid media seem to yo-yo between automatically believing accusers, and then those who are accused.

Jimmy Savile is the most famous person protected by his fame.  But he was far from alone.  So were Rolf Harris and Stuart Hall.  And before them it was Paul Gadd, aka Gary Glitter, and Jonathan King.  This is probably just the tip of a very nasty iceberg.  Much of the entertainment industry of the 1970s and 1980s looks horribly rotten from today’s vantage point.

Religious institutions were similarly infected, with priests and vicars also protected by their status.  Where entertainers used their fame, priests used their influence.  Both sets of people attained positions where they (correctly) believed they were above suspicion.  Father Brendan Smyth was notorious, but the problem went beyond the Catholic church.  The Right Reverand Peter Ball, formerly Bishop of Gloucester, received the backing of the Church of England when he was originally investigated, but was eventually imprisoned for child sex abuse.

Meanwhile, we are only now learning of the extent of child abuse within sport.  Coaches are alone with children for many, many hours and some had children living in their own homes.  They hold positions of authority and trust over the children.  And the children are dependent – or at least feel dependent – on the coaches for their chance of sporting success, glory and wealth.  These factors provided an open door for abuse by some football coaches.

Child care and family social workers had a particular opportunity to abuse.   Those who rose to the top of their profession also had the opportunity to influence the policy framework in which they operated.

Peter Righton, for example, was director at the National Institute of Social Work and helped formulate the national child protection policies that he may have exploited.  (Righton was a founder of the Paedophile Information Exchange.)  Frank Beck – who abused children in Leicestershire and was central to our book Abuse of Trust – influenced policy development locally as an elected politician and basked in a national reputation as a child care worker through the promotion of a bogus ‘regression therapy’ for damaged children.

Some politicians used their privileged positions for personal access to children.  Sir Cyril Smith MP – a very senior political figure in the 1970s and 1980s – established and ran his own home for boys.  It was there that he conducted a regime of physical and sexual abuse of boys in his care.

Allegations were never proven against Greville Janner MP – who died as Lord Janner of Braunstone just over a year ago.  Janner visited children’s homes regularly and took one of the boys in care into his own home on a frequent basis, with the approval of the local authority.  As a senior politician he was, literally, above suspicion.

Some 33 individuals have now claimed they were sexual abused by Janner.  It is accepted by the police, prosecutors and by a judge who examined the records that Janner should have been prosecuted for alleged child sex abuse.   The Independent Inquiry into Child Sexual Abuse (IICSA) was to have heard witnesses making these allegations, but legal action by Janner’s family may now prevent this.

Since the republication of Abuse of Trust, I have received worrying allegations about other former senior politicians.  Many of these allegations are credible and the police are investigating.  In some instances influential politicians are alleged to have turned a blind eye to the abuse of children, in other cases they are alleged to have been actively involved.

Credibility is one thing, proof another.  Forensic evidence is at the heart of much of modern police work.  But with child sex abuse we are talking about allegations of events from 30 or 40 years ago.  People who claim to have been victims were traumatised by their experiences and may want to repress it.  Those who came forward at the time were often treated as if they were the criminals.

Some children who ran away from residential ‘care’ in Leicestershire went to the police – who ignored their complaints and simply returned them back to the children’s homes.  There they were badly physically punished and sexually abused all over again.  Years later, their allegations were believed by the court.

This history proves that children must be listened to and people in authority must be willing to believe them.  The same applies to those adults who now come forward with allegations about the past.  Those who claim to have been abused must be treated with respect and dignity – which is still often not the case.

I have spoken at two meetings of survivors within the House of Commons.  I have never before seen such raw grief, hurt and anger at a public meeting.  For survivors, the trauma of past abuse continues on a daily basis, in many cases still blighting their lives.  The lack of justice is a major factor in their continuing hurt.

Ironically, many survivors are now subject to a new dimension of pain.  IICSA was supposed to be more than a safeguarding exercise in which institutional failures are understood and corrected (though that role is very, very important).  Survivors expected it would also provide recognition of their pain.  Yet many survivors now see the proceedings as marginalising them and a forum where clever lawyers are seeking to prevent them from speaking and making key accusations against institutions and influential individuals.

This is not to say that it was wrong to establish IICSA.  But it is bizarre, to put it politely, that the first two people nominated as inquiry chairs had close personal links to famous people accused of having been abusers.

It looks increasingly unlikely that IICSA will achieve much.  Professional appointments were misguided.  And the structure is too constrained for an inquiry that broad.  I would personally prefer there to be a number of parallel inquiries that examined each of the main areas of institutional failure, reporting to an over-arching chair.

Despite IICSA’s problems to date, I believe it is essential that an inquiry does take place that understands the nature of institutional failings and how well placed individuals have been able to exploit those.  But it is equally essential that any such inquiry wins the trust of survivors and groups representing survivors.  IICSA has yet to prove it can achieve this.

We need to understand why the police failed to investigate many of the allegations of child sex abuse.  Why the prosecuting authorities chose to park cases that evidently warranted going to court.  And what role, if any, the security services played in protecting politicians, diplomats and senior civil servants.  In the case of entertainers and religious leaders, we need to understand why they were able to abuse children in the eye of those around and above them.  To do this, IICSA needs to hear the personal stories of survivors.

There are no grounds for optimism that the institutionalised sexual abuse of children has been stopped.  What we do know is that the sexual abuse of children has been, and still is, one of the most common crimes.  It is a crime that may only be known about by the perpetrator and the victim and which is exceedingly difficult to prove.

That is why the subject of child sex abuse needs greater transparency.  That also means that IICSA has to become survivor friendly if it is to hear the truth.  Making allegations is difficult and often heartbreaking for the survivor.  It is not something anyone does easily or without serious thought and commitment.  Survivors need to be treated with sensitivity and respect.  Sadly, many survivors believe IICSA is incapable of doing that.

• Abuse of Trust by Mark D’Arcy and Paul Gosling was originally published in 1998, examining the institutionalised abuse of children in Leicestershire. It has just been republished by Canbury Press to include an additional chapter on allegations against Greville Janner.