MAYDAY IN THE MAGELLAN – PART III What Caused the Flood?

Ten years ago, the Royal Navy’s Ice Patrol Vessel HMS Endurance catastrophically flooded. Her main engine room filled to the deckhead within 30 minutes. Such was our remoteness our Mayday call went unanswered. The crew and I spent the next 24 hours fighting for our lives.

This article is the final part of a three-part blog focusing on leadership, culture and priorities. Part 1 – Leadership was published in December and can be found here. Part II – Priorities is here. Part III – Culture, looks at what caused the flood and was it inevitable or could more have been done to prevent it? What was it about the ship that set it apart and yet rendered it so flawed? Has the Royal Navy learned the lessons from this incident?

Part III – Culture

Introduction

Joseph Nye in his paper Inevitability and War argues that conflict occurs only when three causal thresholds have been crossed; deep, intermediate and precipitating. His analogy is that of building a fire: “The logs are the deep cause, the kindling and paper are the intermediate cause, and the striking of the match is the precipitating cause.”

By looking at the causes of the flood through these lenses it is hoped that this blog will broaden the findings of the relevant Service Inquiry which tended towards the more precipitating events. These are naturally woollier and more open to interpretation and thus perhaps the reason the Service Inquiry avoided them. My aim here is not to expose or embarrass anyone, but to inform those who are interested in this subject (and there are many). If it helps those currently responsible for running the replacement ship HMS Protector, then so much the better. And if I do raise concerns about the culture onboard prior to the flood, then that must be set against the fact that it was exactly the same culture that so heroically saved the ship as described in Part 1.

Deep Causes (1990 – 2008)

This section covers the 18 year period from purchase-to-incident during which the way the ship was manned, operated and assured all diverged from her design criteria.

People.

MV Polar Circle was designed to have a crew of 38. By the time she became HMS Endurance, she had a complement of 119. It’s not entirely clear why. Merchant vessels generally run with very lean crews when compared to their Royal Navy counterparts for a number of reasons; they are mechanically simpler, they don’t need large numbers of people for battle damage repair, and their equipment and staffing is designed to minimize crew costs. As one would expect, putting 119 people in a ship designed to run with 38 and only occasionally surge to higher numbers put considerable and sustained strain on her systems.

Operations.

MV Polar Circle was built to operate in the Arctic, a relatively short hop from its home port, and to undergo dry-docked maintenance once a year. HMS Endurance operated every austral summer in the Antarctic—a harsher environment, requiring a much longer commute. This had consequences both serious and comical: the heated shower floors, whilst lovely in Antarctica, made a lot less sense off the West Coast of Africa, and couldn’t be turned off. More seriously, crossing the equator with no air conditioning (which Endurance had to do twice yearly) caused regular engineering issues. Most seriously of all, under naval ownership she was dry-docked and refitted only every five years – far beyond her design criteria. Her last docking period, in Falmouth in 2004/5, was generally reckoned to have been a disaster. More on that later. 

There is a phrase in common usage amongst warship drivers: “drive it like you stole it”. It alludes to a sort of Bondian derring-do; cavalier yet sharp – understand the limits of your ship and then vigorously explore them. Sensibly applied to an agile, high-powered warship fitted with systems expecting to be stressed, it’s a good thing. Where it doesn’t work at all is in a large, heavy, relatively underpowered ice-breaker whose machinery is designed to be run for very long periods without excessive loading or alterations. As an example, Endurance had a propulsion configuration called “ice-mode” that directed all power away from non-essential systems through the single shaft in order to optimise her performance in the ice. Not only did it sound cool, it gave you nearly a whole extra knot of top speed. So, of course, the ship had steamed around in ice-mode at full throttle for years. There’s a reason Endurance was nicknamed The Big Red Plum and it wasn’t because of her raked bow, slender lines or wave-piercing prowess – “drive it like you want it to run forever” would have been more sensible.

In every warship in the fleet, breakdown drills are conducted routinely three times a week as an essential part of ensuring that everyone onboard instinctively knows the systems and their reversionary modes. Continuing to provide propulsion to a warship under fire is clearly a core skill, so we did breakdown drills routinely…but we weren’t a warship. A Bergen shipping engineer visited us off Africa after a particularly unreliable passage and discovered that every drill we conducted was damaging the engine control systems: we were starting each serial when the mechanics had reset from the previous drill but, unbeknownst to everyone, the electrical control systems had not; this had been going on for 20 years. The Bergen engineer was a vocal man, or he may have just had a loud voice, but when he shouted at me “would you do these drills in the ice in Antarctica?” and I answered “no” he bellowed, “SO WHY ARE YOU DOING THEM NOW?” My answer “because they support career progression and conform to Fleet Engineering Orders” felt pretty lame.

Assurance.

Integrated Project Teams(IPTs) were part of Naval engineering standards and assurance for many years, normally with a high degree of success. However, it was clear by 2005 that the Minor Warships and Boats IPT (MWAB), of which Endurance was a major element, was not fit for task and thus the Minor, Patrol and Hydrography IPT (MPHIPT) was born. I have received written accounts from engineers who served in Endurance over this period and their assessments of the new group are scathing. The ship was in a constant state of materiel fragility almost across the board. Our Operational Defect list was excessively long and measures were never in place to take the required bold action to fix it.

Between the IPT and the ship’s captain there are two Naval organisations responsible for delivering ships on operations: Flag Officer Sea Training (FOST) and the Flotilla organisation, in our case in Portsmouth Dockyard. Neither were familiar with Endurance or her systems. Their training and inspection methodologies were all configured for grey ships – very few understood the red one. Admittedly, this was compounded over the years by senior ship’s companies (and Commanding Officers) holding these organisations at arms-length on the principle that “we know how to operate our ship – you don’t”. When a ‘wrecker’ (a member of the FOST team responsible for simulating damage in order to train your teams in controlling it) admits that they’re “making it up as they go along when they inspect Endurance” then you know something is wrong.

The ship was formally assessed by a small FOST team a couple of months prior to the incident. In isolation there are some telling comments in their final report:

“Equipment. Despite the best efforts of the engineering department, the equipment is assessed as “below standard” due to the defects on both shaft generators. There are an additional five outstanding high-level defects.”

“Sustainability. The Main focus of the forthcoming period will be defect rectification…”

“Summary. The planned inspection programme’s sea phase was significantly affected by emergent defects…”

But the overall impression of the report was one of “doing OK considering. Nothing fantastic, plenty of caveats, but nothing to worry about.” Of course, all we wanted to do at the time was get rid of the inspection teams and get back to sea.

Culture.

There were two things wrong with the culture in HMS Endurance – “this is the way we’ve always done it” and alcohol consumption. Both of these, anecdotally but believably, predate the ship and perhaps go all the way back to the first HMS Protector, but certainly existed in the previous HMS Endurance. The stories of mishap in these two ships are legion and worn almost as a badge of pride in various blogs and groups.

Prior to taking over as second in command of Endurance I attended an eight-week course to prepare me. This is a standard RN course that refreshes you on all manner of things from your knowledge of the Rules of the Road, to the power and propulsion systems in your upcoming ship, to the latest regulations for dealing with errant sailors. I have never been asked to stay behind after lectures “for a chat” so much in my life. Every time it was to give me examples where standards of maintenance and discipline onboard were of concern. Individually I treated the stories with caution – ‘dits’ like that often magnify in the telling. Collectively, it was clear that there was a problem but I didn’t want to pre-judge too much – and how bad could it really be? Either way, I joined with my eyes and mind open.

Imagine my surprise when I joined to find everyone wearing slippers around the ship. It’s quite hard to describe how inappropriate and unsafe—and simply odd—this is to someone not used to operating in warships and therefore how comprehensively that image encapsulates “The Endurance Way” and all that was wrong with it. I had an exceptional Executive Warrant Officer who was also new to the ship and who had been equally well pre-briefed. One of my early conversations with him was over slipper-gate and what we should do about it. He was also opposed so I suggested that we ban slippers immediately, only to find that it was permitted in Ship’s Standing Orders! Ignoring the advice given on my course to not go in too hard too early, we banned them anyway. A unique ship operating in the most hostile conditions on the planet is bound to diverge from the norm a little, and a bit of character is a good thing. The key, and what often seemed to be missing in Endurance, is knowing where and how.

Drink culture was hardwired into Endurance, more than any other ship I served in. This was principally because the senior ratings who would normally set the tone for this kind of thing were often in their last tour and many harked back to some sort of misplaced notion of ‘the good old days’. Again, this was a common feature across the entire ship’s lifespan and was probably even brought across from the previous Endurance. The fact that she wasn’t a warship per se, certainly lent itself to a more relaxed and informal atmosphere that in turn lead to too much drinking whilst at sea. Tighter control mechanisms were put in place by the Captain and I and the first person to contravene them spent 28 days in Colchester’s service prison. Many conversations took place discussing the dates that the worst offenders were due to leave and how they could be accelerated. All sensible measures. However, with the wisdom of hindsight, maybe we didn’t do enough.  

To summarise the Deep Causes, since 1990, the Royal Navy had been operating the ship contrary to its design criteria. This caused her to age prematurely and the assurance mechanisms were not robust enough to halt the decline. Then, the “Endurance Way” was often misplaced leading to lower standards in many areas.

Intermediate Causes (2005 – 2008)

This section takes us from the refit in Falmouth in 2005 to the flood.

18 Month Deployment.

At the top of the list of intermediate causes was the plan to deploy for 18 months. The then Commander in Chief was, not unreasonably, looking at ways to get more out of the fleet. Endurance’s operating cycle was pretty inefficient, so the notion of keeping her ‘down south’ for an austral winter, rather than transiting c8500 miles back to and from the UK each time, was a good one. The ship would conduct multiple Work Periods in the ice then, as the austral winter set in, head for the ex-Naval dockyard of Simon’s Town, South Africa for her mid-deployment maintenance. From there, she would conduct a period of engagement up the coast of West Africa before returning to the Falklands and then Antarctica for the start of the following austral summer. So far, so good. However, there were two major omissions from the plan. First, no consideration was given to what deploying for this length of time would do to an already fragile platform. Second, with no extra crew assigned, the agreed rotation involved sending one third of the ship’s company home at any given time. This became known as ‘managed gapping’ and was just as chaotic and disruptive as the name implies.

My statement above that Endurance was over-staffed remains true. However, ‘managed gapping’ one third of the crew also meant that the ship was now under-staffed in the sense that the resident expert on any given system might be gone one-third of the time. We had too many bodies overall and yet not enough specialists for key systems. An odd dichotomy. On the day of the flood, the person responsible for the equipment that failed was on leave and whilst his advice was sought by e-mail, the reply (which was “don’t do the work”) came too late to be heeded.

Nothing in the pre-deployment documentation suggests that any of this tautness was challenged and thus we fell foul of a classic case of press-on-itis. Someone between the ship and the Commander-in-Chief should have challenged the way the deployment was configured and the associated cumulative risk associated with deploying a clearly fragile ship for this length of time. The Marine Accident Investigation Branch (MAIB) call this a failure of organizational influence:

“Organizational influence is a two-way exchange: Organizations cannot accomplish their goals if they can’t influence their members to do the right things. And the members, of course, cannot do the right things—and satisfy their needs in the process—if they can’t influence what goes on in their organizations.”

Refit.

In 2004 the ship went into its five-yearly refit in Falmouth. Speaking to uniformed personnel who were onboard either during or just after reveals that this was a substandard refit. On undocking the ship’s stability condition was so poor that she rolled to 45 degrees and had to be quickly bullied back onto the blocks and errant water pumped out – this seemed to set the tone. All defects are logged onboard through a process called OPDEF (Operational Defects) reporting. I have it on good authority that the OPDEF count post 2005 was significantly higher than before and anecdotally, that she never felt the same. It was during this refit that the actuator at the heart of the incident was replaced with a non-compliant valve actuator, such that the actuator air-lines had to be disconnected to remove the sea suction strainer lid. More on that later but this was indicative of the poor standards of workmanship undertaken and accepted during that period.

Engineering Standards.

There is no nice way of saying this. The end result of much of the above was a ship with poor engineering standards. I was supposed to join her in Cape Town but had to divert to the Falklands because on sailing to transit the South Atlantic, and before having left sheltered waters, her shaft bearing seized locking the (only) propeller firmly in position. It turned out that the person whose job it was to check the oil level in the bearing prior to sailing hadn’t bothered. This was a pivotal moment in the story because when I eventually joined, I found that the legal advice from ashore on how to deal with this incident involved sacking a large percentage of the team. The problem with that, and the discussions I had at length with the Captain, was who would replace them? We still had well over a year to run in this deployment and the only RN expertise in the platform was already onboard. In the end we both agreed that the most sensible course of action was to manage the situation we had. But that was only one incident. On sailing from the Falklands again, pipework to the roll reduction tank ruptured and dumped seven tons of water into the engine room causing significant secondary damage. On another occasion, a tank was overfilled, warping the lid and again, flooding the engine room – this was becoming a habit. We eventually made it to South Africa and received our maintenance package, but on sailing into the tropical waters of West Africa it was clear that all was not well as propulsion system failings caused the ship to regularly end up dead in the water.

The Intermediate causes were succinctly summed up in the Service Inquiry:

“Externally, the provision of engineering and management assurance for a unit conducting an unusually lengthy deployment in a remote and challenging environment was insufficient, and the significance of previous incidents suggesting poor engineering management were not recognised.”

Precipitating Causes (7 days leading up to the flood)

So, who did what on the day and immediately prior to the flood? This in some ways is the easiest section to review as the Service Inquiry covers it in detail.

Since leaving South Georgia we had been struggling to produce enough fresh water for the very high number of people on board. However, this was far from critical and we had options for managing it that had yet to be put in place. The order/threat given to the senior engineers that morning, “improve fresh water production or we’re going to the Falklands for Christmas” was therefore clumsy and certainly contravened the Command Aim of “safe passage to Valparaiso”. It also set in train a rather hurried and poorly diagnosed piece of work. As mentioned previously, the ‘owner’ of the valve in question was contacted and advised by return e-mail against doing the work at sea, but the advice was not heeded or even seen.

No risk analysis was conducted prior to commencing the work, nor was there a fixed procedure in place for such a thing. Neither the Engineer nor I knew that we were going to be ‘one valve open to the sea’ whilst the work was going on. In the US Navy, a ship to be in this condition requires sign-off from the captain. Given the sea-state, our relative proximity to Valparaiso and the fact that we were at flying stations, I’m not sure I’d have given it; easy to say now mind.

What happened next is summarized in the Service Inquiry:

“The opening of the hull valve was caused by the incorrect re-connection of the air control lines during the reassembly of the strainer, and a failure to fully isolate the compressed air supply to those lines. There were a number of contributory factors: poor system knowledge among those attempting the maintenance work; the absence of the appropriately trained system maintainer due to the manpower constraints of an extended deployment; management failure to implement a safe system of work including adequate risk assessment and mitigation measures; a failure to apply satisfactory engineering practice and design shortfalls in the valve control system.”

What was not clear then, and never will be, was whether the “failure to fully isolate the compressed air supply to those lines” was due to a defective valve upstream in the system (i.e. it was passing air despite having been closed properly) or human error (i.e. that valve was opened prematurely before the main valve lid had been re-secured). The Technical Investigation suggested the former but the Service Inquiry suggested the latter. By the time the Service Police came along to investigate almost a year after the event (and they were the first team who had the remit to apportion blame) the evidence had been long since contaminated. Because of this, no one person was ever blamed for the incident. I’m glad about this. One of the points of this article is to indicate the breadth and depth of causal issues and therefore if at the end of the investigation one person had been held responsible, then that would have been unfair.

I will at this point defend the Engineer who came under considerable scrutiny during the endless investigations (see Part 2). When it became clear that proving ‘who did what to which valve’ was going to be impossible then the broader business of engineering culture started to be questioned. As highlighted above, this would have been reasonable had it been spread over an appropriate timeline. However, pinning it on the person who had taken charge of the department just 17 days before, was not. He subsequently won a formal letter of guidance from the Admiralty that, in my view, he hadn’t earned. It also, and this is inevitable I’m afraid, cancelled out his impeccable performance as the head of the damage control organization during the flood itself.

Administrative action

After all was said and done, the Navy took very little administrative action. Technically and legally unable to pin the blame on an individual, and seemingly unwilling to dig into the deeper causes, the end result was a handful of ‘letters of guidance’ to various members of the engineering department. Outside of the ship, where many of the causal issues lay, no action was taken.

Lessons

Having said that, much was learned from this incident that has since been transferred into the wider Royal Navy. The service from the Fleet Incident Response Cell was outstanding that day and that system is now firmly in place for future major incidents. The relationship between ships and their many external auditing agencies is now smarter than it was which is good news especially given how ships are getting more complex, living longer than their designed lives and operating in an increasingly resource constrained environment. Damage control lessons have been taken to the school in Whale Island and are routinely taught to ships’ crews as they pass through. However…

HMS Protector

In 2009, about nine months after the flood, I went to sea for a day in MV Polar Bjorn to scope her out as a possible replacement ice breaker as the prohibitive cost of repairing Endurance was starting to become clear. It was a very interesting day, not least of which was because she had a crew of just 17. That is an unremarkable number for a merchant vessel but is almost unimaginably low to those used to operating RN ships. (For example, a ‘lean-manned’ Type 23 Frigate deploys with over 200 crew). Polar Bjorn’s engineering department had just seven people in it. She operated a two-crew system on a five-week rotation so successfully that she lost only 18 hours on-task over seven and a half years due to defect rectification. Their system knowledge, ability to order stores quickly and conduct repairs at sea whilst still on task left me and the visiting engineer from Abbey Wood both deeply impressed and scratching our heads. We were clear that if we were to procure Polar Bjorn to replace Endurance then we should adopt a similar operating model even if this meant deviating from ‘normal’ RN practice.

Protector is now firmly in service and has been operating in the deep south now for some time. My suggestion to mimic her previous operating methodology and create a lean two crew system of about 30 per crew (there was no-way we could manage 17 by the time the Hydrographers and Royal Marines were added) was very quickly ignored, as were my thoughts on the shape and seniority of the engineering department. In fact, she currently has a crew of about 80 operating on a three-watch rotation. Sound familiar? As for other cultural habits that we inherited all those years ago, I am not well placed to judge if we passed any of them on. I do hope that if the notion of “the Protector way” exists onboard, then it is being applied for the right reasons.

As an aside, but for the Foreign Office’s requirement to keep a White Ensign flying in the Southern Ocean, I believe this task would have been handed to the Blue Ensign of the Royal Fleet Auxiliary whose natural tendency to lean-crew ships would have made them a most suitable operator.

Conclusion

To conclude, two questions should be addressed; was the flood inevitable and could we have done more to prevent it?

To answer the first, I don’t think it was inevitable. Since Parts 1 and 2 of this blog were published I’ve had a handful of Endurance ship’s company contact me with stories that could make you think it was. Certainly, when all the woes are compressed in a blog like this you can convince yourself that it was. However, this kind of dit-compression (as I’m now calling it) is misleading. In reality, the gaps between incidents were much longer than the incidents themselves and we did have a knack of muddling through them, making it to the next work period or port on time and always with a healthy post-operational report to submit to the Admiralty. If someone in authority had confronted the larger issues effectively and sooner, and thereby broken the chain leading to the precipitating error, I am convinced that the flood could have been averted.

So why didn’t we do more onboard to see and prevent this happening? Partly because it’s hard-wired into every naval commanding officer to go to sea and, if necessary, bring violence to the enemy. Staying alongside because you think your ship might not be safe is utterly contrary to RN norms and values and would most likely end with you being replaced by someone who would. Ruling out that path—not even considering it seriously—left us on the much less effective path of attempting major repairs-in-place on the ship’s culture and practices while also conducting operations. Both the Captains I served with in Endurance, the Executive Warrant Officer and I spent hours every day discussing how we should tackle both the culture and the poor engineering standards and we had plans in place that reflected the nature of the deployment that we were on and the people involved. Regrettably, they weren’t enough, or at least they weren’t in time.  But here is the rub: the chain should have been broken long before we ever left Portsmouth. B.M. Batalden & A.K. Sydnes’s conference paper, “What causes ‘very serious’ maritime accidents?” written in 2017 summarizes the nature of the most serious failings perfectly:

“The study builds on investigation reports published by the UK’s Marine Accident Investigation Branch (MAIB) published in the period from 01 July 2002 until 01 July 2010. The study investigates 22 very serious accidents and the 133 causal factors identified as leading to them. It concludes that very serious accidents, distinguish themselves by having causal factors that are to be found higher up in organizations, in comparison to other accidents.”        

The decision to deploy a fragile and poorly understood platform for 18 months with almost no consideration given to the cumulative risk is a real culprit here. Another is a misapplied warship-staffing policy that pays too little attention to the distinctive mission and needs of this type of ship. Better decision-making on either of these matters would have reduced the chances of the flood happening. They would not have corrected the cultural issues, but they would have eliminated crucial factors that enabled those to have such an amplified effect.

In the end, no lives were lost on Endurance, and the ship made it to port. There is no guarantee that any future repeat of this type of incident will be as kind.  

To Finish

Any analysis of a disaster such as this inevitably tends toward the negative. It is an appropriate counter-balance, therefore, to finish this three-part series by going back to Part 1 and the heroic actions and deeds of the ship’s company on that cold day in the South Pacific just over 10 years ago. The ship was brilliant but flawed; the flood might have been inevitable or avoidable – one can’t be absolutely sure. What I am absolutely certain of, is that regardless of all the difficulties and shortcomings mentioned above, the sailors onboard that day, from a standing start, conducted themselves with the professionalism and courage that typifies the finest traditions of the Royal Navy. No one underperformed and many performed exceptionally. Some were rewarded for their efforts, others were not. Put yourself in a dark, oily and violently rolling machinery space, with freezing water coming in at a rate of 2400 tons an hour making enough noise to drown out the main engines. A mayday call goes out on the bridge and no-one answers. What would you do? The ship’s company of HMS Endurance ran at it head-on until beaten out of the space, then they ran at the next challenge and the next one, for 48 hours until we were alongside. Because of that, the ship didn’t sink and no one died. Given how this scenario could have played out, that’s all that really matters. Thanks everyone.