The Uncle Gnarley Blog has a new website. Click here to visit www.unclegnarley.ca to view the latest posts!

Thursday, 23 June 2016

MUSKRAT FALLS: A FAILED PROJECT BEFORE IT STARTED (and nothing has changed)

Guest Post written by Frank Wright

I recently had a look online simply using the search criterion, "Muskrat Falls". A NL Govt. website ‎featuring the 'Muskrat Falls Oversight Committee' showed up so I decided to have a look at what this "Oversight Committee" does. I didn't have to look hard to realize that the word 'Oversight' in the name was at best a misnomer.

Amongst other things, the web page states the following: "Chaired by the Clerk of the Executive Council, the committee comprises senior officials from Executive Council and the Departments of Finance, Natural Resources and Justice. It is supported by a working group representing expertise in the areas of law, engineering, project management, accounting and auditing.‎"

‎This sounds like a good beginning for such a Govt. Oversight Committee; the right types of people for undertaking vital governance practices for a mega-project are identified. Apparently, however, the good beginning didn’t get any further than this.

In March 2015, the Oversight Committee "engaged Ernst & Young ("EY") to review Nalcor's cost and schedule processes and controls for the Muskrat Falls Project. EY issued a report entitled “Review of Muskrat Falls Project Cost and Schedule Management Processes and Controls” on October 29, 2015. What the so called "working group" with so much "expertise" to support the Oversight Committee was doing one has to wonder. Looking into cost and schedule processes and controls seems like a typical activity for such a group and at least by function, the right people were assembled. However, and despite all this expertise, the Oversight Committee felt the need to hire an outside organization to do it.  

At first I thought this may be a reflection of the Oversight Committee's view of the abilities of the working group; perhaps a lack of trust in this group or the lack of awareness of the role of such people. Nevertheless, hiring an outside organization for this work is very strange when you are supposedly surrounded by so much key talent.

On looking a little further I discovered where the problem really rests; it is actually the ineptitude of the Oversight Committee itself. And this is borne out in another EY report called “Review of project cost, schedule and related risks, interim report” issued on April 8, 2016 and I will come back to this report later in this article.

The “reporting period” under review by EY in its on October 29, 2015 report was from December 2014 to February 2015. I will also come back to the shocking findings of the EY report, but for now what is most striking about this report is that:

1.  EY took almost eight months to deliver it; and
2.  They stated, "EY's Major Capital Projects practice was engaged to bring additional experience to assist the Oversight Committee in meeting its mandate." [Emphasis added]‎

Let's deal with the eight months to produce a report first. ‎In the project world, and especially mega-projects, relevant and timely information is the backbone of management. This is also true for those undertaking the oversight of a mega-project. The risks are so much greater and the impact of those risks can be disastrous. Best practice in project management and governance demands timely, appropriate and quality information.

A project management team and its oversight body must be able to report on the status of a project with 'real time' or at least, near real time information. Timely status reporting allows everyone involved in a project to have up-to-date knowledge of progress and make informed decisions early, when it is most effective.

Taking eight months to deliver a key project management report that is focused on a timeframe long past, is generally viewed as useless. And most of the time it is. On what basis is a stale-dated report of any value for comparing where a project is, against where it was planned to be? For anyone, whether in the project management business or otherwise, the answer to this rhetorical, but very basic question is obvious. It is usually of no value; and more to the point, such a lag in reporting on a mega-project is extremely dangerous. There is a huge price to be paid for this folly.

By their nature, mega-projects are complex with many dynamic variables that intersect. Timely, actionable information that measures work completed against a well-established baseline is fundamental and offers the only chance to exercise any degree of control. An outside organization that delivers key information so long after the fact is asinine. One could easily say that it is simply throwing good money after bad. I would normally hold this view of the EY report except that in this case, the findings are worth several billion dollars - NL dollars that is - pouring from the public coffers.

Before I get to the matter of EY findings in the October 2015 report, let’s have a quick look at the second point I raised above.‎ EY claimed in its report that it engaged personnel from its Major Capital Projects to bring "additional" experience to bear. 
This statement begs several questions: First, who did they have working on this review if its Major Capital Projects personnel were simply "additional"? Given the nature of the Muskrat Falls Project, it is only people with major capital projects experience who are capable of performing this work. 

Second, anyone experienced in major capital projects has to know that delivering a report on a mega-project eight months after being engaged and even longer after the time frame studied, may be of some value in dealing with lessons learned when the project is completed, but its value is seriously diminished for anyone engaged in managing or overseeing an ongoing project. One has to question what EY means by Major Capital Projects. I suspect it is simply a marketing term of art designed to bring unwarranted comfort to a naive Oversight Committee.

Now let's tackle the content of that October 2015 EY report.

Below is a major excerpt from the report summary. Ordinarily, I would attempt to summarize this but it is so compelling that it deserves to be read in its original form.

"The following observations summarize key aspects of management processes and controls not fully developed and deployed at the time of our review: [Emphasis added]

Key schedule management process and control risks and issues 
1. Certain baseline documents defining contractor schedules as well as the documents defining the control of project schedules were not yet complete. 
2. Contractors’ schedule updates were not being systematically rolled up into the Nalcor Integrated Project Schedule (IPS) that forms the basis of reporting to the Oversight Committee. 
3. A completion date had not been established for finalizing an integrated baseline of contractor and IPS schedules to correct the issues noted in #1 and #2 above. 
4. The IPS development and maintenance process is not fully documented.  

Key cost management process and control risks and issues  
1. The conditions and processes for rebaselining cost and schedule are not defined in the Project’s control processes and procedures. The Oversight Committee’s understanding of such conditions and processes is an important foundation as it conducts its oversight activities.  
2. Nalcor uses a relatively basic approach to its updating of forecasted contingency requirements which in our experience is not consistent with the expected practices for a project of this scale and complexity. Given this, it is not clear whether the cost contingency as forecasted in reports for the Project will be adequate. 
3. The Project does not define thresholds for variance management, reporting, and escalation purposes. We would normally expect these to be in place as they assist in giving clear indications of the severity of issues and the need to escalate to key stakeholders, such as the Oversight Committee. 
4. Fully quantified risks or trends have not been documented for certain significant challenges on the project. The scale of potential challenges is also not quantified in the summary reporting made available to the Oversight Committee."

The summary report from EY goes on to state:
"Until such time as the management process and controls risks and issues identified in this report and the detailed supplementary report are addressed, the completeness and accuracy of Project cost and schedule status reporting to the Oversight Committee cannot be fully verified."

Let's keep in mind that the Muskrat Falls Project was sanctioned by the NL Govt. on December 17, 2012 and apparently construction also commenced in late 2012. This is a disturbing approach to project management when you consider that detailed project engineering work had reportedly only reached 98% completion at the end of 2014. It is not necessary that detailed engineering be complete before early construction works begins. However, it sure begs the question what detailed engineering was done at the time considering that the project received sanction at the same time as construction began.

At the end of the “reporting period” under review (December 2014-February 2015), the Muskrat Falls Project was already into construction for over two years. Yet, as the EY report states, Nalcor hadn't even put the most fundamental and absolutely vital, fully functional project controls (schedule/cost monitoring and reporting, risk management, forensic analysis of schedule/cost, and much more) processes in place for managing the Project, something that should be implemented for any project long before it is sanctioned. This is unforgivable and in the ‘normal’ project management world, the failure to do this would get people fired.

If project controls are not robustly established on the front-end of a project and for all phases of the project life-cycle, there is no mechanism for properly gathering relevant data, analyzing it and then using it to constructively communicate and manage the schedule and cost of a project. In addition, in the absence of robust project controls processes, the ability to undertake any form of analysis to understand when a project is failing is absent.

Robust project controls processes implemented by highly skilled and experienced personnel are the life blood of project management. Successful performance of a project depends on it; it doesn’t guarantee success, but without it, failure is guaranteed. All of the fabulous contracting strategies, plans for safety, quality, communication, interface management, etc. are of little value without it. In a mega-project, you are on a train that is out of control, it is off the tracks (if it was ever on), it is crashing into a ravine, and you won’t even be aware of it until you hit bottom.

This was the case for the Muskrat Falls Project.

And when a mega-project is off the rails, the very best that can be done is to stop the hemorrhaging; the impacts cannot be reversed. Ask anyone in the mega-project management business; the evidence is well documented.

Based on this long stale-dated EY report, is it any wonder that the Muskrat Falls Project is now massively over budget and the schedule is blown out?

The findings in the EY report are gobsmackingly staggering, and should have shocked the Oversight Committee to the core. Indeed, one has to ask: What was the Oversight Committee doing up to this point? It definitely was not oversight. Apparently, the Oversight Committee hasn’t been doing anything related to oversight since this report either. Take a look at the reports from Muskrat Falls Oversight Committee. These documents read more like a project manager’s progress report than an oversight report.

An oversight report should be addressing oversight…’duh’. I went through these reports looking for a section I would expect to see near the beginning with a title such as ‘Critical Project Concerns’, followed by another section with a title such as ‘Mandatory Requirements to Redress Critical Concerns’ (with deadlines) and then a section with a title such as “Recommendations to the NL Government”.  

Unfortunately, there is nothing like these in any of the Oversight Committee reports. The Oversight Committee apparently only has “Observations” coupled with a smattering of repetitions fed to them by Nalcor or taken from the EY reports. This is not oversight; it is simply another rehashed and badly presented progress report. Even a progress report should flag critical issues.

Clerk of the Executive Council
Chair - MF Oversight Committee
On the Muskrat Falls Oversight Committee website it says the “…Government mandated the committee to provide reliable and transparent oversight so that the public can have confidence in the management of project costs, schedule and risks.” Having independent reviews done by a qualified organization with strong mega-project knowledge is a valuable tool for the Committee. It is questionable whether EY is the right organization for this, but that’s another issue. What is apparent and appalling, however, is that the Muskrat Falls Oversight Committee uses EY as a surrogate for its own mandate and responsibility, rather than exercising its stewardship obligations, providing watchful care and imposing accountability on Nalcor. On what basis can the people of NL have confidence in Nalcor’s management of the Muskrat Falls Project when the Oversight Committee doesn’t even discharge its own obligations?

To some, this may seem like nothing more than an upbraiding of the Oversight Committee by a malcontent. Well, consider the following little nugget from the EY interim report recommendations issued in April 2016.

“Project governance and independent oversight should be re-evaluated by the Provincial Government and strengthened at the Project, Nalcor Board and Provincial Government levels; and

Project reporting should be enhanced to support senior management focus on key risks and issues, to communicate more clearly how key risks are reflected in the forecast and to enable more effective Provincial Government oversight.”

It is worth noting, that while not part of its mandate, EY felt the need to make a seriously stinging indictment of the Oversight Committee. That takes chutzpah, especially from the very consultant that was hired by them. You know it must be really bad for EY to do that. Even the consultant couldn’t stand it any longer. Quite frankly, I wonder if the Oversight Committee even recognized the overt effrontery.

The October 2015 EY report should have also shocked Ed Martin, CEO of Nalcor, and Gilbert Bennett, VP responsible for the project, to the core, and should have gotten both of them fired. Of course, to be shocked implies that Ed Martin and Gilbert Bennett actually understand the role of project management. Clearly, that assumption cannot be made.

Mr. Martin got himself conveniently fired, and replaced by Stan Marshall, retired CEO of Fortis, and there has been much hullabaloo about whether Martin should have been fired for ‘cause’. Not having a properly constituted set of project controls processes (the most critical of project management processes) in place and fully functional on the front-end of the Muskrat Falls Project sure has the look and feel of gross negligence. The evidence is in the October 2015 EY report for all to see.

The April 2016 EY interim report also makes some very disturbing revelations. Consider these excerpts:

The overall conclusion of the Review is that the September 2015 Forecast [Nalcor’s forecast] is not reasonable.”

“The  MFG civil works contract is the highest  dollar  value contract. The deliverables on this contract are required to allow progress on other contracts, e.g. installation and commissioning of the turbines and generators, installation of spillway and intake gates and the balance of plant contract….Progress on this contract is significantly behind the original contract schedule….As at December 2015, the proportion of contract value paid to the contractor is significantly greater than the proportion of the concrete that has been placed. [Emphasis added]
.

“HVdc transmission line contract….In the first nine months of the 32-month contract duration, actual progress has been only 50% of plan….Performance to date and the ongoing risks described above create potential for a multiple-month delay to the contract schedule. This potential delay could be greater than the time contingency included in Nalcor’s Project schedule and so presents a risk to overall Project milestones.” [Emphasis added]

“HVdc convertor stations contract….Nalcor and the contractor are currently forecasting delays to the mechanical completion of the convertor stations, with the Muskrat Falls delay being approximately two months. Mitigation plans are being implemented to maintain the forecast and recover this delay; however, the contractor would be required to more than double its rate of progress to date to maintain the forecast schedule.” [Emphasis added]

“The scope of EY’s review did not include a formal review of the Project governance…HoweverEY has observed that certain elements of governance and reporting arrangements have not been effective in respect of the Project’s cost and schedule forecasts. There is a need to strengthen Project governance and reporting to provide more effective oversight and constructive challenge to Project performance and execution, key decisions and forecasting.” [Emphasis added]

There are many more scary comments from EY in this interim report but it not necessary to prolong the agony. The Muskrat Falls Project started out a mess and has not recovered. In fact, it has gotten worse; far worse. Nalcor’s management, the Oversight Committee, and the NL Govt. have to all share the blame. Unfortunately, it is the people of NL who have to pick up the multi-billion dollar tab.

There is one other point that must be considered. The causes for project failures are well documented. They can generally be summarized under the following headings:
1.    Poor/incompetent leadership at all levels
2.    Inadequate business case to support going ahead with the project
3.    Disregard of project warning signs
4.    Poor planning, inadequate project procedures and processes
5.    Inadequate methodologies/personnel to document and track cost, schedule and risks
6.    Failure to set expectations and manage them
7.    Inappropriate contracting strategy
8.    Inexperienced project management personnel
9.    Inadequate/unreliable cost estimate
10.Inadequate/unreliable schedule
11.Lack of appropriate communication
12.Competing priorities

When you read the two reports from EY you can only conclude that the Muskrat Falls Project fits into every one of these headings. With Ed Martin out of the way, the next person in the queue with the responsibility for establishing the groundwork and delivering the project is Gilbert Bennett. Clearly, he has also failed on every front. Even without the damning EY reports, you just have to ask the question: What the hell is Stan Marshall thinking?

Gilbert Bennett - Executive V-P Nalcor
Gilbert Bennett was a key Nalcor executive responsible for the project and the unmitigated mess it is in? Instead of getting fired by Stan Marshall, he gets promoted to Executive VP. An explanation in the press that he is “good people” doesn’t cut it. He may be “good people” but his inability to plan and deliver a project of this magnitude is now well documented and it has cost the people of NL many billions of dollars. You don’t promote this person, you fire him, and for ‘just cause’.

Stan Marshall is expected to provide his own project update by the end of June. It is obvious his initial steps to realign senior management were serious missteps. You don’t have to read between the lines in the EY reports to quickly understand that the most significant failures on the Muskrat Falls Project lie at the feet of Nalcor’s leadership. In fact, the EY reports scream it. However, in this case the missteps involved just a handful of people.

Now Stan Marshall must deal with a larger (approx. 400 personnel) and also seriously underperforming project management team.

Nalcor originally engaged SNC Lavalin as the Engineering, Procurement & Construction Management (“EPCM”) contractor. It is well documented that engaging an EPCM to manage your project is a recipe for disaster. EPCMs are most often paid on a cost reimbursable basis. Even if they are performing under a lump sum contract (which is rare), the business model of an EPCM is totally inconsistent with the needs of a project owner. An EPCM’s motivation is to make more work for itself. More work requires more bums in the seats and therefore, more profit for the EPCM. It has no interest in delivering a well-managed project. As the saying is well known in the project management world, ‘chaos equals cash’ for an EPCM.

In November 2013, Nalcor did get around to changing the management model to an ‘integrated project management team’ using Nalcor staff, SNC Lavalin personnel and other third-party consultants. At first blush, this looks like a good move. However, the recent EY interim report states that one of the “…main drivers [for cost overruns] reported by Nalcor…” is related to “project management execution”.

So what went wrong?

The model of an integrated project management team is sound in principle. However, management models don’t perform project management or make it successful. Quality personnel working with quality processes and procedures (something clearly lacking at Nalcor from the beginning and from the top down) working as an integrated team are what make the model successful.

In an integrated project management team it is standard procedure to utilize personnel from an organization such as SNC Lavalin and third party consultants. Owner’s often do not have sufficient numbers of the right kinds of personnel to fully staff a large project. However, it is fundamental to the success of this model that there are sufficient and highly experienced personnel to fill all key project management positions from the owner’s organization. With the right people in the right positions, the right behaviours can be driven in those who report to them. This is central to the integrated project management model. And if the owner doesn’t have them, it must hire them. They must be beholden to the owner.

Based on the notation from the EY interim report, one can only conclude that Nalcor overlooked this crucial ingredient. Without this, organizations such as SNC Lavalin and the third party consultants are still left running the show. Ultimately, it ends up being no different than having an EPCM managing the project. Get ready to pay and keep the cash pouring.

Stan Marshall - CEO Nalcor
The importance of strong project management at every level is fundamental. When Stan Marshall delivers his own report at the end of June he will have to demonstrate that he personally understands the problems that have plagued this project from the start, especially internal project controls, appropriate and sufficient owner personnel in the right positions, and the demanding process of oversight.

If Stan Marshall’s response to all of those deep seated problems is as superficial as the executive changes he announced last week, get ready for the final project cost to reach a level that will be well beyond every Newfoundlander’s worst nightmare.
__________________________________________________________________
Editor's Note:
Frank Wright is  a seasoned executive with 25 years’ experience delivering large EPC projects around the globe. He possesses immense strategic and technical knowledge of energy construction project development and management. His international experience includes projects in North Africa, Middle East, South East Asia, South America and Mexico. Frank says that focused leadership and prudent management are the key influences on project success. Frank’s curiosity was piqued when he happened upon the Uncle Gnarley Blog. He decided to take a closer look at the Muskrat Falls Project and offered to share his views.