Thursday, 6 October 2016

DARKNL: IT WAS INCOMPETENCE STUPID

When CBC Morning Show Host Anthony Germain invited this blogger to come on the Show and discuss the Public Utilities Board's release of its September 29th Report entitled "Supply Issues and Power Outages On The Island Interconnected System"  - on the root cause of DARKNL and other major outages on the Island transmission system - I could not say, no! Nalcor, the Tories, and most recently Premier Ball, want to hold onto the narrative that covers everyone's derierre - except the public's. That's the old "aging assets" excuse. Sorry folks, that won't do. 

Germain is a demanding interview; he reads this stuff. I like to prepare by writing out answers to possible questions. I usually end up with more material than the interview can afford - which I suppose is not a bad thing. Imagine if I ran out of things to say!  

Still, I thought that Uncle Gnarley readers might want to see a little more detail on what the PUB said in its Report - which would have been better titled "Who Is Responsible for DARKNL?"  So, what Germain asked and what he didn't - you get to read it all. Oh joy!

Has anyone learned anything from two years of investigation by the Liberty Consulting Group, and from the PUB’s Report? Does it matter that DARKNL was not the fault of aging infrastructure? Did Liberty and the PUB convince anyone that the power blackouts resulted from poor management practices and the "operating culture" at Nalcor and NL Hydro? 

With the lights back on and the heat cranked up this may not be the sexiest topic in the morning - any morning - and Nalcor is counting on that. 

So, for that reason, and because the problems at Hydro and Nalcor generally remain a pressing public policy concern, thanks again for the invite Mr. Germain. Des Sullivan

Notes - Interview with CBC Morning Show Host Anthony Germain
October 5, 2016

1) When the PUB says there are issues with regards to hydro's 'generation asset   
    management'... what does that mean?

Essentially it means that the outages - which occurred in January 2014 - were caused by Hydro’s failure to effectively plan for demand on the system and a failure to manage its physical assets - like Holyrood and Hardwood and other thermal assets.

Back in January, 2014 - when the lights went out - we were told by Nalcor and Hydro management that the problems were related to aging assets. That wasn't the case. And for a long time the public have been kept in the dark about what really happened. Based upon a lengthy investigation by the Liberty Group and follow-up by them, the PUB is able to tell us the truth. According to the PUB:

- the rotating outages which began on January 2, 2014 were caused by a 
  shortage of power to meet demand;  

- then, it states, the extended outages, which began on January 4, 2014 were 
  associated with transmission system failures and  equipment failures - In a 
  nutshell, the PUB states those failures were caused by deficient   
  management and deficient maintenance.

In the PUB's view Hydro’s management "did not meet the standard of generally accepted sound public utility practice."

2) What, if anything, has changed or improved at NL Hydro?

The PUB Report states that "A great deal of work has been done since the January 2014 outages" .... but it says that with a big caveat.  

It states: "Hydro has not fully addressed the recommendations set out by Liberty in its March 2015 report"....

More than two years later, the PUB says it still has concerns about Hydro’s transmission and generation and what it calls asset management execution, operating culture and the adequacy and reliability of supply. For example:

- Liberty notes ongoing problems with Hydro’s thermal fleet, Holyrood (2x170 MW, 1x150 MW), Hardwoods C.T. (50 MW), Stephenville C.T. (50MW).

- The Consultancy found Hydro’s load forecasting procedures outdated and likely to underestimate peaks in demand compared to approaches used by large utilities in North America…Liberty takes a more conservative approach demand forecasting…..

BUT: The PUB says it is not satisfied that Hydro has made real progress in addressing the systemic issues that contributed to the outages in January 2013, January 2014, and March 2015 and therefore will review this matter as part of Phase Two of this investigation."

3) The report also says there are problems with NL Hydro's "Operating Culture"... what does THAT mean?

The PUB called it a systemic issue - Operating culture relates to Hydro’s management practices, its approach to mitigating problems, how it strives to meet the Corporation’s mandate.  On a more specific level “operating culture” is about Hydro’s diligence about its maintenance and operations programs, quality control and outage mitigation procedures,  the adequacy of its training systems – and ultimately the diligence with which it pursues its role as a public utility.

Concern over Hydro's operating culture led Liberty’s Group's president - who gave testimony during Hydro’s general rate application in 2015 - to question whether there had been sustained and effective change at Hydro.

Furthermore, Liberty’s Phase Two report confirmed that “many of Hydro’s past issues, including the major outages in 2013 and 2014, and 2015, were as much or more due to organizational issues as they were due to system inadequacies.”
Liberty said it is essential that Hydro develop a plan to address those issues.

The Liberty Group also administered Hydro a number of tests regarding whether it made prudent decisions in the course of managing the Company. It was following this examination that the PUB refused to permit Hydro to levy what it deemed to $26.9 million in unnecessary and avoidable costs.

************************************
P.54 of PUB Report

Hydro’s deferral of preventative maintenance was imprudent ....
weakness in Hydro’s supply planning….

Hydro failed to take action in relation to black start capability at Holyrood until directed by the Board in October 2013, even though Hydro had identified it as critical.

The widespread outage in January 2013 was associated with Hydro’s failure to conduct its own testing to demonstrate that the lube oil system worked as intended and to verify that the required test was performed by its contractor and Hydro had still not addressed the ongoing risk of common mode failure of the lube oil system.

****************************************************
4) QUOTE: The board believes that Hydro let down its customers and the people 
    of this province who collectively will bear the significant financial burdens 
    associated with these failures…What ARE those costs?

When there is an outage of more than a few minutes, everyone pays...

- school closures resulting in additional day care costs
- business closures
- loss of income, lost productivity

There are longer term costs, too:
- frequent or lengthy outages cause a loss of confidence which could affect investment 
  in the province

- people are motivated to purchase alternative sources of heat and electricity 
  i.e wood stoves and generators – robbing purchasing power for other   
  needs…  

- when any system is allowed to deteriorate it often costs more to get it back to 
  a high standard than if it were well maintained in the first place.....

…and there's only one group of ratepayers to pay for all of it….

5) The PUB concludes that the flaws on which it reported "pose a continuing threat to the adequate and reliable supply of power on the Island..." Is the PUB suddenly relevant again... why are we seeing this kind of muscle-flexing now?

The PUB has always been relevant; it is an important institution in the province...as the regulator of electricity transmission system and for its role in levying rates which meet the lowest cost requirements of the Electrical Control Act under which it operates. 

Following DARKNL it ordered this investigation. This Report is its first specifically in response to investigations and three Reports – so far- by the Liberty Consulting Group over a two year period.

In fact, it is the only institution that has had the courage not to comply with Government's expressed wishes...and I'm thinking of Muskrat here, which it refused to endorse.

The public will hear more as the PUB gets into Public Hearings regarding Phase II of its investigation which will largely deal with reliability and security issues in the post-Muskrat Falls era.

6) Concluding comment:

The public should demand accountability from NL Hydro regarding the PUB’s damming indictment of its operations. The Provincial Government, including the Minister of Natural Resources, should end their cone of silence – start governing, begin responding - describe for the public the actions Hydro is taking respond to the PUB's concerns about it laxidaiscal approach to utility management - prove that corrective action is being taken to prevent a repeat of the worst and most costly electicity failures the Province has ever experienced.

Premier Ball did tell reporters following the Liberal fundraiser last week that he isn't as concerned about DARKNL as he was a few months ago. 

The Telegram reported the Premier saying "with the province's aging electricity-producing infrastructure....Muskrat has been even more problematic with ....the delay in getting to reliable power". (my emphasis).

Imagine if the Premier had read the PUB's Report....and discovered that the problem is NOT THE AGING ASSETS but Hydro's managers and the poor "management culture" they have inculcated into that Utility.

And if only he knew it's the same "culture" that is managing Muskrat Falls!

15 comments:

  1. Your conclusion is absolutely correct. The premier is not capable of understanding the electrical problems we face or the skills needed to run this province...That simple...

    ReplyDelete
  2. An insightful read. Thanks for posting.

    ReplyDelete
  3. Great article. I'm one of those who do not have faith in Nalcor to provide reliable and consistent electrical generation, either now or after Muskrat comes on stream. For that reason, I will always have a wood stove and back up generator.

    ReplyDelete
  4. Your comment re Item 1. --- that the causes of the outages were a shortage of power.

    An outage is effectively just that --- a shortage of power. So that is like saying the outages were caused by outages.

    To say a 'shortage of power' leaves an average reader to think that the cause was a need for more generation capacity. WHen the truth is that if Nalcor had properly maintained the generation capacity that we already had, there would have been no need for rotating outages (and Stan Marshall has essentially confirmed that by stating that we are not expected to reach Nalcor's forecast demand at sanction until 2036).

    The problem was and is poor maintenance . Those in charge said they never heard of preventative/maintenance management! I am and never was in the equipment maintenance business, but I became familiar with the need for an effective maintenance management system as far back as the mid 1980's when I was part of Coast Guard's transformation of the province's major lighthouses to automatic/remote monitored operation (and I am no engineer).

    The whole mess is disgraceful abusive to those that government and Nalcor is supposed to be there to serve.

    Overall, a good summary and helpful Des.

    Maurice Adams, Paradise

    ReplyDelete
    Replies
    1. Maurice: Inadequate generation relative to demand will cause an outage; Liberty made the assessment repeatedly that Hydro needed more generation assets and that it used poor forecasting techniques to calculate the probably of reaching peak demand. The new 125 MW generator at Holyrood will help take care of that problem as will the drop in demand...but only if Hydro maintains the other generation assets regarding which Liberty and the PUB were very critical of their management and maintenance practices.

      Delete
    2. Yes, I agree. But Nalcor's existing firm capacity was already several hundred MWs more than the peak demand at the time. Hence, if our already existing firm capacity had been properly maintained no rotating outages would have been necessary.

      Liberty's view was and is that Nalcor's buffer (the capacity over and above our forecast peak demand) was always lower than it should have been. Extra/unneeded capacity is an unnecessary cost to the ratepayers (MF for example)and NL Hydro's historical practice was to keep the buffer at a lower level than Liberty suggests, but well maintained.

      The problem was that Nalcor tried to have both was -- lower buffer and very poor maintenance.

      Hence, we are where we are.

      Properly maintained, we would not have even needed the $110 million combustion turbine. Maurice

      Delete
    3. Liberty is saying we likely need more capacity for this coming winter, Maybe another 60 MW, which time will not allow procurement. The reason is that our thermal generation units are not highly reliable, even of there is good maintenance.Also P90 versus P50 forecasting allows for extreme weather with very cold temperature and high winds,and therefore more capacity requirement, which the power companies did not allow for in the past. On the other hand, more efficient heating could reduce peak demand to avoid some additional gas turbine addition, but this demand management option has been avoided, and is an incremental option taking some years to achieve the goal.Another instance of incompetence.
      On average, every house with efficient heating(mini-split heatpumps) can knock 3 kw off the peak. 20,000 houses, 10 or 15 percent of all houses, so equipped could save the cost of a 60 MW gas turbine (plus the saving of fuel to run them). In 2011, Nova Scotia installed 20,000 units in one year.Did anyone see Nfld Power yesterday on NTV citing the benefits of these! The first of these type were installed in Montreal in 1992! So we are really being kept informed by Take Charge group, as Toni Marie Wiseman asks Nfld Power engineer Wayne Upshall.....What is an heatpump!!!!! And of course Upshall says one unit of heat in gives two units of heat out. In fact one unit in gives on average 3 units out, and an average of 3.5 is possible. Guess they really just hate those heatpumps, as they reduce revenue so much! Demand management has been suggested by Liberty.
      Winston Adams

      Delete
    4. The aging asset "argument" could have been remedied if the PCs committed to decommissioning Holyrood Thermal all together instead of using HRT as the ONLY alternative to Muskrat Falls. When PUB was forced with limited terms of reference they were expected to just rubber stamp MF, instead they rightfully endorsed neither option.
      Nalcor's demand forecasts are 2 decades too early (did they even consult with Power for MF?) fabricated to justify Danny's retirement gift.
      Energy efficiency wasn't part of the Muskrat Plan, in fact they need demand to skyrocket with their take or pay PPA.
      Clyde Wells attempt to privatize Hydro and Roger Grimes HQ-SNC-Ont joint venture look like absolute gems compared to Boondoggle Falls.
      Nalcor's BoD were few in number and severely lacking in utility/power expertise - #DarkNL happened under their astute watch NO WAY! /S

      Delete
  5. What I find most interesting in the PUB report is on page 11, dealing with the cause of Jan 2013 outage that resulted in a loss of 700 MW and affecting 175,000 customers. It states the severe winter storm caused salt contamination that triggered the outages,and then a cascade of other problems.
    For the Muskrat Falls transmission line, Nalcor stated salt was not a contaminant. That is false.
    I have raised the issue of salt contamination in past comments, and has been surprised that, if I recall correctly, neither Liberty or the PUB has previously acknowledged the part played by the salt contamination at the Holyrood station and adjacent lines. This cause would be known by Hydro engineers within 24 hours, and was documented in filed appendix of reports of the events following the outage.I was impressed by that early Hydro analysis, being similar to work I did with Hydro in the 1970s. Yet it seemed to be ignored up to now. I believe Amec later filed a report saying this could happen again at Holyrood as a one in 10 year event.
    It seems to me that, despite all the evidence of mismanagement and incompetence of Hydro, they were not anxious to make known to the public the details of the salt contamination issue, even though it would serve as some defence that it was not all incompetence. I can only suspect that they did not want to bring attention to the risk of reliability issues for Muskrat as to salt contamination for the Great northern Peninsula.
    I suspect rater than a one in 10 year event, salt contamination outages could be a yearly event for that line. This Holyrood outage occurred within a month of the sanction of Muskrat, and should have served as a wake up call on Muskrat reliability. Now the PUB cites the salt problem 3.5 years later, and this will be subject to further study in Phase 2 which will be completed in 2017. Liberty has already given notice of poor reliability expected. Salt issues are not unusual, but are often resolved as temporary loss of a line for a second, but can sometimes endure for hours. The Jan 2013 salt problem lasted 6 or 7 hours.
    So need we spend 12 billion or more for a unreliable power system from Muskrat, as to the likely problem of salt contamination alone, when past experience had already identified this problem. We build first, deny reliability issues, and deal with it once in operation. This is a reality of transmission lines in coastal regions, worsened by high winds.The lines on the GNP cannot be moved inland enough to solve the problem. And this line is to serve the Avalon as the main source of power. I`m with Liberty, reliability of the MF transmission line will be worse than stated by Nalcor. As a homeowner invest in a backup heat system and power generator and be vary of the high winds on the GNP. A boondoggle indeed.

    Winston Adams

    ReplyDelete
    Replies
    1. Mr. Adams, the HVDC grounding lines will also be prone to salt contamination. I believe that Nalcor will have to implement a washing program for these lines as other utilities have done where salt, or some other contaminant, is an issue. That cost was possibly not included in the O&M estimates for the HVDC system.

      I do not know what the statistics for outages caused by salt contamination are for this province but I was surprised to read in the early reports that it was what caused the January 2014 events and that there doesn't seem to be a program here to deal with it before outages occur.

      Cheers.

      Delete
    2. I incorrectly typed 2014 there, apologies. Salt was the cause of the 2013 outage.

      Delete
    3. Yes, the HVDC grounding lines is another reliability issue. Liberty has identified grounding lines as less reliable, for a variety of reasons, than Nalcor suggests. The readers of this blog are aware from Liberty than MF transmission will be unreliable to the extent that thermal generation backup will be needed for the Avalon. But the details of the reliability issues have not been put in concise plain language.
      Having worked at Controls and Protection for Hydro in the past I am aware than the GNP area of the province has salt contamination issues about 10 times worse than most other areas in the province. While most of this is 66 kv, higher voltage AC and DC would also be subject to salt issues as to reliability. DC being steady state voltage can be more of a problem than AC where the voltage has a waveform that has lower voltage part of the cycle. I have no experience with flashovers on DC, but I understand salt creates reliability issues on DC. You seem to have some expertise on power systems.... can you clarify your experience, as few with technical knowledge want to comment.
      Winston Adams

      Delete
    4. I'm an electrical engineer with experience in generator controls, protections, system stability, and modelling.

      Delete
    5. Expected you might be an electrical engineer, what is your general view of MF, expected reliability of power to the Avalon, costs, vs the Isolated option(customer efficiency, wind, island hydro). DO you think it wise to stop or continue to completion. Do you think forecast for demand will be as per recent reduction update, or go lower from efficient heating installations, especially when prices for energy rise.Have you experience with GIC issues, how this might impact the DC line(as I have not seen Liberty address this at all).
      WA

      Delete
  6. I can see a washing program for salt on terminal station equipment, and have heard that Nfld power sometimes do that. But for the lines on the GNP, which are not easily for access in winter, it seems maybe impossible. Also,while the PUB says the 2013 outage from salt in Holyrood was during severe winter weather, I recall checking wind conditions and wind was only moderate and temperatures just about the freezing point, if my memory is correct. I would not call it a severe winter storm. If someone wish to verify this. Moreover, this continuous salt contamination prevented a restart of Holyrood from Hardwoods near St John`s. Holyroods own start gas turbine was unsafe and not operational. Hydro could not image a situation whereby restart could not be accomplished from Hardwoods. This was incompetence, and it was the salt contamination on the lines that prevented this restart. Holyroods own restart gas turbine was an essential component which would allow restart regardless of salt on the adjacent connecting lines of the grid. This high risk dependance on Hardwoods for restart was what Liberty calls imprudence, and Hydro had not even advised the PUB of this risky situation. Salt contamination on the GNP is a very serious problem and well documented. It is therefore rather amazing that in the Manitoba Hydro review they accepted Nalcors statement that salt was not a contaminant. Being 4 years ago, I need to double check this sometime, and wonder if I dreamed that statement, it sees so remarkable. I do know that, at my suggestion last year, Danny D has since asked the question formally, and Nfld Hydro now states that salt is a contaminant for the DC line.

    Winston

    ReplyDelete