Supervisory Parameter Adjustment for Distribution Energy Storage (SPADES)
Energy Storage Systems (ESS), such as electric batteries, are increasingly becomming critical components of the electric power system. As these devices begin to provide grid services, it is critical to ensure that they are resistant to cyber attacks and cannot be exploited as a mechanism to disrupt the operation of the power grid. Tools from nonlinear and optimal control theory can be utilized to develop control policies to re-dispatch setpoints in both the ESS and the surrounding electric grid infrastructure to mitigate the effect of cyber attacks.
Characterize the interaction ESS control systems and Distributed Energy Resource (DER) smart inverter functions on electric grid voltage stability.
Develop a supervisory control system (reinforcement learning-based) that will identify and autonomously update control parameters of the power electronic control systems within ESS devices.
Validate control approaches via Hardware-In-the-Loop (HIL) testing at the LBNL FLEXGRID facility.
Integrate the defensive reinforcement learning-based agent into the NRECA Open Modeling Framework (OMF) simulation tool.
The Supervisory Parameter Adjustment for Distribution Energy Storage (SPADES) project will develop the methodology and tools allowing Energy Storage Systems (ESS) to automatically reconfigure themselves to counteract cyberattacks against both the ESS control system directly and indirectly through the electric distribution grid. This research will begin with an effort to analyze the stability of both ESS control systems and the interaction of the ESS control system and the electric grid, to determine what parameters an attacker would change if a given device (or multiple devices) were to be compromised. Then, the research team will develop a supervisory control framework that utilizes adaptive control and reinforcement learning techniques to adjust ESS control system parameters and ESS active and reactive power injections to actively defend against a variety of cyberattacks. The supervisory control framework will be validated in Hardware-in-the-Loop (HIL) experiments where an independent red team will attempt to alter control parameters of an ESS to prevent the device from providing grid services. The reinforcement learning defensive algorithms will be integrated into the National Rural Electric Cooperative Association (NRECA) Open Modeling Framework (OMF), thereby allowing defensive strategies to be tailored on a utility specific basis. The major outcomes of this project will be the tools to isolate the component of the ESS control system that has been compromised during a cyberattack as well as policies for changing the control parameters of ESS to mitigate a wide variety of cyberattacks on both the ESS device itself and the electric distribution grid.
This project is supported by the U.S. Department of Energy’s Cybersecurity for Energy Delivery Systems (CEDS) program.
Daniel Arnold (PI; LBNL)
Publications resulting from this project:
Losada Carreno, I., Saha, S. S., Scaglione, A., Arnold, D., Ngo, S., and Roberts, C., “Log(v) 3LPF: A Linear Power Flow Formulation for Unbalanced Three-Phase Distribution Systems,” IEEE Transactions on Power Systems, accepted.
Arnold, D., Saha, S. S., Ngo, S., Roberts, C., Scaglione, A., Johnson, N., Peisert, S., and Pinney, D., “Adaptive Control of Distributed Energy Resources for Distribution Grid Voltage Stability,” IEEE Transactions on Power Systems, early access, March 2022, pp. 1-1.
Arnold, D., Ngo, S., Roberts, C., Chen, Y., Scaglione, A., and Peisert, S., “Adam-based Augmented Random Search for Control Policies for Distributed Energy Resource Cyber Attack Mitigation,” American Control Conference, 2022, accepted, available: http://arxiv.org/abs/2201.11825
Roberts, C., Uros M., Arnold, D., and Callaway, D. S., “Malicious Control of an Active Load in an Islanded Mixed-Source Microgrid.” In 2021 IEEE Madrid PowerTech, pp. 1-6. IEEE, 2021.
Roberts, C., Daniel Lara, J., Henriquez-Auba, R., Kameshwar Poolla, B., and Callaway, D. S., “Grid-Coupled Dynamic Response of Battery-Driven Voltage Source Converters.” In 2020 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids (SmartGridComm)