TY - JOUR
T1 - Runtime Adaptation in Wireless Sensor Nodes Using Structured Learning
AU - Sapio, Adrian
AU - Bhattacharyya, Shuvra S.
AU - Wolf, Marilyn
N1 - Funding Information:
This research was sponsored in part by the US National Science Foundation (CNS1514425 and CNS151304). Authors’ addresses: A. Sapio, 3539 Shady Pines Ln, Frederick, MD 21704; email: [email protected]; S. S. Bhattacharyya, Dept. of ECE, University of Maryland, 8223 Paint Branch Dr., College Park, MD 20742; M. Wolf, Dept. of CSE, University of Nebraska Lincoln, 256 Avery Hall, Lincoln NE 68588-0115. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. © 2020 Association for Computing Machinery. 2378-962X/2020/07-ART40 $15.00 https://doi.org/10.1145/3372153
Publisher Copyright:
© 2020 ACM.
Copyright:
Copyright 2020 Elsevier B.V., All rights reserved.
PY - 2020/8
Y1 - 2020/8
N2 - Markov Decision Processes (MDPs) provide important capabilities for facilitating the dynamic adaptation and self-optimization of cyber physical systems at runtime. In recent years, this has primarily taken the form of Reinforcement Learning (RL) techniques that eliminate some MDP components for the purpose of reducing computational requirements. In this work, we show that recent advancements in Compact MDP Models (CMMs) provide sufficient cause to question this trend when designing wireless sensor network nodes. In this work, a novel CMM-based approach to designing self-aware wireless sensor nodes is presented and compared to Q-Learning, a popular RL technique. We show that a certain class of CPS nodes is not well served by RL methods and contrast RL versus CMM methods in this context. Through both simulation and a prototype implementation, we demonstrate that CMM methods can provide significantly better runtime adaptation performance relative to Q-Learning, with comparable resource requirements.
AB - Markov Decision Processes (MDPs) provide important capabilities for facilitating the dynamic adaptation and self-optimization of cyber physical systems at runtime. In recent years, this has primarily taken the form of Reinforcement Learning (RL) techniques that eliminate some MDP components for the purpose of reducing computational requirements. In this work, we show that recent advancements in Compact MDP Models (CMMs) provide sufficient cause to question this trend when designing wireless sensor network nodes. In this work, a novel CMM-based approach to designing self-aware wireless sensor nodes is presented and compared to Q-Learning, a popular RL technique. We show that a certain class of CPS nodes is not well served by RL methods and contrast RL versus CMM methods in this context. Through both simulation and a prototype implementation, we demonstrate that CMM methods can provide significantly better runtime adaptation performance relative to Q-Learning, with comparable resource requirements.
KW - adaptation
KW - LTE-M
KW - Markov decision processes
KW - reinforcement learning
KW - self-awareness
U2 - 10.1145/3372153
DO - 10.1145/3372153
M3 - Article
AN - SCOPUS:85095979541
SN - 2378-962X
VL - 4
JO - ACM Transactions on Cyber-Physical Systems
JF - ACM Transactions on Cyber-Physical Systems
IS - 4
M1 - 40
ER -