Multi-Armed Bandits for Low-Complexity Beam Management in High-Speed mmWave IoT

Beam selection in millimeter-wave (mmWave) Internet of Things (IoT) networks is particularly challenging due to frequent blockages. Multi-Armed Bandit (MAB) algorithms for beam selection offer a promising approach to reduce overhead compared to exhaustive or hierarchical beam searches. With various MAB algorithms available, however, it is unclear which one is best suited for mmWave IoT beam selection. In this paper, we compare non-stationary and contextual MAB algorithms for beam selection in high-speed environments with high blockage probabilities. These lightweight algorithms each exploit a different aspect of the wireless channel – making it hard to evaluate under which conditions one might outperform the other. We assess these algorithms over a range of parameters to show that those based on a contextual Upper Confidence Bound (UCB) approach are particularly well-suited for highly dynamic environments. Our numerical evaluation offers valuable insights into designing adaptable beam selection algorithms for high-speed IoT scenarios.