Offline Reinforcement Learning for Bandwidth Estimation in RTC Using a Fast Actor and Not-So-Furious Critic

🎉 Runner-Up Solution in 2nd Bandwidth Estimation Challenge 🎉

Published In: MMSys '24: Proceedings of the 15th ACM Multimedia Systems Conference - PaperLink

Authors: Ekrem Çetinkaya (Ozyegin University), Ahmet Pehlivanoglu (Ozyegin University), Ihsan U. Ayten (Ozyegin University), Basar Yumakogullari (Ozyegin University), Mehmet E. Ozgun (Ozyegin University) , Yigit K. Erinc (Technical University of Munich), Enes Deniz (Technical University of Munich), Ali C Begen (Ozyegin University)

Abstract: The increasing demand for real-time communication (RTC) applications necessitates robust and reliable systems. Seamless media delivery depends on an accurate assessment of the network conditions, with bandwidth estimation (BWE) being crucial for maintaining system reliability and achieving good quality of experience (QoE) for the users. BWE poses a significant challenge due to dynamic network conditions, limited information availability and computational complexity. The Second Bandwidth Estimation Challenge, organized within ACM MMSys 2024, aims to enhance RTC user QoE by developing a deep learning-based bandwidth estimator using offline reinforcement learning. This paper presents our solution, ranked second in the grand challenge. This solution employs an actor-critic approach to achieve accurate real-time BWE by relying solely on observed network statistics. Due to the offline setting of the challenge, the critic network is trained separately from the actor network to estimate the action quality without interacting with the real environment. Furthermore, the quality prediction by the critic is adjusted by a predefined conservation factor to address overshooting the bandwidth values. The solution's source code is publicly available at https://github.com/streaming-university/FARC.

Keywords: Real-time communication, QoE, WebRTC

@inproceedings{ccetinkaya2024offline,
  title={Offline Reinforcement Learning for Bandwidth Estimation in RTC Using a Fast Actor and Not-So-Furious Critic},
  author={{\c{C}}etinkaya, Ekrem and Pehlivanoglu, Ahmet and Ayten, Ihsan U and Yumakogullari, Basar and Ozgun, Mehmet E and Erinc, Yigit K and Deniz, Enes and Begen, Ali C},
  booktitle={Proceedings of the 15th ACM Multimedia Systems Conference},
  pages={388--393},
  year={2024}
}