Abid and Colleagues Investigate Minimization of Waiting Delay in IoT Applications

As pointed out in a new study by the TSYS School's Mohamed Riduan Abid and his University of Montreal coauthors Maad Ebrahim and Abdelhakim Senhaji Hafid, Fog computing - a technology that extends cloud computing and services to the edge of an enterprise's network, thus allowing data, applications, and other resources to be moved closer to end users - has emerged as a promising paradigm to address the challenges of processing and managing data generated by the Internet of Things. Load balancing - a process that distributes traffic and workloads to ensure that no single server or machine is under-loaded, overloaded, or idle - plays a crucial role in Fog computing environments to optimize overall system performance. It requires efficient resource allocation to improve resource utilization, minimize latency, and enhance the quality of service for end-users. 

     Abid's new study seeks to improve the performance of privacy-aware Reinforcement Learning agents (i.e., agents that complete tasks within an uncertain environment) that optimize the execution delay of IoT applications by minimizing the waiting delay. To maintain privacy, these agents optimize the waiting delay by minimizing the change in the number of queued requests in the whole system.  According to Abid, "Besides improving the performance of these agents, [our paper] propose[s] a lifelong learning framework for these agents, where . . . models are used during deployment to minimize action delay . . . To improve the performance, minimize the training cost, and adapt the agents to those changes, we explore the application of Transfer Learning." 

     Transfer Learning transfers the knowledge acquired from a source domain and applies it to a target domain, enabling the reuse of learned policies and experiences. Transfer Learning can be also used to pre-train the agent in simulation before fine-tuning it in the real environment, a process that significantly reduces failure probability compared to learning from scratch in the real environment. According to Abid, "To our knowledge, there are no existing efforts in the literature that use Transfer Learning to address lifelong learning for Reinforcement Learning-based Fog load balancing. This is one of the main obstacles in deploying Reinforcement Learning load balancing solutions in Fog systems." "In future work, we will study the effect of the number of training steps . . . on achieving more consistent performance using Full agent Transfer Learning. Having consistent performance is vital to provide semi-deterministic outcomes for end users in real-world environments, which is often hard to achieve using machine learning approaches . . . [W]e can explore the practical implementation of our lifelong learning approach in real-world Internet of Things applications, aiming to bridge the gap between theoretical advancements and practical deployment," Abid added.