Machine Learning-Assisted Routing Optimization for Scalable Network-on-Chip Architectures

Abstract

Network-on-Chip (NoC) systems have developed as a scalable communication infrastructure to the modern multicore and manycore systems as a solution to the shortcomings of conventional bus based interconnects. But growing network size and heterogeneity of traffic in a network pose a great challenge in the form of congestion, high latency, low throughput and high power consumption. Traditional deterministic routing algorithms are not usually capable of dynamically changing their routes in response to traffic variations, which constrains the overall system performance and scalability. In this paper, the authors suggest an optimization routing problem that is solved using machine learning in the framework of reinforcement learning, and in particular, through the application of a technique like Q-learning and Deep Q-Network (DQN). Under the proposed framework, the router is an intelligent agent, which learns the best routing decisions in interaction with the network environment. The model takes into account the current states of the network such as the occupancy of buffers and the congestion level to choose the efficient paths of transmitting packets. The methodology will be to train the RL agent through a reward-driven mechanism that will reduce delay and use of overcrowded routes. Comprehensive simulations are performed in different traffic models and network scales to test performance. Experimental evidence shows that there are substantial improvements in terms of decreased average latency, increased throughput and lower power usage than the traditional routing methods. The scheme is also highly scalable as the network size grows. On the whole, the article demonstrates the opportunities of reinforcement learning in facilitating adaptive, efficient, and scalable routing schemes of next-generation NoC architectures.