AI-Enabled Hardware Acceleration Architecture for Real-Time Inference in Resource-Constrained Edge AI Systems

Abstract

The accelerated use of edge artificial intelligence (AI) in sensors in areas like smart-surveillance, health-surveillance, and autonomous systems has raised critical issues of high-inference-latencies, excessive, and unlimited-power usage, and restricted computational academic capabilities of edge machines. This paper attempts to counter these issues by introducing a hardware acceleration architecture powered by AI to implement real-time inference in resource-constrained systems. The suggested method combines a database of a lightweight convolutional neural network (CNN), particularly optimised to be deployed on the edge, with superior model compression algorithms such as quantization and structured pruning. The adaptable VLSI-backed hardware design is developed with a view to effectively map the optimised model onto parallel processing components, and hence, access fewer memories and offer increased computational throughput. The architecture also takes advantage of pipelined processing and reuse of the data to minimise even more the latency and energy consumption. Experimental results show that there is a great improvement in the degree of experimental evaluation as the objectives concluded by a great power consumption as well as the inference latency and also achieved a competitive accuracy level. The relative performance compared to conventional CPU and GPU-based implementations shows that the proposed design is superior in both use of energy and real-time performance. The main value of the work is the fact that the algorithms and hardware are designed to be co-designed, which offers a solution based on efficient scaling and energy consumption to the upcoming generations of edge AI.