Embedded Super-Resolution Acceleration Architectures for Real-Time Recognition-Oriented Vision Systems

Abstract

The concept of super-resolution (SR) has risen to a significant role in the pre-processing section of contemporary vision pipelines and has found application with recognition oriented applications that are characterized by limited sensing capabilities, like low-resolution images, bandwidth, and unfavorable environments. The latest and the most advanced SR algorithms, particularly those based on deep learning, are computationally expensive and inapplicable to the real-time scenarios on embedded systems because they are highly limited with regard to power, memory, and hardware. The paper is a recognition-based embedded super-resolution acceleration architecture that seeks to maximize downstream visual recognition performance and achieve real-time and resource constraints. At this contrast to traditional methods of SR, which nearly solely focus on pixel-wise reconstructive fidelity, the proposed one opts to use a task-conscious method of SR, which exclusively focuses on preserving discriminative features of step-wise recognition tasks. A loosely piped highly parallel architecture is then created with a fixed-point arithmetic, modular processing units, and memory-aware dataflow optimization to reduce the latency and external memory access. The presented architecture is applied to an embedded platform based on FPGA and assessed in regard to throughput, energy efficiency, used resources, and recognition accuracy. It is experimentally shown that the proposed accelerator can reach real time processing with significant frame rate and energy efficiency advantages over CPU-based and GPU-based implementations, and that it is able to maintain recognition accuracy over benchmark data sets with consistent performance. These findings confirm that recognition-focused super-resolution coupled with a software-efficient acceleration architecture is an effective and realistic solution to the embedded vision system employed in autonomous sensing, surveillance and edge case intelligence systems.