Deploying Sparse Ternary Neural Networks on edge devices is in the areas of computational efficiency and energy optimization is a challenging task. This work presents a new FPGA-based accelerator integrating reinforcement learning and neural architecture search to dynamically optimize Sparse Ternary Neural Networks (Sparse TNN) for real-time applications. The design adopts adaptive pruning and quantization techniques for computational complexity and power consumption with the desired accuracy. Experimental evaluation on the Xilinx ZCU102 platform achieves up to 16.46 × speedup compared to dense models at less than 1% accuracy loss and achieves state-of-the-art performance on benchmarks such as Google Net and MobileNetV2. This work holds promise for resource-constrained high-throughput applications, bringing FPGA-based deep learning closer to efficiency and scalability.