We present a two-dimensional GPU-accelerated implementation of the lattice Boltzmann method (LBM) using CUDA, tailored for computational fluid dynamics applications. The numerical framework employs the D2Q9 velocity stencil combined with a regularized Bhatnagar-Gross-Krook collision operator. Our primary objective is the systematic evaluation and optimization of distinct LBM formulations through detailed comparisons between second-order and high-order regularizations, extending the analysis to include implementations up to fourth-order moments. Such high-order regularizations are instrumental in enhancing numerical stability and accuracy. Additionally, the impact of different boundary condition approaches is rigorously assessed through comparative studies between regularized boundary conditions and incompressible regularized boundary conditions. The computational performance and numerical accuracy of each approach are investigated using the classical lid-driven cavity flow configuration as a standard benchmark problem. Exploiting the computational advantages inherent in two-dimensional simulations, we utilize high-resolution grids consisting of 1024, 2048, and 4096 lattice points. These simulations are conducted across a comprehensive range of Reynolds numbers (3,200, 10,000, 50,000, and 100,000), thus demonstrating the robustness of the solver and capability in accurately resolving complex flow features typical of highly inertial regimes.