Consider a group (Formula presented) and construct its power graph, whose vertex set consists of the elements of (Formula presented). Two distinct vertices (elements) are adjacent in the graph if and only if one element can be expressed as an integral power of the other. In this paper, we improve the bounds of the spectral radius of the power graphs of the cyclic group (Formula presented), the dihedral group (Formula presented), and the dicyclic group (Formula presented). For (Formula presented) the power graph of the cyclic group (Formula presented) is not a complete multipartite graph. We obtain lower bounds on the second largest eigenvalue in terms of the clique number. In some cases, we find the bounds are exact. Finally, we establish bounds on the distance spectral radius of the power graphs of the same groups.