Alzheimer’s disease (AD) is a growing global concern due to its increasing prevalence and lack of effective treatments. In this study, we employed computational methods to discover potential glycogen synthase kinase 3 beta (GSK3β) inhibitors, a key therapeutic target in AD. Using a library of compounds from Rosmarinus officinalis L. (Rosemary), we performed homology modeling, virtual screening, drug-likeness analysis, toxicity prediction, molecular dynamics (MD) simulations, and Super_MM-PBSA analysis to identify novel hit molecules. 32 lead compounds were initially selected based on binding affinity to GSK3β. Of these, 17 compounds adhered to Lipinski’s Rule of Five (RO5) and ADME properties. Toxicity analysis further narrowed the selection to four non-toxic compounds—Taraxasterol, Pomolic Acid, Nepetoidin A, and 23-Hydroxybetulinic Acid—which showed no mutagenic or tumorigenic effects. MD simulations and binding free energy calculations revealed significantly favorable binding energies for these compounds, with values of − 150.010 kJ/mol, − 156.511 kJ/mol, − 157.605 kJ/mol, and − 160.265 kJ/mol, respectively, compared to the reference compound (− 134.211 kJ/mol). These findings suggest that the identified R. officinalis compounds are promising candidates for further development as GSK3β inhibitors in the treatment of Alzheimer’s disease.