The charge density wave (CDW), a translational symmetry breaking electronic liquid, plays a pivotal role in correlated quantum materials, such as high-Tc superconductors and topological semimetals. Recently, CDWs that possibly intertwine with superconductivity and magnetism are observed in various kagome metals. However, the nature of CDWs and the role of the Fermi-surface (FS) topology in these materials remain an unresolved challenge. In this paper, we reveal the formation of CDWs in the newly discovered kagome metal LuNb6Sn6. We observe a “yield sign”-like hollow triangular diffuse scattering pattern and nearly complete softening of a flat optical phonon band near QH = (1/3, 1/3, 1/2). The scattering intensity of the diffuse scattering displays divergent behavior as decreasing temperature until TCDW = 70 K, where a competing CDW at QCDW = (1/3, 1/3, 1/3) emerges. Combined with scanning tunneling microscopy/spectroscopy and first-principles calculations, our observations support a Fermi-surface driven frustration that suppresses the leading CDW instability at QH. This frustration is relieved by the emergence of a subleading CDW at QCDW. These results provide insights into the interplay between the Fermi surface, strong electron-phonon coupling, and charge density wave formation in quantum materials.