The domain configurations such as domain size, orientation, and interdomain spacing have a significant influence on the electromechanical properties of piezoelectric materials though their role on mechanical properties is not well understood. In this manuscript, we have systematically varied the domain configuration of polycrystalline lead magnesium niobate-lead titanate (PMN-PT) piezoceramics (by annealing them below and slightly above the Curie temperature, Tc) and determined the nanomechanical properties. Nanoindentation experiments performed on pristine, sub and above-Tc annealed samples in the peak load range of 1 mN–5 mN reveals a strong indentation size effect (ISE) in hardness, H. Further, it is observed that the sub-Tc annealed samples exhibit higher H and elastic modulus, E compared to the above-Tc annealed and pristine samples. In contrast to this, the piezoelectric constant, d33, decreases with increase in annealing temperature eventually approaches to “zero” for the above-Tc annealed samples, though both sub and above-Tc annealed samples have similar crystal structure. The microstructure and domain characterization indicate discernable differences in the domain structure suggesting that the differences in nano-mechanical and piezoelectric properties can be attributed to the changes in domain configurations. These results provide new insights about the novel way to engineer the domain configurations for tailoring mechanical and piezoelectric properties of the piezoceramics.