Crystalline materials that combine electrical polarization and magnetization could be advantageous in applications such as information storage, but these properties are usually considered to have incompatible chemical bonding and electronic requirements. Recent theoretical work on perovskite materials suggested a route for combining both properties. We used crystal chemistry to engineer specific atomic displacements in a layered perovskite, (CaySr1-y)1.15Tb1.85Fe2O7, that change its symmetry and simultaneously generate electrical polarization and magnetization above room temperature. The two resulting properties are magnetoelectrically coupled as they arise from the same displacements.