We generate the non-separable state of polarization and orbital angular momentum using a laser beam. The generated state undergoes a cyclic polarization evolution which introduces a Pancharatnam geometric phase to the polarization state and, in turn, a relative phase in the non-separable state. We experimentally study the violation of the Bell-like inequality for different Pancharatnam phases introduced by various cyclic polarization evolutions with linear and circular states as measurement bases. While measuring in linear bases, the Bell-CHSH parameter oscillates with a Pancharatnam phase. One can overcome this dependence by introducing a relative phase in one of the projecting states. However, for measurements in circular bases, the Pancharatnam phase does not affect the Bell-like inequality violation.