A novel, robust attitude controller for rigid bodies in the presence of time-varying orientation constraints is presented in this paper. Using an error transformation, the dynamic attitude constraints are converted into time-varying quaternion constraints. Subsequently, a robust attitude control law is synthesized using the backstepping philosophy in which barrier Lyapunov functions (BLFs) are used to achieve asymptotic tracking and simultaneously avoid attitude constraint violation. This is accomplished by ensuring the boundedness of BLFs in the closed-loop Lyapunov stability analysis. Besides the nominal scenario, an adaptive control law is also formulated to tackle moment of inertial uncertainties and an unknown, time-varying, bounded disturbance. In this case, the attitude tracking errors are uniformly ultimately bounded, whose bounds can be adjusted by user-defined constants. Note that in both scenarios, the dynamic attitude constraints are not transgressed. Finally, the effectiveness of the proposed controller is demonstrated by carrying out extensive numerical simulations in the presence of parametric uncertainties and disturbance torques.