Electric vehicle (EV) is a promising technology for reducing environmental impacts of road transport. Efficient EV charging control strategies that can affect the impacts and benefits is a potential research problem. Adopting the notion of IoT, in this paper, we present a Cloud-Fog based Battery Swapping Topology (BSS). A QoS ensuring timing model is proposed for defining the charging management of EV batteries across the BSS. For optimal BSS infrastructure planning, we also present a cost optimization framework, considering the timing and architectural constraints. The potential solution approaches for the given optimization formulation is also discussed.