With rising energy storage capacity and distributed generation in the electric grid, optimal power management of converters help to reduce running price and optimize power transfer. This paper develops a particle swarm optimization (PSO) based technique to minimize the cost of energy (CoE) purchased from the grid to reduce the running cost of a multi-port converter system for electric vehicle (EV) charging station. The multi-port system discussed in the paper integrates a photovoltaic (PV) system, battery storage, electric grid, and EV charging system. The developed optimal power management system supervises the controllers of different converters in the multi-port converter system. The performance of the optimal power management system is validated through simulation.