With the increase of the transformer capacity and converter switching frequency, the transformer efficiency, stray parameters, insulation and heat dissipation should be considered when designing a transformer. Based on the transformer losses and stray parameters calculations, this paper focuses on the transformer optimization considering the transformer losses, insulation and leakage under constrained conditions using NSGA-II algorithm. A 220V/3.52kV, 3.52kW/20kHz transformer prototype was built and the experimental results verified the correctness of the calculation and the validity of the proposed optimization method.