The hardware-in-the-loop(HIL) simulation is an effective method to verify the overall function of the flexible HVDC transmission control and protection device. With this method, debugging the control and protection device can make the system run safely and stably after being put into operation. Therefore, a hardware-in-the-loop simulation platform modular multilevel converter(MMC) based on RT-LAB is established in this paper. Data merging between a converter valve control system and the real-time simulator is realized by high-speed optical fiber communication protocol conversion chassis, and the high-speed communication interface is designed to meet the requirements of the communication rate. Aiming at the control performance of the physical device, the test scheme is designed, and the test methods of voltage balance control and circulating current suppression are proposed. The closed-loop test of control and protection device is carried out by the active power step and AC/DC fault test. The above test verifies the validity of the HIL simulation platform of the MMC and the rationality of the testing scheme, and can meet the performance testing requirements of the control and protection device.