The transient behavior of DC systems has much influence on the stability of AC systems under the background of strong DC and weak AC. The strong nonlinearity of DC controllers and the flexible switches of control mode make it difficult to evaluate the transient stability of AC/DC systems using direct method by constructing energy function. From the view of network energy, the algorithm of branches potential energy uses the manifestation of critical branches during transient process to reflect the stability of systems. Based on the mechanism of coupling impact between AC and DC systems, this thesis builds the state space model including the electric variables of coupling point, and gets the stable area of AC/DC systems using the Lyapunov function. The critical energy can be got to recognize the instability when the voltage phase difference of critical branches reach the boundary of stable area. Time domain simulation is used to verify the effectiveness of the evaluation method.