Aiming at the problem that the Siamese network has insufficient ability to express the features of scale-varying targets, a multi-branch structure is constructed by using convolution, pooling branches and pruning operations of different sizes to improve the robustness of features and ensure the translation invariance of the Siamese network. Aiming at the problem that the multi-branch structure brings too many parameters, the multi-branch structure is reparameterized into a single convolution in the tracking stage, which effectively reduces the time cost in the tracking stage. The experimental results show that compared with SiamFC, the accuracy, success rate and tracking speed of the proposed algorithm on the OTB100 datasets are improved by 5.1%, 3% and 30 FPS, respectively. The tracking accuracy and success rate are improved on the UAV123 and Temple-Color-128 datasets.