To reduce the random noise of inertial sensors with low precision in IMU of telemetry system, a variable step-size LMS (VSSLMS) adaptive filter is applied to noise attenuation of IMU signal. An estimated cross-correlation coefficient between the offset error and the input signal is used to control the step size of the algorithm. Then the contradiction among the convergence speed, tracking performance and steady-state accuracy of the fixed step-size NLMS is overcome. Compared with the fixed step-size NLMS algorithm, the faster convergence speed, higher steady-state accuracy and better noise attenuation of dynamic signal can be achieved. It has been shown by experiments that the random noise of slave IMU signal is effectively reduced with the use of VSSLMS adaptive filter, which can offer a technical support for the improvement of measurement precision of telemetry system.