In the case of non-uniformly distributed layers, the vertical difference scheme used in GRAPES (Global/Regional Assimilation and Prediction System) model can only achieve first-order accuracy. A second-order scheme was designed and introduced into the GRAPES model in order to be used in the process of vertical discretization. An ideal test using the 1-D profile showed that the new scheme can improve the accuracy of difference computation obviously. An ideal density flow test was conducted to verify the correctness and stability of the new scheme in GRAPES model. A statistical evaluation of a medium-range forecast using the second-order scheme showed an improvement of forecast skill in large-scale fields, especially for the forecast after 120 h. Additionally, the second-order scheme was tested with a real-case experiment based on the extreme rainfall at South China, which again showed an improvement in the forecast of precipitation after 48 h.