Mobile edge computing (MEC) can be used to reduce the task delay for users with limited computing resources. However, in 6G networks, the diversity of tasks is greatly increased. For those extremely delay-sensitive small-size computing tasks, the inference delay of neural network (NN)-based algorithms such as resource allocation and task offloading cannot be ignored. As a hyperparameter, the inference cost of NN is usually difficult to adjust. Dynamic neural network (DyNN) is an emerging technique that improves the model efficiency by adjusting the network architecture on-demand according to the sample characteristics during inference. In this paper, we propose a DyNN-based resource management method for MEC that dynamically adjusts the depth and width of the NN according to the features of the task, improving computational efficiency and achieving a balance between inference delay and the management performance of computational and communication resources. Furthermore, to reduce the training cost of DyNN, a new training method is proposed in this paper, where all the blocks in DyNN are gradually trained in the order of size. Simulation results demonstrate that the proposed DyNN-based resource management method outperforms the traditional optimization algorithm and the static-NN-based method.