A method and apparatus for adjusting a neural network that classifies a scene of an input image into at least one class is provided. The method generates a feature image having a size that is less than a size of an input image by applying a convolutional network to the input image, determines at least one class corresponding to the feature image, generates a class image having a size corresponding to the size of the input image by applying a deconvolutional network to the feature image, calculates a loss of the class image based on a verification class image preset with respect to the input image, and adjusts the neural network based on the loss.