Spectral imaging systems are used to gather spectral image data on earthen material moving within an earthen material processing system, such as a mineral processing system or cement plant. Machine learning models such as 3D convolutional neural networks may be utilized to process the spectral image data to determine or classify one or more characteristics of the earthen material, such as ore grade, mineral alteration(s), moisture content, lithology and/or mineralogy. Such earthen material characteristics, or classifications thereof, may then be utilized to automatically control one or more operational characteristics of the earthen material processing system, such as rotational speed of milling equipment or flow rates of water or chemicals added to milling equipment or mineral concentration systems.