This disclosure describes adaptive spatial variant interpolation (SVI) techniques for image upscaling. In various embodiments, the interpolation techniques described in this disclosure may support low complexity upscaling of image while promoting high image quality, including enhanced sharpness, higher contrast and more accurate interpolation. The interpolation techniques may be applied using generalized finite impulse response (FIR) filters. In some embodiments, the interpolation techniques may be content-adaptive to provide more accurate interpolation while suppressing significant artifacts associated with sharp edges. In addition, the interpolation techniques may be readily applicable to upscaling of color imagery and video, e.g., in both YCbCr (luminance, blue chrominance, red chrominance) and RGB (red, green, blue) formats.