Systems, methods, and apparatus are provided for adaptively switching interpolation filters during the encoding of video data or the decoding of a video bitstream. In various implementations, a set of interpolation filters can be defined and made available to coding device. The coding device can select an interpolation filter for a given coding unit. The interpolation filter can be selected based on, for example, the coding level of the coding unit, among other things. In some examples, signaling of the selected interpolation filter can be simplified by selecting a subset of the set of interpolation filters for a given coding situation. An index indicating an interpolation filter from the subset can then be signaled. Alternatively, a decoder can derive an identity of the interpolation filter from data provided by a bitstream, in which case the index need not be explicitly signaled in the bitstream.