An encoder (100) comprises a motion estimation processor (103) which generates motion compensated video frames for a video signal. The motion compensated video frames are fed to a wavelet processor (105) which applies an adaptive spatial wavelet transform to the frames to generate spatially transformed frames. An encoding processor (107) generates encoded video data for the video signal in response to the spatially transformed frames. The encoder (100) further comprises an adaptation processor (111) which determines adaptive parameters for the adaptive spatial wavelet transform in response to motion data for the motion compensated video frames. The adaptive parameters may be wavelet coefficients for the adaptive spatial wavelet transform. The decoder (300) performs the inverse operation and determines adaptive parameters for an inverse adaptive spatial wavelet transform based on received motion data. The use of motion information for adaptation of the adaptive spatial wavelet transforms allows improved video quality to data rate ratio without necessitating additional side data.