A vehicle environment detection system (2) that includes a detection device and a processing unit (5), arranged to detect at least two feature points at objects (11) outside a vehicle (1). Each feature point constitutes a retrievable point that has a fixed position (x1, y1; x2, y2) on the object (11). The processing unit (5) is arranged to determine the positions (x1, y1; x2, y2) and the resulting velocities (vr1, vr2) for each one of the feature points for multiple frames by use of a feature point tracker; and determine a reference position (x0, y0), a corresponding reference velocity (vr0) and reference angular velocity (ω), constituting a common motion state for the object (11), by use of the results from the feature point tracker. A feature point tracker is constituted by a tracking algorithm which is arranged to track multiple features and which includes temporal filtering.