Trick mode play for controlling video content playback can be realized using a semantic based criterion in order to achieve a more uniform playback experience for the viewer, when that viewer wishes to observe the video content presented according to a selected semantic. In one embodiment of the method, semantics associated with shorter time intervals in the video content are replayed in trick mode play proportionally slower than semantics associated with longer time intervals, wherein the proportionally is determined, at least in part, to the time interval associated with each semantic. In various embodiments, semantics can include scenes, audio, metadata, tags, and the like. The semantics based trick mode play approach allows for a more uniform viewing experience, on a time duration basis, from one semantic to the next.