Disclosed are various embodiments for adjusting the encoding of a video signal into a video stream based on user attention. A temporary lapse of attention by a user is predicted. The encoding of the video signal into the video stream is adjusted from an initial state to a conservation state in response to predicting the temporary lapse of attention by the user. The conservation state is configured to conserve one or more resources used for the video stream relative to the initial state.