A method, system and computer program product for selecting a bitrate to stream encoded media. A manifest file is fetched, where the manifest file contains metadata for encoded media segments which includes tags signifying the associated encoded media segments being important to the viewer. A playback occupancy level is then increased before reaching these tagged media segment(s). A high bitrate (high quality) is selected to stream the tagged media segment(s) which are then fetched and played at the selected high bitrate in response to the new playback buffer occupancy level being above a threshold level. In this manner, those media segments that are of higher importance to the viewer will be viewed in a higher quality since content information is used to select the bitrate to receive the encoded media segments. As a result, the user's quality of experience in viewing media is improved.