Techniques for generating a personalization value that measures how tailored certain system interactions are for a user are described. A dialog exchange between a user and a skill may be determined, with the dialog exchange including user input data and system output data. It may be determined that the system output data was generated without respect to at least one previous user input or system output of the dialog exchanges. Based on this, a personalization value may be generated and sent to the skill.