An activity stream based interaction model is disclosed. To cause a desired application level action to be performed at a remote system, such as automatically retrieving and displaying a document in a viewer or other portion of a user interface at the remote system, a text-based tag, such as a hash tag, is inserted into an activity stream. The activity stream is sent to the remote system, which is configured to respond to the text-based tag by performing, at least in part automatically, the corresponding application level action.