Identifying multiple languages in a content item

    公开(公告)号:US10180935B2

    公开(公告)日:2019-01-15

    申请号:US15422463

    申请日:2017-02-02

    Applicant: Facebook, Inc.

    Abstract: A system for identifying language(s) for content items is disclosed. The system can identify different languages for content item words segments by identifying segment languages that maximize a probability across the segments. The probability can be a combination of: an author's likelihood for the language identified for the first word; a combination of transition frequencies for selected languages identified for words, the transition frequencies indicating likelihoods that a transition occurred to the selected language from the previous word's language; and a combination of observation probabilities indicating, for a given word in the content item, a likelihood the given word is in the identified language. For an in-vocabulary word, the observation probabilities can be based on learned probability for that word. For an out-of-vocabulary word, the probability can be computed by breaking the word into overlapping n-grams and computing combined learned probabilities that each n-gram is in the given language.

    IDENTIFYING MULTIPLE LANGUAGES IN A CONTENT ITEM

    公开(公告)号:US20180189259A1

    公开(公告)日:2018-07-05

    申请号:US15422463

    申请日:2017-02-02

    Applicant: Facebook, Inc.

    CPC classification number: G06F17/275 G06F17/2294 G06F17/2775

    Abstract: A system for identifying language(s) for content items is disclosed. The system can identify different languages for content item words segments by identifying segment languages that maximize a probability across the segments. The probability can be a combination of: an author's likelihood for the language identified for the first word; a combination of transition frequencies for selected languages identified for words, the transition frequencies indicating likelihoods that a transition occurred to the selected language from the previous word's language; and a combination of observation probabilities indicating, for a given word in the content item, a likelihood the given word is in the identified language. For an in-vocabulary word, the observation probabilities can be based on learned probability for that word. For an out-of-vocabulary word, the probability can be computed by breaking the word into overlapping n-grams and computing combined learned probabilities that each n-gram is in the given language.

Patent Agency Ranking