公开(公告)号：US20120281969A1

公开(公告)日：2012-11-08

申请号：US13099391

申请日：2011-05-03

申请人： Wei Jiang ,  Alexander C. Loui ,  Courtenay Cotton

发明人： Wei Jiang ,  Alexander C. Loui ,  Courtenay Cotton

IPC分类号： G11B27/00

CPC分类号： G11B27/034 ,  G11B27/11

摘要： A method for producing an audio-visual slideshow for a video sequence having an audio soundtrack and a corresponding video track including a time sequence of image frames, comprising: segmenting the audio soundtrack into a plurality of audio segments; subdividing the audio segments into a sequence of audio frames; determining a corresponding audio classification for each audio frame; automatically selecting a subset of the audio segments responsive to the audio classification for the corresponding audio frames; for each of the selected audio segments automatically analyzing the corresponding image frames to select one or more key image frames; merging the selected audio segments to form an audio summary; forming an audio-visual slideshow by combining the selected key frames with the audio summary, wherein the selected key frames are displayed synchronously with their corresponding audio segment; and storing the audio-visual slideshow in a processor-accessible storage memory.

摘要翻译： 一种用于产生具有音频声轨和包括图像帧的时间序列的对应视频轨迹的视频序列的视听幻灯片放映方法，包括：将所述音频音轨分割为多个音频段; 将音频段细分成音频帧序列; 确定每个音频帧的相应音频分类; 响应于相应音频帧的音频分类自动选择音频段的子集; 对于每个所选择的音频片段，自动分析对应的图像帧以选择一个或多个关键图像帧; 合并所选音频片段以形成音频摘要; 通过将所选择的关键帧与音频摘要组合来形成视听幻灯片，其中所选择的关键帧与其对应的音频片段同步显示; 以及将视听幻灯片放映在处理器可访问的存储存储器中。

公开(公告)号：US20120109964A1

公开(公告)日：2012-05-03

申请号：US12912820

申请日：2010-10-27

申请人： Wei Jiang ,  Alexander C. Loui

发明人： Wei Jiang ,  Alexander C. Loui

IPC分类号： G06F17/30

CPC分类号： G06F17/30038

摘要： A method of classifying a set of semantic concepts on a second multimedia collection based upon adapting a set of semantic concept classifiers and updating concept affinity relations that were developed to classify the set of semantic concepts for a first multimedia collection. The method comprises providing the second multimedia collection from a different domain and a processor automatically classifying the semantic concepts from the second multimedia collection by adapting the semantic concept classifiers and updating the concept affinity relations to the second multimedia collection based upon the local smoothness over the concept affinity relations and the local smoothness over data affinity relations.

摘要翻译： 一种基于适应一组语义概念分类器并且更新用于对第一多媒体集合的语义概念集合进行分类而开发的概念亲和度关系来对第二多媒体集合上的一组语义概念进行分类的方法。 该方法包括从不同的域提供第二多媒体集合，并且处理器通过基于概念上的局部平​​滑度来适应语义概念分类器并且将概念兴趣关系更新到第二多媒体集合来自动地对来自第二多媒体集合的语义概念进行分类 亲和关系和数据关联关系的局部平滑度。

公开(公告)号：US09111547B2

公开(公告)日：2015-08-18

申请号：US13591489

申请日：2012-08-22

申请人： Alexander C. Loui ,  Wei Jiang ,  Kevin Michael Gobeyn ,  Charles Parker

发明人： Alexander C. Loui ,  Wei Jiang ,  Kevin Michael Gobeyn ,  Charles Parker

IPC分类号： G06F15/18 ,  G10L25/51

CPC分类号： G10L25/51

摘要： A method for determining a semantic concept associated with an audio signal captured using an audio sensor. A data processor is used to automatically analyze the audio signal using a plurality of semantic concept detectors to determine corresponding preliminary semantic concept detection values, each semantic concept detector being adapted to detect a particular semantic concept. The preliminary semantic concept detection values are analyzed using a joint likelihood model based on predetermined pair-wise likelihoods that particular pairs of semantic concepts co-occur to determine updated semantic concept detection values. One or more semantic concepts are determined based on the updated semantic concept detection values. The semantic concept detectors and the joint likelihood model are trained together with a joint training process using training audio signals, at least some of which are known to be associated with a plurality of semantic concepts.

摘要翻译： 一种用于确定与使用音频传感器捕获的音频信号相关联的语义概念的方法。 数据处理器用于使用多个语义概念检测器自动分析音频信号，以确定对应的初步语义概念检测值，每个语义概念检测器适于检测特定的语义概念。 基于预定的成对似然性，使用联合似然模型来分析初步语义概念检测值，特定的语义概念对共同出现以确定更新的语义概念检测值。 基于更新的语义概念检测值来确定一个或多个语义概念。 语义概念检测器和联合似然模型与使用训练音频信号的联合训练过程一起训练，其中至少一些已知与多个语义概念相关联。

公开(公告)号：US08867891B2

公开(公告)日：2014-10-21

申请号：US13269742

申请日：2011-10-10

申请人： Wei Jiang ,  Alexander C. Loui

发明人： Wei Jiang ,  Alexander C. Loui

IPC分类号： H04N5/92 ,  H04N21/233 ,  H04N21/234 ,  H04N5/93 ,  G11B27/28

CPC分类号： H04N5/93 ,  G11B27/28 ,  H04N21/233 ,  H04N21/23418

摘要： A method for determining a semantic concept classification for a digital video clip, comprising: receiving an audio-visual dictionary including a plurality of audio-visual grouplets, the audio-visual grouplets including visual background and foreground codewords, audio background and foreground codewords, wherein the codewords in a particular audio-visual grouplet were determined to be correlated with each other; analyzing the digital video clip to determine a set of visual features and a set of audio features; determining similarity scores between the digital video clip and each of the audio-visual grouplets by comparing the set of visual features to any visual background and foreground codewords associated with a particular audio-visual grouplet, and comparing the set of audio features to any audio background and foreground codewords associated with the particular audio-visual grouplet; and determining one or more semantic concept classifications using trained semantic classifiers.

摘要翻译： 一种用于确定数字视频剪辑的语义概念分类的方法，包括：接收包括多个视听小组的视听词典，所述视听小组包括视觉背景和前景码字，音频背景和前景码字，其中 确定特定视听小组中的码字彼此相关; 分析数字视频剪辑以确定一组视觉特征和一组音频特征; 通过将视觉特征的集合与与特定视听小组相关联的任何视觉背景和前景码字进行比较，以及将该组音频特征与任何音频背景进行比较来确定数字视频剪辑和每个视听小组之间的相似性得分 以及与特定视听小组相关联的前景码字; 以及使用经训练的语义分类器确定一个或多个语义概念分类。

公开(公告)号：US20140056432A1

公开(公告)日：2014-02-27

申请号：US13591489

申请日：2012-08-22

申请人： Alexander C. Loui ,  Wei Jiang ,  Kevin Michael Gobeyn ,  Charles Parker

发明人： Alexander C. Loui ,  Wei Jiang ,  Kevin Michael Gobeyn ,  Charles Parker

IPC分类号： H04R29/00

CPC分类号： G10L25/51

摘要： A method for determining a semantic concept associated with an audio signal captured using an audio sensor. A data processor is used to automatically analyze the audio signal using a plurality of semantic concept detectors to determine corresponding preliminary semantic concept detection values, each semantic concept detector being adapted to detect a particular semantic concept. The preliminary semantic concept detection values are analyzed using a joint likelihood model based on predetermined pair-wise likelihoods that particular pairs of semantic concepts co-occur to determine updated semantic concept detection values. One or more semantic concepts are determined based on the updated semantic concept detection values. The semantic concept detectors and the joint likelihood model are trained together with a joint training process using training audio signals, at least some of which are known to be associated with a plurality of semantic concepts.

摘要翻译： 一种用于确定与使用音频传感器捕获的音频信号相关联的语义概念的方法。 数据处理器用于使用多个语义概念检测器自动分析音频信号，以确定对应的初步语义概念检测值，每个语义概念检测器适于检测特定的语义概念。 基于预定的成对似然性，使用联合似然模型来分析初步语义概念检测值，特定的语义概念对共同出现以确定更新的语义概念检测值。 基于更新的语义概念检测值来确定一个或多个语义概念。 语义概念检测器和联合似然模型与使用训练音频信号的联合训练过程一起训练，其中至少一些已知与多个语义概念相关联。

公开(公告)号：US08386490B2

公开(公告)日：2013-02-26

申请号：US12912820

申请日：2010-10-27

申请人： Wei Jiang ,  Alexander C. Loui

发明人： Wei Jiang ,  Alexander C. Loui

IPC分类号： G06F7/00

CPC分类号： G06F17/30038

摘要： A method of classifying a set of semantic concepts on a second multimedia collection based upon adapting a set of semantic concept classifiers and updating concept affinity relations that were developed to classify the set of semantic concepts for a first multimedia collection. The method comprises providing the second multimedia collection from a different domain and a processor automatically classifying the semantic concepts from the second multimedia collection by adapting the semantic concept classifiers and updating the concept affinity relations to the second multimedia collection based upon the local smoothness over the concept affinity relations and the local smoothness over data affinity relations.

摘要翻译： 一种基于适应一组语义概念分类器并且更新用于对第一多媒体集合的语义概念集合进行分类而开发的概念亲和度关系来对第二多媒体集合上的一组语义概念进行分类的方法。 该方法包括从不同的域提供第二多媒体集合，并且处理器通过基于概念上的局部平​​滑度来适应语义概念分类器并且将概念兴趣关系更新到第二多媒体集合来自动地对来自第二多媒体集合的语义概念进行分类 亲和关系和数据关联关系的局部平滑度。

公开(公告)号：US10134440B2

公开(公告)日：2018-11-20

申请号：US13099391

申请日：2011-05-03

申请人： Wei Jiang ,  Alexander C. Loui ,  Courtenay Cotton

发明人： Wei Jiang ,  Alexander C. Loui ,  Courtenay Cotton

IPC分类号： G11B27/034 ,  G11B27/11

摘要： A method for producing an audio-visual slideshow for a video sequence having an audio soundtrack and a corresponding video track including a time sequence of image frames, comprising: segmenting the audio soundtrack into a plurality of audio segments; subdividing the audio segments into a sequence of audio frames; determining a corresponding audio classification for each audio frame; automatically selecting a subset of the audio segments responsive to the audio classification for the corresponding audio frames; for each of the selected audio segments automatically analyzing the corresponding image frames to select one or more key image frames; merging the selected audio segments to form an audio summary; forming an audio-visual slideshow by combining the selected key frames with the audio summary, wherein the selected key frames are displayed synchronously with their corresponding audio segment; and storing the audio-visual slideshow in a processor-accessible storage memory.

公开(公告)号：US20130089304A1

公开(公告)日：2013-04-11

申请号：US13269753

申请日：2011-10-10

申请人： Wei Jiang ,  Alexander C. Loui

发明人： Wei Jiang ,  Alexander C. Loui

IPC分类号： H04N5/93

CPC分类号： H04N21/8405 ,  G06F17/30784 ,  G06K9/00718 ,  G06K9/00744 ,  H04N21/2743 ,  H04N21/4332 ,  H04N21/44008

摘要： A method for determining a semantic concept classification for a digital video clip, comprising: receiving an audio-visual dictionary including a plurality of audio-visual grouplets, the audio-visual grouplets including visual background and foreground codewords, audio background and foreground codewords, wherein the codewords in a particular audio-visual grouplet were determined to be correlated with each other; determining reference video codeword similarity scores for a set of reference video clips; determining codeword similarity scores for the digital video clip; determining a reference video similarity score for each reference video clip representing a similarity between the digital video clip and the reference video clip responsive to the audio-visual grouplets, the codeword similarity scores and the reference video codeword similarity scores; and determining one or more semantic concept classifications using trained semantic classifiers responsive to the determined reference video similarity scores.

摘要翻译： 一种用于确定数字视频剪辑的语义概念分类的方法，包括：接收包括多个视听小组的视听词典，所述视听小组包括视觉背景和前景码字，音频背景和前景码字，其中 确定特定视听小组中的码字彼此相关; 确定一组参考视频剪辑的参考视频码字相似性得分; 确定所述数字视频剪辑的码字相似性得分; 响应于所述视听小组，所述码字相似度得分和所述参考视频码字相似性得分，确定表示所述数字视频剪辑和所述参考视频剪辑之间的相似性的每个参考视频剪辑的参考视频相似性分数; 以及响应于所确定的参考视频相似性分数，使用经过训练的语义分类器来确定一个或多个语义概念分类。

公开(公告)号：US07720851B2

公开(公告)日：2010-05-18

申请号：US11615120

申请日：2006-12-22

申请人： Shih-Fu Chang ,  Wei Jiang ,  Alexander C. Loui

发明人： Shih-Fu Chang ,  Wei Jiang ,  Alexander C. Loui

IPC分类号： G06F17/30

CPC分类号： G06F17/30265

摘要： A context-based concept fusion method detects a first concept in an image record. The method includes automatically determining at least one other concept in the image record which has a contextual relationship with the first concept and which is to be labeled by a user of the method; and labeling the at least one other concept by the user with a ground truth label to be used in the context-based concept fusion method to improve detection of the first concept in the image record.

摘要翻译： 基于上下文的概念融合方法检测图像记录中的第一概念。 该方法包括自动确定图像记录中至少一个与第一概念具有上下文关系并且由该方法的用户标记的其它概念; 以及用户使用要在基于上下文的概念融合方法中使用的基本真值标签来标记所述至少一个其他概念，以改进对图像记录中的第一概念的检测。

公开(公告)号：US20090299999A1

公开(公告)日：2009-12-03

申请号：US12408140

申请日：2009-03-20

申请人： Alexander C. Loui ,  Wei Jiang

发明人： Alexander C. Loui ,  Wei Jiang

IPC分类号： G06F17/30

CPC分类号： G06F17/30256 ,  G06F17/30802 ,  G06F17/30805 ,  G06F17/30808 ,  G06K9/00664 ,  G06K9/00711

摘要： A method for facilitating semantic event classification of a group of image records related to an event. The method using an event detector system for providing: extracting a plurality of visual features from each of the image records; wherein the visual features include segmenting an image record into a number of regions, in which the visual features are extracted; generating a plurality of concept scores for each of the image records using the visual features, wherein each concept score corresponds to a visual concept and each concept score is indicative of a probability that the image record includes the visual concept; generating a feature vector corresponding to the event based on the concept scores of the image records; and supplying the feature vector to an event classifier that identifies at least one semantic event classifier that corresponds to the event.

摘要翻译： 一种用于促进与事件相关的一组图像记录的语义事件分类的方法。 该方法使用事件检测器系统来提供：从每个图像记录提取多个视觉特征; 其中所述视觉特征包括将图像记录分割成其中提取所述视觉特征的多个区域; 使用所述视觉特征为每个所述图像记录生成多个概念分数，其中每个概念分数对应于视觉概念，并且每个概念分数指示所述图像记录包括所述视觉概念的概率; 基于所述图像记录的概念分数生成与所述事件相对应的特征向量; 以及将特征向量提供给识别与该事件相对应的至少一个语义事件分类器的事件分类器。