公开(公告)号：US12014548B2

公开(公告)日：2024-06-18

申请号：US17805075

申请日：2022-06-02

Applicant: ADOBE INC.

Inventor： Hijung Shin ,  Xue Bai ,  Aseem Agarwala ,  Joel R. Brandt ,  Jovan Popović ,  Lubomira Dontcheva ,  Dingzeyu Li ,  Joy Oakyung Kim ,  Seth Walker

IPC: G06V20/40 ,  G06F18/231 ,  G10L25/78 ,  G11B27/00 ,  G11B27/10 ,  G11B27/19 ,  G11B27/36

CPC classification number: G06V20/49 ,  G06F18/231 ,  G06V20/41 ,  G06V20/46 ,  G10L25/78 ,  G11B27/002 ,  G11B27/19 ,  G06V20/44

Abstract: Embodiments are directed to segmentation and hierarchical clustering of video. In an example implementation, a video is ingested to generate a multi-level hierarchical segmentation of the video. In some embodiments, the finest level identifies a smallest interaction unit of the video—semantically defined video segments of unequal duration called clip atoms. Clip atom boundaries are detected in various ways. For example, speech boundaries are detected from audio of the video, and scene boundaries are detected from video frames of the video. The detected boundaries are used to define the clip atoms, which are hierarchically clustered to form a multi-level hierarchical representation of the video. In some cases, the hierarchical segmentation identifies a static, pre-computed, hierarchical set of video segments, where each level of the hierarchical segmentation identifies a complete set (i.e., covering the entire range of the video) of disjoint (i.e., non-overlapping) video segments with a corresponding level of granularity.

公开(公告)号：US11630562B2

公开(公告)日：2023-04-18

申请号：US17017366

申请日：2020-09-10

Applicant: ADOBE INC.

Inventor： Seth Walker ,  Joy Oakyung Kim ,  Aseem Agarwala ,  Joel R. Brandt ,  Jovan Popović ,  Lubomira Dontcheva ,  Dingzeyu Li ,  Hijung Shin ,  Xue Bai

IPC: G06F3/04847 ,  G06F3/04845 ,  G06F3/0485

Abstract: Embodiments are directed to techniques for interacting with a hierarchical video segmentation using a video timeline. In some embodiments, the finest level of a hierarchical segmentation identifies the smallest interaction unit of a video—semantically defined video segments of unequal duration called clip atoms, and higher levels cluster the clip atoms into coarser sets of video segments. A presented video timeline is segmented based on one of the levels, and one or more segments are selected through interactions with the video timeline. For example, a click or tap on a video segment or a drag operation dragging along the timeline snaps selection boundaries to corresponding segment boundaries defined by the level. Navigating to a different level of the hierarchy transforms the selection into coarser or finer video segments defined by the level. Any operation can be performed on selected video segments, including playing back, trimming, or editing.

公开(公告)号：US11145333B2

公开(公告)日：2021-10-12

申请号：US16679013

申请日：2019-11-08

Applicant: ADOBE INC.

Inventor： William Hayes Allen ,  Lubomira Dontcheva ,  Haiqing Lu ,  Zachary Platt McCullough ,  David R. Stein ,  Christopher Nuuja ,  Benoit Ambry ,  Joel Richard Brandt ,  Cristin Ailidh Fraser ,  Joy Oakyung Kim ,  Hijung Shin

IPC: G11B27/034 ,  G11B27/036 ,  G06F3/0484

Abstract: Systems and methods provide for capturing and presenting content creation tools of an application used in a video. Application data from the application for the duration of the video is received. The application data includes data identifiers and time markers corresponding to user interaction with an application in a video. The application data is processed to detect tool identifiers identifying tools used in the video based on the data identifiers. For each a tool identifier, a tool label and a corresponding time in the timeline is determined. A tool record storing the tool labels and the corresponding times in association with the video is generated. When a viewer requests to watch the video, the tool record is presented to the viewer in conjunction with the video.

公开(公告)号：US20230161553A1

公开(公告)日：2023-05-25

申请号：US18152055

申请日：2023-01-09

Applicant: Adobe Inc.

Inventor： Lubomira Dontcheva ,  Arjun Srinivasan ,  Seth John WALKER ,  Eytan Adar

IPC: G06F3/16 ,  G06F16/68 ,  G06F16/9032 ,  G06F3/0482 ,  G06F3/0483 ,  G06F3/0487 ,  G06F9/451 ,  G10L15/22 ,  G06F3/0481

CPC classification number: G06F3/167 ,  G06F3/0481 ,  G06F3/0482 ,  G06F3/0483 ,  G06F3/0487 ,  G06F9/451 ,  G06F9/453 ,  G06F16/686 ,  G06F16/90324 ,  G10L15/22 ,  G06F2203/0381 ,  G10L2015/223

Abstract: A framework for generating and presenting verbal command suggestions to facilitate discoverability of commands capable of being understood and support users exploring available commands. A target associated with a direct-manipulation input is received from a user via a multimodal user interface. A set of operations relevant to the target is selected and verbal command suggestions relevant to the selected set of operations and the determined target are generated. At least a portion of the generated verbal command suggestions is provided for presentation in association with the multimodal user interface in one of three interface variants: one that presents command suggestions as a list, one that presents command suggestions using contextual overlay windows, and one that presents command suggestions embedded within the interface. Each of the proposed interface variants facilitates user awareness of verbal commands that are capable of being executed and teaches users how available verbal commands can be invoked.

公开(公告)号：US20220301313A1

公开(公告)日：2022-09-22

申请号：US17805076

申请日：2022-06-02

Applicant: ADOBE INC.

Inventor： Hijung Shin ,  Xue Bai ,  Aseem Agarwala ,  Joel R. Brandt ,  Jovan Popovic ,  Lubomira Dontcheva ,  Dingzeyu Li ,  Joy Oakyung Kim ,  Seth Walker

IPC: G06V20/40 ,  G06K9/62 ,  G11B27/19 ,  G11B27/00 ,  G10L25/78

Abstract: Embodiments are directed to segmentation and hierarchical clustering of video. In an example implementation, a video is ingested to generate a multi-level hierarchical segmentation of the video. In some embodiments, the finest level identifies a smallest interaction unit of the video—semantically defined video segments of unequal duration called clip atoms. Clip atom boundaries are detected in various ways. For example, speech boundaries are detected from audio of the video, and scene boundaries are detected from video frames of the video. The detected boundaries are used to define the clip atoms, which are hierarchically clustered to form a multi-level hierarchical representation of the video. In some cases, the hierarchical segmentation identifies a static, pre-computed, hierarchical set of video segments, where each level of the hierarchical segmentation identifies a complete set (i.e., covering the entire range of the video) of disjoint (i.e., non-overlapping) video segments with a corresponding level of granularity.

公开(公告)号：US20220292831A1

公开(公告)日：2022-09-15

申请号：US17805080

申请日：2022-06-02

Applicant: ADOBE INC.

Inventor： Hijung Shin ,  Xue Bai ,  Aseem Agarwala ,  Joel R. Brandt ,  Jovan Popovic ,  Lubomira Dontcheva ,  Dingzeyu Li ,  Joy Oakyung Kim ,  Seth Walker

IPC: G06V20/40 ,  G06K9/62 ,  G11B27/19 ,  G11B27/00 ,  G10L25/78

Abstract: Embodiments are directed to segmentation and hierarchical clustering of video. In an example implementation, a video is ingested to generate a multi-level hierarchical segmentation of the video. In some embodiments, the finest level identifies a smallest interaction unit of the video—semantically defined video segments of unequal duration called clip atoms. Clip atom boundaries are detected in various ways. For example, speech boundaries are detected from audio of the video, and scene boundaries are detected from video frames of the video. The detected boundaries are used to define the clip atoms, which are hierarchically clustered to form a multi-level hierarchical representation of the video. In some cases, the hierarchical segmentation identifies a static, pre-computed, hierarchical set of video segments, where each level of the hierarchical segmentation identifies a complete set (i.e., covering the entire range of the video) of disjoint (i.e., non-overlapping) video segments with a corresponding level of granularity.

公开(公告)号：US20220292830A1

公开(公告)日：2022-09-15

申请号：US17805075

申请日：2022-06-02

Applicant: ADOBE INC.

Inventor： Hijung Shin ,  Xue Bai ,  Aseem Agarwala ,  Joel R. Brandt ,  Jovan Popovic ,  Lubomira Dontcheva ,  Dingzeyu Li ,  Joy Oakyung Kim ,  Seth Walker

IPC: G06V20/40 ,  G06K9/62 ,  G11B27/19 ,  G11B27/00 ,  G10L25/78

Abstract: Embodiments are directed to segmentation and hierarchical clustering of video. In an example implementation, a video is ingested to generate a multi-level hierarchical segmentation of the video. In some embodiments, the finest level identifies a smallest interaction unit of the video—semantically defined video segments of unequal duration called clip atoms. Clip atom boundaries are detected in various ways. For example, speech boundaries are detected from audio of the video, and scene boundaries are detected from video frames of the video. The detected boundaries are used to define the clip atoms, which are hierarchically clustered to form a multi-level hierarchical representation of the video. In some cases, the hierarchical segmentation identifies a static, pre-computed, hierarchical set of video segments, where each level of the hierarchical segmentation identifies a complete set (i.e., covering the entire range of the video) of disjoint (i.e., non-overlapping) video segments with a corresponding level of granularity.

公开(公告)号：US20220076707A1

公开(公告)日：2022-03-10

申请号：US17330702

申请日：2021-05-26

Applicant: ADOBE INC.

Inventor： Seth Walker ,  Hijung Shin ,  Cristin Ailidh Fraser ,  Aseem Agarwala ,  Lubomira Dontcheva ,  Joel Richard Brandt ,  Jovan Popovic ,  Joy Oakyung Kim ,  Justin Salamon ,  Jui-hsien Wang ,  Timothy Jeewun Ganter ,  Xue Bai ,  Dingzeyu Li

IPC: G11B27/036 ,  G06F3/0486 ,  G06F3/0482

Abstract: Embodiments are directed to a snap point segmentation that defines the locations of selection snap points for a selection of video segments. Candidate snap points are determined from boundaries of feature ranges of the video indicating when instances of detected features are present in the video. In some embodiments, candidate snap point separations are penalized for being separated by less than a minimum duration corresponding to a minimum pixel separation between consecutive snap points on a video timeline. The snap point segmentation is computed by solving a shortest path problem through a graph that models different snap point locations and separations. When a user clicks or taps on the video timeline and drags, a selection snaps to the snap points defined by the snap point segmentation. In some embodiments, the snap points are displayed during a drag operation and disappear when the drag operation is released.

公开(公告)号：US20220076706A1

公开(公告)日：2022-03-10

申请号：US17330689

申请日：2021-05-26

Applicant: ADOBE INC.

Inventor： Seth Walker ,  Hijung Shin ,  Cristin Ailidh Fraser ,  Aseem Agarwala ,  Lubomira Dontcheva ,  Joel Richard Brandt ,  Jovan Popovic ,  Joy Oakyung Kim ,  Justin Salamon ,  Jui-hsien Wang ,  Timothy Jeewun Ganter ,  Xue Bai ,  Dingzeyu Li

IPC: G11B27/036 ,  G06F3/0482 ,  G06F3/0486

Abstract: Embodiments are directed to interactive tiles that represent video segments of a segmentation of a video. In some embodiments, each interactive tile represents a different video segment from a particular video segmentation (e.g., a default video segmentation). Each interactive tile includes a thumbnail (e.g., the first frame of the video segment represented by the tile), some transcript from the beginning of the video segment, a visualization of detected faces in the video segment, and one or more faceted timelines that visualize a category of detected features (e.g., a visualization of detected visual scenes, audio classifications, visual artifacts). In some embodiments, interacting with a particular interactive tile navigates to a corresponding portion of the video, adds a corresponding video segment to a selection, and/or scrubs through tile thumbnails.

公开(公告)号：US11995894B2

公开(公告)日：2024-05-28

申请号：US17017353

申请日：2020-09-10

Applicant: ADOBE INC.

Inventor： Seth Walker ,  Joy Oakyung Kim ,  Hijung Shin ,  Aseem Agarwala ,  Joel R. Brandt ,  Jovan Popović ,  Lubomira Dontcheva ,  Dingzeyu Li ,  Xue Bai

IPC: G06V20/40 ,  G06T7/10

CPC classification number: G06V20/49 ,  G06T7/10 ,  G06V20/41 ,  G06T2207/10016

Abstract: Embodiments are directed to techniques for interacting with a hierarchical video segmentation using a metadata panel with a composite list of video metadata. The composite list is segmented into selectable metadata segments at locations corresponding to boundaries of video segments defined by a hierarchical segmentation. In some embodiments, the finest level of a hierarchical segmentation identifies the smallest interaction unit of a video—semantically defined video segments of unequal duration called clip atoms, and higher levels cluster the clip atoms into coarser sets of video segments. One or more metadata segments can be selected in various ways, such as by clicking or tapping on a metadata segment or by performing a metadata search. When a metadata segment is selected, a corresponding video segment is emphasized on the video timeline, a playback cursor is moved to the first video frame of the video segment, and the first video frame is presented.