-
公开(公告)号:US20240244287A1
公开(公告)日:2024-07-18
申请号:US18154412
申请日:2023-01-13
Applicant: Adobe Inc.
Inventor: Kim Pascal PIMMEL , Stephen Joseph DIVERDI , Jiaju MA , Rubaiat HABIB , Li-Yi WEI , Hijung SHIN , Deepali ANEJA , John G. NELSON , Wilmot LI , Dingzeyu LI , Lubomira Assenova DONTCHEVA , Joel Richard BRANDT
IPC: H04N21/431 , G06F3/04812 , G06F3/0482 , H04N21/4402
CPC classification number: H04N21/4312 , G06F3/04812 , G06F3/0482 , H04N21/440236
Abstract: Embodiments of the present disclosure provide, a method, a system, and a computer storage media that provide mechanisms for multimedia effect addition and editing support for text-based video editing tools. The method includes generating a user interface (UI) displaying a transcript of an audio track of a video and receiving, via the UI, input identifying selection of a text segment from the transcript. The method also includes in response to receiving, via the UI, input identifying selection of a particular type of text stylization or layout for application to the text segment. The method further includes identifying a video effect corresponding to the particular type of text stylization or layout, applying the video effect to a video segment corresponding to the text segment, and applying the particular type of text stylization or layout to the text segment to visually represent the video effect in the transcript.
-
公开(公告)号:US20250139161A1
公开(公告)日:2025-05-01
申请号:US18431134
申请日:2024-02-02
Applicant: ADOBE INC.
Inventor: Deepali ANEJA , Zeyu JIN , Hijung SHIN , Anh Lan TRUONG , Dingzeyu LI , Hanieh DEILAMSALEHY , Rubaiat HABIB , Matthew David FISHER , Kim Pascal PIMMEL , Wilmot LI , Lubomira Assenova DONTCHEVA
IPC: G06F16/783 , G06F16/738 , G06V20/40 , G06V40/16
Abstract: Embodiments of the present invention provide systems, methods, and computer storage media for cutting down a user's larger input video into an edited video comprising the most important video segments and applying corresponding video effects. Some embodiments of the present invention are directed to adding captioning video effects to the trimmed video (e.g., applying face-aware and non-face-aware captioning to emphasize extracted video segment headings, important sentences, quotes, words of interest, extracted lists, etc.). For example, a prompt is provided to a generative language model to identify portions of a transcript (e.g., extracted scene summaries, important sentences, lists of items discussed in the video, etc.) to apply to corresponding video segments as captions depending on the type of caption (e.g., an extracted heading may be captioned at the start of a corresponding video segment, important sentences and/or extracted list items may be captioned when they are spoken).
-
公开(公告)号:US20250140291A1
公开(公告)日:2025-05-01
申请号:US18431139
申请日:2024-02-02
Applicant: ADOBE INC.
Inventor: Hanieh DEILAMSALEHY , Jui-Hsien WANG , Zhengyang MA , Dingzeyu LI , Hijung SHIN , Aseem Omprakash AGARWALA , Kim Pascal PIMMEL , Lubomira Assenova DONTCHEVA
IPC: G11B27/031 , G06V20/40 , G10L15/04 , G10L15/183 , G10L21/0272 , G10L25/57 , G11B27/06 , G11B27/34
Abstract: Embodiments of the present invention provide systems, methods, and computer storage media for identifying the relevant segments that effectively summarize the larger input video and/or form a rough cut, and assembling them into one or more smaller trimmed videos. For example, visual scenes and corresponding scene captions are extracted from the input video and associated with an extracted diarized and timestamped transcript to generate an augmented transcript. The augmented transcript is applied to a large language model to extract sentences that characterize a trimmed version of the input video (e.g., a natural language summary, a representation of identified sentences from the transcript). As such, corresponding video segments are identified (e.g., using similarity to match each sentence in a generated summary with a corresponding transcript sentence) and assembled into one or more trimmed videos. In some embodiments, the trimmed video is generated based on a user's query and/or desired length.
-
公开(公告)号:US20230169795A1
公开(公告)日:2023-06-01
申请号:US17539652
申请日:2021-12-01
Applicant: ADOBE INC.
Inventor: JIYOUNG LEE , Justin Jonathan SALAMOM , Dingzeyu LI
CPC classification number: G06V40/20 , G06V10/82 , G06V20/41 , G06V20/46 , G06V20/49 , G06N3/08 , G06N3/0454
Abstract: A method for detecting a cue (e.g., a visual cue or a visual cue combined with an audible cue) occurring together in an input video includes: presenting a user interface to record an example video of a user performing an act including the cue; determining a part of the example video where the cue occurs; applying a feature of the part to a neural network to generate a positive embedding; dividing the input video into a plurality of chunks and applying a feature of each chunk to the neural network to generate a plurality of negative embeddings; applying a feature of a given one of the chunks to the neural network to output a query embedding; and determining whether the cue occurs in the input video from the query embedding, the positive embedding, and the negative embeddings.
-
公开(公告)号:US20250168442A1
公开(公告)日:2025-05-22
申请号:US19033062
申请日:2025-01-21
Applicant: Adobe Inc.
Inventor: Kim Pascal PIMMEL , Stephen Joseph DIVERDI , Jiaju MA , Rubaiat HABIB , LI-Yi WEI , Hijung SHIN , Deepali ANEJA , John G. NELSON , Wilmot LI , Dingzeyu LI , Lubomira Assenova DONTCHEVA , Joel Richard BRANDT
IPC: H04N21/431 , G06F3/04812 , G06F3/0482 , H04N21/4402
Abstract: Embodiments of the present disclosure provide, a method, a system, and a computer storage media that provide mechanisms for multimedia effect addition and editing support for text-based video editing tools. The method includes generating a user interface (UI) displaying a transcript of an audio track of a video and receiving, via the UI, input identifying selection of a text segment from the transcript. The method also includes in response to receiving, via the UI, input identifying selection of a particular type of text stylization or layout for application to the text segment. The method further includes identifying a video effect corresponding to the particular type of text stylization or layout, applying the video effect to a video segment corresponding to the text segment, and applying the particular type of text stylization or layout to the text segment to visually represent the video effect in the transcript.
-
公开(公告)号:US20240161335A1
公开(公告)日:2024-05-16
申请号:US18055310
申请日:2022-11-14
Applicant: Adobe Inc.
Inventor: Yang ZHOU , Jimei YANG , Jun SAITO , Dingzeyu LI , Deepali ANEJA
IPC: G06T7/73 , G06F16/683 , G06F40/242 , G06T7/207
CPC classification number: G06T7/73 , G06F16/685 , G06F40/242 , G06T7/207
Abstract: Embodiments are disclosed for generating a gesture reenactment video sequence corresponding to a target audio sequence using a trained network based on a video motion graph generated from a reference speech video. In particular, in one or more embodiments, the disclosed systems and methods comprise receiving a first input including a reference speech video and generating a video motion graph representing the reference speech video, where each node is associated with a frame of the reference video sequence and reference audio features of the reference audio sequence. The disclosed systems and methods further comprise receiving a second input including a target audio sequence, generating target audio features, identifying a node path through the video motion graph based on the target audio features and the reference audio features, and generating an output media sequence based on the identified node path through the video motion graph paired with the target audio sequence.
-
公开(公告)号:US20240134909A1
公开(公告)日:2024-04-25
申请号:US17967703
申请日:2022-10-17
Applicant: Adobe Inc.
Inventor: Lubomira Assenova DONTCHEVA , Dingzeyu LI , Kim Pascal PIMMEL , Hijung SHIN , Hanieh DEILAMSALEHY , Aseem Omprakash AGARWALA , Joy Oakyung KIM , Joel Richard BRANDT , Cristin Ailidh Fraser
IPC: G06F16/732
CPC classification number: G06F16/732
Abstract: Embodiments of the present invention provide systems, methods, and computer storage media for a visual and text search interface used to navigate a video transcript. In an example embodiment, a freeform text query triggers a visual search for frames of a loaded video that match the freeform text query (e.g., frame embeddings that match a corresponding embedding of the freeform query), and triggers a text search for matching words from a corresponding transcript or from tags of detected features from the loaded video. Visual search results are displayed (e.g., in a row of tiles that can be scrolled to the left and right), and textual search results are displayed (e.g., in a row of tiles that can be scrolled up and down). Selecting (e.g., clicking or tapping on) a search result tile navigates a transcript interface to a corresponding portion of the transcript.
-
公开(公告)号:US20230043769A1
公开(公告)日:2023-02-09
申请号:US17969536
申请日:2022-10-19
Applicant: Adobe Inc.
Inventor: Seth WALKER , Joy O KIM , Aseem AGARWALA , Joel Richard Brandt , Jovan POPOVIC , Lubomira DONTCHEVA , Dingzeyu LI , Hijung SHIN , Xue Bai
IPC: G06F3/04847 , G06F3/0485 , G06F3/04845
Abstract: Embodiments are directed to techniques for interacting with a hierarchical video segmentation using a video timeline. In some embodiments, the finest level of a hierarchical segmentation identifies the smallest interaction unit of a video—semantically defined video segments of unequal duration called clip atoms, and higher levels cluster the clip atoms into coarser sets of video segments. A presented video timeline is segmented based on one of the levels, and one or more segments are selected through interactions with the video timeline. For example, a click or tap on a video segment or a drag operation dragging along the timeline snaps selection boundaries to corresponding segment boundaries defined by the level. Navigating to a different level of the hierarchy transforms the selection into coarser or finer video segments defined by the level. Any operation can be performed on selected video segments, including playing back, trimming, or editing.
-
公开(公告)号:US20220261573A1
公开(公告)日:2022-08-18
申请号:US17175441
申请日:2021-02-12
Applicant: Adobe Inc.
Inventor: Jimei YANG , Deepali ANEJA , Dingzeyu LI , Jun SAITO , Yang ZHOU
IPC: G06K9/00 , H04N21/845 , H04N21/8547 , G06T7/215 , H04N5/06
Abstract: Embodiments are disclosed for re-timing a video sequence to an audio sequence based on the detection of motion beats in the video sequence and audio beats in the audio sequence. In particular, in one or more embodiments, the disclosed systems and methods comprise receiving a first input, the first input including a video sequence, detecting motion beats in the video sequence, receiving a second input, the second input including an audio sequence, detecting audio beats in the audio sequence, modifying the video sequence by matching the detected motions beats in the video sequence to the detected audio beats in the audio sequence, and outputting the modified video sequence.
-
-
-
-
-
-
-
-