公开(公告)号：US09754571B2

公开(公告)日：2017-09-05

申请号：US14720320

申请日：2015-05-22

Applicant: Smule, Inc.

Inventor： Spencer Salazar ,  Rebecca A. Fiebrink ,  Ge Wang ,  Mattias Ljungstrom ,  Jeffrey C. Smith ,  Perry R. Cook

IPC: G10H1/02 ,  G10H1/36 ,  G10L21/013 ,  G06F3/16 ,  G10L21/0356

CPC classification number: G10H1/366 ,  G06F3/165 ,  G10H1/368 ,  G10H2210/251 ,  G10H2210/331 ,  G10H2220/011 ,  G10H2240/125 ,  G10H2240/175 ,  G10H2240/211 ,  G10H2240/251 ,  G10L21/013 ,  G10L21/0356

Abstract: Techniques have been developed to facilitate (1) the capture and pitch correction of vocal performances on handheld or other portable computing devices and (2) the mixing of such pitch-corrected vocal performances with backing tracks for audible rendering on targets that include such portable computing devices and as well as desktops, workstations, gaming stations, even telephony targets. Implementations of the described techniques employ signal processing techniques and allocations of system functionality that are suitable given the generally limited capabilities of such handheld or portable computing devices and that facilitate efficient encoding and communication of the pitch-corrected vocal performances (or precursors or derivatives thereof) via wireless and/or wired bandwidth-limited networks for rendering on portable computing devices or other targets.

公开(公告)号：US20150379974A9

公开(公告)日：2015-12-31

申请号：US14323667

申请日：2014-07-03

Applicant: Smule, Inc.

Inventor： Ge Wang ,  Jeannie Yang ,  Jieun Oh ,  Tom Lieber

IPC: G10H1/00 ,  G06F3/041

CPC classification number: G10H1/0016 ,  G06F3/041 ,  G10H1/0008 ,  G10H1/0033 ,  G10H2220/096 ,  G10H2220/106 ,  G10H2230/015 ,  G10H2230/075 ,  G10H2240/305

Abstract: Synthetic multi-string musical instruments have been developed for capturing and rendering musical performances on handheld or other portable devices in which a multi-touch sensitive display provides one of the input vectors for an expressive performance by a user or musician. Visual cues may be provided on the multi-touch sensitive display to guide the user in a performance based on a musical score. Alternatively, or in addition, uncued freestyle modes of operation may be provided. In either case, it is not the musical score that drives digital synthesis and audible rendering of the synthetic multi-string musical instrument. Rather, it is the stream of user gestures captured at least in part using the multi-touch sensitive display that drives the digital synthesis and audible rendering.

Abstract translation: 已经开发了合成多弦乐器，用于在手持或其他便携式设备上捕获和呈现音乐表演，其中多点触摸敏感显示器提供用户或音乐家表现性能的输入向量之一。 可以在多触点敏感显示器上提供视觉提示，以引导用户基于乐谱的表演。 或者，或另外，可以提供未经批准的自由式操作模式。 在这两种情况下，不是驱动合成多弦乐器的数字合成和可听渲染的乐谱。 相反，至少部分使用驱动数字合成和可听渲染的多点触摸敏感显示器捕获用户手势的流。

公开(公告)号：US10163428B2

公开(公告)日：2018-12-25

申请号：US15583632

申请日：2017-05-01

Applicant: SMULE, INC.

Inventor： Ge Wang ,  Jeannie Yang ,  Jieun Oh ,  Tom Lieber

IPC: G10H1/06 ,  G10H1/00 ,  G06F3/041

Abstract: Synthetic multi-string musical instruments have been developed for capturing and rendering musical performances on handheld or other portable devices in which a multi-touch sensitive display provides one of the input vectors for an expressive performance by a user or musician. Visual cues may be provided on the multi-touch sensitive display to guide the user in a performance based on a musical score. Alternatively, or in addition, uncued freestyle modes of operation may be provided. In either case, it is not the musical score that drives digital synthesis and audible rendering of the synthetic multi-string musical instrument. Rather, it is the stream of user gestures captured at least in part using the multi-touch sensitive display that drives the digital synthesis and audible rendering.

公开(公告)号：US09754572B2

公开(公告)日：2017-09-05

申请号：US14856152

申请日：2015-09-16

Applicant: Smule, Inc.

Inventor： Spencer Salazar ,  Rebecca A. Fiebrink ,  Ge Wang ,  Mattias Ljungstrom ,  Jeffrey Smith ,  Perry Cook

IPC: G10L21/013 ,  G10H1/36 ,  G06F3/16 ,  G10L21/0356

CPC classification number: G10H1/366 ,  G06F3/165 ,  G10H1/368 ,  G10H2210/251 ,  G10H2210/331 ,  G10H2220/011 ,  G10H2240/125 ,  G10H2240/175 ,  G10H2240/211 ,  G10H2240/251 ,  G10L21/013 ,  G10L21/0356

Abstract: Vocal musical performances may be captured and continuously pitch-corrected at a mobile device for mixing and rendering with backing tracks in ways that create compelling user experiences. In some cases, the vocal performances of individual users are captured in the context of a karaoke-style presentation of lyrics in correspondence with audible renderings of a backing track. Such performances can be pitch-corrected in real-time at the mobile device in accord with pitch correction settings. In some cases, such pitch correction settings code a particular key or scale for the vocal performance or for portions thereof. In some cases, pitch correction settings include a score-coded melody sequence of note targets supplied with, or for association with, the lyrics and/or backing track. In some cases, pitch correction settings are dynamically variable based on gestures captured at a user interface.

公开(公告)号：US09640160B2

公开(公告)日：2017-05-02

申请号：US14323667

申请日：2014-07-03

Applicant: Smule, Inc.

Inventor： Ge Wang ,  Jeannie Yang ,  Jieun Oh ,  Tom Lieber

IPC: G09B15/00 ,  G10H1/00 ,  G06F3/041

CPC classification number: G10H1/0016 ,  G06F3/041 ,  G10H1/0008 ,  G10H1/0033 ,  G10H2220/096 ,  G10H2220/106 ,  G10H2230/015 ,  G10H2230/075 ,  G10H2240/305

Abstract: Synthetic multi-string musical instruments have been developed for capturing and rendering musical performances on handheld or other portable devices in which a multi-touch sensitive display provides one of the input vectors for an expressive performance by a user or musician. Visual cues may be provided on the multi-touch sensitive display to guide the user in a performance based on a musical score. Alternatively, or in addition, uncued freestyle modes of operation may be provided. In either case, it is not the musical score that drives digital synthesis and audible rendering of the synthetic multi-string musical instrument. Rather, it is the stream of user gestures captured at least in part using the multi-touch sensitive display that drives the digital synthesis and audible rendering.

公开(公告)号：US20160071503A1

公开(公告)日：2016-03-10

申请号：US14856152

申请日：2015-09-16

Applicant: Smule, Inc.

Inventor： Spencer Salazar ,  Rebecca A. Fiebrink ,  Ge Wang ,  Mattias Ljungstrom ,  Jeffrey Smith ,  Perry Cook

IPC: G10H1/36 ,  G10L21/0356 ,  G10L21/013

CPC classification number: G10H1/366 ,  G06F3/165 ,  G10H1/368 ,  G10H2210/251 ,  G10H2210/331 ,  G10H2220/011 ,  G10H2240/125 ,  G10H2240/175 ,  G10H2240/211 ,  G10H2240/251 ,  G10L21/013 ,  G10L21/0356

Abstract: Vocal musical performances may be captured and continuously pitch-corrected at a mobile device for mixing and rendering with backing tracks in ways that create compelling user experiences. In some cases, the vocal performances of individual users are captured in the context of a karaoke-style presentation of lyrics in correspondence with audible renderings of a backing track. Such performances can be pitch-corrected in real-time at the mobile device in accord with pitch correction settings. In some cases, such pitch correction settings code a particular key or scale for the vocal performance or for portions thereof. In some cases, pitch correction settings include a score-coded melody sequence of note targets supplied with, or for association with, the lyrics and/or backing track. In some cases, pitch correction settings are dynamically variable based on gestures captured at a user interface.

Abstract translation: 声音音乐表演可以被捕获并且在移动设备上连续地进行音调校正，用于混合和渲染具有创造令人信服的用户体验的后备轨道。 在某些情况下，个人用户的声乐表演是在卡拉OK风格的歌词呈现的背景下被捕获的，与歌曲的背景轨道的可听效果相对应。 可以根据音调校正设置在移动设备上实时地对这些演奏进行音调校正。 在某些情况下，这种音调校正设置代表声乐表演的特定键或音阶或其部分。 在某些情况下，音调校正设置包括与歌词和/或背景音轨一起提供或用于与歌词和/或背景音轨相关联的音符目标的乐谱编曲曲。 在某些情况下，基于在用户界面捕获的手势，音调校正设置是动态变化的。

公开(公告)号：US20140318347A1

公开(公告)日：2014-10-30

申请号：US14323667

申请日：2014-07-03

Applicant: Smule, Inc.

Inventor： Ge Wang ,  Jeannie Yang ,  Jieun Oh ,  Tom Lieber

IPC: G10H1/00 ,  G06F3/041

CPC classification number: G10H1/0016 ,  G06F3/041 ,  G10H1/0008 ,  G10H1/0033 ,  G10H2220/096 ,  G10H2220/106 ,  G10H2230/015 ,  G10H2230/075 ,  G10H2240/305

Abstract: Synthetic multi-string musical instruments have been developed for capturing and rendering musical performances on handheld or other portable devices in which a multi-touch sensitive display provides one of the input vectors for an expressive performance by a user or musician. Visual cues may be provided on the multi-touch sensitive display to guide the user in a performance based on a musical score. Alternatively, or in addition, uncued freestyle modes of operation may be provided. In either case, it is not the musical score that drives digital synthesis and audible rendering of the synthetic multi-string musical instrument. Rather, it is the stream of user gestures captured at least in part using the multi-touch sensitive display that drives the digital synthesis and audible rendering.

Abstract translation: 已经开发了合成多弦乐器，用于在手持或其他便携式设备上捕获和呈现音乐表演，其中多点触摸敏感显示器提供用户或音乐家表现性能的输入向量之一。 可以在多触点敏感显示器上提供视觉提示，以引导用户基于乐谱的表演。 或者，或另外，可以提供未经批准的自由式操作模式。 在这两种情况下，不是驱动合成多弦乐器的数字合成和可听渲染的乐谱。 相反，至少部分使用驱动数字合成和可听渲染的多点触摸敏感显示器捕获用户手势的流。

公开(公告)号：US20140290465A1

公开(公告)日：2014-10-02

申请号：US14231651

申请日：2014-03-31

Applicant: Smule, Inc.

Inventor： Spencer Salazar ,  Ge Wang ,  Perry Cook

IPC: G10H1/00

CPC classification number: G10H1/0033 ,  G10H1/0008 ,  G10H2220/096 ,  G10H2220/361 ,  G10H2220/395 ,  G10H2240/175 ,  G10H2240/251

Abstract: Techniques have been developed for capturing and rendering musical performances on handheld or other portable devices using signal processing techniques suitable given the somewhat limited capabilities of such devices and in ways that facilitate efficient encoding and communication of such captured performances via wireless networks. The developed techniques facilitate the capture, encoding and use of gesture streams for rendering of a musical performance. In some embodiments, a gesture stream encoding facilitates audible rendering of the musical performance locally on the portable device on which the musical performance is captured, typically in real time. In some embodiments, a gesture stream efficiently codes the musical performance for transmission from the portable device on which the musical performance is captured to (or toward) a remote device on which the musical performance is (or can be) rendered. Indeed, is some embodiments, a gesture stream so captured and encoded may be rendered both locally and on remote devices using substantially identical or equivalent instances of a digital synthesis of the musical instrument executing on the local and remote devices.

Abstract translation: 已经开发了用于在手持设备或其他便携式设备上使用适合于使用这种设备的有限能力的信号处理技术来捕获和呈现音乐表演的技术，并且以促进经由无线网络的这种捕获的性能的有效编码和通信的方式。 所开发的技术有助于拍摄，编码和使用手势流来呈现音乐表演。 在一些实施例中，手势流编码有助于在通常实时地捕获音乐表演的便携式设备上本地对音乐表演的可听渲染。 在一些实施例中，手势流有效地编码用于从其上拍摄演奏的便携式设备（或向）呈现演奏（或可以）呈现的远程设备的演奏的演奏。 实际上，在一些实施例中，使用本地和远程设备上执行的乐器的数字合成的基本相同或等同的实例，可以在本地和远程设备上呈现拍摄和编码的手势流。

公开(公告)号：US20210125595A1

公开(公告)日：2021-04-29

申请号：US16898891

申请日：2020-06-11

Applicant: SMULE, INC.

Inventor： Spencer Salazar ,  Rebecca A. Fiebrink ,  Ge Wang ,  Mattias Ljungstrom ,  Jeffrey C. Smith ,  Perry R. Cook

IPC: G10H1/36 ,  G10L21/013 ,  G06F3/16 ,  G10L21/0356

Abstract: Techniques have been developed to facilitate (1) the capture and pitch correction of vocal performances on handheld or other portable computing devices and (2) the mixing of such pitch-corrected vocal performances with backing tracks for audible rendering on targets that include such portable computing devices and as well as desktops, workstations, gaming stations, even telephony targets. Implementations of the described techniques employ signal processing techniques and allocations of system functionality that are suitable given the generally limited capabilities of such handheld or portable computing devices and that facilitate efficient encoding and communication of the pitch-corrected vocal performances (or precursors or derivatives thereof) via wireless and/or wired bandwidth-limited networks for rendering on portable computing devices or other targets.

公开(公告)号：US10685634B2

公开(公告)日：2020-06-16

申请号：US15693946

申请日：2017-09-01

Applicant: SMULE, INC.

Inventor： Spencer Salazar ,  Rebecca A. Fiebrink ,  Ge Wang ,  Mattias Ljungstrom ,  Jeffrey C. Smith ,  Perry R. Cook

IPC: G10H1/02 ,  G10H1/36 ,  G10L21/013 ,  G06F3/16 ,  G10L21/0356

Abstract: Techniques have been developed to facilitate (1) the capture and pitch correction of vocal performances on handheld or other portable computing devices and (2) the mixing of such pitch-corrected vocal performances with backing tracks for audible rendering on targets that include such portable computing devices and as well as desktops, workstations, gaming stations, even telephony targets. Implementations of the described techniques employ signal processing techniques and allocations of system functionality that are suitable given the generally limited capabilities of such handheld or portable computing devices and that facilitate efficient encoding and communication of the pitch-corrected vocal performances (or precursors or derivatives thereof) via wireless and/or wired bandwidth-limited networks for rendering on portable computing devices or other targets.