公开(公告)号：US09473778B2

公开(公告)日：2016-10-18

申请号：US14039871

申请日：2013-09-27

Applicant: Apple Inc.

Inventor： Jim C. Chou ,  Craig M. Okruhlica ,  Guy Cote

IPC: H04N7/26 ,  H04N19/17 ,  H04N19/105 ,  H04N19/147 ,  H04N19/192 ,  H04N19/176 ,  H04N19/186

CPC classification number: H04N19/17 ,  H04N19/105 ,  H04N19/147 ,  H04N19/176 ,  H04N19/186 ,  H04N19/192

Abstract: The video encoders described herein may make an initial determination to designate a macroblock as a skip macroblock, but may subsequently reverse that decision based on additional information. For example, an initial skip mode decision may be based on aggregate distortion metrics for the luma component of the macroblock (e.g., SAD, SATD, or SSD), then reversed based on an individual pixel difference metric, an aggregate or individual pixel metric for a chroma component of the macroblock, or on the position of the macroblock within a macroblock row. The final skip mode decision may be based, at least in part, on the maximum difference between any pixel in the macroblock (or in a region of interest within the macroblock) and the corresponding pixel in a reference frame. The initial skip mode decision may be made during an early stage of a pipelined video encoding process and reversed in a later stage.

Abstract translation: 本文描述的视频编码器可以进行初始确定以将宏块指定为跳过宏块，但是随后可以基于附加信息来反转该决定。 例如，初始跳过模式决定可以基于宏块的亮度分量（例如，SAD，SATD或SSD）的聚合失真度量，然后基于单独的像素差异度量来反转，聚合或单独的像素度量用于 宏块的色度分量，或宏块行内宏块的位置。 至少部分地，最终跳过模式决定基于宏块中的任何像素（或宏块内的感兴趣区域）与参考帧中的相应像素之间的最大差异。 初始跳过模式决定可以在流水线视频编码处理的早期阶段进行，并且在稍后的阶段中反转。

公开(公告)号：US09299122B2

公开(公告)日：2016-03-29

申请号：US14037316

申请日：2013-09-25

Applicant: Apple Inc.

Inventor： Craig M. Okruhlica ,  Guy Cote

IPC: G06T1/20 ,  G06T1/60 ,  H04N19/61 ,  H04N19/423 ,  H04N19/436

CPC classification number: G06T1/20 ,  G06T1/60 ,  H04N19/423 ,  H04N19/436 ,  H04N19/61

Abstract: A block processing pipeline in which blocks are input to and processed according to row groups so that adjacent blocks on a row are not concurrently at adjacent stages of the pipeline. A stage of the pipeline may process a current block according to neighbor pixels from one or more neighbor blocks. Since adjacent blocks are not concurrently at adjacent stages, the left neighbor of the current block is at least two stages downstream from the stage. Thus, processed pixels from the left neighbor can be passed back to the stage for use in processing the current block without the need to wait for the left neighbor to complete processing at a next stage of the pipeline. In addition, the neighbor blocks may include blocks from the row above the current block. Information from these neighbor blocks may be passed to the stage from an upstream stage of the pipeline.

Abstract translation: 块处理流水线，其中块被输入并根据行组进行处理，使得一行上的相邻块不是在管道的相邻阶段同时进行。 流水线的一个阶段可以根据一个或多个相邻块的相邻像素处理当前块。 由于相邻块不是在相邻阶段同时进行，所以当前块的左邻近位于级的下游至少两级。 因此，来自左邻居的经处理的像素可被传回到用于处理当前块的阶段，而不需要等待左邻居在流水线的下一阶段完成处理。 此外，相邻块可以包括来自当前块上方的行的块。 来自这些相邻块的信息可以从流水线的上游级传递到级。

公开(公告)号：US09571846B2

公开(公告)日：2017-02-14

申请号：US14039764

申请日：2013-09-27

Applicant: Apple Inc.

Inventor： Timothy John Millet ,  Mark P. Rygh ,  Craig M. Okruhlica ,  Jim C. Chou ,  Guy Cote ,  Gaurav S. Gulati ,  Joseph J. Cheng ,  Joseph P. Bratt

IPC: H04N19/423 ,  H04N19/56 ,  H04N19/53

CPC classification number: H04N19/423 ,  H04N19/53

Abstract: Block processing pipeline methods and apparatus in which reference data are stored to a memory according to tile formats to reduce memory accesses when fetching the data from the memory. When the pipeline stores reference data from a current frame being processed to memory as a reference frame, the reference samples are stored in macroblock sequential order. Each macroblock sample set is stored as a tile. Reference data may be stored in tile formats for luma and chroma. Chroma reference data may be stored in tile formats for chroma 4:2:0, 4:2:2, and/or 4:4:4 formats. A stage of the pipeline may write luma and chroma reference data for macroblocks to memory according to one or more of the macroblock tile formats in a modified knight's order. The stage may delay writing the reference data from the macroblocks until the macroblocks have been fully processed by the pipeline.

Abstract translation: 块处理管道方法和装置，其中参考数据根据瓦片格式存储到存储器中，以在从存储器取出数据时减少存储器访问。 当流水线将正在处理的当前帧的参考数据存储为参考帧时，参考样本以宏块顺序存储。 每个宏块样本集被存储为一个图块。 参考数据可以以瓦片和色度的瓦片格式存储。 色度参考数据可以以瓦4：2：0,4：2：2和/或4：4：4格式的瓦片格式存储。 流水线的一个阶段可以根据改进的骑士顺序中的一个或多个宏块瓦片格式将宏块的亮度和色度参考数据写入存储器。 该阶段可以延迟从宏块写入参考数据，直到宏块已被管道完全处理。

公开(公告)号：US20160173885A1

公开(公告)日：2016-06-16

申请号：US15050061

申请日：2016-02-22

Applicant: Apple Inc.

Inventor： Guy Cote ,  Craig M. Okruhlica

IPC: H04N19/186 ,  H04N19/593 ,  H04N19/423 ,  H04N19/182 ,  H04N19/159 ,  H04N19/96 ,  H04N19/107 ,  H04N19/52 ,  H04N19/13 ,  H04N19/86 ,  H04N19/176 ,  H04N19/40

CPC classification number: H04N19/186 ,  H04N19/107 ,  H04N19/13 ,  H04N19/159 ,  H04N19/176 ,  H04N19/182 ,  H04N19/40 ,  H04N19/423 ,  H04N19/436 ,  H04N19/439 ,  H04N19/52 ,  H04N19/593 ,  H04N19/86 ,  H04N19/96

Abstract: A block processing pipeline in which macroblocks are input to and processed according to row groups so that adjacent macroblocks on a row are not concurrently at adjacent stages of the pipeline. The input method may allow chroma processing to be postponed until after luma processing. One or more upstream stages of the pipeline may process luma elements of each macroblock to generate luma results such as a best mode for processing the luma elements. Luma results may be provided to one or more downstream stages of the pipeline that process chroma elements of each macroblock. The luma results may be used to determine processing of the chroma elements. For example, if the best mode for luma is an intra-frame mode, then a chroma processing stage may determine a best intra-frame mode for chroma and reconstruct the chroma elements according to the best chroma intra-frame mode.

公开(公告)号：US20150092855A1

公开(公告)日：2015-04-02

申请号：US14039871

申请日：2013-09-27

Applicant: Apple Inc.

Inventor： Jim C. Chou ,  Craig M. Okruhlica ,  Guy Cote

IPC: H04N7/26

CPC classification number: H04N19/17 ,  H04N19/105 ,  H04N19/147 ,  H04N19/176 ,  H04N19/186 ,  H04N19/192

Abstract: The video encoders described herein may make an initial determination to designate a macroblock as a skip macroblock, but may subsequently reverse that decision based on additional information. For example, an initial skip mode decision may be based on aggregate distortion metrics for the luma component of the macroblock (e.g., SAD, SATD, or SSD), then reversed based on an individual pixel difference metric, an aggregate or individual pixel metric for a chroma component of the macroblock, or on the position of the macroblock within a macroblock row. The final skip mode decision may be based, at least in part, on the maximum difference between any pixel in the macroblock (or in a region of interest within the macroblock) and the corresponding pixel in a reference frame. The initial skip mode decision may be made during an early stage of a pipelined video encoding process and reversed in a later stage.

Abstract translation: 本文描述的视频编码器可以进行初始确定以将宏块指定为跳过宏块，但是随后可以基于附加信息来反转该决定。 例如，初始跳过模式决定可以基于宏块的亮度分量（例如，SAD，SATD或SSD）的聚合失真度量，然后基于单独的像素差异度量来反转，聚合或单独的像素度量用于 宏块的色度分量，或宏块行内宏块的位置。 至少部分地，最终跳过模式决定基于宏块中的任何像素（或宏块内的感兴趣区域）与参考帧中的相应像素之间的最大差异。 初始跳过模式决定可以在流水线视频编码处理的早期阶段进行，并且在稍后的阶段中反转。

公开(公告)号：US20150084969A1

公开(公告)日：2015-03-26

申请号：US14037316

申请日：2013-09-25

Applicant: Apple Inc.

Inventor： Craig M. Okruhlica ,  Guy Cote

IPC: G06T1/20

CPC classification number: G06T1/20 ,  G06T1/60 ,  H04N19/423 ,  H04N19/436 ,  H04N19/61

Abstract: A block processing pipeline in which blocks are input to and processed according to row groups so that adjacent blocks on a row are not concurrently at adjacent stages of the pipeline. A stage of the pipeline may process a current block according to neighbor pixels from one or more neighbor blocks. Since adjacent blocks are not concurrently at adjacent stages, the left neighbor of the current block is at least two stages downstream from the stage. Thus, processed pixels from the left neighbor can be passed back to the stage for use in processing the current block without the need to wait for the left neighbor to complete processing at a next stage of the pipeline. In addition, the neighbor blocks may include blocks from the row above the current block. Information from these neighbor blocks may be passed to the stage from an upstream stage of the pipeline.

Abstract translation: 块处理流水线，其中块被输入并根据行组进行处理，使得一行上的相邻块不是在管道的相邻阶段同时进行。 流水线的一个阶段可以根据一个或多个相邻块的相邻像素处理当前块。 由于相邻块不是在相邻阶段同时进行，所以当前块的左邻近位于级的下游至少两级。 因此，来自左邻居的经处理的像素可被传回到用于处理当前块的阶段，而不需要等待左邻居在流水线的下一阶段完成处理。 此外，相邻块可以包括来自当前块上方的行的块。 来自这些相邻块的信息可以从流水线的上游级传递到级。

公开(公告)号：US20150287351A1

公开(公告)日：2015-10-08

申请号：US14247373

申请日：2014-04-08

Applicant: Apple Inc.

Inventor： Peter F. Holland ,  Craig M. Okruhlica

IPC: G09G3/20

CPC classification number: G09G3/20 ,  G09G2330/022 ,  G09G2360/08 ,  G09G2360/14

Abstract: A system includes one or more video processing components and a display processing unit. The display processing unit may include one or more processing pipelines that generate read requests to fetch stored pixel data from a memory for subsequent display on a display unit. The display processing unit may also include a timing control unit that may generate an indication that indicates that the display unit will enter an inactive state. In response to receiving the indication, one or more of the video processing components may enter a low power state.

Abstract translation: 系统包括一个或多个视频处理组件和显示处理单元。 显示处理单元可以包括一个或多个处理管线，其生成读取请求以从存储器提取存储的像素数据，以便随后在显示单元上显示。 显示处理单元还可以包括定时控制单元，其可以生成指示显示单元将进入非活动状态的指示。 响应于接收到指示，一个或多个视频处理组件可以进入低功率状态。

公开(公告)号：US09270999B2

公开(公告)日：2016-02-23

申请号：US14037310

申请日：2013-09-25

Applicant: Apple Inc.

Inventor： Guy Cote ,  Craig M. Okruhlica

IPC: H04N19/86 ,  H04N19/107 ,  H04N19/42 ,  H04N19/159 ,  H04N19/176 ,  H04N19/186 ,  H04N19/436

CPC classification number: H04N19/186 ,  H04N19/107 ,  H04N19/13 ,  H04N19/159 ,  H04N19/176 ,  H04N19/182 ,  H04N19/40 ,  H04N19/423 ,  H04N19/436 ,  H04N19/439 ,  H04N19/52 ,  H04N19/593 ,  H04N19/86 ,  H04N19/96

Abstract: A block processing pipeline in which macroblocks are input to and processed according to row groups so that adjacent macroblocks on a row are not concurrently at adjacent stages of the pipeline. The input method may allow chroma processing to be postponed until after luma processing. One or more upstream stages of the pipeline may process luma elements of each macroblock to generate luma results such as a best mode for processing the luma elements. Luma results may be provided to one or more downstream stages of the pipeline that process chroma elements of each macroblock. The luma results may be used to determine processing of the chroma elements. For example, if the best mode for luma is an intra-frame mode, then a chroma processing stage may determine a best intra-frame mode for chroma and reconstruct the chroma elements according to the best chroma intra-frame mode.

Abstract translation: 块处理流水线，其中宏块被输入到并根据行组进行处理，使得一行上的相邻宏块不是在流水线的相邻阶段同时进行。 输入法可以允许色度处理被推迟直到亮度处理。 流水线的一个或多个上游级可以处理每个宏块的亮度元素以产生亮度结果，例如用于处理亮度元素的最佳模式。 亮度结果可以被提供给处理每个宏块的色度元素的流水线的一个或多个下游阶段。 亮度结果可用于确定色度元素的处理。 例如，如果亮度的最佳模式是帧内模式，则色度处理阶段可以确定用于色度的最佳帧内模式，并且根据最佳色度帧内模式重建色度元素。

公开(公告)号：US09196187B2

公开(公告)日：2015-11-24

申请号：US14247373

申请日：2014-04-08

Applicant: Apple Inc.

Inventor： Peter F. Holland ,  Craig M. Okruhlica

IPC: G09G3/20

CPC classification number: G09G3/20 ,  G09G2330/022 ,  G09G2360/08 ,  G09G2360/14

Abstract: A system includes one or more video processing components and a display processing unit. The display processing unit may include one or more processing pipelines that generate read requests to fetch stored pixel data from a memory for subsequent display on a display unit. The display processing unit may also include a timing control unit that may generate an indication that indicates that the display unit will enter an inactive state. In response to receiving the indication, one or more of the video processing components may enter a low power state.

Abstract translation: 系统包括一个或多个视频处理组件和显示处理单元。 显示处理单元可以包括一个或多个处理管线，其生成读取请求以从存储器提取存储的像素数据，以便随后在显示单元上显示。 显示处理单元还可以包括定时控制单元，其可以生成指示显示单元将进入非活动状态的指示。 响应于接收到指示，一个或多个视频处理组件可以进入低功率状态。

公开(公告)号：US20150092843A1

公开(公告)日：2015-04-02

申请号：US14039764

申请日：2013-09-27

Applicant: Apple Inc.

Inventor： Timothy John Millet ,  Mark P. Rygh ,  Craig M. Okruhlica ,  Jim C. Chou ,  Guy Cote ,  Gaurav S. Gulati ,  Joseph J. Cheng ,  Joseph P. Bratt

IPC: H04N19/57 ,  H04N19/186 ,  H04N19/182

CPC classification number: H04N19/423 ,  H04N19/53

Abstract: Block processing pipeline methods and apparatus in which reference data are stored to a memory according to tile formats to reduce memory accesses when fetching the data from the memory. When the pipeline stores reference data from a current frame being processed to memory as a reference frame, the reference samples are stored in macroblock sequential order. Each macroblock sample set is stored as a tile. Reference data may be stored in tile formats for luma and chroma. Chroma reference data may be stored in tile formats for chroma 4:2:0, 4:2:2, and/or 4:4:4 formats. A stage of the pipeline may write luma and chroma reference data for macroblocks to memory according to one or more of the macroblock tile formats in a modified knight's order. The stage may delay writing the reference data from the macroblocks until the macroblocks have been fully processed by the pipeline.

Abstract translation: 块处理管道方法和装置，其中参考数据根据瓦片格式存储到存储器中，以在从存储器取出数据时减少存储器访问。 当流水线将正在处理的当前帧的参考数据存储为参考帧时，参考样本以宏块顺序存储。 每个宏块样本集被存储为一个图块。 参考数据可以以瓦片和色度的瓦片格式存储。 色度参考数据可以以瓦4：2：0,4：2：2和/或4：4：4格式的瓦片格式存储。 流水线的一个阶段可以根据改进的骑士顺序中的一个或多个宏块瓦片格式将宏块的亮度和色度参考数据写入存储器。 该阶段可以延迟从宏块写入参考数据，直到宏块已被管道完全处理。