知搜-全球专利检索

  1. Login
  2. Sign Up

Search Results

61 records (took 0.036 seconds)

    Sampled Softmax with Random Fourier Features
    31.
    发明申请
    Sampled Softmax with Random Fourier Features 有权

    公开(公告)号：US20210019654A1

    公开(公告)日：2021-01-21

    申请号：US16931862

    申请日：2020-07-17

    Applicant: Google LLC

    Inventor： Xinnan Yu ,  Ankit Singh Rawat ,  Jiecao Chen ,  Ananda Theertha Suresh ,  Sanjiv Kumar

    IPC: G06N20/00 ,  G06F40/20 ,  G06K9/62 ,  G06F17/14 ,  G06F17/18

    Abstract: Systems and methods for low bias negative sampling of classes according to the sampled softmax method are described herein. The systems and methods can include training a machine-learned model for classifying inputs into one or more classes of a plurality of classes, each of the plurality of classes having an associated class embedding in a plurality of class embeddings. The systems and methods can include selecting, by the one or more computing devices, one or more negative classes from the plurality of classes based at least in part on a probability distribution approximating a softmax distribution, wherein the probability distribution is determined based at least in part on a Random Fourier Features map.

    LOCAL ORTHOGONAL DECOMPOSITION FOR MAXIMUM INNER PRODUCT SEARCH
    32.
    发明申请
    LOCAL ORTHOGONAL DECOMPOSITION FOR MAXIMUM INNER PRODUCT SEARCH 审中-公开

    公开(公告)号：US20200257668A1

    公开(公告)日：2020-08-13

    申请号：US16715620

    申请日：2019-12-16

    Applicant: GOOGLE LLC

    Inventor： Xiang Wu ,  David Morris Simcha ,  Sanjiv Kumar ,  Ruiqi Guo

    IPC: G06F16/22 ,  G06F16/27 ,  G06F16/2458 ,  G06F16/28 ,  G06F17/16 ,  G06F16/248

    Abstract: Techniques of indexing a database and processing a query involve decomposing the residual term according to a projection matrix that is based on a given direction v. For example, for each database element of a partition, the residual for that database element is split into a component parallel to a given direction and a component perpendicular to that direction. The parallel component lies in a one-dimensional subspace spanned by the direction and may be efficiently quantized with a scalar quantization. The perpendicular component is quantized using multiscale quantization techniques. The quantized residual components and the center elements of each partition define the indexed database. Upon receipt of a query from a user, the inner products of q with the residual may be computed efficiently using the quantized residual components. From these inner products, the database elements that are most similar to the query are selected and returned to the user.

    Decreasing neural network inference times using softmax approximation
    33.
    发明授权
    Decreasing neural network inference times using softmax approximation 审中-公开

    公开(公告)号：US10671909B2

    公开(公告)日：2020-06-02

    申请号：US16586702

    申请日：2019-09-27

    Applicant: Google LLC

    Inventor： Yang Li ,  Sanjiv Kumar ,  Pei-Hung Chen ,  Si Si ,  Cho-Jui Hsieh

    IPC: G06N3/04 ,  G06F17/16 ,  G06F17/18 ,  G06K9/62

    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for decreasing neural network inference times using softmax approximation. One of the methods includes maintaining data specifying a respective softmax weight vector for each output in a vocabulary of possible neural network outputs; receiving a neural network input; processing the neural network input using one or more initial neural network layers to generate a context vector for the neural network input; and generating an approximate score distribution over the vocabulary of possible neural network outputs for the neural network input, comprising: processing the context vector using a screening model configured to predict a proper subset of the vocabulary for the context input; and generating a respective logit for each output that is in the proper subset, comprising applying the softmax weight vector for the output to the context vector.

    Systems and Methods for Weighted Quantization
    39.
    发明公开
    Systems and Methods for Weighted Quantization 审中-公开

    公开(公告)号：US20240061889A1

    公开(公告)日：2024-02-22

    申请号：US18456688

    申请日：2023-08-28

    Applicant: Google LLC

    Inventor： Ruiqi Guo ,  David Simcha ,  Quan Geng ,  Felix Chern ,  Sanjiv Kumar ,  Xiang Wu

    IPC: G06F16/906 ,  G06F16/2457 ,  G06F16/25 ,  H03M7/30

    CPC classification number: G06F16/906 ,  G06F16/24578 ,  G06F16/258 ,  H03M7/30

    Abstract: Generally, the present disclosure is directed to systems and methods of quantizing a database with respect to a novel loss or quantization error function which applies a weight to an error measurement of quantized elements respectively corresponding to the datapoints in the database. The weight is determined based on the magnitude of an inner product between the respective datapoints and a query compared therewith. In contrast to previous work, embodiments of the proposed loss function are responsive to the expected magnitude of an inner product between the respective datapoints and a query compared therewith and can prioritize error reduction for higher-ranked pairings of the query and the datapoints. Thus, the systems and methods of the present disclosure provide solutions to some of the problems with traditional quantization approaches, which regard all error as equally impactful.

    Multiscale quantization for fast similarity search
    40.
    发明授权
    Multiscale quantization for fast similarity search 有权

    公开(公告)号：US11874866B2

    公开(公告)日：2024-01-16

    申请号：US18081376

    申请日：2022-12-14

    Applicant: Google LLC

    Inventor： Xiang Wu ,  David Simcha ,  Daniel Holtmann-Rice ,  Sanjiv Kumar ,  Ananda Theertha Suresh ,  Ruiqi Guo ,  Xinnan Yu

    IPC: G06F16/33 ,  G06F16/31 ,  G06N20/00

    CPC classification number: G06F16/3347 ,  G06F16/313 ,  G06F16/319 ,  G06N20/00

    Abstract: The present disclosure provides systems and methods that include or otherwise leverage use of a multiscale quantization model that is configured to provide a quantized dataset. In particular, the multiscale quantization model can receive and perform vector quantization of a first dataset. The multiscale quantization model can generate a residual dataset based at least in part on a result of the vector quantization. The multiscale quantization model can apply a rotation matrix to the residual dataset to generate a rotated residual dataset that includes a plurality of rotated residuals. The multiscale quantization model can perform reparameterization of each rotated residual in the rotated residual dataset into a direction component and a scale component. The multiscale quantization model can perform product quantization of the direction components of the plurality of rotated residuals, and perform scalar quantization of the scale components of the plurality of rotated residuals.

Patent Agency Ranking