-
公开(公告)号：US12033080B2
公开(公告)日：2024-07-09
申请号：US16442203
申请日：2019-06-14
申请人： GOOGLE LLC
发明人： Xinnan Yu , Shanshan Wu , Daniel Holtmann-Rice , Dmitry Storcheus , Sanjiv Kumar , Afshin Rostamizadeh
摘要： A sparse dataset is encoded using a data-driven learned sensing matrix. For example, an example method includes receiving a dataset of sparse vectors with dimension d from a requesting process, initializing an encoding matrix of dimension k×d, selecting a subset of sparse vectors from the dataset, and updating the encoding matrix via machine learning. Updating the encoding matrix includes using a linear encoder to generate an encoded vector of dimension k for each vector in the subset, the linear encoder using the encoding matrix, using a non-linear decoder to decode each of the encoded vectors, the non-linear decoder using a transpose of the encoding matrix in a projected subgradient, and adjusting the encoding matrix using back propagation. The method also includes returning an embedding of each sparse vector in the dataset of sparse vectors, the embedding being generated with the updated encoding matrix.
-
公开(公告)号：US11775823B2
公开(公告)日：2023-10-03
申请号：US17014139
申请日：2020-09-08
申请人： Google LLC
摘要： Generally, the present disclosure is directed to systems and methods that perform adaptive optimization with improved convergence properties. The adaptive optimization techniques described herein are useful in various optimization scenarios, including, for example, training a machine-learned model such as, for example, a neural network. In particular, according to one aspect of the present disclosure, a system implementing the adaptive optimization technique can, over a plurality of iterations, employ an adaptive effective learning rate while also ensuring that the effective learning rate is non-increasing.
-
公开(公告)号：US20230123941A1
公开(公告)日：2023-04-20
申请号：US18081376
申请日：2022-12-14
申请人： Google LLC
发明人： Xiang Wu , David Simcha , Daniel Holtmann-Rice , Sanjiv Kumar , Ananda Theertha Suresh , Ruiqi Guo , Xinnan Yu
摘要： The present disclosure provides systems and methods that include or otherwise leverage use of a multiscale quantization model that is configured to provide a quantized dataset. In particular, the multiscale quantization model can receive and perform vector quantization of a first dataset. The multiscale quantization model can generate a residual dataset based at least in part on a result of the vector quantization. The multiscale quantization model can apply a rotation matrix to the residual dataset to generate a rotated residual dataset that includes a plurality of rotated residuals. The multiscale quantization model can perform reparameterization of each rotated residual in the rotated residual dataset into a direction component and a scale component. The multiscale quantization model can perform product quantization of the direction components of the plurality of rotated residuals, and perform scalar quantization of the scale components of the plurality of rotated residuals.
-
公开(公告)号：US11586904B2
公开(公告)日：2023-02-21
申请号：US16130058
申请日：2018-09-13
申请人： Google LLC
摘要： Generally, the present disclosure is directed to systems and methods that perform adaptive optimization with improved convergence properties. The adaptive optimization techniques described herein are useful in various optimization scenarios, including, for example, training a machine-learned model such as, for example, a neural network. In particular, according to one aspect of the present disclosure, a system implementing the adaptive optimization technique can, over a plurality of iterations, employ an adaptive learning rate while also ensuring that the learning rate is non-increasing.
-
公开(公告)号：US20210064634A1
公开(公告)日：2021-03-04
申请号：US17001850
申请日：2020-08-25
申请人： Google LLC
发明人： Ruiqi Guo , David Simcha , Quan Geng , Felix Chern , Sanjiv Kumar , Xiang Wu
IPC分类号： G06F16/25 , G06F16/2457 , H03M7/30
摘要： Generally, the present disclosure is directed to systems and methods of quantizing a database with respect to a novel loss or quantization error function which applies a weight to an error measurement of quantized elements respectively corresponding to the datapoints in the database. The weight is determined based on the magnitude of an inner product between the respective datapoints and a query compared therewith. In contrast to previous work, embodiments of the proposed loss function are responsive to the expected magnitude of an inner product between the respective datapoints and a query compared therewith and can prioritize error reduction for higher-ranked pairings of the query and the datapoints. Thus, the systems and methods of the present disclosure provide solutions to some of the problems with traditional quantization approaches, which regard all error as equally impactful.
-
公开(公告)号：US20200104686A1
公开(公告)日：2020-04-02
申请号：US16586702
申请日：2019-09-27
申请人： Google LLC
发明人： Yang Li , Sanjiv Kumar , Pei-Hung Chen , Si Si , Cho-Jui Hsieh
摘要： Methods, systems, and apparatus, including computer programs encoded on computer storage media, for decreasing neural network inference times using softmax approximation. One of the methods includes maintaining data specifying a respective softmax weight vector for each output in a vocabulary of possible neural network outputs; receiving a neural network input; processing the neural network input using one or more initial neural network layers to generate a context vector for the neural network input; and generating an approximate score distribution over the vocabulary of possible neural network outputs for the neural network input, comprising: processing the context vector using a screening model configured to predict a proper subset of the vocabulary for the context input; and generating a respective logit for each output that is in the proper subset, comprising applying the softmax weight vector for the output to the context vector.
-
公开(公告)号：US10394777B2
公开(公告)日：2019-08-27
申请号：US14951909
申请日：2015-11-25
申请人： Google LLC
发明人： Xinnan Yu , Sanjiv Kumar , Ruiqi Guo
IPC分类号： G06F17/00 , G06F16/22 , G06F16/33 , G06F16/951
摘要： Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for efficiently performing linear projections. In one aspect, a method includes actions for obtaining a plurality of content items from one or more content sources. Additional actions include, extracting a plurality of features from each of the plurality of content items, generating a feature vector for each of the extracted features in order to create a search space, generating a series of element matrices based upon the generated feature vectors, transforming the series of element matrices into a structured matrix such that the transformation preserves one or more relationships associated with each element matrix of the series of element matrices, receiving a search object, searching the enhanced search space based on the received search object, provided one or more links to a content item that are responsive to the search object.
-
公开(公告)号：US20190114343A1
公开(公告)日：2019-04-18
申请号：US15783685
申请日：2017-10-13
申请人： Google LLC
发明人： Ruiqi Guo , Bo Dai , Sanjiv Kumar
摘要： The present disclosure provides systems and methods that perform stochastic generative hashing. According to one example aspect, a machine-learned hashing model that generates a binary hash for an input can be trained in conjunction with a machine-learned generative model that reconstructs the input from the binary hash. The present disclosure provides a novel generative approach to learn hash functions through Minimum Description Length principle such that the learned hash codes maximally compress the dataset. According to another example aspect, the present disclosure provides an efficient learning algorithm based on the stochastic distributional gradient, which avoids the notorious difficulty caused by binary output constraints, to jointly optimize the parameters of the hashing model and the associated generative model. The present disclosure also provides extensive experiments which show that the systems and methods described herein achieve better retrieval results than the existing state-of-the-art methods.
-
公开(公告)号：US12079700B2
公开(公告)日：2024-09-03
申请号：US15793455
申请日：2017-10-25
申请人： GOOGLE LLC
发明人： Daniel Holtmann-Rice , Sanjiv Kumar , Xinnan Yu , Krzysztof Marcin Choromanski , Ananda Theertha Suresh
摘要： Techniques of generating input for a kernel-based machine learning system that uses a kernel to perform classification operations on data involve generating unbiased estimators for gaussian kernels according to a new framework called Structured Orthogonal Random Features (SORF). The unbiased estimator KSORF to the kernel involves a linear transformation matrix WSORF computed using products of a set of pairs of matrices, each pair including an orthogonal matrix and respective diagonal matrix whose elements are real numbers following a specified probability distribution. Typically, the orthogonal matrix is a Walsh-Hadamard matrix, the specified probability distribution is a Rademacher distribution, and there are at least two, usually three, pairs of matrices multiplied together to form the linear transformation matrix WSORF.
-
公开(公告)号：US20240135254A1
公开(公告)日：2024-04-25
申请号：US18488951
申请日：2023-10-17
申请人： Google LLC
发明人： Harikrishna Narasimhan , Wittawat Jitkrittum , Aditya Krishna Menon , Ankit Singh Rawat , Sanjiv Kumar
摘要： Methods, systems, and apparatus, including computer programs encoded on computer storage media, for post-hoc deferral for classification tasks. In particular, a system can perform either post-hoc threshold correction or post-hoc rejector training to account for the cost of deferring model inputs to an expert system for classification.
-
-
-
-
-
-
-
-
-
搜索结果
54 条记录 (耗时 0.024 秒)
