公开(公告)号：US20210347379A1

公开(公告)日：2021-11-11

申请号：US17229350

申请日：2021-04-13

申请人： Stradvision, Inc.

发明人： Hongmo Je ,  Yongjoong Kim ,  Dongkyu Yu ,  Sung An Gweon

IPC分类号： B60W60/00 ,  G06K9/62 ,  G06N20/00

摘要： A method for performing on-device learning of embedded machine learning network of autonomous vehicle by using multi-stage learning with adaptive hyper-parameter sets is provided. The processes include: (a) dividing the current learning into a 1-st stage learning to an n-th stage learning, assigning 1-st stage training data to n-th stage training data, generating a 1_1-st hyper-parameter set candidate to a 1_h-th hyper-parameter set candidate, training the embedded machine learning network in the 1-st stage learning, and determining a 1-st adaptive hyper-parameter set; (b) generating a k_1-st hyper-parameter set candidate to a k_h-th hyper-parameter set candidate, training the (k−1)-th stage-completed machine learning network in the k-th stage learning, and determining a k-th adaptive hyper-parameter set; and (c) generating an n-th adaptive hyper-parameter set, and executing the n-th stage learning, to thereby complete the current learning.

公开(公告)号：US11087175B2

公开(公告)日：2021-08-10

申请号：US16723753

申请日：2019-12-20

申请人： Stradvision, Inc.

发明人： Kye-Hyeon Kim ,  Yongjoong Kim ,  Hak-Kyoung Kim ,  Woonhyun Nam ,  SukHoon Boo ,  Myungchul Sung ,  Dongsoo Shin ,  Donghun Yeo ,  Wooju Ryu ,  Myeong-Chun Lee ,  Hyungsoo Lee ,  Taewoong Jang ,  Kyungjoong Jeong ,  Hongmo Je ,  Hojin Cho

IPC分类号： G06K9/00 ,  G06K9/62 ,  G05D1/00 ,  G05D1/02 ,  G06N3/04 ,  G06N3/08

摘要： A method for learning a recurrent neural network to check an autonomous driving safety to be used for switching a driving mode of an autonomous vehicle is provided. The method includes steps of: a learning device (a) if training images corresponding to a front and a rear cameras of the autonomous vehicle are acquired, inputting each pair of the training images into corresponding CNNs, to concatenate the training images and generate feature maps for training, (b) inputting the feature maps for training into long short-term memory models corresponding to sequences of a forward RNN, and into those corresponding to the sequences of a backward RNN, to generate updated feature maps for training and inputting feature vectors for training into an attention layer, to generate an autonomous-driving mode value for training, and (c) allowing a loss layer to calculate losses and to learn the long short-term memory models.

公开(公告)号：US11042780B2

公开(公告)日：2021-06-22

申请号：US16723753

申请日：2019-12-20

申请人： Stradvision, Inc.

发明人： Kye-Hyeon Kim ,  Yongjoong Kim ,  Hak-Kyoung Kim ,  Woonhyun Nam ,  SukHoon Boo ,  Myungchul Sung ,  Dongsoo Shin ,  Donghun Yeo ,  Wooju Ryu ,  Myeong-Chun Lee ,  Hyungsoo Lee ,  Taewoong Jang ,  Kyungjoong Jeong ,  Hongmo Je ,  Hojin Cho

IPC分类号： G06K9/00 ,  G06K9/62 ,  G05D1/00 ,  G05D1/02 ,  G06N3/04 ,  G06N3/08

摘要： A method for learning a recurrent neural network to check an autonomous driving safety to be used for switching a driving mode of an autonomous vehicle is provided. The method includes steps of: a learning device (a) if training images corresponding to a front and a rear cameras of the autonomous vehicle are acquired, inputting each pair of the training images into corresponding CNNs, to concatenate the training images and generate feature maps for training, (b) inputting the feature maps for training into long short-term memory models corresponding to sequences of a forward RNN, and into those corresponding to the sequences of a backward RNN, to generate updated feature maps for training and inputting feature vectors for training into an attention layer, to generate an autonomous-driving mode value for training, and (c) allowing a loss layer to calculate losses and to learn the long short-term memory models.

公开(公告)号：US10970633B1

公开(公告)日：2021-04-06

申请号：US17135301

申请日：2020-12-28

申请人： Stradvision, Inc.

发明人： Sung An Gweon ,  Yongjoong Kim ,  Bongnam Kang ,  Hongmo Je

IPC分类号： G06N3/02 ,  G06N3/08 ,  G06K9/62 ,  G06N20/10

摘要： A method for optimizing an on-device neural network model by using a Sub-kernel Searching Module is provided. The method includes steps of a learning device (a) if a Big Neural Network Model having a capacity capable of performing a targeted task by using a maximal computing power of an edge device has been trained to generate a first inference result on an input data, allowing the Sub-kernel Searching Module to identify constraint and a state vector corresponding to the training data, to generate architecture information on a specific sub-kernel suitable for performing the targeted task on the training data, (b) optimizing the Big Neural Network Model according to the architecture information to generate a specific Small Neural Network Model for generating a second inference result on the training data, and (c) training the Sub-kernel Searching Module by using the first and the second inference result.

公开(公告)号：US10817777B2

公开(公告)日：2020-10-27

申请号：US16724302

申请日：2019-12-22

申请人： STRADVISION, INC.

发明人： Kye-Hyeon Kim ,  Yongjoong Kim ,  Hak-Kyoung Kim ,  Woonhyun Nam ,  SukHoon Boo ,  Myungchul Sung ,  Dongsoo Shin ,  Donghun Yeo ,  Wooju Ryu ,  Myeong-Chun Lee ,  Hyungsoo Lee ,  Taewoong Jang ,  Kyungjoong Jeong ,  Hongmo Je ,  Hojin Cho

IPC分类号： G06N3/04 ,  G06K9/00 ,  G06K9/62 ,  G06N3/08

摘要： A learning method for generating integrated object detection information by integrating first object detection information and second object detection information is provided. And the method includes steps of: (a) a learning device instructing a concatenating network to generate one or more pair feature vectors; (b) the learning device instructing a determining network to apply FC operations to the pair feature vectors, to thereby generate (i) determination vectors and (ii) box regression vectors; (c) the learning device instructing a loss unit to generate an integrated loss by referring to the determination vectors, the box regression vectors and their corresponding GTs, and performing backpropagation processes by using the integrated loss, to thereby learn at least part of parameters included in the DNN.

公开(公告)号：US10796206B2

公开(公告)日：2020-10-06

申请号：US16739220

申请日：2020-01-10

申请人： StradVision, Inc.

发明人： Kye-Hyeon Kim ,  Yongjoong Kim ,  Hak-Kyoung Kim ,  Woonhyun Nam ,  Sukhoon Boo ,  Myungchul Sung ,  Dongsoo Shin ,  Donghun Yeo ,  Wooju Ryu ,  Myeong-Chun Lee ,  Hyungsoo Lee ,  Taewoong Jang ,  Kyungjoong Jeong ,  Hongmo Je ,  Hojin Cho

IPC分类号： G06K9/62 ,  G06N3/04 ,  G06N3/08 ,  G06K9/00

摘要： A method for integrating images from vehicles performing a cooperative driving is provided. The method includes steps of: a main driving image integrating device on one main vehicle (a) inputting one main driving image into a main object detector to (1) generate one main feature map by applying convolution operation via a main convolutional layer, (2) generate main ROIs via a main region proposal network, (3) generate main pooled feature maps by applying pooling operation via a main pooling layer, and (4) generate main object detection information on the main objects by applying fully-connected operation via a main fully connected layer; (b) inputting the main pooled feature maps into a main confidence network to generate main confidences; and (c) acquiring sub-object detection information and sub-confidences from sub-vehicles, and integrating the main object detection information and the sub-object detection information using the main & the sub-confidences to generate object detection result.

公开(公告)号：US10776542B2

公开(公告)日：2020-09-15

申请号：US16723450

申请日：2019-12-20

申请人： Stradvision, Inc.

发明人： Kye-Hyeon Kim ,  Yongjoong Kim ,  Hak-Kyoung Kim ,  Woonhyun Nam ,  SukHoon Boo ,  Myungchul Sung ,  Dongsoo Shin ,  Donghun Yeo ,  Wooju Ryu ,  Myeong-Chun Lee ,  Hyungsoo Lee ,  Taewoong Jang ,  Kyungjoong Jeong ,  Hongmo Je ,  Hojin Cho

IPC分类号： G06F30/27 ,  G06K9/62 ,  G06N3/08 ,  G06N3/04 ,  G06F111/10

摘要： A method for calibrating a physics engine of a virtual world simulator for learning of a deep learning-based device is provided. The method includes steps of a calibrating device (a) if virtual current frame information corresponding to a virtual current state in virtual environment is acquired, (i) transmitting the virtual current frame information to the deep learning-based device to output virtual action information, (ii) transmitting the virtual current frame information and the virtual action information to the physics engine to output virtual next frame information corresponding to the virtual current frame information and the virtual action information, and (iii) transmitting the virtual current frame information and the virtual action information to a real state network learned to output predicted next frame information in response to action in a real environment to output predicted real next frame information; and (b) optimizing the previous calibrated parameters to generate current calibrated parameters.

公开(公告)号：US10762393B2

公开(公告)日：2020-09-01

申请号：US16739201

申请日：2020-01-10

申请人： StradVision, Inc.

发明人： Kye-Hyeon Kim ,  Yongjoong Kim ,  Hak-Kyoung Kim ,  Woonhyun Nam ,  Sukhoon Boo ,  Myungchul Sung ,  Dongsoo Shin ,  Donghun Yeo ,  Wooju Ryu ,  Myeong-Chun Lee ,  Hyungsoo Lee ,  Taewoong Jang ,  Kyungjoong Jeong ,  Hongmo Je ,  Hojin Cho

IPC分类号： G06K9/62 ,  G06N3/08 ,  G06N3/04

摘要： A method for learning an automatic labeling device for auto-labeling a base image of a base vehicle using sub-images of nearby vehicles is provided. The method includes steps of: a learning device inputting the base image and the sub-images into previous trained dense correspondence networks to generate dense correspondences; and into encoders to output convolution feature maps, inputting the convolution feature maps into decoders to output deconvolution feature maps; with an integer k from 1 to n, generating a k-th adjusted deconvolution feature map by translating coordinates of a (k+1)-th deconvolution feature map using a k-th dense correspondence; generating a concatenated feature map by concatenating the 1-st deconvolution feature map and the adjusted deconvolution feature maps; and inputting the concatenated feature map into a masking layer to output a semantic segmentation image and instructing a 1-st loss layer to calculate 1-st losses and updating decoder weights and encoder weights.

公开(公告)号：US10740593B1

公开(公告)日：2020-08-11

申请号：US16721961

申请日：2019-12-20

申请人： STRADVISION, INC.

发明人： Kye-Hyeon Kim ,  Yongjoong Kim ,  Hak-Kyoung Kim ,  Woonhyun Nam ,  SukHoon Boo ,  Myungchul Sung ,  Dongsoo Shin ,  Donghun Yeo ,  Wooju Ryu ,  Myeong-Chun Lee ,  Hyungsoo Lee ,  Taewoong Jang ,  Kyungjoong Jeong ,  Hongmo Je ,  Hojin Cho

IPC分类号： G06K9/00 ,  G06K9/62 ,  G06K9/46

摘要： A method for face recognition by using a multiple patch combination based on a deep neural network is provided. The method includes steps of: a face-recognizing device, (a) if a face image with a 1-st size is acquired, inputting the face image into a feature extraction network, to allow the feature extraction network to generate a feature map by applying convolution operation to the face image with the 1-st size, and to generate multiple features by applying sliding-pooling operation to the feature map, wherein the feature extraction network has been learned to extract a feature using a face image for training having a 2-nd size and wherein the 2-nd size is smaller than the 1-st size; and (b) inputting the multiple features into a learned neural aggregation network, to allow the neural aggregation network to aggregate the multiple features and to output an optimal feature for the face recognition.

公开(公告)号：US20200242476A1

公开(公告)日：2020-07-30

申请号：US16442691

申请日：2019-06-17

申请人： Stradvision, Inc.

发明人： Kye-Hyeon Kim ,  Yongjoong Kim ,  Insu Kim ,  Hak-Kyoung Kim ,  Woonhyun Nam ,  SukHoon Boo ,  Myungchul Sung ,  Donghun Yeo ,  Wooju Ryu ,  Taewoong Jang ,  Kyungjoong Jeong ,  Hongmo Je ,  Hojin Cho

IPC分类号： G06N3/08 ,  G06N3/04

摘要： A method for on-device continual learning of a neural network which analyzes input data is provided to be used for smartphones, drones, vessels, or a military purpose. The method includes steps of: a learning device, (a) sampling new data to have a preset first volume, instructing an original data generator network, which has been learned, to repeat outputting synthetic previous data corresponding to a k-dimension random vector and previous data having been used for learning the original data generator network, such that the synthetic previous data has a second volume, and generating a batch for a current-learning; and (b) instructing the neural network to generate output information corresponding to the batch. The method can be performed by generative adversarial networks (GANs), online learning, and the like. Also, the present disclosure has effects of saving resources such as storage, preventing catastrophic forgetting, and securing privacy.