Abstract:
Hyper-parameters are selected for training a deep convolutional network by selecting a number of network architectures as part of a database. Each of the network architectures includes one or more local logistic regression layer and is trained to generate a corresponding validation error that is stored in the database. A threshold error for identifying a good set of network architectures and a bad set of network architectures may be estimated based on validation errors in the database. The method also includes choosing a next potential hyper-parameter, corresponding to a next network architecture, based on a metric that is a function of the good set of network architectures. The method further includes selecting a network architecture, from among next network architectures, with a lowest validation error.
Abstract:
A method of updating a set of classifiers includes applying a first set of classifiers to a first set of data. The method further includes requesting, from a remote device, a classifier update based on an output of the first set of classifiers or a performance measure of the application of the first set of classifiers.
Abstract:
Differential encoding in a neural network includes predicting an activation value for a neuron in the neural network based on at least one previous activation value for the neuron. The encoding further includes encoding a value based on a difference between the predicted activation value and an actual activation value for the neuron in the neural network.
Abstract:
A method for selecting a neuron model with a user defined firing rate for operating in a neural network includes selecting the neuron model based on a selected firing rate bandwidth.
Abstract:
Certain aspects of the present disclosure support methods and apparatus for temporal spike encoding for temporal learning in an artificial nervous system. The temporal spike encoding for temporal learning can comprise obtaining sensor data being input into the artificial nervous system, processing the sensor data to generate feature vectors, converting element values of the feature vectors into delays, and causing at least one artificial neuron of the artificial nervous system to spike at times based on the delays.
Abstract:
Certain aspects of the present disclosure support a method and apparatus for conversion of neuron types to a hardware implementation of an artificial nervous system. According to certain aspects, at least one of synapse weights of the artificial nervous system, neuron input channel resistances associated with a neuron model for neuron instances of the artificial nervous system, or neuron input channel potentials associated with the neuron model can be normalized by one or more factors. A linear transformation can be determined for mapping of parameters of the neuron model. Then, the linear transformation can be applied to the parameters of the neuron model to obtain transformed parameters of the neuron model, and at least one of inputs to the neuron instances or dynamics of the neuron model based may be updated based at least in part on the transformed parameters.
Abstract:
Methods and apparatus are provided for processing in an artificial nervous system. According to certain aspects, resolution of one or more functions performed by processing units of a neuron model may be reduced, based at least in part on availability of computational resources or a power target or budget. The reduction in resolution may be compensated for by adjusting one or more network weights.
Abstract:
A method for selecting bit widths for a fixed point machine learning model includes evaluating a sensitivity of model accuracy to bit widths at each computational stage of the model. The method also includes selecting a bit width for parameters, and/or intermediate calculations in the computational stages of the mode. The bit width for the parameters and the bit width for the intermediate calculations may be different. The selected bit width may be determined based on the sensitivity evaluation.
Abstract:
A method of quantizing a floating point machine learning network to obtain a fixed point machine learning network using a quantizer may include selecting at least one moment of an input distribution of the floating point machine learning network. The method may also include determining quantizer parameters for quantizing values of the floating point machine learning network based at least in part on the at least one selected moment of the input distribution of the floating point machine learning network to obtain corresponding values of the fixed point machine learning network.
Abstract:
A method of adaptively selecting a configuration for a machine learning process includes determining current system resources and performance specifications of a current system. A new configuration for the machine learning process is determined based at least in part on the current system resources and the performance specifications. The method also includes dynamically selecting between a current configuration and the new configuration based at least in part on the current system resources and the performance specifications.