org.joone.engine.extenders
Class RpropExtender

java.lang.Object
  extended by org.joone.engine.extenders.LearnerExtender
      extended by org.joone.engine.extenders.DeltaRuleExtender
          extended by org.joone.engine.extenders.RpropExtender

public class RpropExtender
extends DeltaRuleExtender

This class changes the delta value in such a way that it implements the RPROP algorithm.

Author:
Boris Jansen

Field Summary
protected  double[][] theDeltas
          Each weight has its own individual update-value (delta_ij(t)) represented by the next object.
protected  double[][] thePreviousGradients
          The gradient pattern of the previous epoch (dE(t-1)/dW_ij).
protected  RpropParameters theRpropParameters
          The parameters for the RPROP learning algorithm.
protected  double[][] theSummedGradients
          The current som of the gradients of all patterns seen so far.
 
Constructor Summary
RpropExtender()
          Creates a new instance of RpropExtender
 
Method Summary
 double getDelta(double[] currentGradientOuts, int j, double aPreviousDelta)
          Computes the delta value for a bias.
 double getDelta(double[] currentInps, int j, double[] currentPattern, int k, double aPreviousDelta)
          Computes the delta value for a weight.
 RpropParameters getParameters()
          Gets the parameters of this learning algorithm.
 void postBiasUpdate(double[] currentGradientOuts)
          Gives extenders a change to do some post-computing after the biases are updated.
 void postWeightUpdate(double[] currentPattern, double[] currentInps)
          Gives extenders a change to do some post-computing after the weights are updated.
 void preBiasUpdate(double[] currentGradientOuts)
          Gives extenders a change to do some pre-computing before the biases are updated.
 void preWeightUpdate(double[] currentPattern, double[] currentInps)
          Gives extenders a change to do some pre-computing before the weights are updated.
 void reinit()
          (Re)Initializes this RPROP learner.
 void setParameters(RpropParameters aParameters)
          Sets the parameters for this learning algorithm.
protected  double sign(double d)
          Gets the sign of a double.
 
Methods inherited from class org.joone.engine.extenders.LearnerExtender
getLearner, isEnabled, setEnabled, setLearner
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

theDeltas

protected double[][] theDeltas
Each weight has its own individual update-value (delta_ij(t)) represented by the next object. The weight update deltaW_ij(t) is defined as follows (dE(t) / dW_ij is the summed gradient for a single epoch): | -delta_ij(t), if dE(t) / dW_ij > 0 deltaW_ij(t) = | delta_ij(t), if dE(t) / dW_ij < 0 | 0 otherwise The delta_ij values are updated as follows: | eta_inc * delta_ij(t-1), if dE(t-1)/ dW_ij * dE(t)/ dW_ij > 0 delta_ij(t) = | eta_dec * delta_ij(t-1), if dE(t-1)/ dW_ij * dE(t)/ dW_ij < 0 | delta_ij(t-1), otherwise where 0 < eta_dec < 1 < eta_inc


thePreviousGradients

protected double[][] thePreviousGradients
The gradient pattern of the previous epoch (dE(t-1)/dW_ij).


theRpropParameters

protected RpropParameters theRpropParameters
The parameters for the RPROP learning algorithm.


theSummedGradients

protected double[][] theSummedGradients
The current som of the gradients of all patterns seen so far. The number of summed gradients is smaller the "batch size" and will be reset to zero, if the number of sums becomes equal to "batch size" after the weights/biases have been updated.

Constructor Detail

RpropExtender

public RpropExtender()
Creates a new instance of RpropExtender

Method Detail

reinit

public void reinit()
(Re)Initializes this RPROP learner.


getDelta

public double getDelta(double[] currentGradientOuts,
                       int j,
                       double aPreviousDelta)
Description copied from class: DeltaRuleExtender
Computes the delta value for a bias.

Specified by:
getDelta in class DeltaRuleExtender
Parameters:
currentGradientOuts - the back propagated gradients.
j - the index of the bias.
aPreviousDelta - a delta value calculated by a previous delta extender.

getDelta

public double getDelta(double[] currentInps,
                       int j,
                       double[] currentPattern,
                       int k,
                       double aPreviousDelta)
Description copied from class: DeltaRuleExtender
Computes the delta value for a weight.

Specified by:
getDelta in class DeltaRuleExtender
Parameters:
currentInps - the forwarded input.
j - the input index of the weight.
currentPattern - the back propagated gradients.
k - the output index of the weight.
aPreviousDelta - a delta value calculated by a previous delta extender.

postBiasUpdate

public void postBiasUpdate(double[] currentGradientOuts)
Description copied from class: LearnerExtender
Gives extenders a change to do some post-computing after the biases are updated.

Specified by:
postBiasUpdate in class LearnerExtender
Parameters:
currentGradientOuts - the back propagated gradients.

postWeightUpdate

public void postWeightUpdate(double[] currentPattern,
                             double[] currentInps)
Description copied from class: LearnerExtender
Gives extenders a change to do some post-computing after the weights are updated.

Specified by:
postWeightUpdate in class LearnerExtender
Parameters:
currentPattern - the back propagated gradients.
currentInps - the forwarded input.

preBiasUpdate

public void preBiasUpdate(double[] currentGradientOuts)
Description copied from class: LearnerExtender
Gives extenders a change to do some pre-computing before the biases are updated.

Specified by:
preBiasUpdate in class LearnerExtender
Parameters:
currentGradientOuts - the back propagated gradients.

preWeightUpdate

public void preWeightUpdate(double[] currentPattern,
                            double[] currentInps)
Description copied from class: LearnerExtender
Gives extenders a change to do some pre-computing before the weights are updated.

Specified by:
preWeightUpdate in class LearnerExtender
Parameters:
currentPattern - the back propagated gradients.
currentInps - the forwarded input.

getParameters

public RpropParameters getParameters()
Gets the parameters of this learning algorithm.

Returns:
the parameters of this learning algorithm.

setParameters

public void setParameters(RpropParameters aParameters)
Sets the parameters for this learning algorithm.

Parameters:
aParameters - the parameters for this learning algorithm.

sign

protected double sign(double d)
Gets the sign of a double. return the sign of a double (-1, 0, 1).



Submit Feedback to pmarrone@users.sourceforge.net