|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectorg.joone.engine.extenders.LearnerExtender
org.joone.engine.extenders.DeltaRuleExtender
org.joone.engine.extenders.WeightDecayExtender
public class WeightDecayExtender
Weight decay adds a penalty term to the error function. The penalty term penalizes large weights. The weight decay penalty term causes the weights to converge to smaller absolute values than they otherwise would. Smaller weights are expected to improve generalization. The update formula is changed in: Dw(t+1) = dw(t+1) - d x w(t) d is a weight decay value.
Constructor Summary | |
---|---|
WeightDecayExtender()
Creates a new instance of WeightDecayExtender |
Method Summary | |
---|---|
double |
getDecay()
Gets the decay parameter. |
double |
getDelta(double[] currentGradientOuts,
int j,
double aPreviousDelta)
Computes the delta value for a bias. |
double |
getDelta(double[] currentInps,
int j,
double[] currentPattern,
int k,
double aPreviousDelta)
Computes the delta value for a weight. |
void |
postBiasUpdate(double[] currentGradientOuts)
Gives extenders a change to do some post-computing after the biases are updated. |
void |
postWeightUpdate(double[] currentPattern,
double[] currentInps)
Gives extenders a change to do some post-computing after the weights are updated. |
void |
preBiasUpdate(double[] currentGradientOuts)
Gives extenders a change to do some pre-computing before the biases are updated. |
void |
preWeightUpdate(double[] currentPattern,
double[] currentInps)
Gives extenders a change to do some pre-computing before the weights are updated. |
void |
setDecay(double aDecay)
Sets the decay parameter. |
Methods inherited from class org.joone.engine.extenders.LearnerExtender |
---|
getLearner, isEnabled, setEnabled, setLearner |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Constructor Detail |
---|
public WeightDecayExtender()
Method Detail |
---|
public double getDelta(double[] currentGradientOuts, int j, double aPreviousDelta)
DeltaRuleExtender
getDelta
in class DeltaRuleExtender
currentGradientOuts
- the back propagated gradients.j
- the index of the bias.aPreviousDelta
- a delta value calculated by a previous delta extender.public double getDelta(double[] currentInps, int j, double[] currentPattern, int k, double aPreviousDelta)
DeltaRuleExtender
getDelta
in class DeltaRuleExtender
currentInps
- the forwarded input.j
- the input index of the weight.currentPattern
- the back propagated gradients.k
- the output index of the weight.aPreviousDelta
- a delta value calculated by a previous delta extender.public void postBiasUpdate(double[] currentGradientOuts)
LearnerExtender
postBiasUpdate
in class LearnerExtender
currentGradientOuts
- the back propagated gradients.public void postWeightUpdate(double[] currentPattern, double[] currentInps)
LearnerExtender
postWeightUpdate
in class LearnerExtender
currentPattern
- the back propagated gradients.currentInps
- the forwarded input.public void preBiasUpdate(double[] currentGradientOuts)
LearnerExtender
preBiasUpdate
in class LearnerExtender
currentGradientOuts
- the back propagated gradients.public void preWeightUpdate(double[] currentPattern, double[] currentInps)
LearnerExtender
preWeightUpdate
in class LearnerExtender
currentPattern
- the back propagated gradients.currentInps
- the forwarded input.public void setDecay(double aDecay)
aDecay
- the decay parameter value.public double getDecay()
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |