net.sourceforge.jabm.learning
Class StatelessQLearner

java.lang.Object
  extended by net.sourceforge.jabm.learning.AbstractLearner
      extended by net.sourceforge.jabm.learning.StatelessQLearner
All Implemented Interfaces:
java.io.Serializable, java.lang.Cloneable, DiscreteLearner, Learner, StimuliResponseLearner, Prototypeable, Resetable

public class StatelessQLearner
extends AbstractLearner
implements StimuliResponseLearner, Resetable, java.io.Serializable, Prototypeable

A memory-less version of the Q-Learning algorithm. This class implements StimuliResponseLearner instead of MDPLearner, and so can be used in place of, e.g. a RothErevLearner. We use the standard MDP QLearner class, but fool it with this wrapper into thinking that there is only one state.

See Also:
Serialized Form
 

Field Summary
 
Fields inherited from class net.sourceforge.jabm.learning.AbstractLearner
monitor
 
Constructor Summary
StatelessQLearner(int numActions, double learningRate, double discountRate, cern.jet.random.engine.RandomEngine prng)
           
StatelessQLearner(int numActions, cern.jet.random.engine.RandomEngine prng)
           
StatelessQLearner(cern.jet.random.engine.RandomEngine prng)
           
 
Method Summary
 int act()
          Request that the learner perform an action.
 int bestAction()
           
 void dumpState(DataWriter out)
          Write out our state data to the specified data writer.
 ActionSelector getActionSelector()
           
 double getDiscountRate()
           
 double getInitialQValue()
           
 int getLastActionChosen()
           
 double getLearningDelta()
          Return a value indicative of the amount of learning that occured during the last iteration.
 double getLearningRate()
           
 int getNumActions()
           
 int getNumberOfActions()
          Get the number of different possible actions this learner can choose from when it performs an action.
 int getPreviousState()
           
 cern.jet.random.engine.RandomEngine getPrng()
           
 QLearner getqLearner()
           
 int getState()
           
 void initialise()
           
 double maxQ(int newState)
           
 java.lang.Object protoClone()
           
 void reset()
          Reinitialise our state to the original settings.
 void reward(double reward)
          Reward the learning algorithm according to the last action it chose.
 void setActionSelector(ActionSelector actionSelector)
           
 void setDiscountRate(double discountRate)
           
 void setInitialQValue(double initialQ)
           
 void setLearningRate(double learningRate)
           
 void setNumberOfActions(int n)
           
 void setqLearner(QLearner qLearner)
           
 java.lang.String toString()
           
 int worstAction()
           
 
Methods inherited from class net.sourceforge.jabm.learning.AbstractLearner
monitor
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 
Methods inherited from interface net.sourceforge.jabm.learning.Learner
monitor
 

Constructor Detail

StatelessQLearner

public StatelessQLearner(cern.jet.random.engine.RandomEngine prng)

StatelessQLearner

public StatelessQLearner(int numActions,
                         double learningRate,
                         double discountRate,
                         cern.jet.random.engine.RandomEngine prng)

StatelessQLearner

public StatelessQLearner(int numActions,
                         cern.jet.random.engine.RandomEngine prng)
Method Detail

act

public int act()
Description copied from interface: DiscreteLearner
Request that the learner perform an action. Users of the learning algorithm should invoke this method on the learner when they wish to find out which action the learner is currently recommending.

Specified by:
act in interface DiscreteLearner
Returns:
An integer representing the action to be taken.

bestAction

public int bestAction()
Specified by:
bestAction in interface StimuliResponseLearner

worstAction

public int worstAction()
Specified by:
worstAction in interface StimuliResponseLearner

getDiscountRate

public double getDiscountRate()

getLastActionChosen

public int getLastActionChosen()

getLearningRate

public double getLearningRate()

getNumActions

public int getNumActions()

getPreviousState

public int getPreviousState()

getPrng

public cern.jet.random.engine.RandomEngine getPrng()

getState

public int getState()

initialise

public void initialise()

maxQ

public double maxQ(int newState)

setDiscountRate

public void setDiscountRate(double discountRate)

setLearningRate

public void setLearningRate(double learningRate)

toString

public java.lang.String toString()
Overrides:
toString in class java.lang.Object

reward

public void reward(double reward)
Description copied from interface: StimuliResponseLearner
Reward the learning algorithm according to the last action it chose.

Specified by:
reward in interface StimuliResponseLearner

reset

public void reset()
Description copied from interface: Resetable
Reinitialise our state to the original settings.

Specified by:
reset in interface Resetable

getLearningDelta

public double getLearningDelta()
Description copied from interface: Learner
Return a value indicative of the amount of learning that occured during the last iteration. Values close to 0.0 indicate that the learner has converged to an equilibrium state.

Specified by:
getLearningDelta in interface Learner
Specified by:
getLearningDelta in class AbstractLearner
Returns:
A double representing the amount of learning that occured.

getNumberOfActions

public int getNumberOfActions()
Description copied from interface: DiscreteLearner
Get the number of different possible actions this learner can choose from when it performs an action.

Specified by:
getNumberOfActions in interface DiscreteLearner
Returns:
An integer value representing the number of actions available.

setNumberOfActions

public void setNumberOfActions(int n)

getqLearner

public QLearner getqLearner()

setqLearner

public void setqLearner(QLearner qLearner)

getActionSelector

public ActionSelector getActionSelector()

setActionSelector

public void setActionSelector(ActionSelector actionSelector)

dumpState

public void dumpState(DataWriter out)
Description copied from interface: Learner
Write out our state data to the specified data writer.

Specified by:
dumpState in interface Learner
Specified by:
dumpState in class AbstractLearner

protoClone

public java.lang.Object protoClone()
Specified by:
protoClone in interface Prototypeable

setInitialQValue

public void setInitialQValue(double initialQ)

getInitialQValue

public double getInitialQValue()