gum::E_GreedyDecider Class Reference

<agrum/FMDP/decision/E_GreedyDecider.h> More...

#include <E_GreedyDecider.h>

Inheritance diagram for gum::E_GreedyDecider:

Collaboration diagram for gum::E_GreedyDecider:

Public Member Functions
Constructor & destructor.
	E_GreedyDecider ()
	Constructor. More...
	~E_GreedyDecider ()
	Destructor. More...
Initialization
void	initialize (const FMDP< double > *fmdp)
	Initializes the learner. More...
Incremental methods
void	checkState (const Instantiation &newState, Idx actionId)
ActionSet	stateOptimalPolicy (const Instantiation &curState)

Incremental methods
void	setOptimalStrategy (const MultiDimFunctionGraph< ActionSet, SetTerminalNodePolicy > *optPol)
const MultiDimFunctionGraph< ActionSet, SetTerminalNodePolicy > *	_optPol
ActionSet	_allActions

Detailed Description

<agrum/FMDP/decision/E_GreedyDecider.h>

Class to make decision following an epsilon-greedy compromise between exploration and exploitation

Definition at line 53 of file E_GreedyDecider.h.

Constructor & Destructor Documentation

E_GreedyDecider()

gum::E_GreedyDecider::E_GreedyDecider

(

)

Constructor.

Definition at line 45 of file E_GreedyDecider.cpp.

References __sss.

                                   {
    GUM_CONSTRUCTOR(E_GreedyDecider);

    __sss = 1.0;
  }

~E_GreedyDecider()

gum::E_GreedyDecider::~E_GreedyDecider

(

)

Destructor.

Definition at line 57 of file E_GreedyDecider.cpp.

{ GUM_DESTRUCTOR(E_GreedyDecider); }

Member Function Documentation

checkState()

void gum::E_GreedyDecider::checkState ( const Instantiation &  newState, Idx  actionId  )

virtual

Implements gum::IDecisionStrategy.

Definition at line 88 of file E_GreedyDecider.cpp.

References __statecpt, gum::StatesChecker::addState(), gum::StatesChecker::checkState(), gum::StatesChecker::nbVisitedStates(), and gum::StatesChecker::reset().

                                                                  {
    if (__statecpt.nbVisitedStates() == 0)
      __statecpt.reset(reachedState);
    else if (!__statecpt.checkState(reachedState))
      __statecpt.addState(reachedState);
  }

Here is the call graph for this function:

initialize()

void gum::E_GreedyDecider::initialize ( const FMDP< double > *  fmdp )

virtual

Initializes the learner.

Reimplemented from gum::IDecisionStrategy.

Definition at line 69 of file E_GreedyDecider.cpp.

References __sss, gum::FMDP< GUM_SCALAR >::beginVariables(), gum::FMDP< GUM_SCALAR >::endVariables(), and gum::IDecisionStrategy::initialize().

                                                             {
    IDecisionStrategy::initialize(fmdp);
    for (auto varIter = fmdp->beginVariables(); varIter != fmdp->endVariables();
         ++varIter)
      __sss *= (double)(*varIter)->domainSize();
  }

Here is the call graph for this function:

setOptimalStrategy()

void gum::IDecisionStrategy::setOptimalStrategy ( const MultiDimFunctionGraph< ActionSet, SetTerminalNodePolicy > *  optPol )

inlineinherited

Definition at line 87 of file IDecisionStrategy.h.

References gum::IDecisionStrategy::_optPol.

Referenced by gum::SDYNA::makePlanning().

                                                                                {
      _optPol =
         const_cast< MultiDimFunctionGraph< ActionSet, SetTerminalNodePolicy >* >(
            optPol);
    }

Here is the caller graph for this function:

stateOptimalPolicy()

ActionSet gum::E_GreedyDecider::stateOptimalPolicy ( const Instantiation &  curState )

virtual

Reimplemented from gum::IDecisionStrategy.

Definition at line 102 of file E_GreedyDecider.cpp.

References __sss, __statecpt, gum::IDecisionStrategy::_allActions, gum::StatesChecker::nbVisitedStates(), gum::ActionSet::size(), and gum::IDecisionStrategy::stateOptimalPolicy().

                                                                             {
    double explo = (double)std::rand() / (double)RAND_MAX;
    double temp =
       std::pow((__sss - (double)__statecpt.nbVisitedStates()) / __sss, 3.0);
    double exploThreshold = temp < 0.1 ? 0.1 : temp;

    //      std::cout << exploThreshold << std::endl;

    ActionSet optimalSet = IDecisionStrategy::stateOptimalPolicy(curState);
    if (explo > exploThreshold) {
      //        std::cout << "Exploit : " << optimalSet << std::endl;
      return optimalSet;
    }

    if (_allActions.size() > optimalSet.size()) {
      ActionSet ret(_allActions);
      ret -= optimalSet;
      //        std::cout << "Explore : " << ret << std::endl;
      return ret;
    }

    //      std::cout << "Explore : " << _allActions << std::endl;
    return _allActions;
  }

Here is the call graph for this function:

Member Data Documentation

__sss

double gum::E_GreedyDecider::__sss

private

Definition at line 97 of file E_GreedyDecider.h.

Referenced by E_GreedyDecider(), initialize(), and stateOptimalPolicy().

__statecpt

StatesChecker gum::E_GreedyDecider::__statecpt

private

Definition at line 96 of file E_GreedyDecider.h.

Referenced by checkState(), and stateOptimalPolicy().

_allActions

ActionSet gum::IDecisionStrategy::_allActions

protectedinherited

Definition at line 104 of file IDecisionStrategy.h.

Referenced by gum::IDecisionStrategy::initialize(), gum::RandomDecider::stateOptimalPolicy(), stateOptimalPolicy(), and gum::IDecisionStrategy::stateOptimalPolicy().

_optPol

const MultiDimFunctionGraph< ActionSet, SetTerminalNodePolicy >* gum::IDecisionStrategy::_optPol

protectedinherited

Definition at line 101 of file IDecisionStrategy.h.

Referenced by gum::IDecisionStrategy::initialize(), gum::IDecisionStrategy::setOptimalStrategy(), and gum::IDecisionStrategy::stateOptimalPolicy().

---

The documentation for this class was generated from the following files:

• agrum/FMDP/decision/E_GreedyDecider.h
• agrum/FMDP/decision/E_GreedyDecider.cpp