I am trying to implement the following algorithm:
![Posted Image]()
In a nutshell, we iterate over a set of 'states' to calculate a value for each.
Each state has multiple actions, and each multiple action has a 'transition probability' associated with it, such that if an action a is taken, there is a 0.8 chance it will proceed to a state s', and a 0.2 chance it will proceed to a different state.
When working through the algorithm, I am struggling to make a data structure that allows to locate a state, have a multiple actions available within that state, and for each action of that state, a probability matrix that when a particular action is taken to move to a new state, returns the probabilities of moving into a new state.
Any ideas to help me?
Many thanks.
In a nutshell, we iterate over a set of 'states' to calculate a value for each.
Each state has multiple actions, and each multiple action has a 'transition probability' associated with it, such that if an action a is taken, there is a 0.8 chance it will proceed to a state s', and a 0.2 chance it will proceed to a different state.
When working through the algorithm, I am struggling to make a data structure that allows to locate a state, have a multiple actions available within that state, and for each action of that state, a probability matrix that when a particular action is taken to move to a new state, returns the probabilities of moving into a new state.
Any ideas to help me?
Many thanks.