Planning

The collision avoidance system needs a control strategy for responding to obstacles. We will use a partially observable Markov decision process, or POMDP, to encode this strategy (Puterman, 1994). POMDPs have become the semantic model of choice for decision-theoretic planning in the AI planning community. A POMDP models the domain of interest with a finite set of states S. Actions of the system (wheelchair in this case), are drawn from a finite set A, and induce stochastic state transitions, with P(St|At,St-1) denoting the probability with which state St is reached when action At is executed at state St-1. A real-valued reward function R, associates with each state, s, and action, a, its immediate utility R(s,a). Efficient algorithms have been developed to solve such models, generating a control policy π(s) which gives the optimal action to take in each possible state. Finally, an observation function Ω(z,s,a) gives the probability of observing z in state s after action a was taken.

	S -> {MAP,user control} the set of states be the possible configurations of obstacles in the path of the wheelchair, as given by the occupancy grid, and the possible user controls from the joystick.
	A The actions will be those the wheelchair can take (e.g. go forward, stop, or turn right).
	Z -> sensor reading the observations are the readings from the 3D sensors
	P(St|At,St-1) The transition function is the method for updating the map based on the actions of the user and the system. Since the maps we use are just local occupancy grids, this does not involve any global consistency or SLAM.
	Ω(z,s,a) Is relationship between the occupancy grid map and the raw sensor readings. This is essentially the methods of Murray and Little, 1998
	R(s,a) costs for hitting obstacles, rewards for letting the user do what they want. The control policy will therefore optimise satisfaction of user's needs for mobility (their control commands) with the requirements for safety in the wheelchair's location.