Dynamic Programming

One finds that Markov Decision is something familiar- the principle has been used in Optimal Control. For optimal control problems, the solving technique is Dynamic Programming. Assumption of dynamic programming is that if you are at the optimal path, at each point (time t), this is also an optimal path from 0 to t.

Thus, we formulate recursive optimization or define a value function. At the end of the time period, the value function has a simple format. For control problem, it is the cost of the final states. Go back to the previous time step. The value function is the sum of the previous time step value function and the cost function of the current state. This is the way of dynamic programming.

From operation research point of view, this is called sequential programming.

Dynamic programming is a general philosophy, not an algorithm. Model predictive control is a special case of dynamic programming.