Action-value function

The action-value function is a function that represents the agent's estimate of how good it is to take action  in state . It is denoted by . It is related to the state-value function by the following equation: