http://rail.eecs.berkeley.edu/deeprlcourse/
For a more mathematical treatment, there's a beautiful book by Puterman:
https://www.amazon.com/Markov-Decision-Processes-Stochastic-...
http://www.incompleteideas.net/book/the-book-2nd.html