Textbooks
The Elements of Statistical Learning: Data Mining, Inference, and Prediction
by Trevor Hastie, Robert Tibshirani, Jerome Friedman
A very popular book that focuses on the statistical aspect of machine learning.
Pattern Recognition and Machine Learning
by Christopher M. Bishop
This book follows a similar approach as Skratch as it starts by focusing on information theory and model selection.
Machine Learning
by Tom M. Mitchell
This book was first published in 1986 and is a fundamental read when it comes to machine learning.
Information Theory, Inference and Learning Algorithms
by David J.C. MacKay
Great read to have more insight on the information theory aspect of machine learning.