Textbooks

The Elements of Statistical Learning: Data Mining, Inference, and Prediction

by Trevor Hastie, Robert Tibshirani, Jerome Friedman

A very popular book that focuses on the statistical aspect of machine learning.

Pattern Recognition and Machine Learning

by Christopher M. Bishop

This book follows a similar approach as Skratch as it starts by focusing on information theory and model selection.

Machine Learning

by Tom M. Mitchell

This book was first published in 1986 and is a fundamental read when it comes to machine learning.

Information Theory, Inference and Learning Algorithms

by David J.C. MacKay

Great read to have more insight on the information theory aspect of machine learning.