Math3346 -- Data Mining

Maths3346 -- Data Mining, 2nd Semester Course

"If you are looking for a career where your services will be in high demand, you should find something where you provide a scarce, complementary service to something that is getting ubiquitous and cheap. So what's getting ubiquitous and cheap? Data. And what is complementary to data? Analysis. So my recommendation is to take lots of courses about how to manipulate and analyze data: databases, machine learning, econometrics, statistics, visualization, and so on."
[Hal Varian, Chief Economist at Google]     Read the whole interview     Varian speaks - Youtube video

Lectures and laboratories

Since 2006, John Maindonald and Graham Williams have shared responsibilty for this course. There have been 3 hours per week of lectures, and 2 hours of laboratory exercises.

A Perspective on Data Mining

Data mining gives a new twist on data deployment and analysis methodologies that have been developed over the past century or more. A good overview is the online article by Mike Loukides at O'REILLY radar: What is data science?

"The future belongs to the companies and people that turn data into products."

Technological and methodological changes and advances have included: Classification is a major pre-occupation of data mining, with a more limited focus on regression with a continuous outcome variable. The aim is typically prediction rather than the more challenging task of interpretation of model parameters.

The R System

Students intending to take this course will it useful to come with some initial familiarity with the open-source R system for scientific and statistical computing and for graphics. The R system is available without charge for downloading from the internet. It is a marvelous example of what can be achieved when highly skilled specialists co-operate internationally, using the internet for communication and co-ordination.

Background Reading

Ian Ayres 2007, Super Crunchers. Why Thinking-By-Numbers is the New Way to be Smart. Bantam. [This places data mining in a wider context of data-based decision-making in business, government and consumer affairs. While popular in style and short on analysis detail, it offers a useful overview of ways in which applications of data mining and related analytical techniques are developing and changing, in part because of the new opportunities and challenges of the internet.]

Thomas H. Davenport and Jeanne G. Harris 2007, Competing on Analytics: The New Science of Winning. Harvard Business School Press. [Analytics is a buzzword for the application of data mining type approaches in commerce. Davenport and Thomas give a useful overview of issues for the deployment of analytical techniques within organizations - benefits and traps, choice of amenable tasks, the role of management, skill base issues, etc.]

John Maindonald and John Braun 2010, Data Analysis and Graphics Using R - An Example-Based Approach, 3nd edn Cambridge University Press. [Of greatest relevance to the course are Chapter 2 on Styles of Data Analysis, Chapters 5 & 6 (through to 6.3) on Linear Models, Chapter 8 (through to 8.3) on logistic regression, Chapter 11 on Tree-based Methods, and Chapter 12 (through to 12.2) on Multivariate Data Exploration & Discrimination.]

Links

MATH3346: Detailed syllabus and course description
Details as Graduate course (MATH6210)
Course materials in 2005 - 2008    Updated and more complete set of lab exercises
Suggestions for getting started on R     New York Times article on R
Information Management gives a plug for Math3346    See here also
Graham Williams' data mining web page (NB in particular rattle, a GUI interface to a data mining toolkit)
Felix Andrews' web page for playwith; an R package for interaction with graphs. (Felix was a Math3346 student in 2004!)
John Maindonald's data mining talks and papers