Show The Graduate Center Menu

Machine Learning

Instructor: Distinguished Professor Robert M. Haralick

Professor Haralick has made a series of contributions in the field of computer vision. In the high-level vision area, he has worked on inferring 3D geometry from one or more perspective projection views.] He has also identified a variety of vision problems which are special cases of the consistent labeling problem. His papers on consistent labeling, arrangements, relation homomorphism, matching, and tree search translate some specific computer vision problems to the more general combinatorial consistent labeling problem and then discuss the theory of the look-ahead operators that speed up the tree search. The most basic of these is called Forward Checking.  This gives a framework for the control structure required in high-level vision problems. He has also extended the forward-checking tree search technique to propositional logic.

In the low-and mid-level areas, Professor Haralick has worked in image texture analysis using spatial gray tone co-occurrence texture features. These features have been used with success on biological cell images, x-ray images, satellite images, aerial images and many other kinds of images taken at small and large scales. In the feature detection area, Professor Haralick has developed the facet model for image processing. The facet model states that many low-level image processing operations can be interpreted relative to what the processing does to the estimated underlying gray tone intensity surface of which the given image is a sampled noisy version. The facet papers develop techniques for edge detection, line detection, noise removal, peak and pit detection, as well as a variety of other topographic gray tone surface features. For shape analysis and extraction he developed the techniques of mathematical morphology, including the mathematical morphology sampling theorem and recursive morphological operations.

His most recent work is in the machine learning area, particularly in the manifold clustering of high dimensional data sets, the application of pattern recognition to mathematical combinatorial problems. He is current work is in the learning of knowledge and structure through relation decomposition.


Machine learning is a branch of artificial intelligence, concerned with the construction and study of systems that can learn from data. Data may be numeric or symbolic and typically has the form of an N-tuple. The anthro-pomorphic term learning in the machine learning context means being able to predict some unobserved components of an N-tuple given some observed components of the N-tuple. This course provides a detailed explanation of many of the techniques used in machine learning and statistical pattern recognition.


Students are assumed to have learned the concepts taught in basic courses in probability, statistics, linear algebra, and articial intelligence.

Course Objectives

The course objectives are to enable the student to take a real world machine learning problem and

  • Identify A Suitable Algorithm for Making the Required Prediction

  • Design an Experimental Protocol for Making an Unbiased Estimate of the Performance of the Algorithm

  • Program the Solution

  • Validate that the Program Works

  • Carry out the Experiment


Learning Objectives

To achieve these objectives the student must be able to demonstrate a working knowledge of the theoretical foundations of machine learning represented by the topics of

  • Classication

  • Regression

  • Clustering

  • Dimensionality Reduction

  • Performance Characterization


Course Topics

  • Bayesian Classication

    • Class conditional probablities

    • Prior Probabilities

    • Gain Matrix

    • Maximizing Expected Gain

  • Minimax Classication

  • Non-Parametric Probability Models

  • Parametric Probability Models

  • Making Decisions in Context

    • Conditional Independence

    • Hidden Markov Model

    • Forward Backward Algorithm

  • Graphical Models

    • Semi-graphoids

    • Graphoids

  • Decision Trees

  • Nearest Neighbor

  • Linear Regression

  • Principal Component Analysis

  • Logistic Regression

  • Neural Networks

    • The Perceptron Algorithm

    • The Back Propagation Algorithm

  • Linear Decision Rules

    • Fisher Linear Decision Rule

    • Support Vector Machines

    • Kernel Methods

  • Clustering

    • K-Means Clustering

    • Expectation Maximization

    • Linear Manifold Clustering

    • Gaussian Mixture Models

    • Clustering Evaluation Measures

  • Experimental Protocols

    • Training Sets

    • Test Sets

    • Cross-Validation

    • Performance Characterization



Grades will be based on

  • Course attendance and participation 10%

  • Midterm 40%

  • Final Project 50%