Adaptive AI Engine for RTS Games

Discussing the theory and practice

Archive for the ‘Pattern Classification’ Category

Summary of Pattern Classification book

Introduction to Pattern Recognition

Posted by Ogail on October 15, 2009

  • Model is anything that have specific descriptions in mathematical form
  • Segmentation is an operation to isolate objects from each other in images
  • If we think in business wise, you can ignore errors that are acceptable by customers (Salmon, Sea-Bass Example)
  • Decision theory is concerned with which pattern classification is perhaps the most important subfield
  • Feature vector is a vector from feature space that contains the actual values of current model
  • When selecting features in feature vector don’t choose redundant features
  • Though, our satisfaction would be premature because the central aim of designing a classifier is to suggest actions when presented with novel patterns. This is the issue of generalization
  • Sometimes classifier is tuned of the training samples, here one solution is to support the classifier with more training samples for obtaining a better estimate
  • Complex classifiers could be simplified if the process doesn’t require this level of complexity
  • Assuming that we somehow manage to optimize this trade off, can we then predict how well our system will generalize to new patterns? These are some of the central problems in statistical pattern recognition
  • (“Entities are not to be multiplied without necessity”). Decisions based on overly complex models often lead to lower accuracy of the classifier
  • There is no GPS (Simpson, Newell device). We conclude here that there’s no GPS
  • Each recognition technique is suitable for a specific domain of problem:
    • Patterns that have statistical properties are more suitable using statistical pattern recognition
    • Patterns with noise are more suitable for neural network pattern recognition
    • If the model consists of some set of crisp logical rules, then we employ the methods of syntactic pattern recognition
  • A central aspect in PR problem is to achieve good representation
    • Features of each sample are close to its category
    • Simplify number of features
    • Robust features: relatively insensitive to noise or other errors
    • In practical applications we may need the classifier to act quickly, or use few electronic components, memory or processing steps
  • A central technique, when we have insufficient training data, is to incorporate knowledge of the problem domain. Indeed the less the training data the more important is such knowledge (analysis by synthesis)
  • What about classifying patterns on depending on their functions (chair example)
  • In acts of associative memory, the system takes in a pattern and emits another pattern which is representative of a general group of patterns.


Sub Problems of Pattern Recognition

  • an ideal feature extractor would yield a representation that makes the job ofthe classifier trivial
  • How do we know which features are most promising? Are there ways to automatically learn which features are best for the classifier? How many shall we use?
  • We define noise very general terms: any property of the sensed pattern due not to the true underlying model but instead to randomness in the world or the sensors
  • While an overly complex model may allow perfect classification of the training samples, it is unlikely to give good classification of novel patterns — a situation known as overfitting
  • One ofthe most important areas ofresearch in statistical pattern classification is determining how to adjust the complexity ofthe model — not so simple that it cannot explain the differences between the categories, yet not so complex as to give poor classification on novel patterns
  • model selection is concerned with how are we to know to reject a class of models and try another one
  • How can we make model selection automated?
  • Prior knowledge is pre knowledge about the problem helps is classification
  • Occlusion!
  • How should we train a classifier or use one when some features are missing?
  • This is the problem of subsets and supersets — formally part of mereology, the study of part/whole relationships. It is closely related to that of prior knowledge and segmentation. In short, how do we recognize or group together the “proper” number of elements — neither too few nor too many
  • In invariation, Thus here we try to build a classifier that is invariant to transformations such as rotation
  • How might we insure that our pattern recognizer is invariant to such complex changes?
  • Evidence pooling, idea about having several classifiers used to categorize one sample. Here how can we resolve the problem of disagreeing classifying?
    • How should a “super” classifier pool the evidence from the component recognizers to achieve the best decision?
  • Costs and Risks
    • How do we incorporate knowledge about such risks and how will they affect our classification decision?
    • Can we estimate the total risk and thus tell whether our classifier is acceptable even before we field it?
  • Computations Complexity:
    • What is the tradeoff between computational ease and performance?
    • How can we optimize within such constraints?
  • Throughout this book, we shall see again and again how methods of learning relate to these central problems, and are essential in the building of classifiers.


Learning and Adaptation

  • Learning refers to some form of algorithm for reducing the error on a set of training data
  • Learning Forms:
    • Supervised Learning:
      • How can we be sure that a particular learning algorithm is powerful enough to learn the solution to a given problem and that it will be stable to parameter variations?
      • How can we insure that the learning algorithm appropriately favors “simple” solutions rather than complicated ones
    • Unsupervised Learning (Clustering):
      • System forms clusters or “natural groupings” of the input patterns
    • Reinforcement Learning (learning with a critic):
      • This is analogous to a critic who merely states that something is right or wrong, but does not say specifically how it is wrong
  • Team reading: Summary of chapters
  • Questions:
    • What’s
      Hypothesis testing

Posted in Pattern Classification | Leave a Comment »