Good Representation Can Ease Learning

It appears we seem to spend lot more time on picking a learning methodology and ignore pattern representation and feature representation. This generally comes at a cost in terms of more training time and predictive accuracy. I thought I will highlight this practice by showing a very simple example. Slide #1 shows a set of training examples and the corresponding label associated with each example. The task is to learn the labeling pattern so as to able to assign labels to test examples shown in the slide.

Some of you will try to unlock the labeling pattern by looking at the data because we have few examples only. Some of you with access to machine learning software will immediately plug in these training examples and select their favored method of learning, for example decision trees, to figure out the relationship between the numbers and their labels.

Let us look at our problem again. But this time let us use a different representation for our problem as shown in Slide #2.

In this case, you will be easily able to discern the labeling pattern as shown in Slide #3. The label assigned to a number depends upon the logical AND of second and sixth bits of binary representation of each number. Thus, it is useful to remember that a good representation, the choice of features to represent patterns, makes learning easier.