We spent the whole of last month discussing statistical bounds for function classes. The equations kept getting horrible, makes me wonder if we are trying to fix the wrong problem. As of now SVMs can do much better generalization than the human brain. (After all we as humans tend to have superstitions, and we can look at a superstition as a failure in generalizing). So where else can we improve.
What would be nice to have is a system that tries to explain the features logically. Maybe that is where inductive programming comes in, but I don't have much knowledge in that area.