I think saying machine learning is just statistics is a bit misleading. There’s not much statistics going on in deep learning. It’s mostly just “eh, this seems to work I dunno let’s keep doing it and see what happens”.
It’s mostly just “eh, this seems to work I dunno let’s keep doing it and see what happens”.
Yeah, no.
Well, eventually the thing you’re working on falls out of fashion in place for the next trendy thing.
But… you have to create criteria for what qualifies as success vs failure, and it’s a scale, not a boolean true/false. That’s where the statistics come in, especially if you have multiple criteria with different weights etc.
The criteria is a loss function, which can be whatever works best for the situation. Some might have statistical interpretations, but it’s not really a necessity. For Boolean true/false there are many to choose from. Hinge loss and logistic loss are two common ones. The former is the basis for support vector machines.
But the choice of loss is just one small part in the design of a deep learning model. Choice of activation functions, layer connectivity, regularization and optimizer must also be considered. Not all of these have statistical interpretations. Like, what is the statistical interpretation between the choice of Relu and Leaky Relu? People seemed to prefer one over the other because that’s what worked best for them.
