Introductory statistics courses teach us that, when fitting a model to some data, we should have more data than free parameters to avoid the danger of overfitting — fitting noisy data too closely, and thereby failing to fit new data. It is surprising, then, that in modern deep learning the practice is to have orders of magnitude more parameters than data. Despite this, deep networks show good predictive performance, and in fact do better the more parameters they have. Why would that be? It has been known for some time that good performance in machine learning comes from controlling the complexity of networks, which is not just a simple function of the number of free parameters. The complexity of a classifier, such as a neural network, depends on measuring the “size” of the space of functions that this network represents, with multiple technical measures previously suggested: Vapnik–Chervonenkis dimension, covering numbers, Continue reading Demystifying the world of deep networks→

Over the last century, scientists have developed methods to map the structures within the Earth’s crust, in order to identify resources such as oil reserves, geothermal sources, and, more recently, reservoirs where excess carbon dioxide could potentially be sequestered. They do so by tracking seismic waves that are produced naturally by earthquakes or artificially via explosives or underwater air guns. The way these waves bounce and scatter through the Earth can give scientists an idea of the type of structures that lie beneath the surface. There is a narrow range of seismic waves — those that occur at low frequencies of around 1 hertz — that could give scientists the clearest picture of underground structures spanning wide distances. But these waves are often drowned out by Earth’s noisy seismic hum, and are therefore difficult to pick up with current detectors. Specifically generating low-frequency waves would require pumping in enormous amounts Continue reading Machine learning picks out hidden vibrations from earthquake data→

We can’t win the technology wars without the federal government’s help. More details