Abstract In 1961, Minsky perceived a fundamental flaw within the burgeoning field of artificial neural networks. He doubted that such a nonlinear system could be effectively trained using gradient methods, because unless the “structure of the search space is special, the optimization may do more harm than good.” Fast forward to today, and we observe deep neural networks — far more complex than those envisioned at the field's inception — being successfully trained with methods akin to gradient descent. It has, indeed, become evident that the objective function displays a highly benign structure that we are only starting to comprehend. In this lecture, I aim to summarize our current understanding of this enigmatic optimization process. I will explore a diverse array of themes, including intrinsic dimensionality, the optimization landscape, and implicit regularization, and I will highlight key open questions, all within the context of residual networks and generative models.

Abstract Over the past 25 years, network science has revealed that everything is connected—entities, people, and disciplines. Through the lens of networks, a rapidly growing community is extracting coherent insights from complexity. In this talk, I will discuss the potential of network modeling of complex systems to deepen our understanding of emergent phenomena and drive foundational discoveries across science, mathematics, and engineering. Drawing from work developed in my group and at the Center for Network Dynamics, I will also reflect on open research questions and share perspectives on the opportunities ahead.

Abstract Protein structure inference attempts to determine how the primary structure of a protein - a sequence of amino acids - will fold into a complex three-dimensional structure. In the Direct Couplings Analysis approach, statistical information from multiple similar proteins is used to identify positions in the sequence that are in contact, which are then used to fold the sequence. We illustrate that the quality of these contacts exhibits a “double-descent”-like phenomenon, whereby increasing the amount of data initially leads to higher error in the inferred contacts.

Abstract Temperature is a critical sensory modality for navigation in Drosophila melanogaster. Here, we investigate fly navigation in a shallow temperature gradient. We demonstrate that flies are unable to navigate effectively in the absence of temperature cues or when specific central complex neurons are ablated. To model this behavior, we construct a ring attractor network representing the fly's goal direction and implement it in a simulated "vehicle fly". Furthermore, we propose a neuronal circuit that could underlie the implementation of this model. Our findings highlight the importance of temperature in goal-directed behavior.

Abstract We simulate fluid dynamos by solving the magnetohydrodynamic (MHD) equations in an axisymmetric rotating shear flow. In the kinematic (linear) regime, we compute the magnetic instability's critical magnetic Reynolds number and associated azimuthal wavenumber. At large magnetic Reynolds number, the instability's growth rate approaches a finite constant, implying "fast" dynamo action. We also perform nonlinear simulations where the saturated system exhibits oscillations, bistability, and subcritical instability in various regions of parameter space.