Results for: Tags sim Neural Network Remove constraint Tags sim: Neural Network
We introduce a Dimension-Reduced Second-Order Method (DRSOM) for convex and nonconvex (unconstrained) optimization. Under a trust-region-like framework, our method preserves the convergence of the second-order method while using only Hessian-vector products in two directions. Moreover; the computational overhead remains comparable to the first-order such as the gradient descent method. We show that the method has a local super-linear convergence and a global convergence rate of 0(∈-3/2) to satisfy the first-order and second-order conditions under a commonly used approximated Hessian assumption. We further show that this assumption can be removed if we perform one step of the Krylov subspace method at the end of the algorithm, which makes DRSOM the first first-order-type algorithm to achieve this complexity bound. The applicability and performance of DRSOM are exhibited by various computational experiments in logistic regression, L2-Lp minimization, sensor network localization, neural network training, and policy optimization in reinforcement learning. For neural networks, our preliminary implementation seems to gain computational advantages in terms of training accuracy and iteration complexity over state-of-the-art first-order methods including SGD and ADAM. For policy optimization, our experiments show that DRSOM compares favorably with popular policy gradient methods in terms of the effectiveness and robustness.
Event date: 19/09/2022
Speaker: Prof. Yinyu Ye (Stanford University)
Hosted by: Department of Applied Mathematics
In 20 episodes, Jabril will teach you about Artificial Intelligence and Machine Learning! This course is based on a university-level curriculum. By the end of the course, you will be able to: * Define, differentiate, and provide examples of Artificial Intelligence and three types of Machine Learning: supervised, unsupervised, and reinforcement * Understand how different AI and ML approaches can be combined to create compelling applications such as natural language processing, robotics, recommender systems, and web search * Implement several types of AI to classify images, generate text from examples, play video games, and recommend content based on past preferences * Understand the causes of algorithmic bias and audit datasets for several of these causes
If you want to break into AI, this Specialization will help you do so. Deep Learning is one of the most highly sought after skills in tech. We will help you become good at Deep Learning. In five courses, you will learn the foundations of Deep Learning, understand how to build neural networks, and learn how to lead successful machine learning projects. You will learn about Convolutional networks, RNNs, LSTM, Adam, Dropout, BatchNorm, Xavier/He initialization, and more. You will work on case studies from healthcare, autonomous driving, sign language reading, music generation, and natural language processing. You will master not only the theory, but also see how it is applied in industry. You will practice all these ideas in Python and in TensorFlow, which we will teach. You will also hear from many top leaders in Deep Learning, who will share with you their personal stories and give you career advice. AI is transforming multiple industries. After finishing this specialization, you will likely find creative ways to apply it to your work. We will help you master Deep Learning, understand how to apply it, and build a career in AI.