Minimizing Average of Loss Functions Using Gradient Descent and Stochastic Gradient Descent

Authors

  • Md. Rajib Arefin
  • M. Asadujjaman

Keywords:

Gradient Descent, Stochastic Gradient Descent, Convex Function, Unconstrained Optimization Problems.

Abstract

This paper deals with minimizing average of loss functions using Gradient Descent (GD) and Stochastic Gradient
Descent (SGD). We present these two algorithms for minimizing average of a large number of smooth convex functions.
We provide some discussions on their complexity analysis, also illustrate the algorithms geometrically. At the end, we
compare their performance through numerical experiments.

Downloads

Issue

Vol. 64 No. 2 (2016)

Section

Articles