Minimizing Average of Loss Functions Using Gradient Descent and Stochastic Gradient Descent

Md. Rajib Arefin, M. Asadujjaman

Abstract


This paper deals with minimizing average of loss functions using Gradient Descent (GD) and Stochastic Gradient
Descent (SGD). We present these two algorithms for minimizing average of a large number of smooth convex functions.
We provide some discussions on their complexity analysis, also illustrate the algorithms geometrically. At the end, we
compare their performance through numerical experiments.


Keywords


Gradient Descent, Stochastic Gradient Descent, Convex Function, Unconstrained Optimization Problems.

Full Text:

PDF

Refbacks

  • There are currently no refbacks.


Dhaka University Journal of Science ISSN 1022-2502 (Print) 2408-8528 (Online)