Advanced
Theory of Stochastic Approximation
Examine the mathematical foundations of Stochastic Gradient Descent.
Discuss the theoretical foundations of Stochastic Approximation, focusing on the Robbins-Monro conditions. Explain how these conditions regarding the step size (learning rate) ensure convergence in Stochastic Gradient Descent (SGD) despite the noise introduced by random sampling of the gradient.
Beginner
Pareto Efficiency in Multi-Objective Optimization
Define and explore Pareto Optimality for problems with conflicting objectives.
Define the concept of Pareto Optimality (or Pareto Efficiency) in the context of multi-objective optimization. Explain the theoretical difficulty in identifying a single 'best' solution when objectives are conflicting. Describe the Pareto Front and discuss the theoretical trade-offs that occur when moving along this frontier.