AI 용어집
인공지능 완전 사전
Automatic Differentiation
Algorithmic technique to efficiently compute derivatives of functions defined by computer programs, based on the differentiation chain rule and the computational graph of operations.
Computational Graph
Data structure representing dependencies between mathematical operations in a model, enabling forward propagation and automatic gradient computation through backpropagation.
Variance Reduction Techniques
Methods like SVRG (Stochastic Variance Reduced Gradient) or SAGA that reduce variance of gradient estimates to accelerate convergence while maintaining computational efficiency.
Stochastic Newton Methods
Second-order optimization algorithms adapting Newton's method to the stochastic framework, using Hessian matrix approximations for more informative descent directions.
Hessian-vector Products
Efficient computation of the product between the Hessian matrix and a vector without explicitly forming the full Hessian, essential for large-scale second-order optimization methods.