Słownik AI
Kompletny słownik sztucznej inteligencji
Options
Temporal abstraction units consisting of an internal policy, an initiation condition, and a termination condition, enabling variable-duration action sequences in hierarchical reinforcement learning.
Semi-Markov Decision Process (SMDP)
Extension of the Markov decision process where actions can have variable and stochastically distributed durations, mathematically formalizing temporal abstraction in option hierarchies.
Primitive Actions
Basic atomic actions at the lowest level of the hierarchy, executed in a single time step without internal abstraction, serving as foundations for higher-level options.
Intra-Option Policies
Policies defining behavior within an option, specifying how to choose primitive actions or sub-options during option execution until its termination condition.
Option Termination Conditions
Probabilistic functions determining when an option should cease execution, allowing flexible control over the duration of temporal abstractions based on the current state.
HAM (Hierarchy of Abstract Machines)
Formal framework using hierarchical finite state machines to structure policies, where each machine defines subtasks and decision points at different temporal scales.
MAXQ Decomposition
Hierarchical task decomposition method where policies are represented as directed acyclic graphs of reusable subtasks, sharing temporal abstractions between different parts of the problem.
Goal-Oriented Reinforcement Learning
Hierarchical approach where subtasks are defined as achieving specific sub-goals, naturally creating temporal abstractions aligned with the semantic structure of the problem.
Feudal Reinforcement Learning (FRL)
Hierarchical architecture inspired by feudalism where high-level managers set goals for low-level workers, creating multi-scale temporal abstraction through command delegation.
Abstract States
Simplified representations of the environment that group multiple concrete states into unique abstractions, enabling decisions at larger temporal scales without considering each individual state.
Multi-scale Temporal Learning
Paradigm where different system components operate and learn at distinct temporal scales simultaneously, optimizing short-term and long-term decisions in a coordinated manner.
Option Discovery
Automated process of identifying and constructing useful options from experience, detecting temporal regularities and recurring subtasks in the environment.
Subgoal-based Learning
Methodology where hierarchical learning is structured around achieving intermediate subgoals, creating natural temporal abstractions aligned with progress toward the final goal.
Hierarchical Temporal Memory
Information storage and retrieval system organized across multiple temporal levels, enabling agents to maintain representations at different time scales for informed decisions.
Termination Function
Mathematical component of an option defining the probability of terminating the option in each state, explicitly controlling the duration and pace of temporal abstractions.
Initiation Set
Set of states in which an option can be initiated, defining the temporal and contextual validity domain of each abstraction in the decision hierarchy.
Option Policy
High-level policy that selects from available options rather than primitive actions, operating at a coarser temporal scale to plan over longer horizons.
Option-Critic Architecture
Algorithmic framework combining hierarchical reinforcement learning with actor-critic methods, where the actor selects options and the critic evaluates their value at different temporal scales.
Nested Options
Hierarchical structure where high-level options can call other lower-level options, creating recursive temporal abstractions to handle problems of arbitrary complexity.