Model Parallelism
GPipe
A pipeline parallelism implementation that uses micro-batching and activation checkpointing to efficiently train very large neural networks.
← KembaliA pipeline parallelism implementation that uses micro-batching and activation checkpointing to efficiently train very large neural networks.
← Kembali