Welcome to pytorch-optimizer’s documentation!¶
torch-optimizer – collection of optimizers for PyTorch.
Simple example¶
import torch_optimizer as optim
# model = ...
optimizer = optim.DiffGrad(model.parameters(), lr=0.001)
optimizer.step()
Supported Optimizers¶
https://www4.comp.polyu.edu.hk/~cslzhang/paper/CVPR18_PID.pdf |
|
Ranger |
|
RangerQH |
|
RangerVA |
|
https://papers.nips.cc/paper/8186-adaptive-methods-for-nonconvex-optimization |