o h@sjUddlmZmZmZmZddlZddlmmZ ddlm Z gZ ee e d<ejjGdddZdS))DictListOptionalTupleN)Tensor__all__c@sheZdZ       ddeededeeefd ed ed ed ed efddZdee efddZ dS)_FunctionalAdamaxMbP?g?g+?:0yE>Fparamslrbetaseps weight_decayforeachmaximize_allow_empty_param_listc Cs d|ks td|d|kstd|d|dkr"dks,ntd|dd|dkr8dksBntd|dd|ksMtd ||||d|d|d |_||_||_tjttjtt tjffi|_ t |dkr~|s~td d |i|_ dS) Nr zInvalid learning rate: zInvalid epsilon value: rg?z#Invalid beta parameter at index 0: z#Invalid beta parameter at index 1: zInvalid weight_decay value: )rrbeta1beta2rz%optimizer got an empty parameter listr ) ValueErrordefaultsrrtorchjitannotaterrstrstatelen param_group) selfr rrrrrrrr"_/var/www/html/ai/venv/lib/python3.10/site-packages/torch/distributed/optim/functional_adamax.py__init__s, $z_FunctionalAdamax.__init__ gradientsc Cs|jd}g}g}g}g}g}t|t|kr*tddt|ddt|t|jd|D]V\}} | dur|||| ||jvrni|j|<|j|} td| d<tj|tj d| d <tj|tj d| d <|j|} || d || d || dq2t +t j ||||||j d |j d |j d |j d|j d|j|jd WddS1swYdS)Nr zEthe gradients passed in does not equal to the size of the parameters!zParams length: z. zGradients length: r step) memory_formatexp_avgexp_infrrrrr)rrrrrrr)r rrzipappendrrtensor zeros_likepreserve_formatno_gradFadamaxrrr) r!r%r params_with_gradgradsexp_avgsexp_infs state_stepsparamgradientrr"r"r#r&=sb          "z_FunctionalAdamax.stepN)r r r r FFF) __name__ __module__ __qualname__rrfloatrboolr$rr&r"r"r"r#rs4  (r)typingrrrrrtorch.optim._functionaloptim _functionalr0rrr__annotations__rscriptrr"r"r"r#s