o h@sfUddlmZmZmZddlZddlmmZddlm Z gZ ee e d<ej jGdddZdS))DictListOptionalN)Tensor__all__c@sleZdZ         ddeededed ed ed ed ed ededefddZdeeefddZ dS)_FunctionalRMSprop{Gz?Gz?:0yE>Fparamslralphaeps weight_decaymomentumcenteredforeachmaximize_allow_empty_param_listc Csn|||||d|_||_||_| |_t|dkr| stdd|i|_tj t tj t t tj ffi|_ dS)N)r rrrrrz%optimizer got an empty parameter listr )defaultsrrrlen ValueError param_grouptorchjitannotaterrstrstate) selfr r rrrrrrrrr `/var/www/html/ai/venv/lib/python3.10/site-packages/torch/distributed/optim/functional_rmsprop.py__init__s (z_FunctionalRMSprop.__init__ gradientscCs|jd}g}g}g}g}g}|jd}|jd} |jd} |jd} |jd} t|t|krCtddt|d d t|t||D]v\} }|dur|| ||| |jvri|j| <|j| }td |d <tj | tj d |d<| dkrtj | tj d |d<|j rtj | tj d |d<|j| }||d| dkr||d|j r||d|d d7<qHt t j||||||| | | | |j |j|jd WddS1swYdS)Nr r rrrrzEthe gradients passed in does not equal to the size of the parameters!zParams length: z. zGradients length: r step) memory_format square_avgrmomentum_buffergrad_avg)r rrrrrrr)rrrrzipappendrrtensor zeros_likepreserve_formatrno_gradFrmsproprr)rr#r params_with_gradgrads square_avgs grad_avgsmomentum_buffer_listr rrrrparamgradientrr r r!r$6s~                "z_FunctionalRMSprop.stepN) rr r r r FFFF) __name__ __module__ __qualname__rrfloatboolr"rr$r r r r!rs@   !r)typingrrrrtorch.optim._functionaloptim _functionalr0rrr__annotations__rscriptrr r r r!s