o hf@sfUddlmZmZmZddlZddlmmZddlm Z gZ ee e d<ej jGdddZdS))DictListOptionalN)Tensor__all__c@s|eZdZ        ddeededededed ed ed ed efd dZdedeefddZ deeefddZ dS)_FunctionalSGD{Gz?Fparamslrmomentum dampening weight_decaynesterovmaximizeforeach_allow_empty_param_listc Csl||||d|_||_||_||_tjttjtt tjffi|_ t |dkr/| s/t dd|i|_ dS)N)r r r rrz%optimizer got an empty parameter listr )defaultsrrrtorchjitannotaterrstrstatelen ValueError param_group) selfr r r r rrrrrr\/var/www/html/ai/venv/lib/python3.10/site-packages/torch/distributed/optim/functional_sgd.py__init__s $z_FunctionalSGD.__init__paramgradc Cs|jd}|jd}|jd}|jd}|g}g}g} d} |durK| ||jr+d} ||jvr5i|j|<|j|} d| vrD|dn|| dttj|| ||||||j|j | |j d Wdn1smwY|j|} |d } | dur| | d<dSdS) z[Similar to self.step, but operates on a single parameter and its gradient. rr r r FNTmomentum_bufferrr r r rrhas_sparse_gradrr) rappend is_sparserrno_gradFsgdrrr) rr r!rr r r r momentum_buffer_listgradsr$rr"rrr step_param3sL            z_FunctionalSGD.step_param gradientscCsx|jd}g}g}g}|jd}|jd}|jd}|jd} t|t|kr:tddt|dd t|d } t||D]7\} } | durx|| || | jrXd } | |jvrbi|j| <|j| } d | vrq|dqA|| d qAt t j ||||||| |j |j | |jd Wdn1swYt|D]\}}|j|} ||}|dur|| d <qdS)Nr r rr r zEthe gradients passed in does not equal to the size of the parameters!zParams length: z. zGradients length: FTr"r#)rrrrzipr%r&rrr'r(r)rrr enumerate)rr-r params_with_gradr+r*r rr r r$r gradientripr"rrrstepbsh              z_FunctionalSGD.stepN)rr r r FFFF) __name__ __module__ __qualname__rrfloatboolrrr,r4rrrrrs<  /r)typingrrrrtorch.optim._functionaloptim _functionalr(rrr__annotations__rscriptrrrrrs