o h@sddlZddlmZmZddlZddlmZddlmZddl m Z ddl m Z dd l mZGd d d e Zd d ZddZGdddeZGdddeZGdddeZGdddeZdS)N)DictList) variables) unimplemented)istype)VariableTrackerConstantVariablecs(eZdZfddZeddZZS)DistributedVariablec s*tjdi|tstddSdS)Nz+torch.distributed package is not available!)super__init__r is_availabler)selfkwargs __class__r Y/var/www/html/ai/venv/lib/python3.10/site-packages/torch/_dynamo/variables/distributed.pyr s zDistributedVariable.__init__cCs tjSN)torch distributedrr r r rrs z DistributedVariable.is_available)__name__ __module__ __qualname__r staticmethodr __classcell__r r rrr s r cCs,tsdSddlm}t|o||juS)NFr)DTensor)r rtorch.distributed._tensorrinspect isfunction from_local)valuerr r r is_from_locals r$cCs6tsdSddlm}m}||g}t|o||vS)NFr)_get_group_tagget_process_group_ranks)r r"torch.distributed.distributed_c10dr%r&r r!)r#r%r&constant_processgroup_functionsr r ris_constant_pg_functions sr)csBeZdZfddZeddZ     d fd d ZZS)PlacementClassVariablec tjdi|||_dSNr rrr#rr#rrr rr2 zPlacementClassVariable.__init__cCs.tsdSddlm}t|tuot||SNFr) Placement)r r)torch.distributed._tensor.placement_typesr1type issubclassr#r1r r ris_placement_type6s z(PlacementClassVariable.is_placement_typeargsList[VariableTracker]rDict[str, VariableTracker]returnr cst|||}t|jddtjfvr9|jr9t|j}t |fi|}t|jddr9| | |d||St |||S)N__new__r)r propagatevaluesr getattr_staticr#objectr;sourcePlacementVariable add_options call_methodr call_function)rtxr7roptionsnew_objvarrr rrD@s z$PlacementClassVariable.call_functionr7r8rr9r:r )rrrrrr6rDrr r rrr*1s   r*csJeZdZfddZeddZddZ   dfd d ZZS)rAc r+r,r-r.rr rrSr/zPlacementVariable.__init__cC"tsdSddlm}t||Sr0)r rr2r1rr5r r r is_placementW  zPlacementVariable.is_placementcC|jSrr#rr r ras_python_constantaz$PlacementVariable.as_python_constantr7r8rr9r:r c sddlm}t|||}ddg}||vrgzt|j}t|dddus*Jdt||} Wn t y<d} Ynw| t j urI|d i|Sdd|D}d d | D}| |jg|Ri||St ||||S) Nrr r __setattr__ __getattr__zno custom getattr allowed!cSsg|]}|qSr rP).0xr r r }sz1PlacementVariable.call_method..cSsi|] \}}||qSr rT)rUkvr r r ~sz1PlacementVariable.call_method..r)r r r<r=r3r#r r>AttributeErrorr?ritemsrrC) rrEnamer7rr rFallowed_methods value_typemethodrr rrCds(    zPlacementVariable.call_methodrI) rrrrrrKrPrCrr r rrrARs   rAcsFeZdZfddZeddZddZdedeffd d Z Z S) DeviceMeshVariablec r+r,r-r.rr rrr/zDeviceMeshVariable.__init__cCrJ)NFr) DeviceMesh)r r%torch.distributed._tensor.device_meshrcr)r#rcr r ris_device_meshrLz!DeviceMeshVariable.is_device_meshcCrMrrNrOr r rrPrQz%DeviceMeshVariable.as_python_constantr^r:cs"|dkr t|jjSt||S)Nndim)r r#rfr var_getattrrrEr^rr rrgs zDeviceMeshVariable.var_getattr) rrrrrrerPstrr rgrr r rrrbs    rbcsbeZdZdZfddZddZddZ  dfdd ZfddZe ddZ Z S)ProcessGroupVariablea We don't want a ProcessGroup object to end up in our output graph. But it's common for dynamo to intercept a PG that is then used to get info like rank() or world_size(), as well as passed to utility functions in distributed_c10d which desugar it into plain types like a ranklist and tag. For convenience and proper guarding, we construct a variable type. TODO: make it possible to use ProcessGroupVariable as input to simple functions like _expand_group without dynamo complaining about making a proxy for it. It is not a tensor-like type, and we don't want a proxy- but dynamo assumes torch library functions are dealing with tensor-like types and would have proxies for their args. TODO: should we make this inherit VT instead of UDOV? Do we want any of the default behaviors or just graph-break whenever one of our special cases is not hit? c r+r,r-r.rr rrr/zProcessGroupVariable.__init__cCrMrrNrOr r rrPrQz'ProcessGroupVariable.as_python_constantcCs t|jSr)r3r#rOr r r python_types z ProcessGroupVariable.python_typer7r8rr9r:r csB|dkr t|jS|dkrt|jSt||||S)Nranksize)rr r#rlrmrrC)rrEr^r7rrr rrCs z ProcessGroupVariable.call_methodcs2dvrtfddStS)N)rlrmcs||Sr)rC)r7rr^rrEr rsz2ProcessGroupVariable.var_getattr..)rLambdaVariablerBrrgrhrrnrrgsz ProcessGroupVariable.var_getattrcCrJ)NFr) ProcessGroup)r rtorch._C._distributed_c10drqr)r#rqr r ris_process_groups  z%ProcessGroupVariable.is_process_grouprI) rrr__doc__rrPrkrCrgrrsrr r rrrjs  rj)r typingrrrr[rexcrutilsrbaser constantr r r$r)r*rArbrjr r r rs      !3