o h0@s0ddlmZddlmZmZmZmZddlmZm Z GdddZ GdddeZ Gdd d eZ Gd d d eZ Gd d d eZGdddeZdedeedefddZde deee fde fddZdee deee fdee e ffddZde de defdd Zdee d!ee e fdefd"d#Zd$S)%)Enum) NamedTupleDictListSet)Nodemap_argc@sBeZdZdZdeddfddZddZd d Zd d Zd dZ dS) PartitionzPartition class contains all the information about an individual partition. It also provides necessary methods for manipulation the partition. partition_idreturnNcCs4t|_||_t|_t|_d|_d|_g|_dS)Nr)setnodesr parentschildren bfs_levelused_mem_byteslogical_device_ids)selfr r]/var/www/html/ai/venv/lib/python3.10/site-packages/torch/fx/experimental/partitioner_utils.py__init__ s zPartition.__init__cCs t|jSN)strr rrrr__str__s zPartition.__str__cCs,d|_|jD] }|jt||j7_qdS)Nr)rrget_extra_size_of)rnoderrrrecalculate_mem_sizes zPartition.recalculate_mem_sizecsdit|jfddt|jfddD] }|jdvr%|j|q|j||dS)Nc |Sr setdefaultn input_nodesrr z$Partition.add_node..crrr r"r$rrr& r'>get_attr placeholder)rargskwargsopraddr)rrr#rr$radd_nodes    zPartition.add_nodecs|jvrCj|it|jfddt|jfddD]}tfdd|jDr<|jdvr<j|q#dSdS)Ncrrr r"r$rrr&.r'z'Partition.remove_node..crrr r"r$rrr&/r'c3s|]}|jvVqdSr)r.0r#rrr 4s  z(Partition.remove_node..>r(r)) rremoverr*r+allusersr,r)rr input_noder)r%rr remove_node(s      zPartition.remove_node) __name__ __module__ __qualname____doc__intrrrr.r6rrrrr s  r c@s&eZdZUeed<eed<eed<dS)Devicenameavailable_mem_bytes logical_idN)r7r8r9r__annotations__r;rrrrr<;s  r<c@seZdZUeed<eed<dS) NodeLatencymem_latency_seccomputer_latency_secNr7r8r9floatr@rrrrrAAs  rAc@s&eZdZUeed<eed<eed<dS)PartitionLatencyrBrCoverall_latency_secNrDrrrrrFHs  rFc@s eZdZdZdZdZdZdZdS) PartitionModerN)r7r8r9 size_based sparse_nn cost_awarekl_based aot_basedrrrrrHQs rHc@seZdZUeeed<ejZeed<dZ e ed<iZ e e efed<iZe e efed<iZe eeefed<dZeed <d S) PartitionerConfigdevicesmodetransfer_rate_bytes_per_secnode_to_latency_mappingnode_to_partition_mapping#partition_to_logical_device_mappingF saturate_hostN)r7r8r9rr<r@rHrMrTrVrErWrrrArXr;rYrZboolrrrrrRYs  rRrrr csit|jfddt|jfddd}D]}||vr2t|dd}|r.||j7}qtdqt|dd}|rB||j7}|Std)zGiven a node and a set of nodes, this function return the extra size that needed if this node is included in this set. crrr r"r$rrr&kr'z#get_extra_size_of..crrr r"r$rrr&lr'r size_bytesNznode has no size_bytes attr)rr*r+getattr output_size RuntimeError total_size)rrtotal_size_of_input_nodesr#r\rr$rrds     r partitionrWcsxdtdttfdd}dtdtffdd |}tdddd }|D]}|tdddd }|j|jkr9|}q%|S) zVGiven a partition and its nodes' latency, return a PartitionLatency for this partitionrbr cslg}jD].}|jdvr qit|jfddt|jfddtfddDs3||q|S)z>Given a partition, return a list of nodes on the top bfs level>r(r)crrr r"r$rrr&r'zEget_latency_of_one_partition..get_top_nodes..crrr r"r$rrr&r'c3s$|] }|jvo |jdvVqdS)>r(r)N)rr,r/)rbrrr1s  zFget_latency_of_one_partition..get_top_nodes..)rr,rr*r+anyappend)rb top_nodesrr)r%rbr get_top_nodess    z3get_latency_of_one_partition..get_top_nodesrc s|}|jt|j|j}|j|j}|j|j}t|jj}|rDtdddd}|D]}|t|||} | j|jkrA| }q.|St|||S)zyGiven a top node of a partition, this function returns the latency of the critical path in the partition rUrBrCrG) rGmaxrCrBr r4 intersectionrrF) rpartition_latency node_latencyrGrBrCr4 max_latencyr#new_partition_latency dfs_helperrWrbrrros:  z0get_latency_of_one_partition..dfs_helperrUrg)r rrrFrG)rbrWrfrecritical_path_latencyrrjrrnrget_latency_of_one_partitions&,rq partitionscCs$i}|D] }t||}|||<q|S)zGiven all the partitions and node_to_latency_mapping dictionary, return a mapping dictionary of each partition to its overall latency )rq)rrrWpartition_to_latency_mappingrbrjrrr get_partition_to_latency_mappings rtparent_partitionchild_partitionrVcs|jgkr|jgkr|j|jkrdSd}t}|jD]:}it|jfddt|jfddD]}||jvrS||vrSt|dd}|durN||j7}||q4q||S)zfGiven two partitions (parent and child), calculate the communication latency between the two. rUrcrrr r"r$rrr&r'z*get_comm_latency_between..crrr r"r$rrr&r'r\N) rr rrr*r+r]r^r-)rurvrV comm_size visited_nodesrr#r\rr$rget_comm_latency_betweens&      ryrscshdtdtdtffdd dttdttfdd}||}d }|D] }|d }||kr1|}q$|S) zGiven all partitions in a graph, find the critical path among all partitions and return its latency as the latency of the whole graph rblatency_so_far_secr csX||j7}|j}|jr*d}|jD]}t||}|||}||kr'|}q|S|S)zJThis function helps to recursively get the latency of a path of partitionsrU)rGrry)rbrzrmax_latency_secchildcomm_latency_secnew_latency_secrorsrVrrros( z4get_latency_of_partitioned_graph..dfs_helperrrcSs*g}|D]}t|jdkr||q|S)zvThis function is to return all the partitions without parents as the starting points of all the paths r)lenrrd)rrtop_partitionsrbrrrget_top_partitions,s  z.get_top_partitionsrU)r rEr)rrrsrVrrcritical_path_latency_secrb latency_secrrr get_latency_of_partitioned_graph s   rN)enumrtypingrrrr torch.fx.noderrr r<rArFrHrRr;rrqrtrEryrrrrrsL 4    Y    $