o h@sUdZddlmZmZddlZddlmZddlmZgdZee e d<d ed eeeffd d Z Gd ddej j Zd eded efddZGdddej j ZdS)z0Arbitrary dependency between two autograd lanes.)ListTupleN)Tensor) get_phony)forkForkjoinJoin__all__inputreturncCs:tr|jrt|\}}||fSt|jdd}||fS)z7Branches out from an autograd lane of the given tensor.F requires_grad)torchis_grad_enabledrrapplyrdevicer phonyr`/var/www/html/ai/venv/lib/python3.10/site-packages/torch/distributed/pipeline/sync/dependency.pyrs rc @sLeZdZedddedeeeffddZedddededefdd Zd S) rctxr r cCst|jdd}||fS)NFr)rrdetachrr rrrrforwardsz Fork.forward grad_input grad_gradcCs|SNr)rrrrrrbackward"sz Fork.backwardN)__name__ __module__ __qualname__ staticmethodrrrrrrrrrs   rrcCs$tr|js |jrt||}|S)zMerges two autograd lanes.)rrrr rrrrrr 's r c @sLeZdZedddededefddZedddedeedffd d ZdS) r rr rr cCs|Sr)rrrrrr0z Join.forwardrNcCs|dfSrr)rrrrrr4r$z Join.backward)r r!r"r#rrrrrrrrr /s $r )__doc__typingrrrrrrr str__annotations__rautogradFunctionrr r rrrrs