o h>@sddlZddlZddlZddlZddlZddlZddlZddlmZddl m Z ddl m Z ddl mZmZddlmZddlZddlmZmZdd lmZmZmZmZmZmZmZmZmZmZm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&d]d d Z'e'd d dZ(ddZ)ddZ*ddZ+ddZ,ddZ-ddZ.ddZ/ddZ0dd Z1d!d"Z2d#d$Z3d%d&Z4d'd(Z5d)d*Z6d+d,Z7d-d.Z8d/d0Z9d^d1d2Z:d3d4Z;d5d6Zd;d<Z?d=d>Z@eAe$egoe ZBd?d@ZCGdAdBdBejDZEGdCdDdDejDZFGdEdFdFejDZGdGdHZHGdIdJdJZIdKdLZJd_dMeIfdNdOZKd`dMeIfdRdSZLGdTdUdUeMZNdadVeeOfdWdXZPedbdYeMdZeOdMeQfd[d\ZRdS)cN)contextmanager)partial)Path)ListUnion)mock)AcceleratorState PartialState)gatheris_bnb_availableis_clearml_availableis_comet_ml_availableis_datasets_availableis_deepspeed_availableis_dvclive_availableis_mps_availableis_pandas_availableis_tensorboard_availableis_timm_availableis_torch_versionis_tpu_availableis_transformers_availableis_wandb_availableis_xpu_available str_to_boolFcCsVztj|}Wn ty|}Y|Swzt|}W|Sty*td|dw)NzIf set, z must be yes or no.)osenvironKeyErrorr ValueError)keydefaultvalue_valuer$S/var/www/html/ai/venv/lib/python3.10/site-packages/accelerate/test_utils/testing.pyparse_flag_from_env4s   r&RUN_SLOW)r!cCstd|S)z+Decorator that skips a test unconditionallyzTest was skipped)unittestskip test_caser$r$r%r)Gsr)cCttd|S)z Decorator marking a test as slow. Slow tests are skipped by default. Set the RUN_SLOW environment variable to a truthy value to run them. z test is slow)r( skipUnless_run_slow_testsr*r$r$r%slowLsr/cCsttj d|S)zu Decorator marking a test that must be only ran on the CPU. These tests are skipped when a GPU is available. ztest requires only a CPUr(r-torchcuda is_availabler*r$r$r% require_cpuTsr4cCsttjd|S)zo Decorator marking a test that requires CUDA. These tests are skipped when there are no GPU available. test requires a GPUr0r*r$r$r% require_cuda[r6cCttd|S)zn Decorator marking a test that requires XPU. These tests are skipped when there are no XPU available. test requires a XPU)r(r-rr*r$r$r% require_xpubr:cCr8)z Decorator marking a test that requires MPS backend. These tests are skipped when torch doesn't support `mps` backend. z0test requires a `mps` backend support in `torch`)r(r-rr*r$r$r% require_mpsir<cCsttotd|S)zv Decorator marking a test that requires transformers and datasets. These tests are skipped when they are not. z$test requires the Hugging Face suite)r(r-rrr*r$r$r%require_huggingface_suiteqs  r>cCr8)i Decorator marking a test that requires transformers. These tests are skipped when they are not. z&test requires the transformers library)r(r-rr*r$r$r%require_transformerszr;r@cCr8)r?ztest requires the timm library)r(r-rr*r$r$r% require_timmr;rAcCr8)zi Decorator marking a test that requires bitsandbytes. These tests are skipped when they are not. z&test requires the bitsandbytes library)r(r-r r*r$r$r% require_bnbr;rBcCr8)zp Decorator marking a test that requires TPUs. These tests are skipped when there are no TPUs available. ztest requires TPU)r(r-rr*r$r$r% require_tpur;rCcCttjdkd|S)z Decorator marking a test that requires CUDA on a single GPU. These tests are skipped when there are no GPU available or number of GPUs is more than one. r5r(r-r1r2 device_countr*r$r$r%require_single_gpurHcCrD)z Decorator marking a test that requires CUDA on a single XPU. These tests are skipped when there are no XPU available or number of xPUs is more than one. rEr9r(r-r1xpurGr*r$r$r%require_single_xpurIrLcCttjdkd|S)z Decorator marking a test that requires a multi-GPU setup. These tests are skipped on a machine without multiple GPUs. rEztest requires multiple GPUsrFr*r$r$r%require_multi_gpurIrNcCrM)z Decorator marking a test that requires a multi-XPU setup. These tests are skipped on a machine without multiple XPUs. rEztest requires multiple XPUsrJr*r$r$r%require_multi_xpurIrOcCr8)z| Decorator marking a test that requires DeepSpeed installed. These tests are skipped when DeepSpeed isn't installed ztest requires DeepSpeed)r(r-rr*r$r$r%require_deepspeedr;rPcCsttddd|S)zr Decorator marking a test that requires FSDP installed. These tests are skipped when FSDP isn't installed >=z1.12.0z%test requires torch version >= 1.12.0)r(r-rr*r$r$r% require_fsdpr7rRcCs0|dur tt|dSttd|d||S)z Decorator marking that a test requires a particular torch version to be tested. These tests are skipped when an installed torch version is less than the required one. N)versionrQztest requires torch version >= )rrequire_torch_min_versionr(r-r)r+rSr$r$r%rTs rTcCr8)z Decorator marking a test that requires tensorboard installed. These tests are skipped when tensorboard isn't installed ztest requires Tensorboard)r(r-rr*r$r$r%require_tensorboardr=rUcCr8)zt Decorator marking a test that requires wandb installed. These tests are skipped when wandb isn't installed ztest requires wandb)r(r-rr*r$r$r% require_wandbr;rVcCr8)zz Decorator marking a test that requires comet_ml installed. These tests are skipped when comet_ml isn't installed ztest requires comet_ml)r(r-rr*r$r$r%require_comet_mlr;rWcCr8)zx Decorator marking a test that requires clearml installed. These tests are skipped when clearml isn't installed ztest requires clearml)r(r-r r*r$r$r%require_clearmlr;rXcCr8)zx Decorator marking a test that requires dvclive installed. These tests are skipped when dvclive isn't installed ztest requires dvclive)r(r-rr*r$r$r%require_dvcliver;rYcCr8)zv Decorator marking a test that requires pandas installed. These tests are skipped when pandas isn't installed ztest requires pandas)r(r-rr*r$r$r%require_pandasr;rZcCr,)z Decorator marking that a test requires at least one tracking library installed. These tests are skipped when none are installed zYtest requires at least one tracker to be available and for `comet_ml` to not be installed)r(r-_atleast_one_tracker_availabler*r$r$r%require_trackerss r\c@s4eZdZdZdZeddZeddZddZd S) TempDirTestCasea A TestCase class that keeps a single `tempfile.TemporaryDirectory` open for the duration of the class, wipes its data at the start of a test, and then destroyes it at the end of the TestCase. Useful for when a class or API requires a single constant folder throughout it's use, such as Weights and Biases The temporary directory location will be stored in `self.tmpdir` TcCst|_dS)zECreates a `tempfile.TemporaryDirectory` and stores it in `cls.tmpdir`N)tempfilemkdtemptmpdirclsr$r$r% setUpClassszTempDirTestCase.setUpClasscCs"tj|jrt|jdSdS)z1Remove `cls.tmpdir` after test suite has finishedN)rpathexistsr`shutilrmtreerar$r$r% tearDownClassszTempDirTestCase.tearDownClasscCsH|jr t|jdD]}|r|q |rt|q dSdS)zt|ttfr |n|g|_|jD] }|||jqdS)aQ Add custom mocks for tests that should be repeated on each test. Should be called during `MockingTestCase.setUp`, after `super().setUp()`. Args: mocks (`mock.Mock` or list of `mock.Mock`): Mocks that should be added to the `TestCase` after `TestCase.setUpClass` has been run N) isinstancetuplelistr~start addCleanupstop)rnr~mr$r$r% add_mocksJs  zMockingTestCase.add_mocksN) rprqrrrsrrMockrrr$r$r$r%r}8s"r}cCs`t}|d|j}t|}|d}t|jdD] }t |||s-dSq dS)NrFT) r clonetodevicer cpurangeshaper1equal)tensorstatetensorsir$r$r%are_the_same_tensorsYs  rc@seZdZddZdS) _RunOutputcCs||_||_||_dSrv) returncodestdoutstderr)rnrrrr$r$r%__init__es z_RunOutput.__init__N)rprqrrrr$r$r$r%rds rcs& |IdH}|r||ndSqrv)readline)streamcallbackliner$r$r% _read_streamks rreturnc s|r tdd|tj|dg|ddR|tjjtjj|dIdH}ggd fdd tjtt|j fd d tt|j fd d g|d IdHt |IdHS)Nz Running:  rrE)stdinrrenvcs2|d}||st|||ddSdS)Nutf-8)file)decoderstripappendprint)rsinkpipelabel)quietr$r%tees  z_stream_subprocess..teec|tjddS)Nzstdout:r)sysrl)outrr$r%z$_stream_subprocess..cr)Nzstderr:r)rrr)errrr$r%rr)timeout)r) rjoinasynciocreate_subprocess_exec subprocessPIPEwait create_taskrrrr)cmdrrrrechopr$)rrrrr%_stream_subprocessts.  rTc Csbt}|t||||||d}d|}|jdkr/d|j} td|d|jd| |S)N)rrrrrrr 'z' failed with returncode z, The combined stderr from workers follows: )rget_event_looprun_until_completerrrr RuntimeError) rrrrrrloopresultcmd_strrr$r$r%execute_subprocess_asyncs   rc@s eZdZdS)SubprocessCallExceptionN)rprqrrr$r$r$r%rsrcommandc Cs|dur tj}ztj|tj|d}|r"t|dr|d}|WSWdStjyB}zt dd |d|j |d}~ww)z Runs `command` with `subprocess.check_output` and will potentially return the `stdout`. Will also properly capture if an error occured while running `command` N)rrrrz Command `rz$` failed with the following error: ) rrcopyr check_outputSTDOUThasattrrCalledProcessErrorrroutput)r return_stdoutrrer$r$r% run_commands"   rexception_classmsgc csd}zdVd}Wn:tyD}z.t||s#Jd|dt||dur:|t|vs:Jd|dt|dWYd}~nd}~ww|rOtd|d dS) z Context manager to assert that the right `Exception` class was raised. If `msg` is provided, will check that the message is contained in the raised exception. FNTzExpected exception of type z but got zExpected message 'z' to be in exception but got 'rz but ran without issue.) ExceptionrtypestrAssertionError)rrwas_ranrr$r$r%assert_exceptions"&r)F)NN)NNNFF)NNrFT)FNrv)Srrrfrrr^r( contextlibr functoolsrpathlibrtypingrrrr1rr r utilsr r r rrrrrrrrrrrrrrr&r.r)r/r4r6r:r<r>r@rArBrCrHrLrNrOrPrRrTrUrVrWrXrYrZanyr[r\TestCaser]rur}rrrrrrrrrboolrr$r$r$r%sp    L      !!  )