o h@sfdZddlZddlZddlZddlZddlmZmZddZde de fd d Z dd e d e fddZdS)z A collection of utilities for ensuring that training can always occur. Heavily influenced by the [toma](https://github.com/BlackHC/toma) library. N)is_npu_availableis_xpu_availablecGsjt|ts t|}tt|D]}d||<qttr$tj |St r.tj |Stj |S)aN Releases memory from `objects` by setting them to `None` and calls `gc.collect()` and `torch.cuda.empty_cache()`. Returned objects should be reassigned to the same variables. Args: objects (`Iterable`): An iterable of objects Returns: A list of `None` objects to replace `objects` Example: ```python >>> import torch >>> from accelerate.utils import release_memory >>> a = torch.ones(1000, 1000).cuda() >>> b = torch.ones(1000, 1000).cuda() >>> a, b = release_memory(a, b) ``` N) isinstancelistrangelengccollectrtorchxpu empty_cachernpucuda)objectsirM/var/www/html/ai/venv/lib/python3.10/site-packages/accelerate/utils/memory.pyrelease_memorys     r exceptionreturncs:gd}ttrtjdkrtfdd|DSdS)z Checks if `exception` relates to CUDA out-of-memory, CUDNN not supported, or CPU out-of-memory Args: exception (`Exception`): An exception )zCUDA out of memory.z(cuDNN error: CUDNN_STATUS_NOT_SUPPORTED.z*DefaultCPUAllocator: can't allocate memoryrc3s|] }|jdvVqdS)rN)args).0errrrr Osz+should_reduce_batch_size..F)r RuntimeErrorrrany)r _statementsrrrshould_reduce_batch_sizeAsrfunctionstarting_batch_sizecs,dur tjt|dS|fdd}|S)a A basic decorator that will try to execute `function`. If it fails from exceptions related to out-of-memory or CUDNN, the batch size is cut in half and passed to `function` `function` must take in a `batch_size` parameter as its first argument. Args: function (`callable`, *optional*): A function to wrap starting_batch_size (`int`, *optional*): The batch size to try and fit into memory Example: ```python >>> from accelerate.utils import find_executable_batch_size >>> @find_executable_batch_size(starting_batch_size=128) ... def train(batch_size, model, optimizer): ... ... >>> train(model, optimizer) ``` N)r"c sDttr tjntrtjntjt t j }t|t|dkrTdddt|dd|ddD}tdjdjd|d d kr]td z g|Ri|WSty}z+t|rttrtjntrtjntjd nWYd}~nd}~wwqU) Nrz, cSsg|] \}}|d|qS)=r)rargvaluerrr szAfind_executable_batch_size..decorator..zBatch size was passed into `zS` as the first argument when called.Remove this as the decorator already does so: `(z)`Trz-No executable batch size found, reached zero.)r r rr r r rrrrinspect signature parameterskeysrjoinzip TypeError__name__r Exceptionr)rkwargsparamsarg_stre batch_sizer!rr decoratorssF   *      z-find_executable_batch_size..decorator) functoolspartialfind_executable_batch_size)r!r"r8rr6rr;Ss #r;)Nr )__doc__r9r r)r importsrrrr1boolrcallableintr;rrrrs$