o hHX@sdZddlZddlZddlmZmZddlmZddlm Z m Z ddl m Z m Z ddlmZddlmZmZmZmZmZddlZd d lmZmZmZmZerUddlmZGd d d eZ d dZ!ddZ"ddZ#ddZ$ddZ%ddZ&ddZ'ddZ(ddZ)dd Z*d!d"Z+d#d$Z,d%d&Z-d'd(Z.d)d*Z/d+d,Z0d-d.Z1d/d0Z2d1d2Z3d3d4Z4Gd5d6d6eZ5erddl6m7m8Z9d7e5d8eeed9ffd:d;Z:dd?Z;e9GdBdCdCe>Z?GdDdEdEe>Z@GdFdGdGZAdHdIZBdJdKZCdedNedOe=dPe=fdQdRZDe dfdTeEfdUdVZFdgdWdXZGdYdZZHdgd[d\ZId]d^ZJd_d`ZKdadbZLdcddZMdS)hz Generic utilities N) OrderedDictUserDict)MutableMapping) ExitStackcontextmanager)fields is_dataclass)Enum)AnyContextManagerIterableListTuple)is_flax_availableis_tf_availableis_torch_availableis_torch_fx_proxyc@seZdZdZdddZdS)cached_propertyz Descriptor that mimics @property but caches output in member variable. From tensorflow_datasets Built-in in functools from Python 3.8. NcCsX|dur|S|jdurtdd|jj}t||d}|dur*||}t||||S)Nzunreadable attribute __cached_)fgetAttributeError__name__getattrsetattr)selfobjobjtypeattrcachedr P/var/www/html/ai/venv/lib/python3.10/site-packages/transformers/utils/generic.py__get__-s     zcached_property.__get__N)r __module__ __qualname____doc__r"r r r r!r$srcCs.|}|dvr dS|dvrdStd|)zConvert a string representation of truth to true (1) or false (0). True values are 'y', 'yes', 't', 'true', 'on', and '1'; false values are 'n', 'no', 'f', 'false', 'off', and '0'. Raises ValueError if 'val' is anything else. >1tyonyestruer>0fnnoofffalserzinvalid truth value )lower ValueError)valr r r! strtobool<s r6cCsHtt|}|dr dS|drdS|drdS|dr"dSd S) z Tries to guess the framework of an object `x` from its repr (brittle but will help in `is_tensor` to try the frameworks in a smart order, without the need to import the frameworks). zjsz1_get_frameworks_and_test_func..csi|]}||qSr r rB)framework_to_testr r! ksz1_get_frameworks_and_test_func..)is_torch_tensor is_tf_tensor is_jax_tensoris_numpy_arrayr@appendextend)r> frameworksr )rFrDr!_get_frameworks_and_test_funcZs rOcCsTt|}|D] }||rdSqt|rdStr(ddlm}t||r(dSdS)z Tests if `x` is a `torch.Tensor`, `tf.Tensor`, `jaxlib.xla_extension.DeviceArray` or `np.ndarray` in the order defined by `infer_framework_from_repr` Tr)TracerF)rOvaluesrrjax.corerP isinstance)r>framework_to_test_func test_funcrPr r r! is_tensorns   rVcCs t|tjSr#)rSr:ndarrayr>r r r! _is_numpys rYcCst|S)z/ Tests if `x` is a numpy array or not. )rYrXr r r!rKsrKcCddl}t||jSNr)torchrSTensorr>r\r r r! _is_torch r_cCtsdSt|S)z] Tests if `x` is a torch tensor or not. Safe to call even if torch is not installed. F)rr_rXr r r!rHrHcCrZr[)r\rSdevicer^r r r!_is_torch_devicer`rdcCra)z] Tests if `x` is a torch device or not. Safe to call even if torch is not installed. F)rrdrXr r r!is_torch_devicerbrecCs8ddl}t|trt||rt||}ndSt||jS)NrF)r\rSr;hasattrrdtyper^r r r!_is_torch_dtypes     rhcCra)z\ Tests if `x` is a torch dtype or not. Safe to call even if torch is not installed. F)rrhrXr r r!is_torch_dtyperbricCrZr[) tensorflowrSr]r>r8r r r!_is_tensorflowr`rlcCra)zg Tests if `x` is a tensorflow tensor or not. Safe to call even if tensorflow is not installed. F)rrlrXr r r!rIrbrIcCs*ddl}t|dr||St||jkS)Nris_symbolic_tensor)rjrfrmr<r]rkr r r!_is_tf_symbolic_tensors  rncCra)z Tests if `x` is a tensorflow symbolic tensor or not (ie. not eager). Safe to call even if tensorflow is not installed. F)rrnrXr r r!is_tf_symbolic_tensorsrocCsddlm}t||jSr[) jax.numpynumpyrSrW)r>jnpr r r!_is_jaxs  rscCra)zY Tests if `x` is a Jax tensor or not. Safe to call even if jax is not installed. F)rrsrXr r r!rJrbrJcCsddddddddd}t|ttfrdd|DSt|ttfr-d d |DSt|}|D]\}}||rE|||Sq5t|tjrP| S|S) zc Convert a TensorFlow tensor, PyTorch tensor, Numpy array or python list to a python list. cS|Sr#)detachcputolistrr r r!zto_py_obj..cSs |Sr#)rqrwrxr r r!rys cSst|Sr#)r:asarrayrwrxr r r!ryscS|Sr#)rwrxr r r!ryrAcSi|] \}}|t|qSr  to_py_objrCkvr r r!rGzto_py_obj..cSsg|]}t|qSr r)rCor r r!rEszto_py_obj..) rSdictritemslisttuplerOr:numberrw)rframework_to_py_objrT frameworkrUr r r!rs" rcCsddddddddd}t|ttfrdd|DSt|ttfr+t|St|}|D]\}}||rC|||Sq3|S) zc Convert a TensorFlow tensor, PyTorch tensor, Numpy array or python list to a Numpy array. cSrtr#)rurvrqrxr r r!ry rzzto_numpy..cSr|r#)rqrxr r r!ry r}cSs t|Sr#)r:r{rxr r r!rys cSs|Sr#r rxr r r!rysrAcSr~r )to_numpyrr r r!rGrzto_numpy..) rSrrrrrr:arrayrO)rframework_to_numpyrTrrUr r r!rs rcseZdZdZdddZfddZdd Zd d Zd d ZddZ ddZ ddZ fddZ fddZ fddZdeefddZZS) ModelOutputa Base class for all model outputs as dataclass. Has a `__getitem__` that allows indexing by integer or slice (like a tuple) or strings (like a dictionary) that will ignore the `None` attributes. Otherwise behaves like a regular python dictionary. You can't unpack a `ModelOutput` directly. Use the [`~utils.ModelOutput.to_tuple`] method to convert it to a tuple before. returnNcCstr t|ttdSdS)zRegister subclasses as pytree nodes. This is necessary to synchronize gradients when using `torch.nn.parallel.DistributedDataParallel` with `static_graph=True` with modules that output `ModelOutput` subclasses. N)r _torch_pytree_register_pytree_node_model_output_flatten_model_output_unflatten)clsr r r!__init_subclass__.szModelOutput.__init_subclass__csJtj|i||jtk}|r!t|s#t|jd|jjddSdS)N.za is not a dataclasss. This is a subclass of ModelOutput and so must use the @dataclass decorator.)super__init__ __class__rr TypeErrorr$r)rargskwargsis_modeloutput_subclassrr r!r;s  zModelOutput.__init__c st}t|stjjdtdd|ddDs'tjjdt|dj}tfdd|ddD}|rt|st |t rP| }d }nzt |}d }Wn t ycd }Ynw|rt|D]F\}}t |ttfrt|d krt |dts|dkr||dj<ntd |d nt|d|d|ddur|d|d<qjdSdS|dur||dj<dSdS|D]}t|j} | dur| |j<qdS)zeCheck the ModelOutput dataclass. Only occurs if @dataclass decorator has been used. z has no fields.css|]}|jduVqdSr#)defaultrCfieldr r r! Tsz,ModelOutput.__post_init__..rNz. should not have more than one required field.rc3s |] }t|jduVqdSr#rnamerrr r!rXsTFzCannot set key/value for z&. It needs to be a tuple (key, value).)rlenr4rrallrrrVrSrriterr enumeraterrr;r) r class_fields first_fieldother_fields_are_noneiteratorfirst_field_iteratoridxelementrrr rr! __post_init__JsZ          zModelOutput.__post_init__cOtd|jjd)Nz$You cannot use ``__delitem__`` on a instance. Exceptionrrrrrr r r! __delitem__zModelOutput.__delitem__cOr)Nz#You cannot use ``setdefault`` on a rrrr r r! setdefaultrzModelOutput.setdefaultcOr)NzYou cannot use ``pop`` on a rrrr r r!poprzModelOutput.popcOr)NzYou cannot use ``update`` on a rrrr r r!updaterzModelOutput.updatecCs*t|trt|}||S||Sr#)rSr;rrto_tuple)rr inner_dictr r r! __getitem__s   zModelOutput.__getitem__cs4||vr|durt||t||dSr#)keysr __setitem__ __setattr__)rrvaluerr r!rszModelOutput.__setattr__cs t||t||dSr#)rrr)rkeyrrr r!rszModelOutput.__setitem__csJts tSt^}}}tfddtD}||g|RS)Nc3s|] }t|jVqdSr#rrrr r!rsz)ModelOutput.__reduce__..)rr __reduce__rr)rcallable_args remainingrrrr!rs  zModelOutput.__reduce__cstfddDS)za Convert self to a tuple containing all the attributes/keys that are not `None`. c3s|]}|VqdSr#r )rCrrr r!rsz'ModelOutput.to_tuple..)rrrr rr!rszModelOutput.to_tuple)rN)rr$r%r&rrrrrrrrrrrrr r __classcell__r r rr!r s 8   routputrz_torch_pytree.ContextcCs t|t|t|ffSr#)rrQr<r)rr r r!rs rrQcontextcCs |\}}|ditt||S)Nr )rzip)rQr output_typerr r r!rsrc@seZdZdZeddZdS) ExplicitEnumzC Enum with more explicit error message for missing values. cCs$t|d|jdt|j)Nz is not a valid z, please select one of )r4rr_value2member_map_r)rrr r r! _missing_szExplicitEnum._missing_N)rr$r%r& classmethodrr r r r!rsrc@seZdZdZdZdZdZdS)PaddingStrategyz Possible values for the `padding` argument in [`PreTrainedTokenizerBase.__call__`]. Useful for tab-completion in an IDE. longest max_length do_not_padN)rr$r%r&LONGEST MAX_LENGTH DO_NOT_PADr r r r!rs rc@s eZdZdZdZdZdZdZdS) TensorTypez Possible values for the `return_tensors` argument in [`PreTrainedTokenizerBase.__call__`]. Useful for tab-completion in an IDE. r7r8r:r9N)rr$r%r&PYTORCH TENSORFLOWNUMPYJAXr r r r!rs rc@s2eZdZdZdeefddZddZddZd S) ContextManagersz Wrapper for `contextlib.ExitStack` which enters a collection of context managers. Adaptation of `ContextManagers` in the `fastcore` library. context_managerscCs||_t|_dSr#)rrstack)rrr r r!rs zContextManagers.__init__cCs|jD]}|j|qdSr#)rr enter_context)rcontext_managerr r r! __enter__s zContextManagers.__enter__cOs|jj|i|dSr#)r__exit__rr r r!rszContextManagers.__exit__N) rr$r%r&r r rrrr r r r!rs  rcCsnt|}|dkrt|j}n|dkrt|j}nt|j}|jD]}|dkr4|j|jdur4dSq#dS)zr Check if a given model can return loss. Args: model_class (`type`): The class of the model. r8r7 return_lossTF)infer_frameworkinspect signaturecallforward__call__ parametersr) model_classrrpr r r!can_return_losss  rcCsn|j}t|}|dkrt|j}n|dkrt|j}nt|j}d|vr/dd|jDSdd|jDS)zq Find the labels used by a given model. Args: model_class (`type`): The class of the model. r8r7QuestionAnsweringcSs g|] }d|vs |dvr|qS)label)start_positions end_positionsr rCrr r r!rEs zfind_labels..cSsg|]}d|vr|qS)rr rr r r!rEs)rrrrrrrr)r model_namerrr r r! find_labels s rrd parent_key delimitercCsddd}t||||S)z/Flatten a nested dict into a single level dict.rrcssf|D]+\}}|rt||t|n|}|r+t|tr+t|||dEdHq||fVqdS)N)r)rr;rSr flatten_dict)rrrrrrr r r! _flatten_dict$s z#flatten_dict.._flatten_dictNrr)r)rrrrr r r!r!s rF use_temp_dirccsD|rt }|VWddS1swYdS|VdSr#)tempfileTemporaryDirectory) working_dirrtmp_dirr r r!working_or_temp_dir/s  " rcCs~t|r tj||dSt|r|dur|jS|j|St|r*ddl}|j||dSt|r5t j||dSt dt |d)z Framework-agnostic version of `numpy.transpose` that will work on torch/TensorFlow/Jax tensors as well as NumPy arrays. )axesNr)permz"Type not supported for transpose: r) rKr: transposerHTpermuterIrjrJrrr4r<)rrr8r r r!r8srcCsjt|r t||St|r|j|St|r!ddl}|||St|r+t||Stdt |d)z Framework-agnostic version of `numpy.reshape` that will work on torch/TensorFlow/Jax tensors as well as NumPy arrays. rNz Type not supported for reshape: r) rKr:reshaperHrIrjrJrrr4r<)rnewshaper8r r r!r Ks    r cCst|r tj||dSt|r|dur|S|j|dSt|r,ddl}|j||dSt|r7tj||dStdt |d)z Framework-agnostic version of `numpy.squeeze` that will work on torch/TensorFlow/Jax tensors as well as NumPy arrays. axisNdimrz Type not supported for squeeze: r) rKr:squeezerHrIrjrJrrr4r<rrr8r r r!r^srcCspt|r t||St|r|j|dSt|r#ddl}|j||dSt|r.tj||dSt dt |d)z Framework-agnostic version of `numpy.expand_dims` that will work on torch/TensorFlow/Jax tensors as well as NumPy arrays. rrNr $Type not supported for expand_dims: r) rKr: expand_dimsrH unsqueezerIrjrJrrr4r<rr r r!rqs  rcCs^t|r t|St|r|St|rddl}||St|r%|jStdt |d)z| Framework-agnostic version of `numpy.size` that will work on torch/TensorFlow/Jax tensors as well as NumPy arrays. rNrr) rKr:sizerHnumelrIrjrJr4r<)rr8r r r! tensor_sizes  rcs^|D](\}}t|ttfrfdd|D||<q|dur,d|vr,d|||<q|S)zB Adds the information of the repo_id to a given auto map. cs.g|]}|durd|vrd|n|qS)N--r )rCrrepo_idr r!rEs.z.add_model_info_to_auto_map..Nr)rrSrr)auto_maprrrr rr!add_model_info_to_auto_mapsrcCst|D]6}|j}|j}|ds|ds|dkrdS|ds'|dkr*dS|ds8|d s8|d kr;dSqtd |d ) z Infers the framework of a given model without using isinstance(), because we cannot guarantee that the relevant classes are imported or available. rjkerasTFPreTrainedModelr8r\PreTrainedModelr7flaxr9FlaxPreTrainedModelz%Could not infer framework from class r)rgetmror$rr=r)r base_classmodulerr r r!rsrr)Fr#)Nr&rr collectionsrrcollections.abcr contextlibrr dataclassesrrenumr typingr r r r rrqr: import_utilsrrrrrprrpropertyrr6r@rOrVrYrKr_rHrdrerhrirlrIrnrorsrJrrrtorch.utils._pytreeutils_pytreerrrrr;rrrrrrrboolrrr rrrrrr r r r!sz