o hu @sddlZddlZddlZddlZddlZddlZddlZddlmZm Z m Z m Z m Z ddl mZmZmZmZddlmZmZmZddlmZmZmZddlmZddlmZdd lmZm Z m!Z!m"Z"m#Z#m$Z$d d l%m&Z&m'Z'e$(e)Z*e"rzddl+Z+e!rdd l,m-Z-d Z.d2ddZ/dZ0GdddZ1Gddde1Z2Gddde1Z3de1fddZ4ddZ5ddddd d!d"d#d$d%d& Z6d'd(Z7d)d*Z8d3d,d-Z9d.d/Z:Gd0d1d1Z;dS)4N)AnyDictListOptionalUnion) create_repohf_hub_downloadmetadata_update upload_folder)RepositoryNotFoundErrorbuild_hf_headers get_session)custom_object_saveget_class_from_dynamic_module get_imports) is_pil_image) AutoProcessor) CONFIG_NAME cached_fileis_accelerate_availableis_torch_availableis_vision_availablelogging)handle_agent_inputshandle_agent_outputs)send_to_devicetool_config.jsonc Ks|dur|Szt|tfddi|WdStyDzt|tfddi|WYdSty9td|dtyCYYdSwtyMYdSw)N repo_typespacemodel`z9` does not seem to be a valid repo identifier on the Hub.)rTOOL_CONFIG_FILEr EnvironmentError Exception)repo_idr hub_kwargsr(M/var/www/html/ai/venv/lib/python3.10/site-packages/transformers/tools/base.py get_repo_type7s"    r*zufrom transformers import launch_gradio_demo from {module_name} import {class_name} launch_gradio_demo({class_name}) c @seZdZUdZdZeed<dZeed<eeed<eeed<dd Z d d Z d d Z ddZ e   d!dedeedeedefddZ    d"dededeedeeeefdedef ddZedd ZdS)#Toola A base class for the functions used by the agent. Subclass this and implement the `__call__` method as well as the following class attributes: - **description** (`str`) -- A short description of what your tool does, the inputs it expects and the output(s) it will return. For instance 'This is a tool that downloads a file from a `url`. It takes the `url` as input, and returns the text contained in the file'. - **name** (`str`) -- A performative name that will be used for your tool in the prompt to the agent. For instance `"text-classifier"` or `"image_generator"`. - **inputs** (`List[str]`) -- The list of modalities expected for the inputs (in the same order as in the call). Modalitiies should be `"text"`, `"image"` or `"audio"`. This is only used by `launch_gradio_demo` or to make a nice space from your tool. - **outputs** (`List[str]`) -- The list of modalities returned but the tool (in the same order as the return of the call method). Modalitiies should be `"text"`, `"image"` or `"audio"`. This is only used by `launch_gradio_demo` or to make a nice space from your tool. You can also override the method [`~Tool.setup`] if your tool as an expensive operation to perform before being usable (such as loading a model). [`~Tool.setup`] will be called the first time you use your tool, but not at instantiation. zThis is a tool that ... descriptionnameinputsoutputscOs d|_dS)NFis_initializedselfargskwargsr(r(r)__init__ms z Tool.__init__cOstdS)Nz-Write this method in your subclass of `Tool`.)NotImplementedr3r(r(r)__call__psz Tool.__call__cCs d|_dS)z Overwrite this method here for any operation that is expensive and needs to be executed before you start using your tool. Such as loading a big model. TNr1r4r(r(r)setupss z Tool.setupc Cstj|dd|jdkrtd|d|dt||}|jj}|dd}|d|jj}tj |d }tj |rZt |d d d  }t |}Wd n1sTwYni}||j|jd}t |dd d }|t j|ddddWd n1swYtj |d} t | dd d }|tj||jjdWd n1swYtj |d} g} |D] } | t| qtt| } t | dd d }|d | dWd d S1swYd S)a Saves the relevant code files for your tool so it can be pushed to the Hub. This will copy the code of your tool in `output_dir` as well as autogenerate: - a config file named `tool_config.json` - an `app.py` file so that your tool can be converted to a space - a `requirements.txt` containing the names of the module used by your tool (as detected when inspecting its code) You should only use this method to save tools that are defined in a separate module (not `__main__`). Args: output_dir (`str`): The folder in which you want to save your tool. T)exist_ok__main__z We can't save the code defining z in z{ as it's been defined in __main__. You have to put this code in a separate module so we can include it in the saved folder..rrutf-8encodingN) tool_classr,r.wr)indent sort_keys zapp.py) module_name class_namezrequirements.txt)osmakedirs __module__ ValueErrorr __class__split__name__pathjoinisfileopenjsonloadr,r.writedumpsAPP_FILE_TEMPLATEformatextendrlistset) r4 output_dir module_filesrI last_module full_name config_filef tool_configapp_filerequirements_fileimportsmoduler(r(r)savezs@     "z Tool.saveNFr& model_repo_idtokenremotec s|r|durt}||vrtd|d||}gdfdd|D}t|fi||d<t|tfd|i|d d d }|du} |durZt|tfd|i|d d d }|duret|d t|d d  } t | } Wdn1s{wY| sd| vrt|d| d} n| } | d} t | |fd|i|} t | j dkr| d| _ | j | dkrt| jd| d| _ t | jdkr| d| _| j| dkrt| jd| d| _|rt||| dS| |fd|i|S)aw Loads a tool defined on the Hub. Args: repo_id (`str`): The name of the repo on the Hub where your tool is defined. model_repo_id (`str`, *optional*): If your tool uses a model and you want to use a different model than the default, you can pass a second repo ID or an endpoint url to this argument. token (`str`, *optional*): The token to identify you on hf.co. If unset, will use the token generated when running `huggingface-cli login` (stored in `~/.huggingface`). remote (`bool`, *optional*, defaults to `False`): Whether to use your tool by downloading the model or (if it is available) with an inference endpoint. kwargs (additional keyword arguments, *optional*): Additional keyword arguments that will be split in two: all arguments relevant to the Hub (such as `cache_dir`, `revision`, `subfolder`) will be used when downloading the files for your tool, and the others will be passed along to its init. N'Could not infer a default endpoint for :, you need to pass one using the `model_repo_id` argument.) cache_dirforce_downloadresume_downloadproxiesrevisionr subfolderlocal_files_onlycsi|] \}}|vr||qSr(r().0kvhub_kwargs_namesr(r) sz!Tool.from_hub..rrlF)%_raise_exceptions_for_missing_entries'_raise_exceptions_for_connection_errorszY does not appear to provide a valid configuration in `tool_config.json` or `config.json`.rArB custom_toolzO does not provide a mapping to custom tools in its configuration `config.json`.rDrr.z_ implements a different name in its configuration and class. Using the tool configuration name.r,zm implements a different description in its configuration and class. Using the tool configuration description.rlrD)get_default_endpointsrNitemsr*rr#rr$rUrVrWrlenr.loggerwarningrQr, RemoteTool)clsr&rkrlrmr6 endpointsr'resolved_config_fileis_tool_configreaderconfigrrDr(rzr)from_hubs             z Tool.from_hub Upload toolcommit_messageprivate create_prreturnc Cst|||dddd}|j}t|ddgiddt(}||td|d d t |t |||||dd Wd S1sFwYd S) a Upload the tool to the Hub. Parameters: repo_id (`str`): The name of the repository you want to push your tool to. It should contain your organization name when pushing to a given organization. commit_message (`str`, *optional*, defaults to `"Upload tool"`): Message to commit while pushing. private (`bool`, *optional*): Whether or not the repository created should be private. token (`bool` or `str`, *optional*): The token to use as HTTP bearer authorization for remote files. If unset, will use the token generated when running `huggingface-cli login` (stored in `~/.huggingface`). create_pr (`bool`, *optional*, defaults to `False`): Whether or not to create a PR with the uploaded files or directly commit. Tr gradio)r&rlrr<r space_sdktagstoolrz!Uploading the following files to z: ,)r&r folder_pathrlrrN) rr&r tempfileTemporaryDirectoryrjrinforSrKlistdirr )r4r&rrrlrrepo_urlwork_dirr(r(r) push_to_hubs"   "$zTool.push_to_hubcCs Gdddt}|j|_||S)z8 Creates a [`Tool`] from a gradio tool. cseZdZfddZZS)z+Tool.from_gradio..GradioToolWrappercst|j|_|j|_dSN)superr7r.r,)r4 _gradio_toolrOr(r)r7Qs  z4Tool.from_gradio..GradioToolWrapper.__init__)rQrM __qualname__r7 __classcell__r(r(rr)GradioToolWrapperPsr)r+runr9) gradio_toolrr(r(r) from_gradioJszTool.from_gradio)NNF)rNNF)rQrMr__doc__r,str__annotations__r.rr7r9r;rj classmethodrboolrrr staticmethodrr(r(r(r)r+QsT     6 p ,r+c@s2eZdZdZd ddZddZddZd d ZdS) rat A [`Tool`] that will make requests to an inference endpoint. Args: endpoint_url (`str`, *optional*): The url of the endpoint to use. token (`str`, *optional*): The token to use as HTTP bearer authorization for remote files. If unset, will use the token generated when running `huggingface-cli login` (stored in `~/.huggingface`). tool_class (`type`, *optional*): The corresponding `tool_class` if this is a remote version of an existing tool. Will help determine when the output should be converted to another type (like images). NcCs||_t||d|_||_dS)Nrl) endpoint_urlEndpointClientclientrD)r4rrlrDr(r(r)r7is zRemoteTool.__init__c OsB|}t|dkr|jdurbt|jtr|jj}n|jj}t|j }dd| D}|ddkr9|dd}t|t|krSt |jdt|dt|d t ||D]\}}|||<qXn&t|dkrlt d t|dkrt |drd |j|diSd |diS| D]\} } t | r|j| || <qd |iS) aP Prepare the inputs received for the HTTP client sending data to the endpoint. Positional arguments will be matched with the signature of the `tool_class` if it was provided at instantation. Images will be encoded into bytes. You can override this method in your custom class of [`RemoteTool`]. rNcSs*g|]\}}|jtjjtjjfvr|qSr()kindinspect_ParameterKindVAR_POSITIONAL VAR_KEYWORD)rwrxpr(r(r) s z-RemoteTool.prepare_inputs..r4rz only accepts z arguments but z were given.z4A `RemoteTool` can only accept one positional input.r/)copyrrD issubclass PipelineToolencoder9r signature parametersrrNziprr encode_image) r4r5r6r/ call_methodrrargr.keyvaluer(r(r)prepare_inputsns<            zRemoteTool.prepare_inputscCs|S)z You can override this method in your custom class of [`RemoteTool`] to apply some custom post-processing of the outputs of the endpoint. r(r4r0r(r(r)extract_outputsszRemoteTool.extract_outputscOst|i|\}}|jduo|jjdgk}|j|i|}t|tr/|jdi|d|i}n|j||d}t|trLt|dkrLt|dtrL|d}t ||jdurW|jjnd}| |S)Nimage output_image)rrrr() rrDr0r isinstancedictrr]rrr)r4r5r6rr/r0r(r(r)r9s $ zRemoteTool.__call__)NNN)rQrMrrr7rrr9r(r(r(r)rZs  + rcsleZdZdZeZdZeZdZ       dfdd Z fddZ ddZ d d Z d d Z d dZZS)ra0 A [`Tool`] tailored towards Transformer models. On top of the class attributes of the base class [`Tool`], you will need to specify: - **model_class** (`type`) -- The class to use to load the model in this tool. - **default_checkpoint** (`str`) -- The default checkpoint that should be used when the user doesn't specify one. - **pre_processor_class** (`type`, *optional*, defaults to [`AutoProcessor`]) -- The class to use to load the pre-processor - **post_processor_class** (`type`, *optional*, defaults to [`AutoProcessor`]) -- The class to use to load the post-processor (when different from the pre-processor). Args: model (`str` or [`PreTrainedModel`], *optional*): The name of the checkpoint to use for the model, or the instantiated model. If unset, will default to the value of the class attribute `default_checkpoint`. pre_processor (`str` or `Any`, *optional*): The name of the checkpoint to use for the pre-processor, or the instantiated pre-processor (can be a tokenizer, an image processor, a feature extractor or a processor). Will default to the value of `model` if unset. post_processor (`str` or `Any`, *optional*): The name of the checkpoint to use for the post-processor, or the instantiated pre-processor (can be a tokenizer, an image processor, a feature extractor or a processor). Will default to the `pre_processor` if unset. device (`int`, `str` or `torch.device`, *optional*): The device on which to execute the model. Will default to any accelerator available (GPU, MPS etc...), the CPU otherwise. device_map (`str` or `dict`, *optional*): If passed along, will be used to instantiate the model. model_kwargs (`dict`, *optional*): Any keyword argument to send to the model instantiation. token (`str`, *optional*): The token to use as HTTP bearer authorization for remote files. If unset, will use the token generated when running `huggingface-cli login` (stored in `~/.huggingface`). hub_kwargs (additional keyword arguments, *optional*): Any additional keyword argument to send to the methods that will load the data from the Hub. Nc  ststdtstd|dur|jdurtd|j}|dur$|}||_||_||_||_||_ |dur9in||_ |durE||j d<||_ ||j d<t dS)N/Please install torch in order to use this tool.z4Please install accelerate in order to use this tool.zHThis tool does not implement a default checkpoint, you need to pass one. device_maprl)r ImportErrorrdefault_checkpointrNr! pre_processorpost_processordevicer model_kwargsr'rr7) r4r!rrrrrrlr'rr(r)r7s*    zPipelineTool.__init__cst|jtr|jj|jfi|j|_t|jtr)|jj|jfi|j|j|_|j dur3|j|_ nt|j trF|j j|j fi|j|_ |j dur`|j dur\t |jjd|_ nt|_ |j durl|j|j tdS)z^ Instantiates the `pre_processor`, `model` and `post_processor` if necessary. Nr)rrrpre_processor_classfrom_pretrainedr'r! model_classrrpost_processor_classrrr] hf_device_mapvaluesget_default_devicetorr;r:rr(r)r;s         zPipelineTool.setupcC ||S)zQ Uses the `pre_processor` to prepare the inputs for the `model`. )r)r4 raw_inputsr(r(r)r zPipelineTool.encodecCs<t|jdi|WdS1swYdS)z7 Sends the inputs through the `model`. Nr()torchno_gradr!)r4r/r(r(r)forward!s $zPipelineTool.forwardcCr)zG Uses the `post_processor` to decode the model output. )rrr(r(r)decode(rzPipelineTool.decodecOsft|i|\}}|js||j|i|}t||j}||}t|d}||}t||j S)Ncpu) rr2r;rrrrrrr0)r4r5r6encoded_inputsr0decoded_outputsr(r(r)r9.s     zPipelineTool.__call__)NNNNNNN)rQrMrrrrrrrr7r;rrrr9rr(r(rr)rs&% %rrDcsZzddl}Wn tytdw|fdd}|j||j|j|jjddS)z Launches a gradio demo for a tool. The corresponding tool class needs to properly implement the class attributes `inputs` and `outputs`. Args: tool_class (`type`): The class of the tool for which to launch the demo. rNz.fn)rr/r0titlearticle)rr Interfacer/r0rQr,launch)rDgrrr(rr)launch_gradio_demo=s    rcCsXtdts tdtjjrtjjrt dStj r't dSt dS)Nu`get_default_device` is deprecated and will be replaced with `accelerate`'s `PartialState().default_device` in version 4.38 of 🤗 Transformers. rmpscudar) rrrrrbackendsr is_availableis_builtrrr(r(r(r)rYs    rDocumentQuestionAnsweringToolImageCaptioningToolImageQuestionAnsweringToolImageSegmentationToolSpeechToTextToolTextSummarizationToolTextClassificationToolTextQuestionAnsweringToolTextToSpeechToolTranslationTool) zdocument-question-answeringzimage-captioningzimage-question-answeringzimage-segmentationzspeech-to-text summarizationztext-classificationztext-question-answeringztext-to-speech translationcCsLtdddd}t|ddd}t|}Wd|S1swY|S)Nz#huggingface-tools/default-endpointszdefault_endpoints.jsondatasetrr@rArB)rrUrVrW)endpoints_filerdrr(r(r)rws  rcCst}||vSr)r)task_or_repo_idrr(r(r)supports_remote~srFc Ks|tvr?t|}td}|j}t||}|r5|dur.t} || vr*td|d| |}t|||dS||fd|i|Stj |f|||d|S)a Main function to quickly load a tool, be it on the Hub or in the Transformers library. Args: task_or_repo_id (`str`): The task for which to load the tool or a repo ID of a tool on the Hub. Tasks implemented in Transformers are: - `"document-question-answering"` - `"image-captioning"` - `"image-question-answering"` - `"image-segmentation"` - `"speech-to-text"` - `"summarization"` - `"text-classification"` - `"text-question-answering"` - `"text-to-speech"` - `"translation"` model_repo_id (`str`, *optional*): Use this argument to use a different model than the default one for the tool you selected. remote (`bool`, *optional*, defaults to `False`): Whether to use your tool by downloading the model or (if it is available) with an inference endpoint. token (`str`, *optional*): The token to identify you on hf.co. If unset, will use the token generated when running `huggingface-cli login` (stored in `~/.huggingface`). kwargs (additional keyword arguments, *optional*): Additional keyword arguments that will be split in two: all arguments relevant to the Hub (such as `cache_dir`, `revision`, `subfolder`) will be used when downloading the files for your tool, and the others will be passed along to its init. transformersNrnrorrl)rkrlrm) TASK_MAPPING importlib import_moduletoolsgetattrrrNrr+r) rrkrmrlr6tool_class_name main_module tools_modulerDrr(r(r) load_tools     rcsfdd}|S)z< A decorator that adds a description to a function. cs|_|j|_|Sr)r,rQr.)funcr,r(r)innerszadd_description..innerr()r,rr(rr)add_descriptions rc @seZdZddedeefddZeddZedd Z    dd ee ee e ee e efd ee d ee de def ddZdS)rNrrlcCs"it|dddi|_||_dS)Nrz Content-Typezapplication/json)r headersr)r4rrlr(r(r)r7s zEndpointClient.__init__cCs.t}|j|ddt|}|dS)NPNG)r[rA)ioBytesIOrjbase64 b64encodegetvaluer)r_bytesb64r(r(r)rs zEndpointClient.encode_imagecCs8tstdddlm}t|}t|}||S)NzbThis tool returned an image but Pillow is not installed. Please install it (`pip install Pillow`).r)Image) rrPILrr b64decoderrrU) raw_imagerrrr(r(r) decode_images    zEndpointClient.decode_imageFr/paramsdatarrcCsLi}|r||d<|r||d<tj|j|j||d}|r"||jS|S)Nr/r)rrVr#)r postrrr!contentrV)r4r/r"r#rpayloadresponser(r(r)r9s zEndpointClient.__call__r)NNNF)rQrMrrrr7rrr!rrrbytesrrr9r(r(r(r)rs*  rr)NFN)s^       W  6