o hX@sddlmZmZddlmZmZmZmZmZm Z ddl m Z m Z er/ddl mZddlmZer8ddlmZerEdd lZdd lmZeeZee Gd d d e Zd S) )ListUnion)add_end_docstringsis_tf_availableis_torch_availableis_vision_availableloggingrequires_backends)PIPELINE_INIT_ARGSPipeline)Image) load_image)'TF_MODEL_FOR_VISION_2_SEQ_MAPPING_NAMESN)$MODEL_FOR_VISION_2_SEQ_MAPPING_NAMEScsleZdZdZfddZdddZdeeeededfffd d Z dd d Z dd dZ ddZ Z S)ImageToTextPipelinea Image To Text pipeline using a `AutoModelForVision2Seq`. This pipeline predicts a caption for a given image. Example: ```python >>> from transformers import pipeline >>> captioner = pipeline(model="ydshieh/vit-gpt2-coco-en") >>> captioner("https://huggingface.co/datasets/Narsil/image_dummy/raw/main/parrots.png") [{'generated_text': 'two birds are standing next to each other '}] ``` Learn more about the basics of using a pipeline in the [pipeline tutorial](../pipeline_tutorial) This image to text pipeline can currently be loaded from pipeline() using the following task identifier: "image-to-text". See the list of available models on [huggingface.co/models](https://huggingface.co/models?pipeline_tag=image-to-text). cs>tj|i|t|d||jdkrtdStdS)Nvisiontf)super__init__r check_model_type frameworkrr)selfargskwargs __class__Z/var/www/html/ai/venv/lib/python3.10/site-packages/transformers/pipelines/image_to_text.pyr6s  zImageToTextPipeline.__init__NcCszi}i}|dur ||d<|dur||d<|dur||d<|dur8d|vr(i|d<d|dvr2td||dd<||ifS)Nprompttimeoutgenerate_kwargsmax_new_tokenszp'max_new_tokens' is defined twice, once in 'generate_kwargs' and once as a direct parameter, please use only one) ValueError)rr#r"r r!forward_kwargspreprocess_paramsrrr_sanitize_parameters=s"   z(ImageToTextPipeline._sanitize_parametersimagesz Image.Imagec stj|fi|S)a Assign labels to the image(s) passed as inputs. Args: images (`str`, `List[str]`, `PIL.Image` or `List[PIL.Image]`): The pipeline handles three types of images: - A string containing a HTTP(s) link pointing to an image - A string containing a local path to an image - An image loaded in PIL directly The pipeline accepts either a single image or a batch of images. max_new_tokens (`int`, *optional*): The amount of maximum tokens to generate. By default it will use `generate` default. generate_kwargs (`Dict`, *optional*): Pass it to send all of these arguments directly to `generate` allowing full control of this function. timeout (`float`, *optional*, defaults to None): The maximum time in seconds to wait for fetching images from the web. If None, no timeout is set and the call may block forever. Return: A list or a list of list of `dict`: Each result comes as a dictionary with the following key: - **generated_text** (`str`) -- The generated text. )r__call__)rr(rrrrr)SszImageToTextPipeline.__call__cCs$t||d}|duryt|tstdt|d|jjj}|dkrI|j||j d}|j |ddj }|j j g|}t |d}|d |in8|d krW|j|||j d }n*|d krq|j||j d}|j ||j d }||ntd|d|j||j d}|jjjdkr|durd|d <|S)N)r!z&Received an invalid text input, got - zy - but expected a single string. Note also that one single text can be provided for conditional image to text generation.git)r(return_tensorsF)textadd_special_tokensr input_ids pix2struct)r( header_textr+zvision-encoder-decoder)r+z Model type z- does not support conditional text generation)r isinstancestrr$typemodelconfig model_typeimage_processorr tokenizerr. cls_token_idtorchtensor unsqueezeupdate)rimager r!r6 model_inputsr. text_inputsrrr preprocessqs0    zImageToTextPipeline.preprocesscCsjd|vrt|dtrtdd|dDrd|d<|dur i}||jj}|jj|fi||}|S)Nr.css|]}|duVqdSNr).0xrrr sz/ImageToTextPipeline._forward..)r1listallpopr4main_input_namegenerate)rr?r"inputs model_outputsrrr_forwards zImageToTextPipeline._forwardcCs0g}|D]}d|jj|ddi}||q|S)Ngenerated_textT)skip_special_tokens)r8decodeappend)rrLrecords output_idsrecordrrr postprocesss zImageToTextPipeline.postprocess)NNNN)NNrB)__name__ __module__ __qualname____doc__rr'rr2rr)rArMrU __classcell__rrrrrs  &  'r)typingrrutilsrrrrr r baser r PILr image_utilsrmodels.auto.modeling_tf_autorr:models.auto.modeling_autor get_loggerrVloggerrrrrrs