o h@sdZddlZddlmZmZmZddlmZddlm Z ddl m Z ddl m Z mZmZmZdd lmZd d lmZGd d d e ZdS)zq Processor class for InstructBLIP. Largely copy of Blip2Processor with addition of a tokenizer for the Q-Former. N)ListOptionalUnion) BatchFeature) ImageInput)ProcessorMixin)PaddingStrategyPreTokenizedInput TextInputTruncationStrategy) TensorType) AutoTokenizerc$seZdZdZddgZdZdZfddZ       d)d ed e e e e e e e fde de e eefde e eefdeededeedee de de de de de de dee eefdef"ddZdd Zd!d"Zed#d$Zfd%d&Zed'd(ZZS)*InstructBlipProcessora Constructs an InstructBLIP processor which wraps a BLIP image processor and a LLaMa/T5 tokenizer into a single processor. [`InstructBlipProcessor`] offers all the functionalities of [`BlipImageProcessor`] and [`AutoTokenizer`]. See the docstring of [`~BlipProcessor.__call__`] and [`~BlipProcessor.decode`] for more information. Args: image_processor (`BlipImageProcessor`): An instance of [`BlipImageProcessor`]. The image processor is a required input. tokenizer (`AutoTokenizer`): An instance of ['PreTrainedTokenizer`]. The tokenizer is a required input. qformer_tokenizer (`AutoTokenizer`): An instance of ['PreTrainedTokenizer`]. The Q-Former tokenizer is a required input. image_processor tokenizerBlipImageProcessorrcst||||_dSN)super__init__qformer_tokenizer)selfrrr __class__n/var/www/html/ai/venv/lib/python3.10/site-packages/transformers/models/instructblip/processing_instructblip.pyr3s zInstructBlipProcessor.__init__NTFrimagestextadd_special_tokenspadding truncation max_lengthstridepad_to_multiple_ofreturn_attention_maskreturn_overflowing_tokensreturn_special_tokens_maskreturn_offsets_mappingreturn_token_type_ids return_lengthverbosereturn_tensorsreturncKs|dur |dur tdt}|durV|jd |||||||| | | | | |||d|}|||jd |||||||| | | | | |||d|}|d|d<|d|d<|durf|j||d}|||S) a  This method uses [`BlipImageProcessor.__call__`] method to prepare image(s) for the model, and [`BertTokenizerFast.__call__`] to prepare text for the model. Please refer to the docstring of the above two methods for more information. Nz,You have to specify at least images or text.)rrr r!r"r#r$r%r&r'r(r)r*r+r, input_idsqformer_input_idsattention_maskqformer_attention_mask)r,r) ValueErrorrrupdaterpopr)rrrrr r!r"r#r$r%r&r'r(r)r*r+r,kwargsencoding text_encodingqformer_text_encodingimage_encodingrrr__call__9sb  zInstructBlipProcessor.__call__cO|jj|i|S)z This method forwards all its arguments to PreTrainedTokenizer's [`~PreTrainedTokenizer.batch_decode`]. Please refer to the docstring of this method for more information. )r batch_decoderargsr5rrrr<z"InstructBlipProcessor.batch_decodecOr;)z This method forwards all its arguments to PreTrainedTokenizer's [`~PreTrainedTokenizer.decode`]. Please refer to the docstring of this method for more information. )rdecoder=rrrr@r?zInstructBlipProcessor.decodecCs"|jj}|jj}tt||Sr)rmodel_input_namesrlistdictfromkeys)rtokenizer_input_namesimage_processor_input_namesrrrrAsz'InstructBlipProcessor.model_input_namesc sXtj|rtd|dtj|ddtj|d}|j|tj|fi|S)NzProvided path (z#) should be a directory, not a fileT)exist_okr) ospathisfiler2makedirsjoinrsave_pretrainedr)rsave_directoryr5qformer_tokenizer_pathrrrrMs  z%InstructBlipProcessor.save_pretrainedcKs2tj|dd}|j|fi|}||||S)Nr) subfolder)rfrom_pretrained_get_arguments_from_pretrainedappend)clspretrained_model_name_or_pathr5rr>rrrrQs z%InstructBlipProcessor.from_pretrained)NNTFNNrNNFFFFFTN)__name__ __module__ __qualname____doc__ attributesimage_processor_classtokenizer_classrrrr r rboolstrr r rintr rr:r<r@propertyrArM classmethodrQ __classcell__rrrrrs         O   r)rYrHtypingrrrimage_processing_utilsr image_utilsrprocessing_utilsrtokenization_utils_baser r r r utilsr autorrrrrrs