o hJ@sddlZddlZddlmZddlmZddlmZddl m Z e e Z dZ GdddZGdd d ZGd d d eZGd d d eZGdddeZGdddeZGdddeZGdddeZGdddeZGdddeZGdddeZGdddeZGdddeZdS) N)add_start_docstrings) get_loggerad Args: input_ids (`jnp.ndarray` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. Indices can be obtained using [`PreTrainedTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) scores (`jnp.ndarray` of shape `(batch_size, config.vocab_size)`): Prediction scores of a language modeling head. These can be logits for each vocabulary when not using beam search or log softmax for each vocabulary token when using beam search kwargs (`Dict[str, Any]`, *optional*): Additional logits processor specific kwargs. Return: `jnp.ndarray` of shape `(batch_size, config.vocab_size)`: The processed prediction scores. c@4eZdZdZeedejdejdejfddZdS)FlaxLogitsProcessorzSAbstract base class for all logit processors that can be applied during generation. input_idsscoresreturncCt|jd)z"Flax method for processing logits.H is an abstract class. Only classes inheriting this class can be called.NotImplementedError __class__selfrrra/var/www/html/ai/venv/lib/python3.10/site-packages/transformers/generation/flax_logits_process.py__call__5 zFlaxLogitsProcessor.__call__N __name__ __module__ __qualname____doc__r!LOGITS_PROCESSOR_INPUTS_DOCSTRINGjnpndarrayrrrrrr2"rc@r)FlaxLogitsWarperzjAbstract base class for all logit warpers that can be applied during generation with multinomial sampling.rrr cCr )zFlax method for warping logits.r r rrrrr@rzFlaxLogitsWarper.__call__Nrrrrrr=rrc @s8eZdZdZeedejdejdedejfddZ dS) FlaxLogitsProcessorLista. This class can be used to create a list of [`FlaxLogitsProcessor`] or [`FlaxLogitsWarper`] to subsequently process a `scores` input tensor. This class inherits from list and adds a specific *__call__* method to apply each [`FlaxLogitsProcessor`] or [`FlaxLogitsWarper`] to the inputs. rrcur_lenr c s|D]C}t|jj}t|dkr?tfddt|ddDs4tdt|d|j d||||fi}q||||}q|S)Nc3s|]}|vVqdSNr).0argkwargsrr Tsz3FlaxLogitsProcessorList.__call__..rz,Make sure that all the required parameters: z for z$ are passed to the logits processor.) inspect signaturer parameterslenalllistkeys ValueErrorr)rrrr r& processor function_argsrr%rrOs &z FlaxLogitsProcessorList.__call__N) rrrrrrrrintrrrrrrHs&rc@>eZdZdZdefddZdejdejdedejfd d Z d S) FlaxTemperatureLogitsWarperz [`FlaxLogitsWarper`] for temperature (exponential scaling output probability distribution). Args: temperature (`float`): The value used to module the logits distribution. temperaturecCs*t|tr |dkstd|||_dS)Nrz:`temperature` has to be a strictly positive float, but is ) isinstancefloatr/r5)rr5rrr__init__hs z$FlaxTemperatureLogitsWarper.__init__rrr r cCs||j}|Sr")r5rrrr rrrrns z$FlaxTemperatureLogitsWarper.__call__N) rrrrr7r8rrr2rrrrrr4_s$r4c@sReZdZdZed dfdededefddZd ejd ejd ed ejfd dZ dS)FlaxTopPLogitsWarpera= [`FlaxLogitsWarper`] that performs top-p, i.e. restricting to top tokens summing to prob_cut_off <= prob_cut_off. Args: top_p (`float`): If set to < 1, only the smallest set of most probable tokens with probabilities that add up to `top_p` or higher are kept for generation. filter_value (`float`, *optional*, defaults to -inf): All filtered values will be set to this float value. min_tokens_to_keep (`int`, *optional*, defaults to 1): Minimum number of tokens that cannot be filtered. Inftop_p filter_valuemin_tokens_to_keepcCs^t|tr |dks |dkrtd|t|tr|dkr$td|||_||_||_dS)Nrg?z.`top_p` has to be a float > 0 and < 1, but is r<z:`min_tokens_to_keep` has to be a positive integer, but is )r6r7r/r2r=r>r?)rr=r>r?rrrr8s zFlaxTopPLogitsWarper.__init__rrr r c Cst||jd\}}t||j}tjj|ddj dd}||j k}t |d}||j dddf dO}|j ddd|jf d}t|||} tj|| d} | S)Naxisr<rT)laxtop_kshaper full_liker>jaxnnsoftmaxcumsumr=rollatsetr?where sort_key_val) rrrr topk_scores topk_indices mask_scorescumulative_probs score_masktopk_next_scores next_scoresrrrrs  zFlaxTopPLogitsWarper.__call__N rrrrr7r2r8rrrrrrrr:ss" $ r:c@sReZdZdZed dfdededefddZd ejd ejd ed ejfd dZ dS)FlaxTopKLogitsWarpera [`FlaxLogitsWarper`] that performs top-k, i.e. restricting to the k highest probability elements. Args: top_k (`int`): The number of highest probability vocabulary tokens to keep for top-k-filtering. filter_value (`float`, *optional*, defaults to -inf): All filtered values will be set to this float value. min_tokens_to_keep (`int`, *optional*, defaults to 1): Minimum number of tokens that cannot be filtered. r;r<rDr>r?cCs6t|tr |dkrtd|t|||_||_dS)Nrz6`top_k` has to be a strictly positive integer, but is )r6r2r/maxrDr>)rrDr>r?rrrr8s  zFlaxTopKLogitsWarper.__init__rrr r cCs|j\}}t|||j}t|j|jd}t||\}} tt||dddf||f } | } | | } |j |  | }| ||} | S)Nr@) rErfullr>minrDrC broadcast_toarangeflattenrLrMreshape)rrrr batch_size vocab_sizenext_scores_flattopkrPrQshifttopk_scores_flattopk_indices_flatrVrrrrs *  zFlaxTopKLogitsWarper.__call__NrWrrrrrXs" $rXc@s>eZdZdZdefddZdejdejdedejfd d Zd S) !FlaxForcedBOSTokenLogitsProcessorz [`FlaxLogitsProcessor`] that enforces the specified token as the first generated token. Args: bos_token_id (`int`): The id of the token to force as the first generated token. bos_token_idcCs ||_dSr")rh)rrhrrrr8s z*FlaxForcedBOSTokenLogitsProcessor.__init__rrr r cCsNt|jtd }dt|d}t||jdd|jfd|}|SNinfr<r) rrZrEr7bool_rNrLrhrMrrrr new_scores apply_penaltyrrrrs$z*FlaxForcedBOSTokenLogitsProcessor.__call__N rrrrr2r8rrrrrrrrgs$rgc@BeZdZdZdedefddZdejdejded ejfd d Zd S) !FlaxForcedEOSTokenLogitsProcessorae [`FlaxLogitsProcessor`] that enforces the specified token as the last generated token when `max_length` is reached. Args: max_length (`int`): The maximum length of the sequence to be generated. eos_token_id (`int`): The id of the token to force as the last generated token when `max_length` is reached. max_length eos_token_idcCs||_||_dSr")rrrs)rrrrsrrrr8s z*FlaxForcedEOSTokenLogitsProcessor.__init__rrr r cCsTt|jtd }dt||jd}t||jdd|jf d|}|Sri) rrZrEr7rkrrrNrLrsrMrlrrrrs$z*FlaxForcedEOSTokenLogitsProcessor.__call__Nrorrrrrqs $rqc@rp) FlaxMinLengthLogitsProcessora3 [`FlaxLogitsProcessor`] enforcing a min-length by setting EOS probability to 0. Args: min_length (`int`): The minimum length below which the score of `eos_token_id` is set to `-float("Inf")`. eos_token_id (`int`): The id of the *end-of-sequence* token. min_lengthrscCsPt|tr |dkrtd|t|tr|dkr td|||_||_dS)Nrz2`min_length` has to be a positive integer, but is z4`eos_token_id` has to be a positive integer, but is )r6r2r/rurs)rrursrrrr8s  z%FlaxMinLengthLogitsProcessor.__init__rrr r cCsFdt||jdd}t||jdd|jftd |}|S)Nr<rrj)rcliprurNrLrsrMr7rrrr rnrrrrs*z%FlaxMinLengthLogitsProcessor.__call__Nrorrrrrts $ rtc@s&eZdZdZddZdefddZdS)(FlaxSuppressTokensAtBeginLogitsProcessora [`FlaxLogitsProcessor`] supressing a list of tokens as soon as the `generate` function starts generating using `begin_index` tokens. This should ensure that the tokens defined by `begin_suppress_tokens` are not sampled at the begining of the generation. Args: begin_suppress_tokens (`List[int]`): Tokens to not sample. begin_index (`int`): Index where the tokens are suppressed. cCst||_||_dSr")r-begin_suppress_tokens begin_index)rryrzrrrr8s  z1FlaxSuppressTokensAtBeginLogitsProcessor.__init__r cCsBdt||j}t||jdd|jftd |}|S)Nr<rj)rrkrzrNrLryrMr7rwrrrr s*z1FlaxSuppressTokensAtBeginLogitsProcessor.__call__N)rrrrr8r2rrrrrrxs rxc@r3) !FlaxSuppressTokensLogitsProcessorz [`FlaxLogitsProcessor`] suppressing a list of tokens at each decoding step. The processor will set their log probs to be `-inf` so they are not sampled. Args: suppress_tokens (`list`): Tokens to not sample. suppress_tokenscCst||_dSr")r-r|)rr|rrrr82sz*FlaxSuppressTokensLogitsProcessor.__init__rrr r cCs |jd|jftd }|S)N.rj)rLr|rMr7r9rrrr5sz*FlaxSuppressTokensLogitsProcessor.__call__N) rrrrr-r8rrr2rrrrrr{(s $r{c@s8eZdZdZddZdejdejdedejfdd Zd S) FlaxForceTokensLogitsProcessora [`FlaxLogitsProcessor`] that takes a list of pairs of integers which indicates a mapping from generation indices to token indices that will be forced before sampling. The processor will set their log probs to 0 and all other tokens to `-inf` so that they are sampled at their corresponding index. Args: force_token_map (`list`): Map giving token ids and indices where they will be forced to be sampled. cCsbt|}tjt|dtjdd}|D]\}}|dur(|j||}qt||_ dS)Nr<dtyper@) dictronesrYr.int32itemsrLrMforce_token_array)rforce_token_maprindextokenrrrr8Fs z'FlaxForceTokensLogitsProcessor.__init__rrr r csBfddtjjdkfddfddS)NcsXjd}j|}tjjdtd }tj|dfjd}t||d|f}|S)Nrr~rjr<) rErr ones_likerr7zerosrCdynamic_update_slice)generation_idxr` current_tokenrmupdates)rrrr _force_tokenRs  z=FlaxForceTokensLogitsProcessor.__call__.._force_tokenrcSr"rrrrr^z9FlaxForceTokensLogitsProcessor.__call__..cs*tjdkfddfddS)NrcsSr"rr)rr rrrcszKFlaxForceTokensLogitsProcessor.__call__....crr"rrrrrrer)rCcondrrrr rrrrr`s    )rCrrrEr9rrrrQs   z'FlaxForceTokensLogitsProcessor.__call__N) rrrrr8rrr2rrrrrr};s $ r}c@s eZdZdZddZddZdS)#FlaxWhisperTimeStampLogitsProcessora{ Whisper specific Processor. This processor can be used to force a list of tokens. The processor will set their log probs to `inf` so that they are sampled at their corresponding index. Args: generate_config (`GenerateConfig`): The generate config used to generate the output. The following parameters are required: eos_token_id (`int`, *optional*, defaults to 50257): The id of the *end-of-sequence* token. no_timestamps_token_id (`int`, *optional*, defaults to 50363): The id of the `"<|notimestamps|>"` token. max_initial_timestamp_index (`int`, *optional*, defaults to 1): Used to set the maximum value of the initial timestamp. This is used to prevent the model from predicting timestamps that are too far in the future. cCsp|j|_|j|_|jd|_|d|_|jr|jd7_t|dr'|j|_n|j|_|jdur6|j|_dSdS)Nr<rmax_initial_timestamp_index)rsno_timestamps_token_idtimestamp_beginrzis_multilingualhasattrrra)rgenerate_config model_configdecoder_input_lengthrrrr8|s      z,FlaxWhisperTimeStampLogitsProcessor.__init__c s|jddjftd }fdd}t|||}tjkdd}tj du|d}j j }t||jdd|ddftd |}tj j |dd}fd d }t|||}|S) Nrjc stjdkdd}t|djk|d}tjdkdd}t|djkd|}t|t|dk|jjdtd |jdjtd |S)Nr<TFrrrj)rrNrzrrLrMr7rs) input_ids_kscores_klast_was_timestamppenultimate_was_timestampr rrr handle_pairss*zBFlaxWhisperTimeStampLogitsProcessor.__call__..handle_pairsTFr<r@rAcsXtjj|jddd}t|dj}t||k|jdjt d |S)Nr@rArj) rGrH logsumexprrrYrNrLrMr7) logprobs_krtimestamp_logprobmax_text_token_logprob)rrrhandle_cumulative_probsszMFlaxWhisperTimeStampLogitsProcessor.__call__..handle_cumulative_probs) rLrrMr7rGvmaprrNrzrrrH log_softmax) rrrr rapply_max_initial_timestamp last_allowedlogprobsrrrrrs&  $  z,FlaxWhisperTimeStampLogitsProcessor.__call__N)rrrrr8rrrrrrks r)r(rGjax.laxrC jax.numpynumpyrutilsr utils.loggingrrloggerrrrr-rr4r:rXrgrqrtrxr{r}rrrrrs*      ,#0