o óÜÓhxuã@sddlZddlZddlZddlmZddlmZddlmZe  e ¡Z Gdd„dej ƒZ Gdd „d ej ƒZGd d „d ej ƒZGd d „d ej ƒZGdd„dej ƒZGdd„dej ƒZd$dd„Zd$dd„Zd$dd„ZGdd„deƒZGdd„deƒZGdd„deƒZd%d d!„ZGd"d#„d#eƒZdS)&éN)Únn)ÚFunctioné)Úloggingcs>eZdZdZ         d ‡fdd„ Zd d d „Z‡ZS) ÚQuantEmbeddingaÞ Quantized version of `torch.nn.Embedding`. Adds quantization-specific arguments on top of `torch.nn.Embedding`. Args: weight_bit (`int`, *optional*, defaults to `8`): Bitwidth for the quantized weight. momentum (`float`, *optional*, defaults to `0.95`): Momentum for updating the activation quantization range. quant_mode (`bool`, *optional*, defaults to `False`): Whether or not the layer is quantized. Nç@Féçffffffî?c s”tƒ ¡||_||_||_||_||_||_||_t   t   ||g¡¡|_ | dt   d¡¡| dt  |j ¡¡| |_| |_| |_d|_tj|_dS)NÚweight_scaling_factoréÚweight_integerF)ÚsuperÚ__init__Únum_ÚdimÚ padding_idxÚmax_normÚ norm_typeÚscale_grad_by_freqÚsparserÚ ParameterÚtorchÚzerosÚweightÚregister_bufferÚ zeros_likeÚ weight_bitÚmomentumÚ quant_modeÚpercentile_modeÚSymmetricQuantFunctionÚapplyÚweight_function) ÚselfÚnum_embeddingsÚ embedding_dimrrrrrÚ_weightrrr©Ú __class__©ú]/var/www/html/ai/venv/lib/python3.10/site-packages/transformers/models/ibert/quant_modules.pyr,s  zQuantEmbedding.__init__c Csº|jstj ||j|j|j|j|j|j ¡dfS|j}|j   ¡}|  ¡  d¡}| ¡  d¡}t|j||dƒ|_| |j|j|j|j¡|_tj ||j|j|j|j|j|j ¡}||j|jfS)Nr F)rrÚ functionalÚ embeddingrrrrrrÚdataÚdetachÚminÚexpandÚmaxÚ$symmetric_linear_quantization_paramsrr r"rr ) r#ÚxÚ positionsÚincremental_stateÚwÚ w_transformÚw_minÚw_maxÚemb_intr)r)r*ÚforwardMs<ù ö ÿù zQuantEmbedding.forward) NNrFFNrr F©NN)Ú__name__Ú __module__Ú __qualname__Ú__doc__rr;Ú __classcell__r)r)r'r*rsô!rcs>eZdZdZd ‡fdd„ Zdd„Z     d d d „Z‡ZS) ÚQuantActap Quantizes the given activation. Args: activation_bit (`int`): Bitwidth for the quantized activation. act_range_momentum (`float`, *optional*, defaults to `0.95`): Momentum for updating the activation quantization range. per_channel (`bool`, *optional*, defaults to `False`): Whether to or not use channel-wise quantization. channel_len (`int`, *optional*): Specify the channel length when set the *per_channel* True. quant_mode (`bool`, *optional*, defaults to `False`): Whether or not the layer is quantized. r FNcs”tƒ ¡||_||_||_||_d|_tj|_ |jsF|  dt   d¡¡|  dt   d¡¡|  dt   d¡¡|j d8_ |jd7_dStdƒ‚)NFÚx_minr Úx_maxÚact_scaling_factorgñh㈵øä>ú;per-channel mode is not currently supported for activation.)r rÚactivation_bitÚact_range_momentumrÚ per_channelÚ percentiler r!Ú act_functionrrrrCrDÚNotImplementedError)r#rGrHrIÚ channel_lenrr'r)r*rƒs zQuantAct.__init__c Cs:|jj›d|j›d|j›d|j ¡d›d|j ¡d›d S)Nz(activation_bit=z, quant_mode: z , Act_min: z.2fz , Act_max: ú))r(r=rGrrCÚitemrD)r#r)r)r*Ú__repr__–sÿ ÿ þÿzQuantAct.__repr__c Cs¦|dur|n||}|jr†|jrJdƒ‚|jrJdƒ‚|j ¡}|j ¡} |  ¡ ¡dkr5| ¡ ¡dks9Jdƒ‚|j ¡dkrT|j  ¡dkrT|j||_|j | |_ n2|j dkrjt  |j|¡|_t  |j | ¡|_ n|j|j |d|j |_|j |j | d|j |_ |j s|dfS|dur”|jn|}|dur|j n|} t |j|| |jd |_|durº| ||j|j|j¡} n t |||j|j||¡} |j d¡} | | |jfS) Nz:percentile mode is not currently supported for activation.rFrz5NaN detected when computing min/max of the activationg¢&ú|”ç¾g¢&ú|”ç>éÿÿÿÿr )rI)ÚtrainingrJrIr-r/r1ÚisnanÚsumrCrDrHrrr2rGrErKÚ FixedPointMulr!Úview) r#r3Úpre_act_scaling_factorÚidentityÚidentity_scaling_factorÚ specified_minÚ specified_maxÚx_actrCrDÚ quant_act_intÚcorrect_output_scaler)r)r*r;sH   "ÿ   ÿú zQuantAct.forward)r FNF)NNNNN©r=r>r?r@rrPr;rAr)r)r'r*rBrs ùrBcs:eZdZdZ d ‡fdd„ Z‡fdd „Zdd d „Z‡ZS)Ú QuantLineara8 Quantized version of `torch.nn.Linear`. Adds quantization-specific arguments on top of `torch.nn.Linear`. Args: weight_bit (`int`, *optional*, defaults to `8`): Bitwidth for the quantized weight. bias_bit (`int`, *optional*, defaults to `32`): Bitwidth for the quantized bias. per_channel (`bool`, *optional*, defaults to `False`): Whether or not to use channel-wise quantization. quant_mode (`bool`, *optional*, defaults to `False`): Whether or not the layer is quantized. Tré Fcs®tƒ ¡||_||_t t ||g¡¡|_|  dt  |j¡¡|  dt |j¡¡|r?t t |¡¡|_ |  dt  |j ¡¡||_ ||_ ||_||_||_ d|_tj|_dS)Nr Úfc_scaling_factorÚ bias_integerF)r rÚ in_featuresÚ out_featuresrrrrrrrÚbiasrrrIÚbias_bitrr r!r")r#rdrerfrrgrIrr'r)r*rës  zQuantLinear.__init__cs*tƒ ¡}d|›d|j›d|j›d}|S)Nú(z weight_bit=z , quant_mode=rN)r rPrr)r#Úsr'r)r*rPs zQuantLinear.__repr__Nc Cs |jstjj||j|jddfS|dur|jdksJdƒ‚|j}|j ¡}|j r=t j |ddd\}}t j |ddd\}}n|  ¡  d¡}|  ¡  d¡}t|j|||j ƒ|_| |j|j|j|j¡|_|j|}|jdurw| |j|jd|¡|_| dd¡}||} tjj| |j|jd||fS)N)rrf)r z«Input activation to the QuantLinear layer should be globally (non-channel-wise) quantized. Please add a QuantAct layer with `per_channel = True` before this QuantAct layerr )rÚoutFrQ)rrr+ÚlinearrrfÚshaper-r.rIrr/r1r0r2rrbr"rr rgrcrV) r#r3Úprev_act_scaling_factorr6r7r8Ú_r9Úbias_scaling_factorÚx_intr)r)r*r;s0ÿ ÿ   þzQuantLinear.forward)TrraFF©Nr_r)r)r'r*r`Üs ÿ r`cs4eZdZdZd ‡fdd„ Zdd„Zd d d „Z‡ZS) ÚIntGELUa} Quantized version of `torch.nn.GELU`. Adds quantization-specific arguments on top of `torch.nn.GELU`. Args: quant_mode (`bool`, *optional*, defaults to `False`): Whether or not the layer is quantized. force_dequant (`str`, *optional*, defaults to `"none"`): Force dequantize the layer if either "gelu" or "nonlinear" is given. TÚnonecsjtƒ ¡||_|dvrt d¡d|_|jst ¡|_d|_d|_ gd¢|_ |j d|j d<dS) N)Ú nonlinearÚgeluzForce dequantize geluFg à- ö?é)g]mÅþ²{Ò¿gçû©ñÒMü¿r ér) r rrÚloggerÚinforÚGELUÚ activation_fnÚkÚconstÚcoeff)r#rÚ force_dequantr'r)r*r7s    zIntGELU.__init__cCsšt |jd|¡}t |jd|d¡}t |¡}t t |¡| ¡}|||d|}|d|jd}t |d|j¡}|d|j}||fS©Nr rwr) rÚfloorr~Úsignr/ÚabsÚ floor_ster!r})r#rpÚscaling_factorÚb_intÚc_intr‚Úabs_intÚy_intr)r)r*Úint_erfGs zIntGELU.int_erfNcCs^|js | |¡dfS||}| |||j¡\}}d|}|||}||d}|||fS)Nçð?rw)rr{rŠr|)r#r3r…rpÚ sigmoid_intÚsigmoid_scaling_factorÚ shift_intr)r)r*r;Vs   zIntGELU.forward)Trsrq)r=r>r?r@rrŠr;rAr)r)r'r*rr,s  rrcs:eZdZdZd ‡fdd„ Zdd„Zdd „Zd d „Z‡ZS) Ú IntSoftmaxaØ Quantized version of `torch.nn.Softmax`. Adds quantization-specific arguments on top of `torch.nn.Softmax`. Args: output_bit (`int`): Bitwidth for the layer output activation. quant_mode (`bool`, *optional*, defaults to `False`): Whether or not the layer is quantized. force_dequant (`str`, *optional*, defaults to `"none"`): Force dequantize the layer if either "softmax" or "nonlinear" is given. FrscsŽtƒ ¡||_d|_||_|dvrt d¡d|_td|jd|_d|_ d|_ gd ¢|_ |j d |j d <|j d |j d <dS) Nra)rtÚsoftmaxzForce dequantize softmaxFé©rgvq à-æ¿é)gN„ª$ôëÖ?g¾Ã'|:ï?r‹r rrw) r rÚ output_bitÚmax_bitrrxryrBÚactÚx0r}Úcoef)r#r”rrr'r)r*rrs   zIntSoftmax.__init__cCs~t ¡t |jd|¡}t |jd|d¡}Wdƒn1s%wY||||}|jd|d}||fSr€)rÚno_gradrr˜)r#rpr…r†r‡Úzr)r)r*Úint_polynomialƒs þzIntSoftmax.int_polynomialcCs¬t ¡t |j|¡}Wdƒn1swYt ||j|¡}t ||¡}|||}| ||¡\}}tj t |d|j|¡dd}|d|j}||fS)Nrwr©r/) rr™rr—r1r}r„r!r›Úclamp)r#rpr…Úx0_intÚqÚrÚexp_intÚexp_scaling_factorr)r)r*Úint_exp‹s ÿ "zIntSoftmax.int_expc Cs¾|js tjj|dddfS||}|jddd\}}||}| ||¡\}}| ||¡\}}||}|jddd} t  d|j | ¡} t  || d|j |j ¡}dd|j }|||fS)NrQ©rT)rÚkeepdimrwr ) rrr+rr1r£r–rTr„r!r•r”) r#r3r…rpÚ x_int_maxrnr¡r¢ÚexpÚ exp_int_sumÚfactorr)r)r*r;—s zIntSoftmax.forward)Frs) r=r>r?r@rr›r£r;rAr)r)r'r*res   rcs<eZdZdZd‡fdd„ Zdd„Zd d „Zdd d „Z‡ZS)Ú IntLayerNormaû Quantized version of `torch.nn.LayerNorm`. Adds quantization-specific arguments on top of `torch.nn.LayerNorm`. Args: output_bit (`int`, *optional*, defaults to `8`): Bitwidth for the layer output activation. quant_mode (`bool`, *optional*, defaults to `False`): Whether or not the layer is quantized. force_dequant (`str`, *optional*, defaults to `"none"`): Force dequantize the layer if either "layernorm" or "nonlinear" is given. rFrscs’tƒ ¡||_||_t t |¡¡|_t t |¡¡|_ ||_ |dvr,t   d¡d|_ |  dt d¡¡||_d|_d|_t|j|j d|_dS)N)rtÚ layernormzForce dequantize layernormFÚshiftr rar’)r rÚnormalized_shapeÚepsrrrrrrfrrxryrr”r•Údim_sqrtrBÚ activation)r#r­r®r”rrr'r)r*r¹s  zIntLayerNorm.__init__cCsžt ¡A|d}tj|ddd}t t |d|j¡¡ ¡ ¡}|j}t |j|¡|_t   dt |ƒ›dt |jƒ›¡WdƒdS1sHwYdS)NrwT©Úaxisr¥zDynamic shift adjustment: z -> ) rr™rTÚlog2Úsqrtr•Úceilr1r¬rxryÚint)r#r‰Úy_sq_intÚvar_intr¬Ú shift_oldr)r)r*Ú set_shiftÌs """úzIntLayerNorm.set_shiftcCs:| |¡t |d|j¡}|d}tj|ddd}|S)z± This fallback function is called when overflow is detected during training time, and adjusts the `self.shift` to avoid overflow in the subsequent runs. rwTr±)rºr„r!r¬rrT)r#r‰Ú y_int_shiftedr·r¸r)r)r*Úoverflow_fallbackÕs zIntLayerNorm.overflow_fallbackNcCs¬|js.|jddd}||}tj|dddd}|t |j|¡}||j|j}|dfS|jdurHtj|j dtj d}t |¡  |j ¡|_||}t  |jddd¡}||} t | d|j¡} | d} tj| ddd} |jr|  ¡d|jkr| | ¡} |  ¡d|jdksJdƒ‚t t | ¡¡d|j} t d| ¡}t | |d¡} |jd}|jj ¡|jj ¡}t ||¡}| |} ||j}| |}||fS) NrwTr±)Údtypegš™™™™™¹?zfError detected in overflow handling: `var_int` exceeds `self.max_bit` (the maximum possible bit width)li@)rÚmeanrr´r®rrfr¯ÚtensorrlÚfloatÚtoÚdeviceÚ round_ster!r„r¬rTrRr1r•r¼r-r.)r#r3r…r¾ÚyÚvarÚnrpÚmean_intr‰r»r·r¸Ústd_intr©rfÚbias_intr)r)r*r;às@  ÿ  zIntLayerNorm.forward)rFrsrq) r=r>r?r@rrºr¼r;rAr)r)r'r*rª¬s    rªFc Cs€|jd}t|d|dƒ}t||dƒ}tj||dj}|dkr(|d}n tj| |dj }|s<| ¡}| ¡}||fS)aÆ Calculate the percentile max and min values in a given tensor Args: input (`torch.Tensor`): The target tensor to calculate percentile max and min. lower_percentile (`float`): If 0.1, means we return the value of the smallest 0.1% value in the tensor as percentile min. upper_percentile (`float`): If 99.9, means we return the value of the largest 0.1% value in the tensor as percentile max. output_tensor (`bool`, *optional*, defaults to `False`): If True, this function returns tensors, otherwise it returns values. Returns: `Tuple(torch.Tensor, torch.Tensor)`: Percentile min and max value of *input* rr g{®Gáz„?)r|)rlÚroundrÚkthvalueÚvaluesrO) ÚinputÚlower_percentileÚupper_percentileÚ output_tensorÚ input_lengthÚ lower_indexÚ upper_indexÚ upper_boundÚ lower_boundr)r)r*Úget_percentile_min_maxs  rÖcCs¢t|jƒdkr| dddd¡}| dddd¡}nt|jƒdkr,| dd¡}| dd¡}n | d¡}| d¡}|rF| d|¡ |¡ ¡|St d|||¡S)a? Quantize single-precision input tensor to integers with the given scaling factor and zeropoint. Args: input (`torch.Tensor`): Single-precision input tensor to be quantized. scale (`torch.Tensor`): Scaling factor for quantization. zero_pint (`torch.Tensor`): Shift for quantization. inplace (`bool`, *optional*, defaults to `False`): Whether to compute inplace or not. Returns: `torch.Tensor`: Linearly quantized value of *input* according to *scale* and *zero_point*. érQr rwr‹)ÚlenrlrVÚmul_Úadd_Úround_rrÊ)rÍÚscaleÚ zero_pointÚinplacer)r)r*Úlinear_quantize5s   rßcCs²t ¡Kd|dd}|r-tjtj| ¡| ¡gdddd\}}tj|dd|}nt| ¡| ¡ƒ}tj|dd|}Wdƒ|SWdƒ|S1sRwY|S)a/ Compute the scaling factor with the given quantization range for symmetric quantization. Args: saturation_min (`torch.Tensor`): Lower bound for quantization range. saturation_max (`torch.Tensor`): Upper bound for quantization range. per_channel (`bool`, *optional*, defaults to `False`): Whether to or not use channel-wise quantization. Returns: `torch.Tensor`: Scaling factor that linearly quantizes the given range between *saturation_min* and *saturation_max*. rwr r¤g:Œ0âŽyE>rœN)rr™r1Ústackrƒr)Únum_bitsÚsaturation_minÚsaturation_maxrIrÆrÜrnr)r)r*r2Xs ( ÷ ú û õ r2c@ó(eZdZdZedd„ƒZedd„ƒZdS)r zw Class to quantize the given floating-point values using symmetric quantization with given range and bitwidth. cCsPt d¡ |j¡}d|dd}t|||dd}t || |d¡}||_|S)a6 Args: x (`torch.Tensor`): Floating point tensor to be quantized. k (`int`): Quantization bitwidth. percentile_mode (`bool`): Whether or not to use percentile calibration. scale (`torch.Tensor`): Pre-calculated scaling factor for *x*. Note that the current implementation of SymmetricQuantFunction requires pre-calculated scaling factor. Returns: `torch.Tensor`: Symmetric-quantized value of *input*. grwr F)rÞ)rr¿rÁrÂrßrrÜ)Úctxr3r|rrÜrÝrÆÚ new_quant_xr)r)r*r;}s zSymmetricQuantFunction.forwardcCsb|j}t|jƒdkr| dddd¡}nt|jƒdkr!| dd¡}n| d¡}| ¡|ddddfS)Nr×rQr rw)rÜrØrlrVÚclone)råÚ grad_outputrÜr)r)r*Úbackward—s zSymmetricQuantFunction.backwardN©r=r>r?r@Ú staticmethodr;rér)r)r)r*r xs  r c@rä)r„z; Straight-through Estimator(STE) for torch.floor() cCó t |¡Srq)rr©rår3r)r)r*r;ªó zfloor_ste.forwardcCó| ¡Srq©rç©rårèr)r)r*ré®ózfloor_ste.backwardNrêr)r)r)r*r„¥ó  r„c@rä)rÃz; Straight-through Estimator(STE) for torch.round() cCrìrq)rrÊrír)r)r*r;¸rîzround_ste.forwardcCrïrqrðrñr)r)r*ré¼ròzround_ste.backwardNrêr)r)r)r*rórórÃécCs®| ¡}| d¡}t | ¡ ¡¡\}}g}|D]}tt |d|¡j t d¡tj dƒ}|  |¡qt  |¡}t |ƒ|}t |¡ |j¡ |¡t |¡ |j¡ |¡fS)zü Decompose the scaling factor into mantissa and twos exponent. Args: scaling_factor (`torch.Tensor`): Target scaling factor to decompose. Returns: ``Tuple(torch.Tensor, torch.Tensor)`: mantisa and exponent rQrwÚ1)Úrounding)ÚsizerVÚnpÚfrexpÚcpuÚnumpyr¶ÚdecimalÚDecimalÚquantizeÚ ROUND_HALF_UPÚappendÚarrayrÀrÚ from_numpyrÁrÂ)Úinputsr•Úshape_of_inputÚoutput_mÚoutput_eÚtmp_mÚmÚ int_m_shiftedr)r)r*Ú batch_frexpÁs "ÿ   þr c@s.eZdZdZe  ddd„ƒZedd„ƒZdS)rUaQ Function to perform fixed-point arithmetic that can match integer arithmetic on hardware. Args: pre_act (`torch.Tensor`): Input tensor. pre_act_scaling_factor (`torch.Tensor`): Scaling factor of the input tensor *pre_act*. bit_num (`int`): Quantization bitwidth. z_scaling_factor (`torch.Tensor`): Scaling factor of the output tensor. identity (`torch.Tensor`, *optional*): Identity tensor, if exists. identity_scaling_factor (`torch.Tensor`, *optional*): Scaling factor of the identity tensor *identity*, if exists. Returns: `torch.Tensor`: Output tensor(*pre_act* if *identity* is not given, otherwise the addition of *pre_act* and *identity*), whose scale is rescaled to *z_scaling_factor*. NcCs”t|jƒdkr dd„}ndd„}||_d|dd}t ¡¡||ƒ}|dur,||ƒ}||_t ||¡} | tj¡} | tj ¡ tj¡} | | } || ƒ} t | ƒ\} }|  tj¡|  tj¡}t |d|¡}|dur«t ||¡}| tj¡} | tj ¡ tj¡} | | } || ƒ} t | ƒ\}}| tj¡| tj¡}t |d|¡}||}t  | tj ¡| d|¡WdƒS1sÃwYdS)NrcSs|Srqr)©r3r)r)r*Úsz'FixedPointMul.forward..cSs| ddd¡S)Nr rQ)rVr r)r)r*r srwr r) rØrlrXrr™Úz_scaling_factorrÊÚtypeÚdoublerÀr r)råÚpre_actrWÚbit_numr rXrYÚreshaperÆÚz_intÚ_AÚ_BÚ new_scalerÚeÚoutputÚwx_intÚm1Úe1Úoutput1r)r)r*r;ús<      $ßzFixedPointMul.forwardcCs8d}|jdur| ¡|j}| ¡|jdddd|dfSrq)rXrçr )rårèÚ identity_gradr)r)r*ré/s zFixedPointMul.backwardr<rêr)r)r)r*rUãs ù4rU)F)rô)rürûrørrÚtorch.autogradrÚutilsrÚ get_loggerr=rxÚModulerrBr`rrrrªrÖrßr2r r„rÃr rUr)r)r)r*Ús*    SjP9G e $ # - "