o hC@sdZddlmZddlZddlZddlmZmZddlmZm Z m Z ddl m Z ddl mZmZmZdd lmZmZmZdd lmZdd lmZd d lmZeeZdZdZgdZ dZ!dZ"dgZ#Gdddej$Z%Gdddej$Z&Gdddej$Z'Gdddej$Z(Gdddej$Z)Gdddej$Z*Gdddej$Z+Gd d!d!ej$Z,Gd"d#d#eZ-d$Z.d%Z/ed&e.Gd'd(d(e-Z0ed)e.Gd*d+d+e-Z1dS),z PyTorch RegNet model.)OptionalN)Tensornn)BCEWithLogitsLossCrossEntropyLossMSELoss)ACT2FN)add_code_sample_docstringsadd_start_docstrings%add_start_docstrings_to_model_forward)BaseModelOutputWithNoAttention(BaseModelOutputWithPoolingAndNoAttention$ImageClassifierOutputWithNoAttention)PreTrainedModel)logging) RegNetConfigrzfacebook/regnet-y-040)ri@rztabby, tabby catcsLeZdZ    ddededededed eef fd d Zd d ZZS)RegNetConvLayerrrrelu in_channels out_channels kernel_sizestridegroups activationc sXttj|||||d|dd|_t||_|dur%t||_ dSt|_ dS)NF)rrpaddingrbias) super__init__rConv2d convolution BatchNorm2d normalizationr Identityr)selfrrrrrr __class__`/var/www/html/ai/venv/lib/python3.10/site-packages/transformers/models/regnet/modeling_regnet.pyr!8s  $zRegNetConvLayer.__init__cCs"||}||}||}|SN)r#r%rr' hidden_stater*r*r+forwardNs   zRegNetConvLayer.forward)rrrr) __name__ __module__ __qualname__intrstrr!r/ __classcell__r*r*r(r+r7s&rcs.eZdZdZdeffdd ZddZZS)RegNetEmbeddingszP RegNet Embedddings (stem) composed of a single aggressive convolution. configcs0tt|j|jdd|jd|_|j|_dS)Nrr)rrr)r r!r num_channelsembedding_size hidden_actembedderr'r7r(r*r+r!Zs  zRegNetEmbeddings.__init__cCs*|jd}||jkrtd||}|S)NrzeMake sure that the channel dimension of the pixel values match with the one set in the configuration.)shaper8 ValueErrorr;)r' pixel_valuesr8r.r*r*r+r/as   zRegNetEmbeddings.forward)r0r1r2__doc__rr!r/r5r*r*r(r+r6Usr6csBeZdZdZd dededeffdd Zded efd d ZZS) RegNetShortCutz RegNet shortcut, used to project the residual features to the correct size. If needed, it is also used to downsample the input using `stride=2`. rrrrcs0ttj||d|dd|_t||_dS)NrF)rrr)r r!rr"r#r$r%)r'rrrr(r*r+r!rs zRegNetShortCut.__init__inputreturncCs||}||}|Sr,)r#r%)r'rBr.r*r*r+r/ws  zRegNetShortCut.forward)r) r0r1r2r@r3r!rr/r5r*r*r(r+rAlsrAcs2eZdZdZdedeffdd ZddZZS) RegNetSELayerz| Squeeze and Excitation layer (SE) proposed in [Squeeze-and-Excitation Networks](https://arxiv.org/abs/1709.01507). rreduced_channelsc sLttd|_ttj||ddttj||ddt|_ dS)Nrrr)r) r r!rAdaptiveAvgPool2dpooler Sequentialr"ReLUSigmoid attention)r'rrEr(r*r+r!s   zRegNetSELayer.__init__cCs ||}||}||}|Sr,)rHrL)r'r.pooledrLr*r*r+r/s  zRegNetSELayer.forward)r0r1r2r@r3r!r/r5r*r*r(r+rD}s rDc <eZdZdZd dedededeffdd Zd d ZZS) RegNetXLayerzt RegNet's layer composed by three `3x3` convolutions, same as a ResNet bottleneck layer with reduction = 1. rr7rrrc st||kp |dk}td||j}|rt|||dnt|_tt ||d|j dt |||||j dt ||ddd|_ t |j |_ dS)Nrrrrrrr)r r!max groups_widthrArr&shortcutrIrr:layerr rr'r7rrrshould_apply_shortcutrr(r*r+r!s zRegNetXLayer.__init__cC.|}||}||}||7}||}|Sr,rVrUrr'r.residualr*r*r+r/    zRegNetXLayer.forwardr r0r1r2r@rr3r!r/r5r*r*r(r+rOs rOc rN) RegNetYLayerzC RegNet's Y layer: an X layer with Squeeze and Excitation. rr7rrrc st||kp |dk}td||j}|rt|||dnt|_tt ||d|j dt |||||j dt |t t |ddt ||ddd|_t|j |_dS)NrrPrQrR)rE)r r!rSrTrArr&rUrIrr:rDr3roundrVr rrWr(r*r+r!s zRegNetYLayer.__init__cCrYr,rZr[r*r*r+r/r]zRegNetYLayer.forwardr^r_r*r*r(r+r`s r`c sDeZdZdZ  d dededededef fdd Zd d ZZS) RegNetStagez4 A RegNet stage composed by stacked layers. rr7rrrdepthcsZtjdkr tnttj||dgfddt|dDR|_dS)NxrPcsg|]}qSr*r*).0_r7rVrr*r+ sz(RegNetStage.__init__..r) r r! layer_typerOr`rrIrangelayers)r'r7rrrrdr(rhr+r!s zRegNetStage.__init__cCs||}|Sr,)rlr-r*r*r+r/s zRegNetStage.forward)rrr_r*r*r(r+rcs  rcc s@eZdZdeffdd Z d dededed efd d ZZ S) RegNetEncoderr7c sttg|_|jt||j|jd|j rdnd|j ddt |j|jdd}t ||j ddD]\\}}}|jt||||dq9dS)Nrrr)rrd)rd) r r!r ModuleListstagesappendrcr9 hidden_sizesdownsample_in_first_stagedepthszip)r'r7in_out_channelsrrrdr(r*r+r!s    zRegNetEncoder.__init__FTr.output_hidden_states return_dictrCcCsb|rdnd}|jD] }|r||f}||}q |r||f}|s+tdd||fDSt||dS)Nr*css|] }|dur|VqdSr,r*)rfvr*r*r+ sz(RegNetEncoder.forward..)last_hidden_state hidden_states)rotupler )r'r.rvrwr{ stage_moduler*r*r+r/s      zRegNetEncoder.forward)FT) r0r1r2rr!rboolr r/r5r*r*r(r+rmsrmc@s$eZdZdZeZdZdZddZdS)RegNetPreTrainedModelz An abstract class to handle weights initialization and a simple interface for downloading and loading pretrained models. regnetr?cCs^t|tjrtjj|jddddSt|tjtjfr-tj|jdtj|j ddSdS)Nfan_outr)mode nonlinearityrr) isinstancerr"initkaiming_normal_weightr$ GroupNorm constant_r)r'moduler*r*r+ _init_weights s z#RegNetPreTrainedModel._init_weightsN) r0r1r2r@r config_classbase_model_prefixmain_input_namerr*r*r*r+rs  raH This model is a PyTorch [torch.nn.Module](https://pytorch.org/docs/stable/nn.html#torch.nn.Module) subclass. Use it as a regular PyTorch Module and refer to the PyTorch documentation for all matter related to general usage and behavior. Parameters: config ([`RegNetConfig`]): Model configuration class with all the parameters of the model. Initializing with a config file does not load the weights associated with the model, only the configuration. Check out the [`~PreTrainedModel.from_pretrained`] method to load the model weights. aK Args: pixel_values (`torch.FloatTensor` of shape `(batch_size, num_channels, height, width)`): Pixel values. Pixel values can be obtained using [`AutoImageProcessor`]. See [`ConvNextImageProcessor.__call__`] for details. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~file_utils.ModelOutput`] instead of a plain tuple. zOThe bare RegNet model outputting raw features without any specific head on top.c s\eZdZfddZeeeeee de d d de de e de e d efd d ZZS) RegNetModelcs>t|||_t||_t||_td|_ | dS)NrF) r r!r7r6r;rmencoderrrGrH post_initr<r(r*r+r!Gs     zRegNetModel.__init__vision) checkpoint output_typermodalityexpected_outputNr?rvrwrCcCs||dur|n|jj}|dur|n|jj}||}|j|||d}|d}||}|s6||f|ddSt|||jdS)Nrvrwrr)rz pooler_outputr{)r7rvuse_return_dictr;rrHrr{)r'r?rvrwembedding_outputencoder_outputsrz pooled_outputr*r*r+r/Ps    zRegNetModel.forward)NN)r0r1r2r!r REGNET_INPUTS_DOCSTRINGr _CHECKPOINT_FOR_DOCr_CONFIG_FOR_DOC_EXPECTED_OUTPUT_SHAPErrr~r/r5r*r*r(r+rAs(  rz RegNet Model with an image classification head on top (a linear layer on top of the pooled features), e.g. for ImageNet. c speZdZfddZeeeeee e d    d de e j de e jde ede ed ef d d ZZS) RegNetForImageClassificationcs^t||j|_t||_tt|jdkr#t|j d|jnt |_ | dS)Nr) r r! num_labelsrrrrIFlattenLinearrqr& classifierrr<r(r*r+r!}s  $ z%RegNetForImageClassification.__init__)rrrrNr?labelsrvrwrCc Csb|dur|n|jj}|j|||d}|r|jn|d}||}d}|dur|jjdurP|jdkr6d|j_n|jdkrL|jtj ksG|jtj krLd|j_nd|j_|jjdkrnt } |jdkrh| | | }n+| ||}n%|jjdkrt } | |d|j|d}n|jjdkrt} | ||}|s|f|dd} |dur|f| S| St|||jd S) a0 labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the image classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). Nrr regressionsingle_label_classificationmulti_label_classificationrr)losslogitsr{)r7rrrr problem_typerdtypetorchlongr3rsqueezerviewrrr{) r'r?rrvrwoutputsrrrloss_fctoutputr*r*r+r/s6    "       z$RegNetForImageClassification.forward)NNNN)r0r1r2r!r rr _IMAGE_CLASS_CHECKPOINTrr_IMAGE_CLASS_EXPECTED_OUTPUTrr FloatTensor LongTensorr~r/r5r*r*r(r+rts0  r)2r@typingrrtorch.utils.checkpointrrtorch.nnrrr activationsr file_utilsr r r modeling_outputsr rrmodeling_utilsrutilsrconfiguration_regnetr get_loggerr0loggerrrrrr$REGNET_PRETRAINED_MODEL_ARCHIVE_LISTModulerr6rArDrOr`rcrmrREGNET_START_DOCSTRINGrrrr*r*r*r+sR      !& .