o h\ @sdZddlmZmZmZddlZddlZddlmZddlm Z m Z m Z ddl m Z ddlmZmZmZmZdd lmZdd lmZmZmZmZmZdd lmZd d lmZee Z!dZ"dZ#gdZ$dZ%dZ&dgZ'd5dej(de)de*dej(fddZ+Gdddej,Z-Gdddej,Z.Gdddej,Z/Gd d!d!ej,Z0Gd"d#d#ej,Z1Gd$d%d%ej,Z2Gd&d'd'ej,Z3Gd(d)d)eZ4d*Z5d+Z6ed,e5Gd-d.d.e4Z7ed/e5Gd0d1d1e4Z8ed2e5Gd3d4d4e4eZ9dS)6z PyTorch ConvNextV2 model.)OptionalTupleUnionN)nn)BCEWithLogitsLossCrossEntropyLossMSELoss)ACT2FN)BackboneOutputBaseModelOutputWithNoAttention(BaseModelOutputWithPoolingAndNoAttention$ImageClassifierOutputWithNoAttention)PreTrainedModel)add_code_sample_docstringsadd_start_docstrings%add_start_docstrings_to_model_forwardloggingreplace_return_docstrings) BackboneMixin)ConvNextV2Configrzfacebook/convnextv2-tiny-1k-224)rirztabby, tabby catFinput drop_probtrainingreturncCsd|dks|s|Sd|}|jdfd|jd}|tj||j|jd}||||}|S)aF Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks). Comment by Ross Wightman: This is the same as the DropConnect impl I created for EfficientNet, etc networks, however, the original name is misleading as 'Drop Connect' is a different form of dropout in a separate paper... See discussion: https://github.com/tensorflow/tpu/issues/494#issuecomment-532968956 ... I've opted for changing the layer and argument names to 'drop path' rather than mix DropConnect as a layer name and use 'survival rate' as the argument. rrr)r)dtypedevice)shapendimtorchrandrrfloor_div)rrr keep_probr random_tensoroutputr)h/var/www/html/ai/venv/lib/python3.10/site-packages/transformers/models/convnextv2/modeling_convnextv2.py drop_path@s r+csTeZdZdZd deeddffdd Zdejdejfdd Z de fd d Z Z S) ConvNextV2DropPathzXDrop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).Nrrcst||_dSN)super__init__r)selfr __class__r)r*r/Xs  zConvNextV2DropPath.__init__ hidden_statescCst||j|jSr-)r+rrr0r3r)r)r*forward\szConvNextV2DropPath.forwardcCs d|jS)Nzp={})formatr)r0r)r)r* extra_repr_s zConvNextV2DropPath.extra_reprr-) __name__ __module__ __qualname____doc__rfloatr/r"Tensorr5strr7 __classcell__r)r)r1r*r,Us r,cs<eZdZdZdeffdd ZdejdejfddZZ S) ConvNextV2GRNz)GRN (Global Response Normalization) layerdimcs>tttddd||_ttddd||_dS)Nr)r.r/r Parameterr"zerosweightbias)r0rAr1r)r*r/fs zConvNextV2GRN.__init__r3rcCsDtj|dddd}||jdddd}|j|||j|}|S)N)rrFT)prAkeepdim)rArHư>)r"normmeanrDrE)r0r3global_features norm_featuresr)r)r*r5kszConvNextV2GRN.forward) r8r9r:r;intr/r" FloatTensorr5r?r)r)r1r*r@csr@cs8eZdZdZd fdd Zdejdejfdd ZZS) ConvNextV2LayerNormaALayerNorm that supports two data formats: channels_last (default) or channels_first. The ordering of the dimensions in the inputs. channels_last corresponds to inputs with shape (batch_size, height, width, channels) while channels_first corresponds to inputs with shape (batch_size, channels, height, width). rJ channels_lastcs`ttt||_tt||_||_ ||_ |j dvr*t d|j |f|_ dS)N)rRchannels_firstzUnsupported data format: ) r.r/rrBr"onesrDrCrEeps data_formatNotImplementedErrornormalized_shape)r0rXrUrVr1r)r*r/{s   zConvNextV2LayerNorm.__init__xrcCs|jdkrtjj||j|j|j|j}|S|jdkr]|j }| }|j ddd}|| dj ddd}||t ||j}|j|d}|jddddf||jddddf}|S)NrRrSrT)rHrF)r)rVr"r functional layer_normrXrDrErUrr<rLpowsqrtto)r0rY input_dtypeusr)r)r*r5s   ,zConvNextV2LayerNorm.forward)rJrR) r8r9r:r;r/r"r=r5r?r)r)r1r*rQus rQcs6eZdZdZfddZdejdejfddZZ S)ConvNextV2EmbeddingszThis class is comparable to (and inspired by) the SwinEmbeddings class found in src/transformers/models/swin/modeling_swin.py. csLttj|j|jd|j|jd|_t|jdddd|_ |j|_dS)Nr kernel_sizestriderJrSrUrV) r.r/rConv2d num_channels hidden_sizes patch_sizepatch_embeddingsrQ layernormr0configr1r)r*r/s  zConvNextV2Embeddings.__init__ pixel_valuesrcCs4|jd}||jkrtd||}||}|S)NrzeMake sure that the channel dimension of the pixel values match with the one set in the configuration.)r rh ValueErrorrkrl)r0rorh embeddingsr)r)r*r5s    zConvNextV2Embeddings.forward r8r9r:r;r/r"rPr=r5r?r)r)r1r*rbs rbcs8eZdZdZd fdd ZdejdejfddZZ S) ConvNextV2Layera5This corresponds to the `Block` class in the original implementation. There are two equivalent implementations: [DwConv, LayerNorm (channels_first), Conv, GELU,1x1 Conv]; all in (N, C, H, W) (2) [DwConv, Permute to (N, H, W, C), LayerNorm (channels_last), Linear, GELU, Linear]; Permute back The authors used (2) as they find it slightly faster in PyTorch. Args: config ([`ConvNextV2Config`]): Model configuration class. dim (`int`): Number of input channels. drop_path (`float`): Stochastic depth rate. Default: 0.0. rcsttj||dd|d|_t|dd|_t|d||_t |j |_ t d||_ td|||_|dkrAt||_dSt|_dS)Nrr )rdpaddinggroupsrJrUr)r.r/rrgdwconvrQrlLinearpwconv1r hidden_actactr@grnpwconv2r,Identityr+)r0rnrAr+r1r)r*r/s  $zConvNextV2Layer.__init__r3rcCsr|}||}|dddd}||}||}||}||}||}|dddd}|||}|S)NrrFr r)rxpermuterlrzr|r}r~r+)r0r3rrYr)r)r*r5s      zConvNextV2Layer.forward)rrrr)r)r1r*rss  rscs8eZdZdZd fdd Zdejdejfdd ZZ S) ConvNextV2StageaConvNeXTV2 stage, consisting of an optional downsampling layer + multiple residual blocks. Args: config ([`ConvNextV2Config`]): Model configuration class. in_channels (`int`): Number of input channels. out_channels (`int`): Number of output channels. depth (`int`): Number of residual blocks. drop_path_rates(`List[float]`): Stochastic depth rates for each layer. rFNc st|ks |dkr!tt|dddtj|||d|_nt|_p,dg|tjfddt|D|_ dS) NrrJrSrfrcrcsg|] }t|dqS))rAr+)rs).0jrndrop_path_rates out_channelsr)r* sz,ConvNextV2Stage.__init__..) r.r/r SequentialrQrgdownsampling_layerrrangelayers)r0rn in_channelsrrdredepthrr1rr*r/s    zConvNextV2Stage.__init__r3rcCs||}||}|Sr-)rrr4r)r)r*r5s  zConvNextV2Stage.forward)rFrFrFNrrr)r)r1r*rs rc sNeZdZfddZ  d dejdeedeedee e ffd d Z Z S) ConvNextV2Encoderc stt|_ddtd|jt|j  |j D}|j d}t |j D]$}|j |}t||||dkr9dnd|j |||d}|j||}q(dS)NcSsg|]}|qSr))tolist)rrYr)r)r*rsz.ConvNextV2Encoder.__init__..rrFr)rrrerr)r.r/r ModuleListstagesr"linspacedrop_path_ratesumdepthssplitrir num_stagesrappend)r0rnrprev_chsiout_chsstager1r)r*r/s&     zConvNextV2Encoder.__init__FTr3output_hidden_states return_dictrcCsj|rdnd}t|jD]\}}|r||f}||}q |r"||f}|s/tdd||fDSt||dS)Nr)css|] }|dur|VqdSr-r))rvr)r)r* sz,ConvNextV2Encoder.forward..)last_hidden_stater3) enumeratertupler )r0r3rrall_hidden_statesr layer_moduler)r)r*r5s    zConvNextV2Encoder.forward)FT) r8r9r:r/r"rPrboolrrr r5r?r)r)r1r*rs  rc@s$eZdZdZeZdZdZddZdS)ConvNextV2PreTrainedModelz An abstract class to handle weights initialization and a simple interface for downloading and loading pretrained models. convnextv2rocCstt|tjtjfr#|jjjd|jjd|j dur!|j j dSdSt|tj r8|j j |jj ddSdS)zInitialize the weightsr)rLstdNg?) isinstancerryrgrDdatanormal_rninitializer_rangerEzero_ LayerNormfill_)r0moduler)r)r* _init_weights3s   z'ConvNextV2PreTrainedModel._init_weightsN) r8r9r:r;r config_classbase_model_prefixmain_input_namerr)r)r)r*r)s  raL This model is a PyTorch [torch.nn.Module](https://pytorch.org/docs/stable/nn.html#torch.nn.Module) subclass. Use it as a regular PyTorch Module and refer to the PyTorch documentation for all matter related to general usage and behavior. Parameters: config ([`ConvNextV2Config`]): Model configuration class with all the parameters of the model. Initializing with a config file does not load the weights associated with the model, only the configuration. Check out the [`~PreTrainedModel.from_pretrained`] method to load the model weights. aI Args: pixel_values (`torch.FloatTensor` of shape `(batch_size, num_channels, height, width)`): Pixel values. Pixel values can be obtained using [`ConvNextImageProcessor`]. See [`ConvNextImageProcessor.__call__`] for details. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. zSThe bare ConvNextV2 model outputting raw features without any specific head on top.c sjeZdZfddZeeeeee de d   d de j de ede ed eeeffd d ZZS) ConvNextV2ModelcsJt|||_t||_t||_tj|j d|j d|_ | dS)NrIrv) r.r/rnrbrqrencoderrrrilayer_norm_epsrl post_initrmr1r)r*r/^s    zConvNextV2Model.__init__vision) checkpoint output_typermodalityexpected_outputNrorrrcCs|dur|n|jj}|dur|n|jj}|durtd||}|j|||d}|d}||ddg}|sC||f|ddSt|||j dS)Nz You have to specify pixel_valuesrrrrIr)r pooler_outputr3) rnruse_return_dictrprqrrlrLr r3)r0rorrembedding_outputencoder_outputsr pooled_outputr)r)r*r5ks( zConvNextV2Model.forward)NNN)r8r9r:r/rCONVNEXTV2_INPUTS_DOCSTRINGr_CHECKPOINT_FOR_DOCr _CONFIG_FOR_DOC_EXPECTED_OUTPUT_SHAPEr"rPrrrrr5r?r)r)r1r*rXs,    rz ConvNextV2 Model with an image classification head on top (a linear layer on top of the pooled features), e.g. for ImageNet. csteZdZfddZeeeeee e d    d de j de e jde ede ed eeeff d d ZZS) ConvNextV2ForImageClassificationcsRt||j|_t||_|jdkrt|jd|jnt|_ | dS)NrrI) r.r/ num_labelsrrrryrir classifierrrmr1r)r*r/s  $ z)ConvNextV2ForImageClassification.__init__)rrrrNrolabelsrrrc Csb|dur|n|jj}|j|||d}|r|jn|d}||}d}|dur|jjdurP|jdkr6d|j_n|jdkrL|jtj ksG|jtj krLd|j_nd|j_|jjdkrnt } |jdkrh| | | }n+| ||}n%|jjdkrt } | |d|j|d}n|jjdkrt} | ||}|s|f|dd} |dur|f| S| St|||jd S) a labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the image classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). Nrr regressionsingle_label_classificationmulti_label_classificationrIrF)losslogitsr3)rnrrrr problem_typerrr"longrOrsqueezerviewrrr3) r0rorrroutputsrrrloss_fctr(r)r)r*r5s>    "       z(ConvNextV2ForImageClassification.forward)NNNN)r8r9r:r/rrr_IMAGE_CLASS_CHECKPOINTrr_IMAGE_CLASS_EXPECTED_OUTPUTr"rPr LongTensorrrrr5r?r)r)r1r*rs0  rzT ConvNeXT V2 backbone, to be used with frameworks like DETR and MaskFormer. c sZeZdZfddZeeeeed  d de j de e de e defd d Z ZS) ConvNextV2Backbonecst|t|t||_t||_|jdg|j|_i}t |j |j D] \}}t |dd||<q)t ||_|dS)NrrS)rV)r.r/_init_backbonerbrqrrri num_featureszip _out_featureschannelsrQr ModuleDicthidden_states_normsr)r0rnrrrhr1r)r*r/s      zConvNextV2Backbone.__init__)rrNrorrrc Cs|dur|n|jj}|dur|n|jj}||}|j|d|d}|r&|jn|d}d}t|j|D]\}} ||jvrG|j || } || f7}q2|sV|f} |rT| |f7} | St ||r_|ddSdddS)a Returns: Examples: ```python >>> from transformers import AutoImageProcessor, AutoBackbone >>> import torch >>> from PIL import Image >>> import requests >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> processor = AutoImageProcessor.from_pretrained("facebook/convnextv2-tiny-1k-224") >>> model = AutoBackbone.from_pretrained("facebook/convnextv2-tiny-1k-224") >>> inputs = processor(image, return_tensors="pt") >>> outputs = model(**inputs) ```NTrrr)) feature_mapsr3 attentions) rnrrrqrr3r stage_names out_featuresrr ) r0rorrrrr3rr hidden_stater(r)r)r*r5s:    zConvNextV2Backbone.forward)NN)r8r9r:r/rrrr rr"r=rrr5r?r)r)r1r*rs  r)rF):r;typingrrrr"torch.utils.checkpointrtorch.nnrrr activationsr modeling_outputsr r r rmodeling_utilsrutilsrrrrrutils.backbone_utilsrconfiguration_convnextv2r get_loggerr8loggerrrrrr(CONVNEXTV2_PRETRAINED_MODEL_ARCHIVE_LISTr=r<rr+Moduler,r@rQrbrsrrrCONVNEXTV2_START_DOCSTRINGrrrrr)r)r)r*s\       ,!0  ;L