o hU @sdZddlmZmZmZddlZddlZddlmZddlm Z m Z m Z ddl m Z ddlmZmZmZmZdd lmZdd lmZmZmZmZmZdd lmZd d lmZee Z!dZ"dZ#gdZ$dZ%dZ&dgZ'd3dej(de)de*dej(fddZ+Gdddej,Z-Gdddej,Z.Gdddej,Z/Gd d!d!ej,Z0Gd"d#d#ej,Z1Gd$d%d%ej,Z2Gd&d'd'eZ3d(Z4d)Z5ed*e4Gd+d,d,e3Z6ed-e4Gd.d/d/e3Z7ed0e4Gd1d2d2e3eZ8dS)4z PyTorch ConvNext model.)OptionalTupleUnionN)nn)BCEWithLogitsLossCrossEntropyLossMSELoss)ACT2FN)BackboneOutputBaseModelOutputWithNoAttention(BaseModelOutputWithPoolingAndNoAttention$ImageClassifierOutputWithNoAttention)PreTrainedModel)add_code_sample_docstringsadd_start_docstrings%add_start_docstrings_to_model_forwardloggingreplace_return_docstrings) BackboneMixin)ConvNextConfigrzfacebook/convnext-tiny-224)rirztabby, tabby catFinput drop_probtrainingreturncCsd|dks|s|Sd|}|jdfd|jd}|tj||j|jd}||||}|S)aF Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks). Comment by Ross Wightman: This is the same as the DropConnect impl I created for EfficientNet, etc networks, however, the original name is misleading as 'Drop Connect' is a different form of dropout in a separate paper... See discussion: https://github.com/tensorflow/tpu/issues/494#issuecomment-532968956 ... I've opted for changing the layer and argument names to 'drop path' rather than mix DropConnect as a layer name and use 'survival rate' as the argument. rrr)r)dtypedevice)shapendimtorchrandrrfloor_div)rrr keep_probr random_tensoroutputr)d/var/www/html/ai/venv/lib/python3.10/site-packages/transformers/models/convnext/modeling_convnext.py drop_path@s r+csTeZdZdZd deeddffdd Zdejdejfdd Z de fd d Z Z S) ConvNextDropPathzXDrop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).Nrrcst||_dSN)super__init__r)selfr __class__r)r*r/Xs  zConvNextDropPath.__init__ hidden_statescCst||j|jSr-)r+rrr0r3r)r)r*forward\szConvNextDropPath.forwardcCs d|jS)Nzp={})formatr)r0r)r)r* extra_repr_s zConvNextDropPath.extra_reprr-) __name__ __module__ __qualname____doc__rfloatr/r"Tensorr5strr7 __classcell__r)r)r1r*r,Us r,cs8eZdZdZd fdd Zdejdejfdd ZZS) ConvNextLayerNormaALayerNorm that supports two data formats: channels_last (default) or channels_first. The ordering of the dimensions in the inputs. channels_last corresponds to inputs with shape (batch_size, height, width, channels) while channels_first corresponds to inputs with shape (batch_size, channels, height, width). ư> channels_lastcs`ttt||_tt||_||_ ||_ |j dvr*t d|j |f|_ dS)N)rBchannels_firstzUnsupported data format: ) r.r/r Parameterr"onesweightzerosbiaseps data_formatNotImplementedErrornormalized_shape)r0rLrIrJr1r)r*r/is   zConvNextLayerNorm.__init__xrcCs|jdkrtjj||j|j|j|j}|S|jdkr]|j }| }|j ddd}|| dj ddd}||t ||j}|j|d}|jddddf||jddddf}|S)NrBrCrT)keepdim)r)rJr"r functional layer_normrLrFrHrIrr<meanpowsqrtto)r0rM input_dtypeusr)r)r*r5ss   ,zConvNextLayerNorm.forward)rArB) r8r9r:r;r/r"r=r5r?r)r)r1r*r@cs r@cs6eZdZdZfddZdejdejfddZZ S)ConvNextEmbeddingszThis class is comparable to (and inspired by) the SwinEmbeddings class found in src/transformers/models/swin/modeling_swin.py. csLttj|j|jd|j|jd|_t|jdddd|_ |j|_dS)Nr kernel_sizestriderArCrIrJ) r.r/rConv2d num_channels hidden_sizes patch_sizepatch_embeddingsr@ layernormr0configr1r)r*r/s  zConvNextEmbeddings.__init__ pixel_valuesrcCs4|jd}||jkrtd||}||}|S)NrzeMake sure that the channel dimension of the pixel values match with the one set in the configuration.)r r_ ValueErrorrbrc)r0rfr_ embeddingsr)r)r*r5s    zConvNextEmbeddings.forward r8r9r:r;r/r" FloatTensorr=r5r?r)r)r1r*rYs rYcs8eZdZdZd fdd ZdejdejfddZZ S) ConvNextLayera3This corresponds to the `Block` class in the original implementation. There are two equivalent implementations: [DwConv, LayerNorm (channels_first), Conv, GELU,1x1 Conv]; all in (N, C, H, W) (2) [DwConv, Permute to (N, H, W, C), LayerNorm (channels_last), Linear, GELU, Linear]; Permute back The authors used (2) as they find it slightly faster in PyTorch. Args: config ([`ConvNextConfig`]): Model configuration class. dim (`int`): Number of input channels. drop_path (`float`): Stochastic depth rate. Default: 0.0. rcsttj||dd|d|_t|dd|_t|d||_t |j |_ td|||_ |j dkrAtj|j t|dd nd|_|d krOt||_dSt|_dS) Nrr )r[paddinggroupsrArIrT) requires_gradr)r.r/rr^dwconvr@rcLinearpwconv1r hidden_actactpwconv2layer_scale_init_valuerDr"rElayer_scale_parameterr,Identityr+)r0redimr+r1r)r*r/s   $zConvNextLayer.__init__r3rcCs||}||}|dddd}||}||}||}||}|jdur-|j|}|dddd}|||}|S)NrrOr r)rqpermutercrsrurvrxr+)r0r3rrMr)r)r*r5s       zConvNextLayer.forward)rrir)r)r1r*rks rkcs8eZdZdZd fdd Zdejdejfdd ZZ S) ConvNextStageaConvNeXT stage, consisting of an optional downsampling layer + multiple residual blocks. Args: config ([`ConvNextConfig`]): Model configuration class. in_channels (`int`): Number of input channels. out_channels (`int`): Number of output channels. depth (`int`): Number of residual blocks. drop_path_rates(`List[float]`): Stochastic depth rates for each layer. rONc st|ks |dkr!tt|dddtj|||d|_nt|_p,dg|tjfddt|D|_ dS) NrrArCr]rZrcsg|] }t|dqS))rzr+)rk).0jredrop_path_rates out_channelsr)r* sz*ConvNextStage.__init__..) r.r/r Sequentialr@r^downsampling_layerryrangelayers)r0re in_channelsrr[r\depthrr1rr*r/s    zConvNextStage.__init__r3rcCs||}||}|Sr-)rrr4r)r)r*r5s  zConvNextStage.forward)rOrOrONrir)r)r1r*r|s r|c sNeZdZfddZ  d dejdeedeedee e ffd d Z Z S) ConvNextEncoderc stt|_ddtd|jt|j  |j D}|j d}t |j D]$}|j |}t||||dkr9dnd|j |||d}|j||}q(dS)NcSsg|]}|qSr))tolist)r}rMr)r)r*rsz,ConvNextEncoder.__init__..rrOr)rrr\rr)r.r/r ModuleListstagesr"linspacedrop_path_ratesumdepthssplitr`r num_stagesr|append)r0rerprev_chsiout_chsstager1r)r*r/s&     zConvNextEncoder.__init__FTr3output_hidden_states return_dictrcCsj|rdnd}t|jD]\}}|r||f}||}q |r"||f}|s/tdd||fDSt||dS)Nr)css|] }|dur|VqdSr-r))r}vr)r)r* sz*ConvNextEncoder.forward..)last_hidden_stater3) enumeratertupler )r0r3rrall_hidden_statesr layer_moduler)r)r*r5s    zConvNextEncoder.forward)FT) r8r9r:r/r"rjrboolrrr r5r?r)r)r1r*rs  rc@s$eZdZdZeZdZdZddZdS)ConvNextPreTrainedModelz An abstract class to handle weights initialization and a simple interface for downloading and loading pretrained models. convnextrfcCstt|tjtjfr#|jjjd|jjd|j dur!|j j dSdSt|tj r8|j j |jj ddSdS)zInitialize the weightsr)rRstdNg?) isinstancerrrr^rFdatanormal_reinitializer_rangerHzero_ LayerNormfill_)r0moduler)r)r* _init_weightss   z%ConvNextPreTrainedModel._init_weightsN) r8r9r:r;r config_classbase_model_prefixmain_input_namerr)r)r)r*rs  raJ This model is a PyTorch [torch.nn.Module](https://pytorch.org/docs/stable/nn.html#torch.nn.Module) subclass. Use it as a regular PyTorch Module and refer to the PyTorch documentation for all matter related to general usage and behavior. Parameters: config ([`ConvNextConfig`]): Model configuration class with all the parameters of the model. Initializing with a config file does not load the weights associated with the model, only the configuration. Check out the [`~PreTrainedModel.from_pretrained`] method to load the model weights. aF Args: pixel_values (`torch.FloatTensor` of shape `(batch_size, num_channels, height, width)`): Pixel values. Pixel values can be obtained using [`AutoImageProcessor`]. See [`ConvNextImageProcessor.__call__`] for details. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. zQThe bare ConvNext model outputting raw features without any specific head on top.c sjeZdZfddZeeeeee de d   d de j de ede ed eeeffd d ZZS) ConvNextModelcsJt|||_t||_t||_tj|j d|j d|_ | dS)Nrn) r.r/rerYrhrencoderrrr`layer_norm_epsrc post_initrdr1r)r*r/Is    zConvNextModel.__init__vision) checkpoint output_typermodalityexpected_outputNrfrrrcCs|dur|n|jj}|dur|n|jj}|durtd||}|j|||d}|d}||ddg}|sC||f|ddSt|||j dS)Nz You have to specify pixel_valuesrrrrr)r pooler_outputr3) reruse_return_dictrgrhrrcrRr r3)r0rfrrembedding_outputencoder_outputsr pooled_outputr)r)r*r5Vs( zConvNextModel.forward)NNN)r8r9r:r/rCONVNEXT_INPUTS_DOCSTRINGr_CHECKPOINT_FOR_DOCr _CONFIG_FOR_DOC_EXPECTED_OUTPUT_SHAPEr"rjrrrrr5r?r)r)r1r*rDs,    rz ConvNext Model with an image classification head on top (a linear layer on top of the pooled features), e.g. for ImageNet. csteZdZfddZeeeeee e d    d de j de e jde ede ed eeeff d d ZZS) ConvNextForImageClassificationcsRt||j|_t||_|jdkrt|jd|jnt|_ | dS)Nrr) r.r/ num_labelsrrrrrr`ry classifierrrdr1r)r*r/s  $ z'ConvNextForImageClassification.__init__)rrrrNrflabelsrrrc Csb|dur|n|jj}|j|||d}|r|jn|d}||}d}|dur|jjdurP|jdkr6d|j_n|jdkrL|jtj ksG|jtj krLd|j_nd|j_|jjdkrnt } |jdkrh| | | }n+| ||}n%|jjdkrt } | |d|j|d}n|jjdkrt} | ||}|s|f|dd} |dur|f| S| St|||jd S) a labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the image classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). Nrr regressionsingle_label_classificationmulti_label_classificationrrO)losslogitsr3)rerrrr problem_typerrr"longintrsqueezerviewrrr3) r0rfrrroutputsrrrloss_fctr(r)r)r*r5s>    "       z&ConvNextForImageClassification.forward)NNNN)r8r9r:r/rrr_IMAGE_CLASS_CHECKPOINTrr_IMAGE_CLASS_EXPECTED_OUTPUTr"rjr LongTensorrrrr5r?r)r)r1r*rs0  rzQ ConvNeXt backbone, to be used with frameworks like DETR and MaskFormer. c sZeZdZfddZeeeeed  d de j de e de e defd d Z ZS) ConvNextBackbonecst|t|t||_t||_|jdg|j|_i}t |j |j D] \}}t |dd||<q)t ||_|dS)NrrC)rJ)r.r/_init_backbonerYrhrrr` num_featureszip _out_featureschannelsr@r ModuleDicthidden_states_normsr)r0rerrr_r1r)r*r/s      zConvNextBackbone.__init__)rrNrfrrrc Cs|dur|n|jj}|dur|n|jj}||}|j|d|d}|r&|jn|d}d}t|j|D]\}} ||jvrG|j || } || f7}q2|sV|f} |rT| |f7} | St ||r_|ddSdddS)az Returns: Examples: ```python >>> from transformers import AutoImageProcessor, AutoBackbone >>> import torch >>> from PIL import Image >>> import requests >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> processor = AutoImageProcessor.from_pretrained("facebook/convnext-tiny-224") >>> model = AutoBackbone.from_pretrained("facebook/convnext-tiny-224") >>> inputs = processor(image, return_tensors="pt") >>> outputs = model(**inputs) ```NTrrr)) feature_mapsr3 attentions) rerrrhrr3r stage_names out_featuresrr ) r0rfrrrrr3rr hidden_stater(r)r)r*r5s:    zConvNextBackbone.forward)NN)r8r9r:r/rrrr rr"r=rrr5r?r)r)r1r*rs  r)rF)9r;typingrrrr"torch.utils.checkpointrtorch.nnrrr activationsr modeling_outputsr r r rmodeling_utilsrutilsrrrrrutils.backbone_utilsrconfiguration_convnextr get_loggerr8loggerrrrrr&CONVNEXT_PRETRAINED_MODEL_ARCHIVE_LISTr=r<rr+Moduler,r@rYrkr|rrCONVNEXT_START_DOCSTRINGrrrrr)r)r)r*sZ       , / ;L