o hC@sdZddlmZmZmZmZddlZddlmZddlm Z ddl m Z ddl m Z dd lmZdd lmZmZmZd d lmZd gZdZGdddejZGdddejZGdddejZGdddejZGdddejZGdddeZdZdZ edeGdddeZ!dS) zs PyTorch UperNet model. Based on OpenMMLab's implementation, found in https://github.com/open-mmlab/mmsegmentation.)ListOptionalTupleUnionN)nn)CrossEntropyLoss) AutoBackbone)SemanticSegmenterOutput)PreTrainedModel)add_start_docstrings%add_start_docstrings_to_model_forwardreplace_return_docstrings) UperNetConfigzopenmmlab/upernet-convnext-tinyrcseZdZdZ   ddededeeeeeffdeeeeefefd ed eeeeeffd d ffd d Z de j d e j fddZ Z S)UperNetConvModulez A convolutional block that bundles conv/norm/activation layers. This block simplifies the usage of convolution layers, which are commonly used with a norm layer (e.g., BatchNorm) and activation layer (e.g., ReLU). rFr in_channels out_channels kernel_sizepaddingbiasdilationreturnNcs<ttj||||||d|_t||_t|_dS)N)rrrrrr) super__init__rConv2dconv BatchNorm2d batch_normReLU activation)selfrrrrrr __class__b/var/www/html/ai/venv/lib/python3.10/site-packages/transformers/models/upernet/modeling_upernet.pyr-s  zUperNetConvModule.__init__inputcCs"||}||}||}|SN)rrr )r!r&outputr$r$r%forwardBs   zUperNetConvModule.forward)rFr)__name__ __module__ __qualname____doc__intrrstrboolrtorchTensorr) __classcell__r$r$r"r%r's* rcsDeZdZdedededdffdd Zdejdejfd d ZZS) UperNetPyramidPoolingBlock pool_scalerchannelsrNcsLtt|t||ddg|_t|jD] \}}|t||qdS)Nrr) rrrAdaptiveAvgPool2drlayers enumerate add_moduler/)r!r5rr6ilayerr"r$r%rKs  z#UperNetPyramidPoolingBlock.__init__r&cCs|}|jD]}||}q|Sr')r9)r!r& hidden_stater=r$r$r%r)Ts  z"UperNetPyramidPoolingBlock.forward) r*r+r,r.rr1r2r)r3r$r$r"r%r4Js r4c sXeZdZdZdeedfdedededdf fd d Zd ej de ej fd d Z Z S)UperNetPyramidPoolingModulea} Pyramid Pooling Module (PPM) used in PSPNet. Args: pool_scales (`Tuple[int]`): Pooling scales used in Pooling Pyramid Module. in_channels (`int`): Input channels. channels (`int`): Channels after modules, before conv_seg. align_corners (`bool`): align_corners argument of F.interpolate. pool_scales.rr6 align_cornersrNcsht||_||_||_||_g|_t|D]\}}t|||d}|j || t ||qdS)N)r5rr6) rrr@rArr6blocksr:r4appendr;r/)r!r@rr6rAr<r5blockr"r$r%rjs  z$UperNetPyramidPoolingModule.__init__xcCsHg}|jD]}||}tjj||ddd|jd}||q|S)NbilinearsizemoderA)rBr functional interpolaterIrArC)r!rEppm_outsppmppm_outupsampled_ppm_outr$r$r%r)vs  z#UperNetPyramidPoolingModule.forward) r*r+r,r-rr.r0rr1r2rr)r3r$r$r"r%r?[s*" r?csNeZdZdZfddZddZddZdd Zd ej d ej fd d Z Z S) UperNetHeadz Unified Perceptual Parsing for Scene Understanding. This head is the implementation of [UPerNet](https://arxiv.org/abs/1807.10221). cst||_|j|_||_|j|_d|_tj |j|j dd|_ t |j|jd|j|jd|_ t|jdt|j|j|jddd|_t|_t|_|jddD] }t||jdd}t|j|jddd}|j||j|qVtt|j|j|jddd|_dS)NFrr7)rArrr)rrconfigr@r hidden_sizer6rArr num_labels classifierr? psp_modulesrlen bottleneck ModuleList lateral_convs fpn_convsrCfpn_bottleneck)r!rTrl_convfpn_convr"r$r%rs@     zUperNetHead.__init__cC||jdSr'apply _init_weightsr!r$r$r% init_weightszUperNetHead.init_weightscCDt|tjr|jjjd|jjd|jdur |jj dSdSdSNg)meanstd isinstancerrweightdatanormal_rTinitializer_rangerzero_r!moduler$r$r%rd  zUperNetHead._init_weightscCs:|d}|g}|||tj|dd}||}|S)NrRrdim)extendrXr1catrZ)r!inputsrEpsp_outsr(r$r$r% psp_forwards  zUperNetHead.psp_forwardencoder_hidden_statesrcsfddtjDt}t|dddD]$}|djdd}|dtjj ||dj d|d<q fd dt|dD}|dt|dddD]}tjj |||djdddj d||<qbt j |dd } |}|}|S) Ncsg|] \}}||qSr$r$).0r< lateral_conv)r}r$r% sz'UperNetHead.forward..rrrRrFrGrHcsg|] }j||qSr$)r])r~r<)lateralsr!r$r%rsrv)r:r\rCr|rYrangeshaperrKrLrAr1ryr^rW)r!r}used_backbone_levelsr< prev_shapefpn_outsr(r$)r}rr!r%r)s$   zUperNetHead.forward) r*r+r,r-rrfrdr|r1r2r)r3r$r$r"r%rQs ' rQc sleZdZdZ ddededeeeeeffdd ffd d Zd d ZddZ de j de j fddZ Z S)UperNetFCNHeada Fully Convolution Networks for Semantic Segmentation. This head is the implementation of [FCNNet](https://arxiv.org/abs/1411.4038>). Args: config: Configuration. in_channels (int): Number of input channels. kernel_size (int): The kernel size for convs in the head. Default: 3. dilation (int): The dilation rate for convs in the head. Default: 1. rFrrin_indexrrrNc st||_|j|_|j|_|j|_|j |_ ||_ |d|}g}| t |j|j|||dt|jdD]}| t |j|j|||dq8|jdkrTt|_ntj||_|j rmt |j|j|j||dd|_tj|j|jdd|_dS)NrF)rrrrrrSr7)rrrTauxiliary_in_channelsrauxiliary_channelsr6auxiliary_num_convs num_convsauxiliary_concat_input concat_inputrrCrrrIdentityconvs Sequentialconv_catrrVrW)r!rTrrr conv_paddingrr<r"r$r%rs8     zUperNetFCNHead.__init__cCrar'rbrer$r$r%rfrgzUperNetFCNHead.init_weightscCrhrirlrsr$r$r%rdruzUperNetFCNHead._init_weightsr}cCs@||j}||}|jr|tj||gdd}||}|S)Nrrv)rrrrr1ryrW)r!r} hidden_statesr(r$r$r%r)s   zUperNetFCNHead.forward)rFrr)r*r+r,r-r.rrrrfrdr1r2r)r3r$r$r"r%rs$rc@s(eZdZdZeZdZddZddZdS)UperNetPreTrainedModelz An abstract class to handle weights initialization and a simple interface for downloading and loading pretrained models. pixel_valuescCs>t|tr|j|j|jdur|jdSdSdSr')rmrbackbonerf decode_headauxiliary_headrsr$r$r%rd.s    z$UperNetPreTrainedModel._init_weightscCs0|j|j|jdur|jdSdS)zInitialize the weightsN)rrfrrrer$r$r%rf5s   z#UperNetPreTrainedModel.init_weightsN) r*r+r,r-r config_classmain_input_namerdrfr$r$r$r%r%s  raI Parameters: This model is a PyTorch [torch.nn.Module](https://pytorch.org/docs/stable/nn.html#torch.nn.Module) sub-class. Use it as a regular PyTorch Module and refer to the PyTorch documentation for all matter related to general usage and behavior. config ([`UperNetConfig`]): Model configuration class with all the parameters of the model. Initializing with a config file does not load the weights associated with the model, only the configuration. Check out the [`~PreTrainedModel.from_pretrained`] method to load the model weights. ax Args: pixel_values (`torch.FloatTensor` of shape `(batch_size, num_channels, height, width)`): Pixel values. Padding will be ignored by default should you provide it. Pixel values can be obtained using [`AutoImageProcessor`]. See [`SegformerImageProcessor.__call__`] for details. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers in case the backbone has them. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers of the backbone. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. zMUperNet framework leveraging any vision backbone e.g. for ADE20k, CityScapes.cseZdZfddZeedeee d     dde e j de e de e d e e j d e e d eeeff d d ZZS)UperNetForSemanticSegmentationcsLt|t|j|_t||jjd|_|j rt |nd|_ | dS)N)r) rrr from_configbackbone_configrrQr6ruse_auxiliary_headrr post_init)r!rTr"r$r%r\s  z'UperNetForSemanticSegmentation.__init__zbatch_size, sequence_length) output_typerNroutput_attentionsoutput_hidden_stateslabels return_dictrcCsd|dur|n|jj}|dur|n|jj}|dur|n|jj}|jj|||d}|j}||}tj j ||j ddddd}d} |j durY| |} tj j | |j ddddd} d} |dur|jj dkritdt|jjd } | ||} | dur| | |} | |jj| 7} |s|r|f|dd} n |f|dd} | dur| f| S| St| ||j|jd S) a labels (`torch.LongTensor` of shape `(batch_size, height, width)`, *optional*): Ground truth semantic segmentation maps for computing the loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels > 1`, a classification loss is computed (Cross-Entropy). Returns: Examples: ```python >>> from transformers import AutoImageProcessor, UperNetForSemanticSegmentation >>> from PIL import Image >>> from huggingface_hub import hf_hub_download >>> image_processor = AutoImageProcessor.from_pretrained("openmmlab/upernet-convnext-tiny") >>> model = UperNetForSemanticSegmentation.from_pretrained("openmmlab/upernet-convnext-tiny") >>> filepath = hf_hub_download( ... repo_id="hf-internal-testing/fixtures_ade20k", filename="ADE_val_00000001.jpg", repo_type="dataset" ... ) >>> image = Image.open(filepath).convert("RGB") >>> inputs = image_processor(images=image, return_tensors="pt") >>> outputs = model(**inputs) >>> logits = outputs.logits # shape (batch_size, num_labels, height, width) >>> list(logits.shape) [1, 150, 512, 512] ```N)rrrFrGFrHrz/The number of labels should be greater than one) ignore_index)losslogitsr attentions)rTuse_return_dictrrrforward_with_filtered_kwargs feature_mapsrrrKrLrrrV ValueErrorrloss_ignore_indexauxiliary_loss_weightr rr)r!rrrrroutputsfeaturesrauxiliary_logitsrloss_fctauxiliary_lossr(r$r$r%r)hsH(      z&UperNetForSemanticSegmentation.forward)NNNNN)r*r+r,rr UPERNET_INPUTS_DOCSTRINGformatrr _CONFIG_FOR_DOCrr1r2r0rtupler)r3r$r$r"r%rWs,   r)"r-typingrrrrr1rtorch.nnrr modeling_outputsr modeling_utilsr utilsr r rconfiguration_upernetr%UPERNET_PRETRAINED_MODEL_ARCHIVE_LISTrModulerr4r?rQrrUPERNET_START_DOCSTRINGrrr$r$r$r%s4      #&]G