o h@sddlZddlZddlZddlZddlmZddZGdddeZdd d Z d e d ee e ffd dZ dgddfddZ ddZ ddZdS)N)Tuplec CsFt|tjjs Jt}||j}||jvrt |j}|j ||j |t |j dkrR|j ddkrH|j d|jjdksHJ|j d|j|<dS|dd|djd}t |ddd}t|ddkjttt |djdd}t|dkd}t||g}||j|<dS|jD]}|qdS)Nr) reduction_dimzscore)dim) isinstancetorchnnLinear OutlierTracer get_instance get_hvalueweighthvalue2outlier_idxfind_outlier_dimsoutliersappendhvalueslennumelmaxshapeviewabssumlistrangewherecatuniquehooksremove) moduleinputtracerhvalue outlier_idxmergeddims outlier_idx2hookr.H/var/www/html/ai/venv/lib/python3.10/site-packages/bitsandbytes/utils.py outlier_hooks(     0  r0c@sDeZdZdZddZddZddZdd Zd d Ze d d Z dS)rNcCstd)NzCall get_instance() instead) RuntimeErrorselfr.r.r/__init__+szOutlierTracer.__init__cCs`d|_d|_g|_g|_i|_d|_g|_|D]\}}t|t j j r-|j | tqdS)NT)last_wcurrent_outlier_dimsrrr initializedr# named_modulesr r r r rregister_forward_pre_hookr0)r3modelnmr.r.r/ initialize.szOutlierTracer.initializecCs t|ddS)Nr7F)getattrr2r.r.r/is_initialized;s zOutlierTracer.is_initializedcCs|jSN)datastoragedata_ptr)r3rr.r.r/r>szOutlierTracer.get_hvaluecCs6|s tddS||}||jvr|j|SdS)Nz$Outlier tracer is not initialized...)r?printrr)r3rr(r.r.r/ get_outliersAs   zOutlierTracer.get_outlierscCs|jdur |||_|jSr@) _instance__new__)clsr.r.r/rKs  zOutlierTracer.get_instance) __name__ __module__ __qualname__rFr4r=r?rrE classmethodrr.r.r.r/r(s  r@FcCs|rtjd|jd|f|jdS||}|}|}|||}||} | } | } | | | } |durJtj| |dd\} }|St | |kd}|S)Nrr)sizedevice)kr ) r randintrrOlongmeanstdtopkrr )rrrrUrdmr<mmmstdzmrTstdmstdstdzstdvalidxr.r.r/rQs     rcommand_stringreturncs(ddfdd}||\}}||fS)NcSstdd|DS)Ncss|] }|dVqdS)zUTF-8N)decodestrip).0 to_decoder.r.r/ js  z6execute_and_return.._decode..)tuple)subprocess_err_out_tupler.r.r/_decodeisz#execute_and_return.._decodecs"tjt|tjtjdS)N)stdoutstderr) subprocessPopenshlexsplitPIPE communicate)r_rhr.r/&execute_and_return_decoded_std_streamsoszBexecute_and_return..execute_and_return_decoded_std_streamsr.)r_rrstd_outstd_errr.rqr/execute_and_returnhs  rulm_headc Cs|D]V\}}tt|dkrt|||||t|tjjrZ||vrZ|j |}||j |j |j du|j |<|rH|j |j |_ |j |j |_ |durZt||d}|durZ||q|S)a Replace linear modules with a new Linear module. Parameters: model (`torch.nn.Module`): Input model or `torch.nn.Module` as the function is run recursively. linear_replacement (`torch.nn.Module`): The linear module that replaces the old one. Only expects standard arguments. If other arguments need to be passed, use a lambda. skip_modules (`List[str]`, *optional*, defaults to `lm_head`): List of modules names not to convert. Defaults to `lm_head`. copy_weights (`bool`): Copy the weights from the old linear module to the new one post_processing_fun_name (`str`): A function name of the replacement linear class that is called after processing. rN)named_childrenrrchildrenreplace_linearr r r r _modules in_features out_featuresbiasrr>) r:linear_replacement skip_modules copy_weightspost_processing_functionnamer% old_modulefuncr.r.r/ry}s$   rycCs,t|}|d}tjt|tjd}|S)z Pack a dictionary into a torch tensor for storing quant_state items in state_dict. Parameters: - source_dict: The dictionary to be packed. Returns: A torch tensor containing the packed data. utf-8)dtype)jsondumpsencoder tensorruint8) source_dictjson_str json_bytes tensor_datar.r.r/pack_dict_to_tensors rcCs$t|}|d}t|}|S)z Unpack a torch tensor into a Python dictionary. Parameters: - tensor_data: The torch tensor containing the packed data. Returns: A Python dictionary containing the unpacked data. r)bytesnumpyrarloads)rrr unpacked_dictr.r.r/unpack_tensor_to_dicts  r)rrMNF)rrmrkr typingrr0objectrrstrruryrrr.r.r.r/s ! )&