o h_ @sdZddlZddlZddlZddlZddlZddlZddlZddlm Z m Z m Z m Z m Z mZmZddlZddlmZddlmZddlmZddlmZddlmZmZddlmZmZmZdd lm Z m!Z!m"Z"m#Z#dd l$m%Z%m&Z&m'Z'm(Z(m)Z)dd l*m+Z+m,Z,m-Z-dd l.m/Z/dd l0m1Z1m2Z2m3Z3ddl4m5Z5m6Z6ddl7m8Z8ddl9m:Z:ea;eejeejejstebeXe\rvqWeWeXe\e]qWd?d@ZchdAZddBdCZeecejQj/D]'\Z\Zfe\ad>se\gdDrqe\eYjZvre\edvreWeYefjheee\qnGdEd6d6ZTGdFd:d:ejQj/ZXGdGd=d=eXZYdHdIZidJdKZjdLdMZkdqdNdOZl    drdPee ee e e efdffdQdRZmdSdTZndUdVZodWdXZpdYdZZqd[d\Zrd]d^Zsejz+OrderedDictWrapper.keys..itemsrDr,r,r-keyszOrderedDictWrapper.keyscCrF)NcSg|]\}}|qSr,r,rGr,r,r-rKrLz-OrderedDictWrapper.values..rMrOr,r,r-valuesrQzOrderedDictWrapper.valuescCs t|Sr3)lenrSrOr,r,r-__len__r:zOrderedDictWrapper.__len__cCstd)Nz6cannot delete methods or parameters of a script module RuntimeErrorrDrIr,r,r- __delitem__szOrderedDictWrapper.__delitem__cC |jSr3)rCrNrOr,r,r-rNr/zOrderedDictWrapper.itemscCs(||vr td||j||dS)NzICan't add a new parameter after ScriptModule construction. Tried to add ')rWrCsetattrrDrIrJr,r,r- __setitem__s zOrderedDictWrapper.__setitem__cCs |j|Sr3)rCcontainsrXr,r,r- __contains__r:zOrderedDictWrapper.__contains__cCs||vrt||j|Sr3)KeyErrorrCr7rXr,r,r- __getitem__s zOrderedDictWrapper.__getitem__N) __name__ __module__ __qualname__rErPrSrUrYrNr]r_rar,r,r,r-rAs rAcs<eZdZfddZddZddZddZd d ZZS) OrderedModuleDictcsttj|||_dSr3)superrEtorch_C ModuleDict_python_modules)rDmodule python_dictr;r,r-rEs zOrderedModuleDict.__init__cCs|j}|Sr3)rjrNrDrr,r,r-rNs zOrderedModuleDict.itemscCs ||jvSr3rjrXr,r,r-r_r/zOrderedModuleDict.__contains__cCs:t|tr|j||||j|<dStd|d|)NzgCannot re-assign modules in a ScriptModule with non-scripted module, tried to replace existing module 'z': ) isinstance ScriptModulerCr[rjrWr\r,r,r-r]s zOrderedModuleDict.__setitem__cC |j|Sr3rprXr,r,r-rar/zOrderedModuleDict.__getitem__) rbrcrdrErNr_r]ra __classcell__r,r,rmr-res   recseZdZfddZZS) ScriptMetac si_ttdd_t|D]"}t|diD] \}}|j|<qt|dt}j|_qt|D]\}}t|t rNt ||j|j j <q9tddr]t |||Stddd tfd d }|_t |||dS) N __constants__r,_methods_constants_set_disable_script_metaFrEcSdSr3r,rOr,r,r-*sz%ScriptMeta.__init__..c stj}|g|Ri|tj|k}t|krWdd}tjjj||| d|jd<|jj }| D]}t ||q7| D] \}}t ||qCdD] }t ||qOdSdS)NcSs2t|}t|drddt|jDSt|S)NrwcSrRr,r,rGr,r,r-rK7rLzUScriptMeta.__init__..init_then_script..make_stubs..)r2r>sortedrwrNr)rkr+r,r,r- make_stubs4s zAScriptMeta.__init__..init_then_script..make_stubs) share_types_actual_script_module) _parameters_buffers_modules) rTrwr2rgjit _recursivecreate_script_moduler<r_concrete_typeget_attributesdelattr get_modules) rDargskwargs num_methodsadded_methods_in_initr} concrete_typer8_r+ original_initr,r-init_then_script,s(       z-ScriptMeta.__init__..init_then_script)rwsetr7rxreversedrNunionr|rqrroriginal_methodrbrfrE functoolswraps) r+r8basesattrsbaserIrJbase_constantsrrmrr-rEs&     zScriptMeta.__init__rbrcrdrErtr,r,rmr-rusruc@seZdZddZdS)_CachedForwardcC |dS)Nforward) __getattr__)rDobjr+r,r,r-__get__Qr/z_CachedForward.__get__N)rbrcrdrr,r,r,r-rPs rc@ eZdZdS) ScriptWarningNrbrcrdr,r,r,r-rUrcCs0ts|Stjdd}t||jdd}t|||S)N frames_uprr) self_name)r _jit_internal!createResolutionCallbackFromFramerrbr)fn_rcbastr,r,r- script_methodYs   rc@seZdZddZddZdS)ConstMapcCrBr3 const_mapping)rDrr,r,r-rEnr/zConstMap.__init__cCrsr3rrDattrr,r,r-rqr/zConstMap.__getattr__N)rbrcrdrErr,r,r,r-rms rimporterscript_module_idreturncCsHt|jtjjs tdtj}tj||j|jt |j |}t |S)z Called by ``torch.package.PackageImporter``'s Pickler's ``persistent_load`` function. Performs work of loading and returning a ScriptModule from a ``torch.package`` archive. z{Loading ScriptObjects from a PackageImporter created from a directory is not supported. Use a package archive file instead.) rq zip_readerrgrhPyTorchFileReaderrWCompilationUnit_import_ir_module_from_packagestorage_contextr%last_map_locationr)rrcu cpp_moduler,r,r-unpackage_script_moduleus r)__iter__rU__neg____mul__r___add____sub____pow__ __truediv____mod____ne____eq____lt____gt____le____ge____and____or____xor__rar]__call____int__ __float____bool____str__ __enter____exit__csPeZdZdZfddZfddZfddZdd Zd d Zd d Z Z S)RecursiveScriptClassa An analogue of RecursiveScriptModule for regular objects that are not modules. This class is a wrapper around a torch._C.ScriptObject that represents an instance of a TorchScript class and allows it to be used in Python. Attributes: _c [torch._C.ScriptObject]: The C++ object to which attribute lookups and method calls are forwarded. _props [Dict[str, property]]: A dictionary of properties fetched from self._c and exposed on this wrppaer. cs>td|jd<||_dd|jD|_d|jd<dS)NT _initializingcSsi|] }|jt|j|jqSr,)r8propertygettersetter)rHpropr,r,r- sz1RecursiveScriptClass.__init__..F)rfrEr<rC _properties_props)rD cpp_classrmr,r-rEs  zRecursiveScriptClass.__init__csDd|jvr|jdrt|S||jvr|j|St|j|SNr)r<rfrrfgetr7rCrrmr,r-rs    z RecursiveScriptClass.__getattr__csNd|jvr|jdrt||S||jvr|j||St|j||dSr)r<rf __setattr__rfsetr[rCrDrr1rmr,r-rs  z RecursiveScriptClass.__setattr__cOs*|j|s t||}||i|Sr3)rC _has_method TypeErrorrrD method_namerr self_methodr,r,r-forward_magic_methods  z)RecursiveScriptClass.forward_magic_methodcCr&)NzScriptClasses cannot be pickledr'rOr,r,r- __getstate__r/z!RecursiveScriptClass.__getstate__cCs$|jdr |d|S|d|S)N__iadd__r)rCrr)rDotherr,r,r-rs   zRecursiveScriptClass.__iadd__) rbrcrd__doc__rErrrrrrtr,r,rmr-rs  rcOs|jtg|Ri|Sr3)rrrDrrr,r,r-method_templatercsveZdZUdZgdZfddZeZede fe d<fddZ fd d Z d d Z d dZdefddZZS)rrz A wrapper around C++ ``torch::jit::Module``. ``ScriptModule``\s contain methods, attributes, parameters, and constants. These can be accessed the same way as on a normal ``nn.Module``. )codecode_with_constantsgraph inlined_graph original_namectdSr3rfrErOrmr,r-rEScriptModule.__init__.rcs"d|jvr t|St|j|S)Nr)r<rfrr7rrrmr,r-r s   zScriptModule.__getattr__csZd|jvr$t|trd|jjvri|j_|j|j|<|j}t||St |j ||dS)Nr__annotations__) r<rqr0r;rr2r1rfrr[rrrmr,r-rs    zScriptModule.__setattr__cCsJd|jvr |j|Stjdd}tj|}t||d|j | j <dS)Nrr$r) r<rdefinerrrgrh_parse_source_defrrwr8)rDsrcrcbrr,r,r-r%s   zScriptModule.definecCrZr3)r_replicate_for_data_parallelrOr,r,r-r;r/z)ScriptModule._replicate_for_data_parallelexportercCs&|}|j|jt|t|ffS)a} Called by ``torch.package.PackageExporter``'s Pickler's ``persistent_id`` when saving TorchScript objects. Performs act of saving a ScriptModule inside of a ``torch.package`` archive. Returns method to load the ScriptModule from a ``torch.package.PackageImporter``'s Pickler's ``persistent_load`` function. ) get_unique_idscript_module_serializer serializerCintr)rDrrr,r,r-__reduce_package__>s  zScriptModule.__reduce_package__)rbrcrdr__jit_unused_properties__rErrrrrrrrrr!r rtr,r,rmr-rrs    rr) metaclasscs,eZdZdZdZfddZeddZeddZd d Z e d d Z e d dZ e ddZ e ddZddZddZddZddZddZddZdd Ze d!d"Zd#d$Zfd%d&Zfd'd(Zd)d*Zd+d,Zd-d.Zd/d0Zd1d2Zd3d4Zd5d6Z fd7d8Z!d9d:Z"d;d<Z#Z$S)=RecursiveScriptModulea# The core data structure in TorchScript is the ``ScriptModule``. It is an analogue of torch's ``nn.Module`` and represents an entire model as a tree of submodules. Like normal modules, each individual module in a ``ScriptModule`` can have submodules, parameters, and methods. In ``nn.Module``\s methods are implemented as Python functions, but in ``ScriptModule``\s methods are implemented as TorchScript functions, a statically-typed subset of Python that contains all of PyTorch's built-in Tensor operations. This difference allows your ``ScriptModule``\s code to run without the need for a Python interpreter. ``ScriptModule``\s should not be created manually, instead use either :func:`tracing ` or :func:`scripting `. Tracing and scripting can be applied incrementally and :ref:`composed as necessary `. * Tracing records the tensor operations as executed with a set of example inputs and uses these operations to construct a computation graph. You can use the full dynamic behavior of Python with tracing, but values other than Tensors and control flow aren't captured in the graph. * Scripting inspects the Python code of the model and compiles it to TorchScript. Scripting allows the use of many `types`_ of values and supports dynamic control flow. Many, but not all features of Python are supported by the compiler, so changes to the source code may be necessary. Tcs(d|jd<||_tt|ddS)NTrtraining)r<rCrfrEr)rDrrmr,r-rEgs  RecursiveScriptModule.__init__cCst|}||t||S)a Construct a RecursiveScriptModule that's ready for use. PyTorch code should use this to construct a RecursiveScriptModule instead of instead of calling `__init__` directly, as it makes sure the object is properly finalized (and in the future, we may take control of how the RecursiveScriptModule instance is created). Args: cpp_module: The C++ Module that will hold the actual state of this RecursiveScriptModule instance. init_fn: Lambda that initializes the RecursiveScriptModule passed to it. )r_finalize_scriptmodule)rinit_fn script_moduler,r,r- _constructps z RecursiveScriptModule._constructcCsBttj|j|_ttj|j|_t|j|j |_ d|_ dS)NF) rArgrh ParameterDictrCr BufferDictrrerrrr,r,r-rs   z,RecursiveScriptModule._finalize_scriptmodulecCs||tjj|j|_i}tj|j D] \}}t |||<qt |j||_ t tj|j|_t tj|j|_dd|j D|_d|jd<dS)z Re-construct an instance of RecursiveScriptModule using an instance of a C++ module. Args: cpp_module: The C++ module that this RecursiveScriptModule will be rebuilt around. cSs$i|]\}}t|tjjs||qSr,)rqrgrh ScriptMethodrGr,r,r-rs  z6RecursiveScriptModule._reconstruct..FrN)rErgrhConcreteModuleType from_jit_typerC_typerrirNrrerrArrrrr<)rDrmodulesr8r,r,r- _reconstructs z"RecursiveScriptModule._reconstructcCs|jdjS)z Returns a string representation of the internal graph for the ``forward`` method. See :ref:`interpreting-graphs` for details. r)rC _get_methodrrOr,r,r-rszRecursiveScriptModule.graphcC|jjS)z Returns a string representation of the internal graph for the ``forward`` method. This graph will be preprocessed to inline all function and method calls. See :ref:`interpreting-graphs` for details. )rrrOr,r,r-rz#RecursiveScriptModule.inlined_graphcCr)z Returns a pretty-printed representation (as valid Python syntax) of the internal graph for the ``forward`` method. See :ref:`inspecting-code` for details. )rrrOr,r,r-rr zRecursiveScriptModule.codecCs|jj}|dt|dfS)a Returns a tuple of: [0] a pretty-printed representation (as valid Python syntax) of the internal graph for the ``forward`` method. See `code`. [1] a ConstMap following the CONSTANT.cN format of the output in [0]. The indices in the [0] output are keys to the underlying constant's values. See :ref:`inspecting-code` for details. rr$)rrrrnr,r,r-rs z)RecursiveScriptModule.code_with_constantscKs|jjt|fi|S)aO save(f, _extra_files={}) See :func:`torch.jit.save ` witch accepts a file-like object. This function, torch.save(), converts the object to a string, treating it as a path. DO NOT confuse these two functions when it comes to the 'f' parameter functionality. )rCsavestr)rDfrr,r,r-r!szRecursiveScriptModule.savecO|jj|i|S)az _save_for_lite_interpreter(f) Add (or update) the bytecode session to the script model. The updated model is used in lite interpreter for mobile applications. Args: f: a string containing a file name. _extra_files: Map from filename to contents which will be stored as part of 'f'. )rC_save_for_mobilerr,r,r-_save_for_lite_interpreters z0RecursiveScriptModule._save_for_lite_interpretercOr$r3)rC_save_to_buffer_for_mobilerr,r,r-$_save_to_buffer_for_lite_interpreterrQz:RecursiveScriptModule._save_to_buffer_for_lite_interpretercOr$r3)rCsave_to_bufferrr,r,r-r)rQz$RecursiveScriptModule.save_to_buffercOrZr3)rCget_debug_staterr,r,r-r*r/z%RecursiveScriptModule.get_debug_statecCs d|jS)Nzoriginal_name=)rrOr,r,r- extra_reprr:z RecursiveScriptModule.extra_reprcOs|jj|g|Ri|Sr3)r graph_forrr,r,r-r,szRecursiveScriptModule.graph_forcCs0t|t|jkrdSt|jS)N)r2r"rCrr8rOr,r,r-rsz#RecursiveScriptModule.original_namecCs"tjdd}|j|j||dS)Nr$r)rrrC_definer)rDrrr,r,r-r s zRecursiveScriptModule.definecsd|jvr td|jrt|S||jvr|j|S|j|r(|j|S|j |r;|j |}||j|<|St|S)NrzKScriptModule has not been initialized, did you forget to call super's init?) r<rWrrfrrrCr>r7rr)rDrrrmr,r-rs          z!RecursiveScriptModule.__getattr__cs|jr t||S||jvr||j|<dS|j|r%|j||dSt|dr>||j vr>t d|d|dt||S)Nrz+Cannot mutate TorchScript constant value: 'z '. Value: '') rrfrrrCr>r[r get_constantsrPAttributeErrorrrmr,r-r-s   z!RecursiveScriptModule.__setattr__cCstjjt|jSr3)rgrrrcopyrCrOr,r,r-__copy__HszRecursiveScriptModule.__copy__cCstjjt|j|Sr3)rgrrrr2deepcopyrC)rDmemor,r,r- __deepcopy__Krz"RecursiveScriptModule.__deepcopy__cOs4t||}t|ddtt|krt||i|S)N__func__)r7rNotImplementedErrorrr,r,r-rRs  z*RecursiveScriptModule.forward_magic_methodcCr)NrrrOr,r,r-rZr/zRecursiveScriptModule.__iter__cC |d|S)Nrar9)rDidxr,r,r-ra]r:z!RecursiveScriptModule.__getitem__cCr)NrUr9rOr,r,r-rU`r/zRecursiveScriptModule.__len__cCr:)Nr_r9)rDkeyr,r,r-r_cr:z"RecursiveScriptModule.__contains__cs&|j}|jttdkrtS|S)N__dir__)r=r7r9rrfrDrrmr,r-r=hs  zRecursiveScriptModule.__dir__cCs |j}|jttdkr dS|S)NrT)rr7r9rr>r,r,r-rts zRecursiveScriptModule.__bool__cCsdd}t|j|S)NcSrzr3r,rr,r,r-rr6zCRecursiveScriptModule._replicate_for_data_parallel..init_fn)rrrCr)rDrr,r,r-r}s z2RecursiveScriptModule._replicate_for_data_parallel)%rbrcrdrryrE staticmethodrrrrrrrrr!r&r(r)r*r+r,rrrrr3r6rrrarUr_r=rrrtr,r,rmr-rKsN              r__csddlj|fdddS)Nrcs|p |Sr3) isfunctionismethodxinspectr,r-r{rLz_get_methods..) predicate)rF getmembersr*r,rEr- _get_methodss rI>%tocpucudaevalhalfr2applyfloattrain_applydoublebuffersrrchildren _get_name zero_grad add_moduler+ parameters state_dict share_memory _slow_forward _tracing_name named_buffers named_modules_named_membersnamed_childrenget_extra_stateload_state_dictregister_bufferregister_moduleset_extra_statenamed_parametersregister_parameter_save_to_state_dict_load_from_state_dictcsfdd}|S)Ncs td)Nz" is not supported on ScriptModulesrVrr8r,r-failr:z_make_fail..failr,)r8rlr,rkr- _make_fails rm _call_implc@r)rNrr,r,r,r-rrceZdZdfdd ZZS)rrNcrr3rrDargrmr,r-rErrr3rr,r,rmr-rrcro)rNcrr3rrprmr,r-rErrr3rr,r,rmr-rrrcCst|tjjs |St|}||vr|t|St|dr |n|}|||<i}|jD]5\}}|dkrJ|D] \}}t ||||<q9|||<q-t|tjjr^t|t s^t ||||<q-|||<q-|D] \}}||j|<qg|S)N__prepare_scriptable__r) rqrgnnridr>rsr<rN!call_prepare_scriptable_func_implrr)rr5obj_id new_obj_dictr8 sub_modulerIrJr,r,r-rvs*    rvcCsi}t||Sr3)rv)rr5r,r,r-call_prepare_scriptable_func s rzcC tj|S)a Create a ``torch._C.ScriptDict`` instance with the data from ``obj``. Args: obj (dict): The Python dictionary that is used to initialize the ``ScriptDict`` returned by this function. Returns: An instance of ``torch._C.ScriptDict`` that has the same data as ``obj`` and can be passed between Python and TorchScript with reference semantics and zero copy overhead. )rgrh ScriptDict)rr,r,r-create_script_dicts r}cCr{)a Create a ``torch._C.ScriptList`` instance with the data from ``obj``. Args: obj (dict): The Python list that is used to initialize the ``ScriptList`` returned by this function. Returns: An instance of ``torch._C.ScriptList`` that has the same data as ``obj`` and can be passed between Python and TorchScript with reference semantics and zero copy overhead. )rgrh ScriptList)r type_hintr,r,r-create_script_lists rexample_inputscCsts|S|dur tdt|tr|St|tr|St|tr"|S|rrtat rmt t}t |3t|t rJ| D] \}}|D]}||qAq;nt|t rY|D]} || qQntdWdn1sgwYntdt|tjjrt|}tjj|tjjjSt|dr|n|}t|trt|St|trt|St|rt|} t |tjjrt!d|dt |t"j#r|St$|st!dt%|&d krt!d |durt'(|d }t)||| |St*|st+|rLt|} t|d r |j,}t'-|}t|d rt!d|j.t/|t0|} | r#| St1||j2} |dur3t'-|}tj34| | |t5|} |j6| _6|| _7t8|| | Stjj9|S)a Scripting a function or ``nn.Module`` will inspect the source code, compile it as TorchScript code using the TorchScript compiler, and return a :class:`ScriptModule` or :class:`ScriptFunction`. TorchScript itself is a subset of the Python language, so not all features in Python work, but we provide enough functionality to compute on tensors and do control-dependent operations. For a complete guide, see the :ref:`language-reference`. Scripting a dictionary or list copies the data inside it into a TorchScript instance than can be subsequently passed by reference between Python and TorchScript with zero copy overhead. ``torch.jit.script`` can be used as a function for modules, functions, dictionaries and lists and as a decorator ``@torch.jit.script`` for :ref:`torchscript-classes` and functions. Args: obj (Callable, class, or nn.Module): The ``nn.Module``, function, class type, dictionary, or list to compile. example_inputs (Union[List[Tuple], Dict[Callable, List[Tuple]], None]): Provide example inputs to annotate the arguments for a function or ``nn.Module``. Returns: If ``obj`` is ``nn.Module``, ``script`` returns a :class:`ScriptModule` object. The returned :class:`ScriptModule` will have the same set of sub-modules and parameters as the original ``nn.Module``. If ``obj`` is a standalone function, a :class:`ScriptFunction` will be returned. If ``obj`` is a ``dict``, then ``script`` returns an instance of `torch._C.ScriptDict`. If ``obj`` is a ``list``, then ``script`` returns an instance of `torch._C.ScriptList`. **Scripting a function** The ``@torch.jit.script`` decorator will construct a :class:`ScriptFunction` by compiling the body of the function. Example (scripting a function): .. testcode:: import torch @torch.jit.script def foo(x, y): if x.max() > y.max(): r = x else: r = y return r print(type(foo)) # torch.jit.ScriptFunction # See the compiled graph as Python code print(foo.code) # Call the function using the TorchScript interpreter foo(torch.ones(2, 2), torch.ones(2, 2)) .. testoutput:: :hide: ... ****Scripting a function using example_inputs** Example inputs can be used to annotate a function arguments. Example (annotating a function before scripting): .. testcode:: import torch def test_sum(a, b): return a + b # Annotate the arguments to be int scripted_fn = torch.jit.script(test_sum, example_inputs=[(3, 4)]) print(type(scripted_fn)) # torch.jit.ScriptFunction # See the compiled graph as Python code print(scripted_fn.code) # Call the function using the TorchScript interpreter scripted_fn(20, 100) .. testoutput:: :hide: ... **Scripting an nn.Module** Scripting an ``nn.Module`` by default will compile the ``forward`` method and recursively compile any methods, submodules, and functions called by ``forward``. If a ``nn.Module`` only uses features supported in TorchScript, no changes to the original module code should be necessary. ``script`` will construct :class:`ScriptModule` that has copies of the attributes, parameters, and methods of the original module. Example (scripting a simple module with a Parameter): .. testcode:: import torch class MyModule(torch.nn.Module): def __init__(self, N, M): super().__init__() # This parameter will be copied to the new ScriptModule self.weight = torch.nn.Parameter(torch.rand(N, M)) # When this submodule is used, it will be compiled self.linear = torch.nn.Linear(N, M) def forward(self, input): output = self.weight.mv(input) # This calls the `forward` method of the `nn.Linear` module, which will # cause the `self.linear` submodule to be compiled to a `ScriptModule` here output = self.linear(output) return output scripted_module = torch.jit.script(MyModule(2, 3)) Example (scripting a module with traced submodules): .. testcode:: import torch import torch.nn as nn import torch.nn.functional as F class MyModule(nn.Module): def __init__(self): super().__init__() # torch.jit.trace produces a ScriptModule's conv1 and conv2 self.conv1 = torch.jit.trace(nn.Conv2d(1, 20, 5), torch.rand(1, 1, 16, 16)) self.conv2 = torch.jit.trace(nn.Conv2d(20, 20, 5), torch.rand(1, 20, 16, 16)) def forward(self, input): input = F.relu(self.conv1(input)) input = F.relu(self.conv2(input)) return input scripted_module = torch.jit.script(MyModule()) To compile a method other than ``forward`` (and recursively compile anything it calls), add the :func:`@torch.jit.export ` decorator to the method. To opt out of compilation use :func:`@torch.jit.ignore ` or :func:`@torch.jit.unused `. Example (an exported and ignored method in a module):: import torch import torch.nn as nn class MyModule(nn.Module): def __init__(self): super().__init__() @torch.jit.export def some_entry_point(self, input): return input + 10 @torch.jit.ignore def python_only_fn(self, input): # This function won't be compiled, so any # Python APIs can be used import pdb pdb.set_trace() def forward(self, input): if self.training: self.python_only_fn(input) return input * 99 scripted_module = torch.jit.script(MyModule()) print(scripted_module.some_entry_point(torch.randn(2, 2))) print(scripted_module(torch.randn(2, 2))) Example ( Annotating forward of nn.Module using example_inputs):: import torch import torch.nn as nn from typing import NamedTuple class MyModule(NamedTuple): result: List[int] class TestNNModule(torch.nn.Module): def forward(self, a) -> MyModule: result = MyModule(result=a) return result pdt_model = TestNNModule() # Runs the pdt_model in eager model with the inputs provided and annotates the arguments of forward scripted_model = torch.jit.script(pdt_model, example_inputs={pdt_model: [([10, 20, ], ), ], }) # Run the scripted_model with actual inputs print(scripted_model([20])) Nz]`optimize` is deprecated and has no effect. Use `with torch.jit.optimized_execution() insteadzError: Unable to infer types. Please format the inputs to type `List[Tuple]` or `Dict[Callable, List[Tuple]]` to be run with MonkeyType.zWarning: monkeytype is not installed. Please install https://github.com/Instagram/MonkeyType to enable Profile-Directed Typing in TorchScript. Refer to https://github.com/Instagram/MonkeyType/blob/master/README.rst to install MonkeyType. rszType 'zO' cannot be compiled since it inherits from nn.Module, pass an instance insteadzLTorchScript classes must be new-style classes. Please inherit from 'object'.rz\TorchScript classes does not support inheritance yet. Please directly inherit from 'object'.r$__script_if_tracing_wrapper__script_unsupportedzTorchScript error: ):rwarningswarnrqrrrScriptFunctionrr4rrrrNr ValueErrorrgrtrrzrrrrr>rsdictr}listrrFisclassr issubclassrWenumEnumr@rTmrorrrrArB __original_fn#createResolutionCallbackFromClosurer"_check_directly_compile_overloadedrrrbrh_jit_script_compilerr_torchdynamo_inlinercreate_script_class)roptimize _frames_uprrmonkeytype_configrk example_inputexampleexamplesqualified_namemaybe_already_compiled_fnrrr,r,r-script,sN                       rcCs@|D]\}}||vs|||krtjj|d|qdS)NzDefault parameters on overloads do not affect the runtime so they must equal to the default parameter on the implementation function. Found on parameter )rNrgrfrontend FrontendError) impl_defaultsoverload_defaultslocr8overload_valuer,r,r-_check_overload_defaultsvsrc Cszt||j}tjj|ddt|}t||j}t |}t |}t |}t ||| tj||||||} | Sr3)rrbdeclrgr annotations get_signaturerFrBrrrrrangerh_jit_script_compile_overload) overload_fn qual_nameimpl_fn overload_decloverload_signatureimpl_astrimplementation_defaultsrrr,r,r-_compile_function_with_overloads(   rcCst|}t|}t|}|dur|S||vrttd|g}|D] }|t|||q#|r5||}t||t ||S)Nfunction) rr r_get_fn_overloadsrW,get_overload_no_implementation_error_messageappendrr_clear_fn_overloads)rexisting_compiled_fnsruncompiled_overloads compiled_fnsrr,r,r-_get_overloadss&     rcCs.t|}t|s t|rtd|ddS)Nz Function z cannot be directly compiled because it is overloaded. It must be used in a context of a function where its inputs can determine which overload to call.)r rrrrW)rrr,r,r-rs  rcCst|s tdt|stdt|tjjot| dk}|s.t| dkr.tdt |}t d}t ||j}tj||||}||_|S)Nz$interface must be applied to a classz1TorchScript interfaces must inherit from 'object'rzmTorchScript interface does not support inheritance yet. Please directly inherit from 'object' or 'nn.Module'.r$)rFrrWr@rrgrtrrTrr rrrrbrh_jit_script_interface_compile__torch_script_interface__)ris_module_interfacerrrmangled_classnamer,r,r- interfaces"   rcCs,t|}tj||}t|}t|||Sr3)r rgrh CallStackr'createResolutionCallbackForClassMethodsr)rr _qual_name error_stackrr,r,r-_recursive_compile_classs  r spaddingoffsetcharcs<|t|kr |t|8}dfddt||D|S)Nr-csg|]}qSr,r,)rHrrr,r-rKszpad..)rTjoinr)rrrrr,rr-pads  $rc@s>eZdZddededefddZded efd d Zd d ZdS)_ScriptProfileColumnrheader alignmentrcCs||_||_||_i|_dSr3)rrrrows)rDrrrr,r,r-rEs z_ScriptProfileColumn.__init__linenor1cCs||j|<dSr3)r)rDrr1r,r,r-add_rowrz_ScriptProfileColumn.add_rowcstj}g}jD]\}}t|}|||ftt||}q jdkr5|jj8ndfdd|D}tjj |fS)Nrcs"g|] \}}|t|jfqSr,)rr)rHr<cellrrDr,r-rK"z4_ScriptProfileColumn.materialize..) rTrrrNr"rmaxrrr)rD max_lengthrr<r1rr,rr- materializes   z _ScriptProfileColumn.materializeN)rr) rbrcrdr"r rErrrr,r,r,r-rs rc@s.eZdZdeedeefddZddZdS)_ScriptProfileTablecols source_rangecCs||_||_dSr3)rr)rDrrr,r,r-rE s z_ScriptProfileTable.__init__c Csg}g}d}|jD]}|\}}||7}||t|fq |||tdt|dd|jD]'}d}|D]\}}||} | durP|tdt|7}q9|| 7}q9||q3d|S)Nr-r= ) rrrrrrTrgetr) rDoutputscells header_buffercolrrline row_bufferrr,r,r- dump_strings$         z_ScriptProfileTable.dump_stringN)rbrcrdrrr rErr,r,r,r-r s rc@s:eZdZddZddZddZdefdd Zd d Zd S) _ScriptProfilecCstj|_dSr3)r profilingrprofilerOr,r,r-rE)z_ScriptProfile.__init__cC|jdSr3)renablerOr,r,r-r,rz_ScriptProfile.enablecCrr3)rdisablerOr,r,r-r/rz_ScriptProfile.disablercs g}|jD]}|}|}tdd|Dfdd|D}|}|t|}t||}t d}t d} t d} t ddd } | } |D]+} | | | | | || || | }|durw| | | | | |qLt|| | | gt|}||qd |S) NcSs"g|] }t|t|dqS)r)rTlstriprHrr,r,r-rK7rz._ScriptProfile.dump_string..csg|]}|dqSr3r,rdedentr,r-rK8szLine #Hitsz Time (ns)z Line Contentsrr$z )r _dump_statssourcetext splitlinesminstarting_linenorTrrline_maprrcount duration_nsrrrrr)rDr source_stats source_ref source_lines start_lineend_linerrhitstime_ns line_contentsstatsrstattabler,rr-r2s6       z_ScriptProfile.dump_stringcCst|dSr3)printrrOr,r,r-dumpPrz_ScriptProfile.dumpN) rbrcrdrErrr"rr r,r,r,r-r(s  rcCs|dusJd|S)NzUnwrapping null optionalr,rCr,r,r-_unwrap_optionalTsr zaten::_unwrap_optionalzaten::is_scriptingzaten::has_torch_functionr3)NrNN)rr)|r collectionsr2rrrFr(rtypingrrrrrrrrgtorch._jit_internalrtorch._classesr r torch.jit._builtinsr torch.jit._fuserr r torch.jit._monkeytype_configrrrtorch.jit._recursiverrrrtorch.jit._staterrrrrtorch.jit.frontendrrrtorch.nnrtorch.overridesrrr torch.packager!r" torch.utilsr#_serializationr%r4rhrr,rr. __reduce__ namedtupler0r5r9r@rArer2rurWarningrrrr"rtr_magic_methodsrrrr[rrrr<rNr8itemcallablerqr startswithr>rI_compiled_methods_allowlistrmmethodendswithrbrvrzr}rrrrrrrrrr rrrrr  is_scriptingr,r,r,r-s$          L#1= > RD(   $  L   ,