o
h @ s^ d dl mZmZ ddlmZ ddlmZ e r#ddlmZ ddl m
Z
dZ G d d
d
eZ d S ) )ListUnion )is_torch_available )Pipeline)%MODEL_FOR_TEXT_TO_SPECTROGRAM_MAPPING)SpeechT5HifiGanzmicrosoft/speecht5_hifiganc sn e Zd ZdZddd fdd
Zdd Zdd Zd
eee e f f fd d Z
dd
dZ dd Z Z
S )TextToAudioPipelinea5
Text-to-audio generation pipeline using any `AutoModelForTextToWaveform` or `AutoModelForTextToSpectrogram`. This
pipeline generates an audio file from an input text and optional other conditional inputs.
Example:
```python
>>> from transformers import pipeline
>>> pipe = pipeline(model="suno/bark-small")
>>> output = pipe("Hey it's HuggingFace on the phone!")
>>> audio = output["audio"]
>>> sampling_rate = output["sampling_rate"]
```
Learn more about the basics of using a pipeline in the [pipeline tutorial](../pipeline_tutorial)
You can specify parameters passed to the model by using [`TextToAudioPipeline.__call__.forward_params`] or
[`TextToAudioPipeline.__call__.generate_kwargs`].
Example:
```python
>>> from transformers import pipeline
>>> music_generator = pipeline(task="text-to-audio", model="facebook/musicgen-small", framework="pt")
>>> # diversify the music generation by adding randomness with a high temperature and set a maximum music length
>>> generate_kwargs = {
... "do_sample": True,
... "temperature": 0.7,
... "max_new_tokens": 35,
... }
>>> outputs = music_generator("Techno music with high melodic riffs", generate_kwargs=generate_kwargs)
```
This pipeline can currently be loaded from [`pipeline`] using the following task identifiers: `"text-to-speech"` or
`"text-to-audio"`.
See the list of available models on [huggingface.co/models](https://huggingface.co/models?filter=text-to-speech).
N)vocoder
sampling_ratec s t j|i | | jdkrtdd | _| jjt v r.|d u r+t
t | jj
n|| _|| _| jd ur<| jjj| _| jd u rj| jj}| jjdd }|d urX|| dD ]}t||d }|d uri|| _qZd S d S )Ntfz5The TextToAudioPipeline is only available in PyTorch.generation_config) sample_rater )super__init__ framework
ValueErrorr model __class__r valuesr from_pretrainedDEFAULT_VOCODER_IDtodevicer config__dict__getupdateto_dictgetattr)selfr r argskwargsr
gen_configsampling_rate_namer Z/var/www/html/ai/venv/lib/python3.10/site-packages/transformers/pipelines/text_to_audio.pyr L s0
z TextToAudioPipeline.__init__c K sh t |tr|g}| jjjdkr%| jjjddddddd}|| |}| j |fi |dd i}|S )
Nbarkmax_input_semantic_length FT
max_length)r, add_special_tokensreturn_attention_maskreturn_token_type_idspaddingreturn_tensorspt)
isinstancestrr r
model_typer semantic_configr r tokenizer)r! textr#
new_kwargsoutputr' r' r(
preprocessk s
z TextToAudioPipeline.preprocessc K s | j || jd}|d }|d }| j r.| j || jd}|| | jjdi ||}nt|r;td| | jdi ||d }| j d urQ| |}|S )N)r forward_paramsgenerate_kwargsa\ You're using the `TextToAudioPipeline` with a forward-only model, but `generate_kwargs` is non empty.
For forward-only TTA models, please use `forward_params` instead of of
`generate_kwargs`. For reference, here are the `generate_kwargs` used here:
r r' )
_ensure_tensor_on_devicer r can_generater generatelenr keysr )r! model_inputsr# r< r= r: r' r' r( _forward s"
z TextToAudioPipeline._forward text_inputsc s t j|fi |S )a
Generates speech/audio from the inputs. See the [`TextToAudioPipeline`] documentation for more information.
Args:
text_inputs (`str` or `List[str]`):
The text(s) to generate.
forward_params (`dict`, *optional*):
Parameters passed to the model generation/forward method. `forward_params` are always passed to the
underlying model.
generate_kwargs (`dict`, *optional*):
The dictionary of ad-hoc parametrization of `generate_config` to be used for the generation call. For a
complete overview of generate, check the [following
guide](https://huggingface.co/docs/transformers/en/main_classes/text_generation). `generate_kwargs` are
only passed to the underlying model if the latter is a generative model.
Return:
A `dict` or a list of `dict`: The dictionaries have two keys:
- **audio** (`np.ndarray` of shape `(nb_channels, audio_length)`) -- The generated audio waveform.
- **sampling_rate** (`int`) -- The sampling rate of the generated audio waveform.
)r __call__)r! rE r< r& r' r( rF s z TextToAudioPipeline.__call__c C s4 |r|ni |r |ni d}|d u ri }i }|||fS )N)r< r= r' )r! preprocess_paramsr< r= paramspostprocess_paramsr' r' r( _sanitize_parameters s
z(TextToAudioPipeline._sanitize_parametersc C s& i }| |d< | j|d< |S )Naudior )cpufloatnumpyr )r! waveform output_dictr' r' r( postprocess s
z TextToAudioPipeline.postprocess)NNN)__name__
__module__ __qualname____doc__r r; rD r r4 r rF rJ rQ
__classcell__r' r' r& r( r
s 0
r
N)
typingr r utilsr baser models.auto.modeling_autor !models.speecht5.modeling_speecht5r r r
r' r' r' r( s