o i̖i\@sddlmZddlZddlZddlZddlmZddlmZddl m Z ddl Z ddl Z ddlmZmZddlmZddlmZmZdd lmZd d lmZd d lmZd ZeGdddZGdddejjZGdddejj Z dS)) annotationsN) dataclass)Path)Literal)agentsrtc)utils) NOT_GIVEN NotGivenOr)is_given) onnx_modelloggerg?c@sFeZdZUded<ded<ded<ded<ded<ded<ded <d S) _VADOptionsfloatmin_speech_durationmin_silence_durationprefix_padding_durationmax_buffered_speechactivation_thresholddeactivation_thresholdint sample_rateN)__name__ __module__ __qualname____annotations__rr\/var/www/html/livekit_bhavya/venv/lib/python3.10/site-packages/livekit/plugins/silero/vad.pyr)s  rc seZdZdZedddddddeeed d-ddZd.fd d! Zed/d#d$Z ed/d%d&Z d0d(d)Z eeeeeed*d1d+d,Z Z S)2VADz Silero Voice Activity Detection (VAD) class. This class provides functionality to detect speech segments within audio data using the Silero VAD model. g?g?g?gN@i>T) rrrrrr force_cpuonnx_file_pathrpadding_durationrrrrrrrLiteral[8000, 16000]r!boolr"NotGivenOr[Path | str]rNotGivenOr[float]r#returnc Cs|tjvr tdt| rtd| }t| r | dkr tdtj||p&dd} t|||||| p7t|dd|d } || | d S) a Load and initialize the Silero VAD model. This method loads the ONNX model and prepares it for inference. When options are not provided, sane defaults are used. **Note:** This method is blocking and may take time to load the model into memory. It is recommended to call this method inside your prewarm mechanism. **Example:** ```python def prewarm(proc: JobProcess): proc.userdata["vad"] = silero.VAD.load() async def entrypoint(ctx: JobContext): vad = (ctx.proc.userdata["vad"],) # your agent logic... if __name__ == "__main__": cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint, prewarm_fnc=prewarm)) ``` Args: min_speech_duration (float): Minimum duration of speech to start a new speech chunk. min_silence_duration (float): At the end of each speech, wait this duration before ending the speech. prefix_padding_duration (float): Duration of padding to add to the beginning of each speech chunk. max_buffered_speech (float): Maximum duration of speech to keep in the buffer (in seconds). activation_threshold (float): Threshold to consider a frame as speech. sample_rate (Literal[8000, 16000]): Sample rate for the inference (only 8KHz and 16KHz are supported). onnx_file_path (Path | str | None): Path to the ONNX model file. If not provided, the default model will be loaded. This can be helpful if you want to use a previous version of the silero model. force_cpu (bool): Force the use of CPU for inference. deactivation_threshold (float): Negative threshold (noise or exit threshold). If model's current state is SPEECH, values BELOW this value are considered as NON-SPEECH. Default is max(activation_threshold - 0.15, 0.01). padding_duration (float | None): **Deprecated**. Use `prefix_padding_duration` instead. Returns: VAD: An instance of the VAD class ready for streaming. Raises: ValueError: If an unsupported sample rate is provided. z4Silero VAD only supports 8KHz and 16KHz sample ratesz`padding_duration is deprecated and will be removed in 1.5.0, use prefix_padding_duration insteadrz-deactivation_threshold must be greater than 0N)r"g333333?g{Gz?)rrrrrrr)sessionopts) r SUPPORTED_SAMPLE_RATES ValueErrorr rwarningnew_inference_sessionrmax) clsrrrrrrr!r"rr#r)r*rrrload;s( < zVAD.loadr)onnxruntime.InferenceSessionr*rNonecs6tjtjjddd||_||_tjt |_ dS)NgMb?)update_interval) capabilities) super__init__rvadVADCapabilities _onnx_session_optsweakrefWeakSet VADStream_streams)selfr)r* __class__rrr7sz VAD.__init__strcCdS)Nsilerorr@rrrmodelz VAD.modelcCrD)NONNXrrFrrrproviderrHz VAD.providerr>cCs.t||jtj|j|jjd}|j||S)z Create a new VADStream for processing audio data. Returns: VADStream: A stream object for processing audio input and detecting speech. ) onnx_sessionr)r>r;r OnnxModelr:rr?add)r@streamrrrrNs  z VAD.streamrrrrrrc Cst|r||j_t|r||j_t|r||j_t|r ||j_t|r(||j_t|r0||j_|jD] }|j ||||||dq3dS)a Update the VAD options. This method allows you to update the VAD options after the VAD object has been created. Args: min_speech_duration (float): Minimum duration of speech to start a new speech chunk. min_silence_duration (float): At the end of each speech, wait this duration before ending the speech. prefix_padding_duration (float): Duration of padding to add to the beginning of each speech chunk. max_buffered_speech (float): Maximum duration of speech to keep in the buffer (in seconds). activation_threshold (float): Threshold to consider a frame as speech. rON) r r;rrrrrrr?update_options)r@rrrrrrrNrrrrPs, zVAD.update_options)rrrrrrrrrrrr$r!r%r"r&rr'r#r'r(r )r)r2r*rr(r3)r(rC)r(r>rr'rr'rr'rr'rr'rr'r(r3)rrr__doc__ classmethodr r1r7propertyrGrJrNrP __classcell__rrrArr 4s6S   r csPeZdZdfd d Zeeeeeed dddZejje ddddZ Z S)r>r8r r*rrGonnx_model.OnnxModelr(r3csNt||||_|_t|_tjdd|_ d|_ d|_ d|_ d|_ dS)Ngffffff?)alpharF)r6r7r;_modelasyncioget_event_loop_loopr ExpFilter _exp_filter_input_sample_rate_speech_buffer_speech_buffer_max_reached_prefix_padding_samples)r@r8r*rGrArrr7s   zVADStream.__init__rOrr'rrrrrcCs|jj}t|r ||j_t|r||j_t|r||j_t|r$||j_t|r,||j_t|r4||j_|jrc|j dus>Jt |jj|j|_ |j t |jj|j|j |jj|kred|_ dSdSdS)aJ Update the VAD options. This method allows you to update the VAD options after the VAD object has been created. Args: min_speech_duration (float): Minimum duration of speech to start a new speech chunk. min_silence_duration (float): At the end of each speech, wait this duration before ending the speech. prefix_padding_duration (float): Duration of padding to add to the beginning of each speech chunk. max_buffered_speech (float): Maximum duration of speech to keep in the buffer (in seconds). activation_threshold (float): Threshold to consider a frame as speech. deactivation_threshold (float): Negative threshold (noise or exit threshold). If model's current state is SPEECH, values BELOW this value are considered as NON-SPEECH. NF)r;rr rrrrrr^r_rraresizer`)r@rrrrrrold_max_buffered_speechrrrrPs6    zVADStream.update_optionsrcstjjjtjd}dd}d}d}d}d}d}d}g} g} d} d} d} j2z>3dHW}t|tjs6q(j so|j _ t j j j _tjt j jj jtjd_j j j krntjj j j tjjd} n j |j kr{tdq(jdusJ| || dur| | |n| | t}tdd | D}|jjkrnt| }t| }tj|j djjt!tjj"|tjd j#$dj|IdH}j%j&d |d }jjj j }|jj7}||7}j jj }jj|| }t |}||} t'j}t(||}|dkr0|j d|j|<|7n j)sgsz(VADStream._main_task..)outrdg?)expsamplezOmax_buffered_speech reached, ignoring further data for the current speech inputz!inference is slower than realtimedelay)extrar(r3csPjdusJjkrdSjj}d_|jdj<jdS)NF)r_rar`) padding_datar@speech_buffer_indexrr_reset_write_cursors   z1VADStream._main_task.._reset_write_cursorrtc.AudioFramecs4jdusJjd}tjjd|dS)Nr )r num_channelsrhdata)r_tobytesr AudioFramer^) speech_datarrrr_copy_speech_buffersz1VADStream._main_task.._copy_speech_bufferr )rwrrvrh) type samples_index timestampsilence_durationspeech_duration probabilityinference_durationframesspeakingraw_accumulated_silenceraw_accumulated_speech)r|r}r~rrrrr(r3)r(ru):npemptyrXwindow_size_samplesfloat32 _input_ch isinstancerryr^rrr;rrarint16r_AudioResamplerAudioResamplerQualityQUICKrerrorappendextendpushtime perf_countersumrcombine_framesdividerwiinfor/r[run_in_executorr]applylenminr`r-SLOW_INFERENCE_THRESHOLD _event_ch send_nowaitrr8VADEvent VADEventTypeINFERENCE_DONErxrrrSTART_OF_SPEECHr END_OF_SPEECH)r@inference_f32_data pub_speakingpub_speech_durationpub_silence_durationpub_current_sample pub_timestampspeech_threshold_durationsilence_threshold_duration input_framesinference_frames resamplerinput_copy_remaining_fractextra_inference_time input_frame start_timeavailable_inference_samplesinference_framepwindow_durationresampling_ratioto_copy to_copy_intavailable_spaceto_copy_bufferrrtr{rwrrrr _main_task"s~                         zVADStream._main_task)r8r r*rrGrVr(r3rQr) rrrr7r rPrrlog_exceptionsrrrUrrrArr>s 5r>)! __future__rrYrr< dataclassesrpathlibrtypingrnumpyr onnxruntimelivekitrrlivekit.agentsrlivekit.agents.typesr r livekit.agents.utilsr r logrrrr8r r>rrrrs*         .