o i o@sdZddlZddlZddlZddlZddlZddlmZddlm Z m Z ddl m Z ddl mZddlmZGdd d e ZdS) z ElevenLabs WebSocket Service Integrates ElevenLabs Conversational AI WebSocket API for service_type 4 Handles real-time audio streaming between Mcube and ElevenLabs N)State)OptionalCallable)ABC)Config)Logc@s,eZdZdZd5deedeefddZdeefdd Zdefd d Zd6de de fddZ ddZ  d7dee de dee fddZdefddZd8dedede fdd Zd!d"Zde fd#d$Zd8d%e d&e fd'd(Zd)ee gdffd*d+Zd%e fd,d-Zd.ede fd/d0Zde fd1d2Zd3d4ZdS)9ElevenLabsWebSocketServicez ElevenLabs WebSocket service for real-time conversational AI. Handles bidirectional audio streaming and message processing. Nagent_idapi_keycCsttdr |p tjnd|_|ptj|_|jstd|js!tdd|_d|_d|_ d|_ d|_ d|_ d|_ d|_d|_d|_d|_d|_d|_d|_d|_d|_d|_d|_d|_d|_td|jdd d dS) z Initialize ElevenLabs WebSocket service. Args: agent_id: ElevenLabs agent ID (from config or bot_config) api_key: ElevenLabs API key (from config) ELEVENLABS_AGENT_IDNz@ELEVENLABS_AGENT_ID is required. Set it in config or bot_config.z1ELEVENLABS_API_KEY is required. Set it in config.Fr{Gzt?u=🎙️ ElevenLabs WebSocket service initialized - Agent ID: ...)hasattrrr r ELEVENLABS_API_KEYr ValueError websocketkeep_alive_taskrunning connectedlast_reconnect_timelast_audio_send_timemin_audio_intervalagent_audio_formatuser_audio_formatconversation_idon_audio_receivedon_transcript_receivedon_response_receivedon_vad_score_receivedon_interruption_receivedon_tool_request_receivedon_connection_closedon_automated_greeting_detectedon_tool_abandonedon_conversation_id_receivedrinfo)selfr r r(J/var/www/html/live_calls/homebook/services/elevenlabs_websocket_service.py__init__s:  z#ElevenLabsWebSocketService.__init__returnc s|jsdSzddl}WntytdYdSwzd|j}d|ji}|jddd}|jd d }|j||d 4IdHi}|j ||d 4IdHF}|j d krs| IdH}| dWdIdHWdIdHWStd|j  WdIdHWdIdHWdS1IdHswYWdIdHWdS1IdHswYWdSt y} ztd| WYd} ~ dSd} ~ ww)z-Get signed URL for private agent (if needed).Nrz;aiohttp not installed. Install it with: pip install aiohttpzIhttps://api.elevenlabs.io/v1/convai/conversation/get-signed-url?agent_id= xi-api-keyd)limitlimit_per_hostg$@)total) connectortimeout)headers signed_urlzFailed to get signed URL: zError getting signed URL: ) r aiohttp ImportErrorrerrorr TCPConnector ClientTimeout ClientSessiongetstatusjson Exception) r'r7urlr4r2r3sessionresponsedataer(r(r)get_signed_urls@       2z)ElevenLabsWebSocketService.get_signed_urlcCs&|jr d|jd|jSd|jS)z]Get WebSocket URL for ElevenLabs. Includes conversation_id if resuming existing conversation.z8wss://api.elevenlabs.io/v1/convai/conversation?agent_id=z&conversation_id=)rr r'r(r(r)get_websocket_urls z,ElevenLabsWebSocketService.get_websocket_urlFuse_signed_url max_retriesc sv|}d|ji}|jrtd|jdddntdd}d}td |d D]}zx|d krT|d |d }td |d |d |ddt|IdH|jrgtd|d|d |dntd|ddd|d |dtjt j ||dddd|dIdH|_ d|_ d|_ td|dt||_WdStjy||krtd|d|d |dYq,d|d|d} t| d |_ t| ty/} zJ||krd!dl} td"|d |d#| td$| WYd} ~ q,d!dl} d%|d&| } t| td'| d |_ t| | d} ~ wwd |_ td%|d)(a Connect to ElevenLabs WebSocket with timeout and retry logic. Args: use_signed_url: Whether to use signed URL (for private agents) max_retries: Maximum number of connection retry attempts (default: 3) r,u'🔁 Resuming ElevenLabs conversation: Nr ru)🆕 Starting NEW ElevenLabs conversationg4@g?u"🔄 Retrying connection (attempt /z) after z.1fz s delay...z$Connecting to ElevenLabs WebSocket: z (attempt )Pz ... (attempt )additional_headers ping_interval ping_timeout close_timeout)r3Tu<✅ Connected to ElevenLabs WebSocket successfully (attempt u⚠️ Connection timeout (zs) on attempt z, will retry...uC❌ Failed to connect to ElevenLabs WebSocket: Connection timeout (z s) after z attemptsFru#⚠️ Connection error on attempt z: Traceback: u4❌ Failed to connect to ElevenLabs WebSocket after z attempts: Full traceback: )rHr rrr&rangeasynciosleepwait_for websocketsconnectrrr create_task _keep_aliver TimeoutErrorwarningr9r@ tracebackdebug format_exc) r'rJrKrAr4connection_timeout base_delayattemptdelay error_msgrErbr(r(r)r] sp    &      z"ElevenLabsWebSocketService.connectc sld}d}d}|jr4ztdIdH|jsWdS|js/|d7}||kr-tdWdSWq|sE|d7}||krCtdWdSWqd}z|IdHt}t tddkrbt dWn|t j j y}z.t|d ru|jnd }t|d r|jr|jnd }td |d|dd|_WYd}~WdSd}~wty}z5|d7}|r|jrtd|d|d|||kr|stdWYd}~WdSWYd}~Wqd}~wwWnMtjyYdSty.}z6|d7}|jr td|d|d|||kr$|jr|s$tdWYd}~dSWYd}~qd}~ww|js dSdS)zzSend keep-alive messages every 5 seconds using user_activity to prevent idle timeouts without interfering with bot speech.rNrLu3⚠️ Keep-alive: WebSocket is None, stopping taskuP⚠️ Keep-alive: connection not connected after multiple checks, stopping tasku5💓 Keep-alive sent successfully (connection active)codeunknownreasonz No reasonu8⚠️ Keep-alive: connection closed during send (Code: z , Reason: rOFu!⚠️ Keep-alive error (attempt rN): u<❌ Keep-alive: too many consecutive failures, stopping tasku,⚠️ Keep-alive unexpected error (attempt )rrYrZrrra is_connectedsend_user_activitytimeintrcr\ exceptionsConnectionClosedrrlrnrr@r9CancelledError)r'consecutive_failuresmax_consecutive_failureslast_success_timerE close_code close_reasonr(r(r)r_Zsv      z&ElevenLabsWebSocketService._keep_alivecustom_llm_extra_body text_onlydynamic_variablescsddi}dddddddd did }|r$d |vri|d <d|d d <||d <|r.||d<|r4||d<||IdHtddS)a Send conversation initiation message. Args: custom_llm_extra_body: Custom LLM configuration text_only: Whether to use text-only mode (False for audio) dynamic_variables: Dynamic variables for the conversation type#conversation_initiation_client_data ulaw_8000Tg?i,)enable thresholdprefix_padding_mssilence_duration_ms)output_audio_formatvadr)agentttsrr}conversation_config_overrider|r~Nu/📤 Sent conversation initiation to ElevenLabs) _send_messagerr&)r'r|r}r~messageconversation_configr(r(r)send_conversation_initiations0 z7ElevenLabsWebSocketService.send_conversation_initiation audio_datac s|jr|js dSzBddl}|}||j}d}||kr/||}t|IdH||_n||_t|d}d|d}|j |ddIdHWdSt yUYdSw) z Send audio chunk to ElevenLabs with rate limiting to prevent 1008 policy violations. Args: audio_data: Raw audio bytes (PCM format expected) Nrr zutf-8user_audio_chunk)rrTsilent) rrrrrrYrZbase64 b64encodedecoderr@) r'rrr current_timetime_since_last_send min_interval sleep_time audio_base64rr(r(r)send_audio_chunks*    z+ElevenLabsWebSocketService.send_audio_chunk tool_call_idresultis_errorcs$d|||d}||IdHdS)zSend tool result to ElevenLabs.client_tool_result)rrrrN)r)r'rrrrr(r(r)send_client_tool_resultsz2ElevenLabsWebSocketService.send_client_tool_resultcs>|jr|js dS|sdSddi}|j|ddIdHdS)z)Send user_activity message as keep-alive.Nr user_activityTr)rrrprr'rr(r(r)rqs z-ElevenLabsWebSocketService.send_user_activitycsV|jr|js dSzddttd}||IdHWdSty*YdSw)NF interrupt interrupt_revent_idT)rrrsrrrr@rr(r(r)send_interruption s  z,ElevenLabsWebSocketService.send_interruptionrrc s|jsdSzt|}|j|IdHWdStjjyE}z d|_|s:t d|j d|j WYd}~dSWYd}~dSd}~wtjj ym}zd|_|sbt d|WYd}~dSWYd}~dSd}~wt y}zRt|t|j}t|ttfpdvp|dv}|rtfddd Drd|_|st d |WYd}~dSWYd}~dSWYd}~dSWYd}~dSd}~ww) NFu2⚠️ WebSocket connection closed while sending: z - u#⚠️ WebSocket in invalid state: r)connectionerroroserrorc3|]}|vVqdSNr().0phrase error_strr(r) Kz;ElevenLabsWebSocketService._send_message..)zconnection closedzconnection lostz not connectedzwebsocket is closedzwebsocket is closingu"⚠️ Connection error detected: )rr?dumpssendr\rtrurrrarlrn InvalidStater@strlowerr__name__ isinstanceConnectionErrorOSErrorany)r'rr message_jsonrE error_typeis_connection_errorr(rr)r-sD ( z(ElevenLabsWebSocketService._send_message on_messagec s|jstdz;zJ|j2zv3dHW}z@t|}|dd}|dkrC|di}|d}|rAd|d }|j|d d IdHWqt|rP||IdHn||Wqty}z$d dl} t d |t d|t d| WYd}~qd}~ww6|jrTzt |jdd} t |jddpd} | dus| r t d| d| dd|_|| } | rt d| dn t d| d|jr zt|jr|| pd| | IdHn || pd| | Wnty } zt d| WYd} ~ nd} ~ wwWnHtyS}z;t d|d|_|jrIzt|jr6|dddIdHn|dddWn tyHYnwWYd}~nd}~wwWntjjy}zd|_t|drl|jnd} t|d r{|jr{|jnd!} t d| d| d|| } | rt d| dn t d| d|jrzt|jr|| | | IdHn|| | | Wnty} zt d| WYd} ~ nd} ~ ww|jr|jst d"WYd}~n;d}~wty#}zt d#|d dl} t d$| d|_WYd}~nd}~wwWd|_t d%dSWd|_t d%dSWd|_t d%dSd|_t d%w)&z Receive messages from ElevenLabs WebSocket. Args: on_message: Callback function to handle received messages zWebSocket not connectedNrrmping ping_eventrpongrTrru❌ Error processing message: zMessage data: rWrzr{zConnection closed cleanlyu6⚠️ ElevenLabs WebSocket connection closed - Code: z , Reason: ''Fu2🛑 Call ended legitimately (ElevenLabs reason: 'z') - reconnections disabledu#⚠️ Non-terminal close reason: 'z' - reconnection may be allowedi)❌ Error in connection_closed callback: z+Could not get close reason from WebSocket: z"Connection closed (reason unknown)rlrnzNo reason provideduW⚠️ Connection closed while keep-alive was active - may indicate server-side closureu❌ Error receiving messages: rVz&ElevenLabs receive_messages task ended)rr@r?loadsr=rrYiscoroutinefunctionrbrr9rdgetattrrar_is_terminal_call_end_reasonr&rcr"r\rtrurrlrnrdoner)r'rrrDmsg_typerrpong_msgrErbrzr{is_terminal_reasoncallback_errorr(r(r)receive_messagesXs            z+ElevenLabsWebSocketService.receive_messagesc" s |dd}|dkrn|di}|d|_|jr^td|jddd|jr^zt|jr<||jIdHn||jWnty]}ztd |WYd}~nd}~ww|d d |_ |d d |_ dS|dkr|di}td||j rt|j r| |IdHdS| |dSdS|dkr|didd}td||j rt|j r| |IdHdS| |dSdS|dkr|didd}|rt |}|jrt|jr||IdHdS||dSdSdS|dkr0|didd}|jr.t|jr'||IdHdS||dSdS|dkrI|di} | dd} | dd} dS|dkr)|ddpVd}|} d | vstd!| vstd"| vstd#| vstd$| vrtd%|dd&d'n td(|dd)d'|rd*| vr!d+dl} | d,|| j| jB}|s| d-|| j| jB}|r#|d.}td/||gd0}tfd1d2|Dr%td3|jr'zt|jr|j|d4d5IdHn |j|d4d5WdSWdSty}ztd6|WYd}~dSd}~wwdSdSdSdSdS|d7krV|d8id7d9}|jrTt|jrM||IdHdS||dSdS|d:kr|d:i}|dd} |dd} |d;d}|dd<}|d=i}td>| d?| d@|dA|dB tdC|tdD||jrt|jr|| | |IdHdS|| | |dSdS|dEkr|dFi}|dd}|dGi}tdH|dI|dS|dJkr#|dKi}|dLd}|dMd}|dd}|rtdN|dO|dP|dQdStdN|dBdS|dRkr|dRi}|dd} |dSd}|dTdU}|rG|nd}dV|vpVdW|vpVdX|v}|rtdY| dI|ri|dd)ndZ|jrzt|jr|| |IdHn || |WdSWdSty}ztd[|WYd}~dSd}~wwdStd\| d]|dI|r|dd^ndZdS|d_ks|d`krZ|daip|dbi}|dcdp|ddpt|}tdd||rT|gde}tfdfd2|DrVtdg|jrXzt|jr+|j|d4d5IdHn |j|d4d5WdSWdStyS}ztd6|WYd}~dSd}~wwdSdSdS|dhkr|dhi}t|} tdi|| | rtdj|j!rzt|j!r|!dh| d4IdHn |!dh| d4WdSWdSty}ztdk|WYd}~dSd}~wwdSdSgdl}!||!vrtdm|dSdS)nNrrm conversation_initiation_metadata&conversation_initiation_metadata_eventru📝 Received conversation_id: r z... (will reuse on reconnect)u0❌ Error in conversation_id_received callback: agent_output_audio_format mulaw_8000user_input_audio_formatpcm_8000user_transcriptuser_transcription_eventu🎤 [User Transcript]: agent_responseagent_response_eventu🤖 [Agent Response]: audio audio_event audio_base_64 interruptioninterruption_eventrclient_tool_call tool_namercontextual_updatetextzagent changed the languagezchanged the language tozthe user is speaking inzuser is speaking inz in flag tamilu%📝 [Contextual Update - Language]: ru📝 [Contextual Update]: r5zautomated greeting detectedrz.automated greeting detected:\s*['\"](.*?)['\"]z#automated greeting detected:\s*(.+)rLu:📞 [Automated Greeting Detected via Contextual Update]: ) voicemailleave a messagerecord your message not available at the tone please recordafter the tonefinished recordingperson you are trying to reach%call has been forwarded to voice mailforwarded to voice mailzyour call has been forwardedc3rrr(rpatterngreeting_lowerr(r)r:rz.uE🛑 Voicemail detected via contextual update - marking call as endedT) is_voicemailu3❌ Error in automated_greeting_detected callback: vad_scorevad_score_eventgagent_tool_request tool_typezN/A parametersu🔧 [Agent Tool Request]: z (id: z, type: z , event_id: rOu2 📋 Tool Parameters received from ElevenLabs: u 📦 Full tool request: agent_response_metadataagent_response_metadata_eventmetadatau*📊 [Agent Response Metadata] (event_id: roagent_response_correctionagent_response_correction_eventcorrected_agent_responseoriginal_agent_responseu,🔧 [Agent Response Correction] (event_id: z): 'z' -> 'ragent_tool_responserrF abandonedztool execution was abandonedzfailed to end the callu1⚠️ [Tool Execution Abandoned] (tool_call_id: z No resultu&❌ Error in tool_abandoned callback: u*🔧 [Agent Tool Response] (tool_call_id: z , is_error: r-automated_greeting_detectedautomated_greeting!automated_greeting_detected_eventautomated_greeting_event greeting_textu$📞 [Automated Greeting Detected]: ) rrrrrrrrrrrc3rrr(rrr(r)rruF🛑 Voicemail detected via automated greeting - marking call as endedr9u❌ [Error from ElevenLabs]: u?🛑 Error indicates terminal call end - reconnections disabledr)rragent_chat_response_partu📨 [Unknown Message Type]: )"r=rrr&r%rYrr@r9rrrrr b64decoderr rrcresearch IGNORECASEDOTALLgroupstriprarr#rr!r$rrr")"r'rrmetadata_eventrrr audio_bytesr tool_callrr text_lowerr matchrvoicemail_patternsscore tool_requestr tool_paramsrcorrection_eventcorrected_text original_text tool_responserr result_lower is_abandonedgreeting_eventr9rknown_unhandled_typesr(rr)handle_messages                           $         "      (2 $      z)ElevenLabsWebSocketService.handle_messager{cCs`|sdS|}d|vr|dd}t|dkr|d}gd}|D] }||vr-dSq$dS)a Check if the WebSocket close reason indicates a terminal call end. Terminal reasons mean the call legitimately ended and should not reconnect. Args: close_reason: The close reason string from ElevenLabs (may include "How the call ended: " prefix) Returns: True if this is a terminal call end reason (should not reconnect) Fzhow the call ended:rL)zconnection closed cleanlyzclient ended callz client endedzuser ended callz user hung upzuser disconnectedzagent ended callz agent endedzagent ended the callzcall exceeded maximum durationzmaximum durationzcall ended due to silencesilencezvoicemail detectedrz call endedzcall terminatedzconversation endedz session endedT)rsplitlenr)r'r{ reason_lowerpartsterminal_patternsrr(r(r)rs    !z7ElevenLabsWebSocketService._is_terminal_call_end_reasonc Cs|jsdSz7t|jdr9t|jjdr'|jjj}|dk}||jkr$||_|WS|jjtjk}||jkr6||_|WSW|jStyH|jYSty\}z |jWYd}~Sd}~ww)z>Check if WebSocket is connected with improved state detection.FstatenameOPENN) rrr'r(rrr)AttributeErrorr@)r' state_nameis_openrEr(r(r)rps,     z'ElevenLabsWebSocketService.is_connectedcsd|_d|_|jr9|js9|jz z|jIdHWntjy'Yn ty/YnwWd|_nd|_w|jrRz |j IdHWdStyQYdSwdS)NF) rrrrcancelrYrvr@rcloserGr(r(r)r.1s*   z ElevenLabsWebSocketService.close)NN)FrI)NFN)F)r __module__ __qualname____doc__rrr*rFrHboolrsr]r_dictrbytesrrrqrrrrr rrpr.r(r(r(r)rs:+ QR 5(  +t ? r)r1rYr?rrrr\websockets.protocolrtypingrrabcrconfigrservices.log_utilsrrr(r(r(r)s&#