U Izi{o@sdZddlZddlZddlZddlZddlZddlmZddlm Z m Z ddl m Z ddl mZddlmZGdd d e ZdS) z ElevenLabs WebSocket Service Integrates ElevenLabs Conversational AI WebSocket API for service_type 4 Handles real-time audio streaming between Mcube and ElevenLabs N)State)OptionalCallable)ABC)Config)Logc@seZdZdZd/eeeedddZeedddZedd d Zd0e e d ddZ ddZ d1ee e ee dddZedddZd2eee dddZddZe dddZd3e e dd d!Zee gdfd"d#d$Ze d%d&d'Zee d(d)d*Ze dd+d,Zd-d.ZdS)4ElevenLabsWebSocketServicez ElevenLabs WebSocket service for real-time conversational AI. Handles bidirectional audio streaming and message processing. N)agent_idapi_keycCsttdr|ptjnd|_|p"tj|_|js4td|jsBtdd|_d|_d|_ d|_ d|_ d|_ d|_ d|_d|_d|_d|_d|_d|_d|_d|_d|_d|_d|_d|_d|_td|jdd d dS) z Initialize ElevenLabs WebSocket service. Args: agent_id: ElevenLabs agent ID (from config or bot_config) api_key: ElevenLabs API key (from config) ELEVENLABS_AGENT_IDNz@ELEVENLABS_AGENT_ID is required. Set it in config or bot_config.z1ELEVENLABS_API_KEY is required. Set it in config.Fr{Gzt?u=🎙️ ElevenLabs WebSocket service initialized - Agent ID: ...)hasattrrr r ELEVENLABS_API_KEYr ValueError websocketkeep_alive_taskrunning connectedlast_reconnect_timelast_audio_send_timeZmin_audio_intervalagent_audio_formatuser_audio_formatconversation_idon_audio_receivedon_transcript_receivedon_response_receivedon_vad_score_receivedon_interruption_receivedon_tool_request_receivedon_connection_closedon_automated_greeting_detectedon_tool_abandonedon_conversation_id_receivedrinfo)selfr r r'J/var/www/html/live_calls/homebook/services/elevenlabs_websocket_service.py__init__s6 z#ElevenLabsWebSocketService.__init__)returnc s~|js dSz ddl}Wn tk r6tdYdSXzd|j}d|ji}|jddd}|jd d }|j||d 4IdH}|j ||d 4IdH}|j d kr| IdH}| dW5QIdHRW5QIdHRWStd|j W5QIdHRW5QIdHRWdSW5QIdHRXW5QIdHRXWn:t k rx} ztd| WYdSd} ~ XYnXdS)z-Get signed URL for private agent (if needed).Nrz;aiohttp not installed. Install it with: pip install aiohttpzIhttps://api.elevenlabs.io/v1/convai/conversation/get-signed-url?agent_id= xi-api-keyd)limitZlimit_per_hostg$@)total) connectortimeout)headersZ signed_urlzFailed to get signed URL: zError getting signed URL: ) r aiohttp ImportErrorrerrorr Z TCPConnectorZ ClientTimeoutZ ClientSessiongetstatusjson Exception) r&r4urlr2r0r1sessionresponsedataer'r'r(get_signed_urls,      0Jz)ElevenLabsWebSocketService.get_signed_urlcCs&|jrd|jd|jSd|jS)z]Get WebSocket URL for ElevenLabs. Includes conversation_id if resuming existing conversation.z8wss://api.elevenlabs.io/v1/convai/conversation?agent_id=z&conversation_id=)rr r&r'r'r(get_websocket_urlsz,ElevenLabsWebSocketService.get_websocket_urlF)use_signed_url max_retriesc s|}d|ji}|jr6td|jdddn tdd}d}td |d D].}z|d kr|d |d }td |d |d |ddt|IdH|jrtd|d|d |dn&td|ddd|d |ddd|D} tj t j || dddd|dIdH|_ d|_ d|_td|dt||_WdStjk r||krtd|d|d |dYqVn*d|d |d!} t| d"|_t| YqVtk r} z||kr.d#dl} td$|d |d%| td&| WYNqVnFd#dl} d'|d(| } t| td)| d"|_t| | W5d} ~ XYqVXqVd"|_td'|d!dS)*a Connect to ElevenLabs WebSocket with timeout and retry logic. Args: use_signed_url: Whether to use signed URL (for private agents) max_retries: Maximum number of connection retry attempts (default: 3) r+u'🔁 Resuming ElevenLabs conversation: Nr ru)🆕 Starting NEW ElevenLabs conversationg4@g?u"🔄 Retrying connection (attempt /z) after z.1fz s delay...z$Connecting to ElevenLabs WebSocket: z (attempt )Pz ... (attempt cSsg|]\}}||fqSr'r').0kvr'r'r( /sz6ElevenLabsWebSocketService.connect.. ) extra_headers ping_interval ping_timeout close_timeout)r1Tu<✅ Connected to ElevenLabs WebSocket successfully (attempt u⚠️ Connection timeout (zs) on attempt z, will retry...uC❌ Failed to connect to ElevenLabs WebSocket: Connection timeout (z s) after z attemptsFru#⚠️ Connection error on attempt z: Traceback: u4❌ Failed to connect to ElevenLabs WebSocket after z attempts: Full traceback: )rBr rrr%rangeasynciosleepitemswait_for websocketsconnectrrr create_task _keep_aliver TimeoutErrorwarningr6r: tracebackdebug format_exc) r&rDrEr;r2connection_timeoutZ base_delayattemptdelayZextra_headers_list error_msgr?rar'r'r(r\ sl    &       z"ElevenLabsWebSocketService.connectc sd}d}d}|jrztdIdH|js4Wq|js^|d7}||krZtdWqWq |s|d7}||krtdWqWq d}z:|IdHt}t tddkrt dWnt j j k rL}z`t|d r|jnd }t|d r|jr|jnd }td |d|dd|_WY WqW5d}~XYntk r}zn|d7}|r|jrtd|d|d|||kr|stdWYWqWYWq W5d}~XYnXWq tjk rYqYq tk r}zh|d7}|jr8td|d|d|||krh|jrT|shtdWYqWYq W5d}~XYq Xq dS)zzSend keep-alive messages every 5 seconds using user_activity to prevent idle timeouts without interfering with bot speech.rNrFu3⚠️ Keep-alive: WebSocket is None, stopping taskuP⚠️ Keep-alive: connection not connected after multiple checks, stopping tasku5💓 Keep-alive sent successfully (connection active)codeunknownreasonz No reasonu8⚠️ Keep-alive: connection closed during send (Code: z , Reason: rIFu!⚠️ Keep-alive error (attempt rH): u<❌ Keep-alive: too many consecutive failures, stopping tasku,⚠️ Keep-alive unexpected error (attempt )rrWrXrrr` is_connectedsend_user_activitytimeintrbr[ exceptionsConnectionClosedrrjrlrr:r6CancelledError)r&Zconsecutive_failuresZmax_consecutive_failuresZlast_success_timer? close_code close_reasonr'r'r(r^^sd        z&ElevenLabsWebSocketService._keep_alive)custom_llm_extra_body text_onlydynamic_variablescsddi}dddddddd did }|rFd |kr:i|d <d|d d <||d <|rZ||d<|rf||d<||IdHtddS)a Send conversation initiation message. Args: custom_llm_extra_body: Custom LLM configuration text_only: Whether to use text-only mode (False for audio) dynamic_variables: Dynamic variables for the conversation typeZ#conversation_initiation_client_data ulaw_8000Tg?i,)enable thresholdZprefix_padding_msZsilence_duration_ms)output_audio_formatZvadr~)agentZttsrrxZconversation_config_overriderwryNu/📤 Sent conversation initiation to ElevenLabs) _send_messagerr%)r&rwrxrymessageZconversation_configr'r'r(send_conversation_initiations2 z7ElevenLabsWebSocketService.send_conversation_initiation) audio_datac s|jr |jsdSzddl}|}||j}d}||kr\||}t|IdH||_n||_t|d}d|d}|j |ddIdHWnt k rYnXdS) z Send audio chunk to ElevenLabs with rate limiting to prevent 1008 policy violations. Args: audio_data: Raw audio bytes (PCM format expected) Nrr zutf-8user_audio_chunk)rzrTsilent) rrrprrWrXbase64 b64encodedecoderr:) r&rrp current_timeZtime_since_last_sendZ min_intervalZ sleep_time audio_base64rr'r'r(send_audio_chunks&   z+ElevenLabsWebSocketService.send_audio_chunk) tool_call_idresultis_errorcs"d|||d}||IdHdS)zSend tool result to ElevenLabs.Zclient_tool_result)rzrrrN)r)r&rrrrr'r'r(send_client_tool_result s z2ElevenLabsWebSocketService.send_client_tool_resultcs<|jr |jsdS|sdSddi}|j|ddIdHdS)z)Send user_activity message as keep-alive.NrzZ user_activityTr)rrrnrr&rr'r'r(ros z-ElevenLabsWebSocketService.send_user_activitycsZ|jr |jsdSz.ddttd}||IdHWdStk rTYdSXdS)NFZ interruptZ interrupt_rzevent_idT)rrrqrprr:rr'r'r(send_interruption$s z,ElevenLabsWebSocketService.send_interruption)rrc sN|js dSz"t|}|j|IdHWntjjk rv}z(d|_|sft d|j d|j W5d}~XYntjj k r}zd|_|st d|W5d}~XYnt k rH}zvt|t|j}t|ttfpdkp|dk}|r8tfddd Dr8d|_|s8t d |W5d}~XYnXdS) NFu2⚠️ WebSocket connection closed while sending: z - u#⚠️ WebSocket in invalid state: r)ZconnectionerrorZoserrorc3s|]}|kVqdSNr')rKphrase error_strr'r( Osz;ElevenLabsWebSocketService._send_message..)zconnection closedzconnection lostz not connectedzwebsocket is closedzwebsocket is closingu"⚠️ Connection error detected: )rr9dumpssendr[rrrsrrr`rjrl InvalidStater:strlowerrz__name__ isinstanceConnectionErrorOSErrorany)r&rrZ message_jsonr? error_typeZis_connection_errorr'rr(r1s2 *  z(ElevenLabsWebSocketService._send_message) on_messagec sl|jstdzDz|j2z3dHW}zt|}|dd}|dkr|di}|d }|rd |d }|j|d d IdHWqt |r||IdHn||Wqtk r}z@ddl } t d|t d|t d| W5d}~XYqXq6|jrz t|jdd} t|jddp:d} | dk sL| r td| d| dd|_|| } | rtd| dntd| d|jr zBt |jr|| pd| | IdHn|| pd| | Wn4tk r} zt d| W5d} ~ XYnXWntk r}zptd|d|_|jrz6t |jrz|dddIdHn|dddWntk rYnXW5d}~XYnXWntjjk r}zd|_t|d r|jnd} t|d!r|jr|jnd"} td| d| d|| } | rHtd| dntd| d|jrz6t |jr|| | | IdHn|| | | Wn4tk r} zt d| W5d} ~ XYnX|jr|jstd#W5d}~XYnVtk rP}z6t d$|ddl } t d%| d|_W5d}~XYnXW5d|_tdXdS)&z Receive messages from ElevenLabs WebSocket. Args: on_message: Callback function to handle received messages zWebSocket not connectedFz&ElevenLabs receive_messages task endedNrzrkping ping_eventrpongrTrru❌ Error processing message: zMessage data: rUrurvzConnection closed cleanlyu6⚠️ ElevenLabs WebSocket connection closed - Code: z , Reason: ''u2🛑 Call ended legitimately (ElevenLabs reason: 'z') - reconnections disabledu#⚠️ Non-terminal close reason: 'z' - reconnection may be allowedi)❌ Error in connection_closed callback: z+Could not get close reason from WebSocket: z"Connection closed (reason unknown)rjrlzNo reason provideduW⚠️ Connection closed while keep-alive was active - may indicate server-side closureu❌ Error receiving messages: rT)rr:rrr%r9loadsr7rrWiscoroutinefunctionrar6rcgetattrr`r_is_terminal_call_end_reasonrbr!r[rrrsrrjrlrdone)r&rrr>msg_typerrZpong_msgr?rarurvis_terminal_reasoncallback_errorr'r'r(receive_messages\s       * & "z+ElevenLabsWebSocketService.receive_messages)rc" sb |dd}|dkr|di}|d|_|jrtd|jddd|jrz0t|jrv||jIdHn ||jWn2tk r}ztd |W5d}~XYnX|d d |_ |d d |_ n|dkr4|di}td||j r^t|j r&| |IdHn | |n*|dkr|didd}td||j r^t|j r| |IdHn | |n|dkr|didd}| r^t |}|j r^t|jr||IdHn ||nb|dkrP|didd}|j r^t|jrB||IdHn ||n|dkr|di} | dd} | dd} n|dkr:|ddpd}|} d | ksd!| ksd"| ksd#| ksd$| krtd%|dd&d'ntd(|dd)d'| r^d*| k r^d+dl} | d,|| j| jB}|sX| d-|| j| jB}| r^|d.}td/||d0d1d2d3d4d5d6d7d8d9d:d;g }tfd|j r^z6t|jr|j|d?d@IdHn|j|d?d@Wn4tk r4}ztdA|W5d}~XYnXn$|dBkr|dCidBdD}|j r^t|jr||IdHn ||n|dEkrb|dEi}|dd} |dd} |dFd}|ddG}|dHi}tdI| dJ| dK|dL|dM tdN|tdO||j r^t|jrP|| | |IdHn|| | |n|dPkr|dQi}|dd}|dRi}tdS|dT|n|dUkr |dVi}|dWd}|dXd}|dd}|r tdY|dZ|d[|d\ntdY|dMn>|d]krZ|d]i}|dd} |d^d}|d_d`}|rh|nd}da|kpdb|kpdc|k}|r(tdd| dT|r|dd)nde|jrVz2t|jr|| |IdHn || |Wn4tk r$}ztdf|W5d}~XYnXn.tdg| dh|dT|rL|ddinden|djksn|dkkr||dlip|dmi}|dndp|ddpt|}tdo|| r^|d0d1d2d3d4d5d6d7d8d9d:g }tfdpd=|D r^tdq|j r^z6t|jr4|j|d?d@IdHn|j|d?d@Wn4tk rx}ztdA|W5d}~XYnXn|drk r:|dri}t|} tds|| |  r^tdt|j! r^z6t|j!r|!dr| d?IdHn|!dr| d?Wn4tk r6}ztdu|W5d}~XYnXn$dvdwdxg}!||!k r^tdy|dS)zNrzrkZ conversation_initiation_metadataZ&conversation_initiation_metadata_eventru📝 Received conversation_id: r z... (will reuse on reconnect)u0❌ Error in conversation_id_received callback: Zagent_output_audio_format mulaw_8000Zuser_input_audio_formatZpcm_8000user_transcriptZuser_transcription_eventu🎤 [User Transcript]: Zagent_responseZagent_response_eventu🤖 [Agent Response]: ZaudioZ audio_eventZ audio_base_64Z interruptionZinterruption_eventrZclient_tool_call tool_namerZcontextual_updatetextzagent changed the languagezchanged the language tozthe user is speaking inzuser is speaking inz in flag tamilu%📝 [Contextual Update - Language]: ru📝 [Contextual Update]: r3zautomated greeting detectedrz.automated greeting detected:\s*['\"](.*?)['\"]z#automated greeting detected:\s*(.+)rFu:📞 [Automated Greeting Detected via Contextual Update]: voicemailzleave a messagezrecord your messagez not availablez at the tonez please recordzafter the tonezfinished recordingzperson you are trying to reachz%call has been forwarded to voice mailzforwarded to voice mailzyour call has been forwardedc3s|]}|kVqdSrr'rKpatternZgreeting_lowerr'r(r>sz.uE🛑 Voicemail detected via contextual update - marking call as endedT) is_voicemailu3❌ Error in automated_greeting_detected callback: Z vad_scoreZvad_score_eventgZagent_tool_request tool_typezN/A parametersu🔧 [Agent Tool Request]: z (id: z, type: z , event_id: rIu2 📋 Tool Parameters received from ElevenLabs: u 📦 Full tool request: Zagent_response_metadataZagent_response_metadata_eventmetadatau*📊 [Agent Response Metadata] (event_id: rmZagent_response_correctionZagent_response_correction_eventZcorrected_agent_responseZoriginal_agent_responseu,🔧 [Agent Response Correction] (event_id: z): 'z' -> 'rZagent_tool_responserrFZ abandonedztool execution was abandonedzfailed to end the callu1⚠️ [Tool Execution Abandoned] (tool_call_id: z No resultu&❌ Error in tool_abandoned callback: u*🔧 [Agent Tool Response] (tool_call_id: z , is_error: r,Zautomated_greeting_detectedZautomated_greetingZ!automated_greeting_detected_eventZautomated_greeting_event greeting_textu$📞 [Automated Greeting Detected]: c3s|]}|kVqdSrr'rrr'r(rsuF🛑 Voicemail detected via automated greeting - marking call as endedr6u❌ [Error from ElevenLabs]: u?🛑 Error indicates terminal call end - reconnections disabledrrrZagent_chat_response_partu📨 [Unknown Message Type]: )"r7rrr%r$rWrr:r6rrrrr b64decoderrrrbresearch IGNORECASEDOTALLgroupstripr`rr"rr r#rrr!)"r&rrZmetadata_eventrrr audio_bytesrZ tool_callrrZ text_lowerrmatchrvoicemail_patternsscore tool_requestr tool_paramsrZcorrection_eventZcorrected_textZ original_textZ tool_responserrZ result_lowerZ is_abandonedZgreeting_eventr6rZknown_unhandled_typesr'rr(handle_messagesv    "           &        $                ($2$ $     $  z)ElevenLabsWebSocketService.handle_message)rvr*cCs|sdS|}d|kr<|dd}t|dkr<|d}dddddd d d d d ddddddddg}|D]}||krhdSqhdS)a Check if the WebSocket close reason indicates a terminal call end. Terminal reasons mean the call legitimately ended and should not reconnect. Args: close_reason: The close reason string from ElevenLabs (may include "How the call ended: " prefix) Returns: True if this is a terminal call end reason (should not reconnect) Fzhow the call ended:rFzclient ended callz client endedzuser ended callz user hung upzuser disconnectedzagent ended callz agent endedzagent ended the callzcall exceeded maximum durationzmaximum durationzcall ended due to silenceZsilencezvoicemail detectedrz call endedzcall terminatedzconversation endedz session endedT)rsplitlenr)r&rvZ reason_lowerpartsZterminal_patternsrr'r'r(rs<    z7ElevenLabsWebSocketService._is_terminal_call_end_reasonc Cs|js dSzjt|jdrrt|jjdrN|jjj}|dk}||jkrH||_|WS|jjtjk}||jkrl||_|WSWnDtk r|jYStk r}z|jWYSd}~XYnX|jS)z>Check if WebSocket is connected with improved state detection.FstatenameOPENN) rrrrrrrAttributeErrorr:)r&Z state_nameZis_openr?r'r'r(rns&      z'ElevenLabsWebSocketService.is_connectedcsd|_d|_|jrn|jsn|jz>z|jIdHWn(tjk rNYntk r`YnXW5d|_X|jrz|j IdHWntk rYnXdS)NF) rrrrcancelrWrtr:rcloserAr'r'r(r3s   z ElevenLabsWebSocketService.close)NN)FrC)NFN)F)F)r __module__ __qualname____doc__rrr)r@rBboolrqr\r^dictrbytesrrrorrrrrrrnrr'r'r'r(rs4+ UR 5(  +t =r)rrWr9rrpr[Zwebsockets.protocolrtypingrrabcrconfigrservices.log_utilsrrr'r'r'r(s