[Paid] 🌟 ChatGPT extension to create fantastic conversations with ChatGPT

[Black_Knight]

New update

@Black_Knight

RequestAudioSpeech Function

Description: ​This function asynchronously requests audio speech synthesis from OpenAI's Audio Speech API, utilizing specified parameters such as API key, model, input text, voice, optional instructions, folder path, and file name. The synthesized speech is saved as an MP3 file in the designated location.​

Parameters:

• apiKey (String): Your API key for accessing OpenAI's Audio Speech API.​
• model (String): The model used for speech synthesis. Options include:​
  • tts-1: Optimized for real-time text-to-speech applications.
  • tts-1-hd: Optimized for higher-quality speech synthesis
  • gpt-4o-mini-tts: Built on GPT-4o mini, offering fast and powerful text-to-speech capabilities.
• textInput (String): The text to be synthesized into speech.​
• voice (String): The voice selection for audio generation. Supported voices are:​
• alloy​
• ash​
• ballad
• coral​
• echo​
• fable​
• onyx​
• nova​
• sage​
• shimmer​
• instructions (String, optional): You can provide specific instructions to potentially influence the tone, style, or delivery of the speech (e.g., "Speak in a cheerful and positive tone."). Leave blank if not needed.
• folderPath (String): The directory path where the resulting MP3 file will be saved.​
• fileName (String): The name assigned to the saved MP3 file.

StreamAudioSpeech

Description

Streams audio speech synthesis from OpenAI in real-time and plays it directly through the device speaker. This function is optimized for low latency playback, starting audio almost immediately as it arrives.

It specifically uses the gpt-4o-mini-tts model and requests audio in PCM format, which is required for the direct playback mechanism. The function incorporates pre-buffering, frame alignment, boundary smoothing (cross-fading), and a final fade-out to minimize glitches and provide a smoother listening experience.

Use the separate StopStream block to halt playback manually.

Parameters

• apiKey (Type: Text)
  • Required. Your unique API key for accessing OpenAI services.
• textInput (Type: Text)
  • Required. The text content you want the AI to speak.
• voice (Type: Text)
  • Required. The desired voice for the speech synthesis. Valid options include: alloy, echo, fable, onyx, nova, shimmer. (Note: The underlying model might support more voices than the older TTS-1).
• instructions (Type: Text)
  • Optional. You can provide specific instructions to potentially influence the tone, style, or delivery of the speech (e.g., "Speak in a cheerful and positive tone."). Leave blank if not needed.
• Note: The model and responseFormat are fixed internally by this function:
  • Model: gpt-4o-mini-tts
  • Response Format: pcm (Raw Pulse Code Modulation audio data)

Events Triggered

This function can trigger the following events:

• StartedAudioStream()
  • Fired shortly after the request is made, once enough initial audio data (pre-buffer) has been received and playback is about to begin. Use this to indicate to the user that the audio is starting.
• FinishedAudioStream()
  • Fired only when the entire audio stream has been successfully received from OpenAI and played through to the end naturally (without being manually stopped).
• AudioStreamError(errorMessage Text)
  • Fired if any error occurs during the process:
    • Invalid API key or network issues.
    • Errors reported by the OpenAI API.
    • Problems initializing the device's audio player (AudioTrack).
    • Errors during data writing or processing.
  • The errorMessage parameter provides details about the failure.
• StoppedAudioStream()
  • Important: This event is NOT directly fired by StreamAudioSpeech itself upon completion. It is fired when you explicitly call the StopStream block to manually interrupt the ongoing audio playback. If the stream is stopped manually, FinishedAudioStream will not be fired.

How It Works (Simplified)

1. Request: Sends your text and parameters to OpenAI, specifically requesting a pcm audio stream from the gpt-4o-mini-tts model.
2. Pre-buffer: Receives the first small portion of audio data from OpenAI and holds it temporarily.
3. Initialize Player: Sets up the device's low-level audio player (AudioTrack) configured for the specific PCM format (e.g., 44.1kHz, 16-bit Mono).
4. Write & Play: Writes the pre-buffered data to the player and immediately starts playback.
5. Stream Loop: Continuously receives subsequent small chunks of audio data from OpenAI.
6. Smooth & Play: For each incoming chunk:

• It checks if there's a large, potentially audible jump (discontinuity) between the end of the last played chunk and the start of the new one.
• If a jump is detected, it applies a very short fade-in to the beginning of the new chunk to smooth the transition, reducing clicks/pops.
• It ensures only complete audio frames (pairs of bytes for 16-bit PCM) are sent to the player.
• The processed chunk is sent to the player to be heard.

7. Final Fade-Out: When the stream from OpenAI ends naturally, a short fade-out is applied to the very last segment of audio for a less abrupt ending.
8. Finish/Error/Stop: Triggers the appropriate event (FinishedAudioStream, AudioStreamError, or relies on StopStream triggering StoppedAudioStream).

Audio Generation: File Download vs. Real-time Stream

[Filip_Wozniak]

Hi @Black_Knight , I have just tried to use the GPT Vision using your extension with this code:

and I am getting the error message:
{ "error": { "message": "The model gpt-4-vision-preview has been deprecated, learn more here: https://platform.openai.com/docs/deprecations", "type": "invalid_request_error", "param": null, "code": "model_not_found" }}

Could you please switch the default model for Vision functions to a valid one or provide a parameter for model selection?

I have also tried to pass the images to gpt-4o this way:

but I am getting the error message: "No value for id" in this case.

Could you please provide a way to use GPT Vision again as this was the reason why I bought your extension.

[Black_Knight]

The first issue is because the model was deprecated,
So I have updated the aix and will send the extension for you at pm but first send the payment transaction ID of the extension

And for the second one
Here is how to use the block

@Black_Knight

[Filip_Wozniak]

I will contact you on PM for the fix.
My idea with the second solution was to simply follow the structure from the OpenAI Documentation:

https://platform.openai.com/docs/guides/images?api-mode=responses&lang=javascript&format=base64-encoded
This could have worked when serialization would be used in the extension to map any structure to a JSON - just an idea which I wanted to share!

[Black_Knight]

@Filip_Wozniak I have updated the extension and send it for you at PM

[Black_Knight]

@Filip_Wozniak

Could you please provide a way to use GPT Vision again as this was the reason why I bought your extension.

The vision blocks was already existed I didn't remove them .