Introduction
Hello everyone I am here today to introduce my new extension called ChatGPT,
Unlock the potential of artificial intelligence in your App Inventor projects with the ChatGPT Extension. This powerful tool empowers you to seamlessly integrate OpenAI's cutting-edge language models, enabling you to build sophisticated and engaging AI-powered features within your apps.
Why Choose the ChatGPT Extension?
-
Unleash the Power of AI: Effortlessly incorporate advanced natural language processing into your apps.
-
Dynamic Conversations: Create interactive chatbots that remember past interactions, providing a more natural and engaging user experience.
-
Streaming for Speed: Handle large responses efficiently with streaming support, ensuring a smooth and responsive user interface.
-
Robust Error Handling: Built-in error management allows you to gracefully handle unexpected issues and provide informative feedback to users.
-
Audio Capabilities: Transcribe and translate audio using OpenAI's powerful Whisper API, expanding the possibilities for accessibility and multilingual support.
-
Image Generation: Create images from text descriptions, generate images inside your app now using Open AI's DALL-E models.
-
Image analysis: Analyze images using OpenAI's ChatGpt Vision API, extract valuable insights from images.
-
Embeddings: Generate text embeddings for various NLP tasks, understand the meaning and relationships between words.
-
Easy Integration: Designed specifically for MIT App Inventor, making integration a breeze.
-
Cost-Effective: Access a wealth of AI features for just $5 (including a Tutorial AIA file)!
Blocks
The SendMessage
block is responsible for sending a conversation to the ChatGPT and processing the response. Here's a breakdown of the code:
- Block Description: This Block allows users to interact with the OpenAI ChatGPT and receive structured API-style responses.
- Function Parameters:
prompts
: A list of conversation prompts provided by the user like in the block above .model
: The name of the OpenAI model to be used.apiKey
: The API key for authorization.maxTokens
: The maximum number of tokens in the response.temperature
: A value controlling the randomness of the response.
The RespondedToChat event is triggered when the OpenAI API successfully provides a response to a user's prompt sent through the SendMessage function. This event delivers the content of the response along with detailed metadata, including token usage information.
Parameters:
-
responseId (String): A unique identifier for the response generated by the OpenAI API.
-
responseType (String): The type of the response object. It usually indicates the nature of the response data structure (e.g., "chat.completion").
-
createdTimestamp (Number - long): The timestamp (in Unix epoch time, milliseconds) when the response was created by the OpenAI API.
-
responseModel (String): The specific OpenAI model that generated the response (e.g., "gpt-3.5-turbo", "gpt-4").
-
choiceIndex (Number - int): The index of the selected choice within the response. The OpenAI API may offer multiple response choices; this parameter indicates which one is being presented (typically 0 for the first and usually only choice).
-
role (String): The role associated with the message in the conversation. Common roles are "system" (for initial instructions), "user" (for the user's prompt), and "assistant" (for the AI's response).
-
content (String): The text content of the response generated by the OpenAI model.
-
finishReason (String): Indicates why the response generation process finished. It can be one of the following:
-
stop: The model reached a natural stopping point or a stop sequence was generated.
-
length: The maximum number of tokens (maxTokens parameter) was reached.
-
content_filter: Content was omitted due to a flag from OpenAI's content filters.
-
null: The API response is still in progress or incomplete (this value might be present if there are issues in receiving the complete response).
-
-
promptTokens (Number - int): The number of tokens used in the user's prompt. This value is now calculated before sending the request using a basic word-count approximation. It will not be the exact prompt token count but will be a close estimate.
-
completionTokens (Number - int): The number of tokens used in the generated response completion. This value is returned by the OpenAI API. If the key is not provided, it will default to 0.
-
totalTokens (Number - int): The total number of tokens used in the entire request and response (prompt tokens + completion tokens). This value is returned by the OpenAI API. If the key is not provided, it will default to 0.
The SendStreamedMessage
function is designed to retrieve a response in chunks from the ChatGPT model. It allows for ongoing communication with the model and is specifically used for streaming responses,
Parameters:
- The function takes several parameters:
id
(integer): An identifier for the stream.prompts
(YailList): A list of prompts (messages) that constitute the conversation with the model.model
(String): The model code used for the conversation.apiKey
(String): The API key required for authentication.maxTokens
(integer): The maximum number of tokens for the response.temperature
(double): A value that controls the randomness of the response.
The StopStream
Block and the associated StoppedStream
event are used in the context of managing streaming operations in the code.
The StoppedStream
Block is an essential component in managing streaming operations and is triggered when the streaming process is manually stopped by calling the StopStream
Block
The GotStream event is fired repeatedly during an ongoing streaming conversation initiated by the SendStreamedMessage function. Each time the OpenAI API sends a chunk of the response, this event is triggered, delivering the partial response content along with updated token usage information.
Parameters:
-
id (Number - int): A user-defined identifier that was originally passed to the SendStreamedMessage function. This ID helps to distinguish between different streaming conversations if multiple are happening concurrently.
-
stream (String): The partial text content of the response received in the current chunk from the OpenAI API.
-
completionTokens (Number - int): An approximate number of tokens in the current stream chunk. This value is calculated using a basic word-count method similar to the prompt token approximation.
-
totalTokens (Number - int): The running total of estimated tokens (prompt + completion) up to this point in the stream. The prompt tokens are approximated before sending the request, and the completion tokens are accumulated as each chunk is received.
The FinishedStream
event is used to notify when all chunks of a stream have been returned through the GotStream
event, indicating the completion of the streaming conversation.
RequestModeration
Description: This function asynchronously requests content moderation using the OpenAI Moderation API. It takes an API key and input text as parameters, sends a POST request to the API endpoint, and processes the response.
Parameters:
apiKey
(String): The API key for accessing the OpenAI Moderation API.input
(String): The input text or content to be moderated.
ModerationResult
Description: This event is triggered when the moderation result is received from the OpenAI Moderation API. It provides information about whether the content is flagged, categories, and category scores as parameters.
Parameters:
flagged
(boolean): Indicates whether the content is flagged.categories
(String): JSON representation of the detected categories.categoryScores
(String): JSON representation of the scores for each category.
Usage: Handle this event to perform actions based on the moderation result, such as updating the user interface or taking appropriate actions based on the moderation outcome.
RequestAudioSpeech
Function
Description: This function is responsible for asynchronously requesting audio speech synthesis from OpenAI's Audio Speech API. It takes various parameters such as API key, input text, model, voice, folder path, and file name. The resulting MP3 content is then written to a file.
Parameters:
-
apiKey
(String): The API key for accessing OpenAI's Audio Speech API. -
text
(String): The input text to be synthesized into speech. -
model
(String): The model to be used for speech synthesis. models tts-1 or tts-1-hd -
voice
(String): The voice to use when generating the audio. Supported voices arealloy
,echo
,fable
,onyx
,nova
, andshimmer
-
folderPath
(String): The path to the folder where the MP3 file will be saved. -
fileName
(String): The name of the MP3 file to be saved.
Exambles -
Alloy :
- Echo :
- Fable :
- Onyx :
You can try other voices
SpeechFileSaved
Event
Description: This event is fired when the MP3 file has been successfully saved. It provides the file path as a parameter.
Parameters:
filePath
(String): The path where the MP3 file has been saved.
Usage: Handle this event to perform actions after the MP3 file has been successfully saved.
SpeechSynthesisError
Event
Description: This event is fired when an error occurs during the audio speech synthesis process. It provides an error message as a parameter.
Parameters:
errorMessage
(String): The error message describing the issue encountered.
Usage: Handle this event to capture and handle errors during the speech synthesis process.
RequestAudioTranscription
Block is responsible for making a request to OpenAI's Audio Transcriptions API to transcribe audio from a provided audio file. (Transcribes audio into the input language.)
The Blcok takes four parameters:
-
apiKey
(API key for authentication), -
audioFilePath
(path to the audio file to be transcribed), -
model
(model configuration), you can set it aswhisper-1
-
responseFormat
( The format of the transcript output, in one of these options:json
,text
,srt
,verbose_json
, orvtt
).
The AudioTranscriptionReceived
block, its purpose is to notify the application when audio transcription data is received
The RequestAudioTranslation
block is designed to request audio translation from OpenAI's Audio Translations API and return the "text" value from the response.
Parameters:
The function takes three parameters:
-
apiKey
(API key for authorization), -
audioFilePath
(path to the audio file to be translated), -
model
(the model used for translation).
This ReturnAudioTranslation
event is triggered when the audio translation response is received.
RequestDALL_EImages
Description
This function initiates a request to the OpenAI DALL-E Images API to generate images based on a given prompt.
Parameters
-
apiKey
(String): The API key for authentication. -
model
(String): (Optional) The model to use for image generation, defaults to "dall-e-2". -
prompt
(String): A text description of the desired image(s) (Required). The maximum length is 1000 characters for dall-e-2 and 4000 characters for dall-e-3. -
n
(int): (Optional) The number of images to generate, defaults to 1. Must be between 1 and 10. For dall-e-3, only n=1 is supported. -
size
(String): (Optional) The size of the generated images, defaults to "1024x1024
". Must be one of "256x256
", "512x512
", or "1024x1024
" for dall-e-2. Must be one of "1024x1024
", "1792x1024
", or "1024x1792
" for dall-e-3 models.
Events
DALL_EImagesGenerated
(List imageUrls): Fired when the DALL-E Images API successfully generates images. Returns a list of image URLs.
DALL_EImagesError
(String errorMessage): Fired when an error occurs during the DALL-E Images API request. Returns an error message.
Function: RequestChatGPTVision(String apiKey, String imageUrl, String prompt)
Purpose: This function sends a request to OpenAI's ChatGPT vision API to analyze an image and provide insights based on the given prompt.
Parameters:
-
apiKey: Your OpenAI API key.
-
imageUrl: The URL of the image to analyze.
-
prompt: A text prompt to guide the analysis (e.g., "What's in this image?").
Function: RequestChatGPTVisionMultipleImages(String apiKey, YailList imageUrls, String prompt)
Purpose: This function sends a request to OpenAI's ChatGPT vision API to analyze multiple images and provide insights based on the given prompt.
Parameters:
-
apiKey: Your OpenAI API key.
-
imageUrls: A YailList containing the URLs of the images to analyze.
-
prompt: A text prompt to guide the analysis (e.g., "Compare these images").
RequestChatGPTVisionFromFile
Purpose: Analyzes a single local image file and provides insights based on a text prompt.
Parameters:
-
apiKey: Your OpenAI API key.
-
imagePath: The file path of the image to analyze.
-
prompt: A text prompt to guide the analysis (e.g., "What's in this image?").
-
detail: The desired level of detail for the analysis (
low
,high
, orauto
). -
maxTokens: The maximum number of tokens allowed in the API response.
RequestChatGPTVisionMultipleImagesFromFile
Purpose: Analyzes multiple local image files and provides insights based on a text prompt.
Parameters:
-
apiKey: Your OpenAI API key.
-
imagePaths: A YailList containing the file paths of the images to analyze.
-
prompt: A text prompt to guide the analysis (e.g., "Compare these images").
-
detail: The desired level of detail for the analysis (e.g., "high").
-
maxTokens: The maximum number of tokens allowed in the API response.
Events:
-
ChatGPTVisionResponseReceived : This event is fired when the API response is successfully received and parsed. It provides the following parameters:
-
id: The unique ID of the response.
-
object: The type of object returned ("chat.completion").
-
model: The model used to generate the response.
-
role: The role of the response ("assistant").
-
content: The main content of the response, containing the analysis of the image.
-
-
ChatGPTVisionError(String errorMessage) : This event is fired if an error occurs during the API request. It provides the error message.
-
The response content will vary depending on the image and the prompt provided.
ChatGPT Extension- Embeddings Functionality
1. GetEmbeddings(String apiKey, String text, String model)
-
Description: This function sends a request to OpenAI's Embeddings API to get the numerical representation (embedding) of a given text.
-
Parameters:
-
apiKey: Your OpenAI API key (required for authentication).
-
text: The text string you want to embed.
-
model: The specific embedding model you want to use (e.g., "text-embedding-ada-002, text-embedding-3-small ,text-embedding-3-lar").
-
-
Functionality:
-
It constructs an API request with your text and the chosen model.
-
It sends this request to OpenAI's server.
-
It then calls the processEmbeddingsAPIResponse function to handle the server's response.
-
-
Events Triggered:
-
EmbeddingsReceived: Fired upon a successful response, containing the embeddings.
-
EmbeddingsError: Fired if an error occurs during the request.
-
2. EmbeddingsReceived(String embeddings)
-
Description: This event is fired when the GetEmbeddings function successfully receives a response from the OpenAI API.
-
Parameter:
- embeddings: The text's embedding, returned as a string representation of a list of numbers.
3. EmbeddingsError(String errorMessage)
-
Description: This event is fired when an error occurs at any point during the embedding request process.
-
Parameter:
- errorMessage: A descriptive error message to help you understand the issue.
preview :
I also use this Extension in this project :
Aix file :
you can purchase the AIX and the AIA file from here via PayPal the two files cost 5$ after you pay you will be automatically redirected to the download URL of the extension