[PAID] :google:emini Extension to interact with the Gemini-pro and PaLM 2 models from :google:oogle

[_Ahmed ((Mr. Koder)‬‏)]

The Gemini extension for AI2 interacts with Google Gemini-Pro & Gemini-Pro-Vision models (used by Bard) for text generation and streaming.

Features of the Gemini Extension for AI2:

• Gemini API Text Generation: Use various Gemini models.
• Image Generation: Create new images from text prompts.
• Image Editing: Modify existing images via text (path/Base64).
• Streaming Text Generation: Real-time, interactive responses.
• Vision Support: Text from images, video thumbnails, PDFs (URL).
• Code Execution (Optional): Run code within API responses.
• Structured JSON Output: Define schemas for predictable results.
• File Handling: Base64 encoding (optimized/standard), path/URI handling, MIME detection.
• PaLM API Integration: Basic text generation via PaLM.
• List Models: Fetch available Gemini model names.
• Stream Control: Start/Stop streaming.
• Error Handling: Events for API/file/JSON errors, stream completion/stop.
• Asynchronous Operations: Keeps UI responsive.

Benefits:

• Integrate advanced AI text/multimodal generation into AI2 apps easily.
• Enhance user engagement with streaming & interactive features.
• Generate diverse content (text, code, based on images/videos/PDFs).
• Reliably process/generate structured data using JSON Schema.
• Work seamlessly with local files for richer AI interactions.
• Flexibility: Choose between Gemini and PaLM APIs/models.
• Easy to use in App Inventor, extensible.

Use Cases:

• Chatbots: Understand text/images for context-aware conversations.
• Content Tools: Generate articles, posts, stories (with/without image prompts).
• Media Analysis: Analyze images/videos, generate descriptions.
• Doc Processing: Process PDFs from URLs for summaries/Q&A.
• Code Tools: Generate/assist code (optional execution).
• Data Structuring: Extract info into structured JSON via schemas.
• Edu/Creative: Interactive learning, story generation.
  The potential is vast!

Blocks

Explanation

---

Generating Content (Non-Streaming)

Use GenerateGeminiContent block.

• modelName (String): Gemini model (e.g., “gemini-1.5-flash”). See docs.
• apiKey (String): Your Google API key.
• contents: List of conversation turn dictionaries (role, parts list with text).

Blocks example:

---

---

RespondedToGemini event handles the response.

• apiResponse: Raw API response.
• textParts: List of generated text strings.
• role: Response role.
• finishReason: Why generation stopped.
• index: Content index.
• safetyRatings: List of safety rating dicts (category, probability).

---

---

Function: StreamGenerateGeminiContent

Stream content using Gemini API, with optional Code Execution.

Parameters:

• contents: List of conversation turn dictionaries.
• apiKey (String): Your Google API key.
• modelName (String): Gemini model (e.g., “gemini-1.5-flash”). See docs.
• enableCodeExecution (boolean): Enable code execution.
  
  
  

Blocks example

Functionality: Streams responses via SSE. Triggers GotGeminiStream (text/code chunks), StreamFinished (completion), ErrorOccurred (errors).
Callbacks: GotGeminiStream(textValue), StreamFinished(), ErrorOccurred(errorMessage, component).
Notes: For streaming text/code. Handles multi-turn/images. Use events for results/errors. Needs internet & API key.

---

---

GotGeminiStream` event.

Triggered with each chunk of streamed data.

• text: Generated text chunk (String).

StopStream block manually stops. StoppedStream event fires when stopped.

IsStreaming block checks if a stream is active.

---

---

Function: GenerateGeminiThinkingContent

Generate content using Gemini 1.5 Flash model (non-streaming).

• prompt (String): Text prompt.
• apiKey (String): Google API key.

Triggers GotGeminiStream with full response or ErrorOccurred.

---

---

Function: StreamGenerateGeminiThinkingContent

Stream content using Gemini 1.5 Flash model. Retrieves in chunks.

• prompt (String): Text prompt.
• apiKey (String): Google API key.

Triggers GotGeminiStream, StreamFinished, and ErrorOccurred.

---

Function: StreamGenerateContentFromPdfUrl

Stream content based on a PDF file from a URL.

• pdfUrl (String): URL of PDF.
• prompt (String): Text prompt related to PDF.
• apiKey (String): Google API key.
• modelName (String): Gemini model (e.g., “gemini-pro-vision”).

Handles download, upload, streaming. Triggers GotGeminiStream, StreamFinished, ErrorOccurred.

---

---

Function: StreamGenerateGeminiStructuredContent

-----------------------

Stream structured JSON content matching a schema.

• contents (List): Conversation turns.
• apiKey (String): Google API key.
• modelName (String): Gemini model (e.g., “gemini-pro”).
• scheme (String): JSON Schema string (use CreateJsonSchema).

Triggers GotGeminiStream (JSON chunks), StreamFinished, ErrorOccurred.

---

Function: CreateJsonSchema

Builds a JSON Schema string.

• propertyNames (List): Property names.
• propertyTypes (List): Corresponding types (“string”, “number”, etc.).
• propertyDescriptions (List): Property descriptions.
• requiredProperties (List): Required property names.

Returns formatted JSON Schema string or empty string on error (triggers ErrorOccurred).

---

Generating Content with Images (Streaming)
Use StreamGenerateGeminiVisionContent.

• contents: List of dicts (can include text and inlineData parts with image Base64).
• apiKey: Google API key.

Blocks example:

Results arrive via the GotGeminiStream event.

StreamGenerateGeminiFileContentFromBase64

Streams content based on Base64 files and text.

• apiKey (String): Google API key.
• modelName (String): Gemini model. See docs.
• fileBase64List (List): Base64 file strings.
• mimeTypeList (List): Corresponding MIME types.
• additionalText (String): Text prompt.
  
  
  

GenerateImage

Creates image from text.

• prompt (Text): Description.
• apiKey (Text): Google API Key.
• modelName (Text): Image generation model. Check Google docs.
  
  

EditImage

Modifies image from Base64.

• prompt (Text): Instructions.
• inputImageBase64 (Text): Base64 image string.
• inputMimeType (Text): Input MIME type.
• apiKey (Text): Google API Key.
• modelName (Text): Image editing model.
  
  

EditImageFromPath

Modifies image from file path.

• prompt (Text): Instructions.
• inputImagePath (Text): Path to image file.
• apiKey (Text): Google API Key.
• modelName (Text): Image editing model.
  
  

EditMultipleImagesSimple

Advanced edit/gen using multiple images (URL/Path/Base64).

• prompt (Text): Instructions.
• imageSourceStrings (List): Image sources.
• apiKey (Text): Google API Key.
• modelName (Text): Multi-image model.
  
  

DisplayBase64Image

Helper to display Base64 on Image component.

• base64Data (Text): Base64 data.
• mimeType (Text): Image MIME type.
• imageComponent (Component): Target Image component.
  
  

Event: GotImageResponse

Fires on successful image task completion.

• imageBase64 (Text): Result image Base64.
• mimeType (Text): Result MIME type.
• responseText (Text): API text response.
• rawApiResponse (Text): Full JSON response.
• imagePath (Text): Path where result was saved (ASD).

Examples of generating and editing with Gemini

(upload://2O0F26UstT6K9AAcJOOUHWWdv87.jpeg)

• StreamGenerateContentFromLocalVideoPath
  • Parameters:
    • videoPath (String): The local file path to a video file.
    • prompt (String): The text prompt related to the video content.
    • apiKey (String): Your Google AI API Key.
    • modelName (String): The Gemini model to use.
    • systemInstructionsValue (String): Optional system instructions.
    • jsonSchemaString (String): Optional JSON schema for structured output.
  • Description: Uploads a local video file using the File API, polls until the file is processed (“ACTIVE”), and then starts a streaming request based on the video content and prompt. Optionally includes system instructions and/or requests structured output via a JSON schema. Response chunks arrive via GotGeminiStream. Triggers StreamFinished when done or ErrorOccurred on failure.

• StreamGenerateContentFromLocalVideoPathWithInstructions
  • Parameters:
    • videoPath (String): The local file path to a video file.
    • prompt (String): The text prompt related to the video content.
    • apiKey (String): Your Google AI API Key.
    • modelName (String): The Gemini model to use.
    • systemInstructionsValue (String): Optional system instructions.
  • Description: Similar to StreamGenerateContentFromLocalVideoPath, but only includes the option for system instructions (no structured output schema). Uploads the video, waits for processing, then starts the streaming request. Response chunks arrive via GotGeminiStream. Triggers StreamFinished when done or ErrorOccurred on failure. Uses standard Designer Properties for generation config.

• StreamGenerateContentFromYouTubeUrl (Overload 1 - Basic)
  • Parameters:
    • youtubeUrl (String): Public URL of a YouTube video (including Shorts).
    • prompt (String): Text prompt relating to the video.
    • apiKey (String): Your Google AI API Key.
    • modelName (String): The Gemini model to use.
  • Description: Starts a streaming analysis request using a YouTube URL and prompt. Uses default generation settings from Designer Properties. Response chunks arrive via GotGeminiStream. Triggers StreamFinished when done or ErrorOccurred on failure.

• StreamGenerateStructuredContentFromYouTubeUrl (Overload 2 - Advanced)
  • Parameters:
    • youtubeUrl (String): Public URL of a YouTube video (including Shorts).
    • prompt (String): Text prompt relating to the video.
    • apiKey (String): Your Google AI API Key.
    • modelName (String): The Gemini model to use.
    • systemInstructionsValue (String): Optional system instructions.
    • jsonSchemaString (String): Optional JSON schema for structured output.
  • Description: Starts a streaming analysis request using a YouTube URL and prompt. Optionally includes system instructions and/or requests structured output via a JSON schema. Response chunks arrive via GotGeminiStream. Triggers StreamFinished when done or ErrorOccurred on failure.

• GenerateSingleSpeakerAudio

  • Parameters:

    • text_input (String): The text content to be converted into speech. This can include natural language prompts to guide the style, accent, pace, and tone (e.g., “Say cheerfully: Have a wonderful day!”).

    • api_key (String): Your Google AI API Key (used to initialize the client).

    • model_name (String): The specific Gemini model to use for speech generation (e.g., “gemini-2.5-flash-preview-tts”, “gemini-2.5-pro-preview-tts”).

    • voice_name (String): The desired prebuilt voice for the audio output (e.g., ‘Kore’, ‘Puck’, ‘Zephyr’). A list of available voices can be found in the Gemini API documentation.

    • output_filename (String): (Optional) The desired filename to save the generated audio (e.g., “output.wav”). The method of saving might vary based on implementation.

  • Description: Converts a given text input into audio spoken by a single synthesized voice. The API allows for control over the speech style through prompts and selection from a variety of prebuilt voices. The generated audio can then be streamed or saved to a file.

---

• Single audio examples

with style (whispering)

with style (Acting)

without style

• GenerateMultiSpeakerAudio

  • Parameters:

    • script_input (String): A text script that includes dialogue for multiple speakers. Speaker names should be clearly indicated in the script (e.g., “Joe: Hello! Jane: Hi there!”). This input can also include natural language prompts to guide the style and tone for each speaker (e.g., “Make Speaker1 sound tired and Speaker2 sound excited: Speaker1: … Speaker2: …”).

    • api_key (String): Your Google AI API Key (used to initialize the client).

    • model_name (String): The specific Gemini model to use for speech generation (e.g., “gemini-2.5-flash-preview-tts”, “gemini-2.5-pro-preview-tts”).

    • speaker_configurations (List of Objects): A list where each object defines a speaker and their voice. Each object should contain:

      • speaker_tag (String): The identifier for the speaker as used in the script_input (e.g., “Joe”, “Speaker1”).

      • voice_name (String): The desired prebuilt voice for this specific speaker (e.g., ‘Kore’, ‘Puck’).

    • output_filename (String): (Optional) The desired filename to save the generated multi-speaker audio (e.g., “dialogue.wav”). The method of saving might vary based on implementation.

  • Description: Generates audio from a text script involving up to two distinct speakers. Each speaker can be assigned a unique prebuilt voice. The API supports prompts within the script to control the style, tone, and delivery for each speaker individually. The output can be streamed or saved.

---

blocks examble

for detailed guide how to use GeminiTTS visit this guide

---

• Multible audio examle

GetGeminiModelNames

Retrieves available Gemini model names.

• apiKey (String): Google API key.

Events:

• GotGeminiModelNames(modelNames as List): Triggered on success with list of names.

• ErrorOccurred(message, component): Triggered on API request error.

Encoding Images to Base64
EncodeImageToBase64 block encodes image path to Base64 (no line breaks).

• imagePath: Path to image file.

Returns Base64 string.

Error Handling

ErrorOccurred event signals errors.

• message: Error description.
• component: Component source of error.

---

Examples

Generate text (non-streaming):

---

Generate text (streaming):

Generate text with images (streaming):

Generate text with images in FreeForm Prompt (streaming):
Use TextFormater extension for layout.

Freeform preview:

---

Applications using this extension:

{ChatGPT & DALLE2 & Gemini}[paid]: The Ultimate AI Chatbot Solution with Free Quotes and API Service Chat completions API

[Description of image] Chat GPT APP in this Android app is an exciting feature that allows users to chat with an AI language model within the app chatGPT, In addition to integrating Chat GPT API, the Android app also has the ability to generate custom images using DALL-E 2. DALL-E 2 is an advanced image generation model that uses the text input from the user to generate a unique and realistic image. features easy to use Chat completions (Chat with connected messages) you can start w…

Videos preview:

Aix_file:

Compare PAID vs FREE versions

PAID_file
Price: $5.99
Purchase: PayPal Link or You can pay HERE using your credit card or You can pay HERE using your credit card
In both cases, you will be redirected to the download page after successful payment. Contact me for issues.

FREE_file
Gemini.aix (11.6 KB)

Have Inquiries?
Contact via PM on Telegram.

Note :

Get your Gemini API key from Google AI Studio.

[AyanDeveloper]

Wow Nice Extension and nice work

[Okeditse_Nare]

Can you please get back to me on the features i asked for because i have a deadline and i don’t know if you’re working on it or not, kindly respond asap. Thanks

[_Ahmed ((Mr. Koder)‬‏)]

Thank you very much

[_Ahmed ((Mr. Koder)‬‏)]

check your PM please

[_Ahmed ((Mr. Koder)‬‏)]

@_Ahmed

PaLM_2 blocks

PaLM 2 NewBlocks added

[Lima1 (Lima)]

I purchased the Gemini extension for AI2 from the MIT APP Inventor community, but PayPal did not redirect me to download the file. The purchase was made on May 17th of this year. I have already left some messages in the MIT APP Inventor community. What is the procedure for me to download the file?

[_Ahmed ((Mr. Koder)‬‏)]

I think your problem solved right now
and thanks for your report.

[_Ahmed ((Mr. Koder)‬‏)]

New update for the Extension to meet the latest updates of the Gemini API .

Here’s a summary of the updates made to the Gemini.aix compared to the initial version.

1. Model Selection:

• The GenerateGeminiContent, StreamGenerateGeminiContent, and functions now all accept a modelName parameter, allowing the user to specify which Gemini model to use for the request. This provides flexibility in choosing the appropriate model for different tasks.

2. StreamGenerateGeminiFileContentFromBase64 Function:

• New Function: A new function called StreamGenerateGeminiFileContentFromBase64 has been added.
  

  
  
  
  

• Base64 File Input: This function accepts a list of Base64 encoded files (fileBase64List) and a corresponding list of MIME types (mimeTypeList).

• Generic File Handling: It handles various file types (not just images) by using the MIME type information.

• Streaming Response: It uses streaming to receive the response from the Gemini API and triggers the GotGeminiStream event for each chunk of text received.

3. GetGeminiModelNames Function:

• New Function: A new function called GetGeminiModelNames has been added.

• Retrieving Model Names: It retrieves a list of available Gemini model names from the API and triggers the GotGeminiModelNames event with the list.

4. GetFilePathFromDataURI Function:

• New Function: A new function called GetFilePathFromDataURI has been added.

• Data URI to File Path: It converts a Data URI (representing a file) to a local file path. It handles content://, file://, and data:// URI schemes.

5. getMimeType Function:

• New Function: A new function called getMimeType has been added.

• Get MIME Type: It takes a file path as input and returns the MIME type of the file using Files.probeContentType(path).

6. Code Cleanup and Improvements:

• Removed Redundant Parameter: The contents parameter in the StreamGenerateGeminiVisionContentFromPathsAndText function was removed as it became unnecessary after adding separate parameters for images and text.

• Error Handling: The code now includes more robust error handling, using try-catch blocks and triggering the ErrorOccurred event when necessary.

Overall, the updated code is more versatile, efficient, and user-friendly:

• More Features: It provides functions to retrieve model names, handle various file types, and work with Data URIs.

• Flexibility: Users can now choose specific Gemini models and send different file types to the API.

• Efficiency: Streaming responses allow for better handling of large data.

• Improved Usability: The code is more organized and includes better documentation and error handling.

These updates enhance the functionality and make the extension more useful for a wider range of applications within App Inventor.

[AyanDeveloper]

Text response like this

Hi! \n\nHow can I help you today? \n

Why

[anon92712962]

Just enable that html text(something like that, can’t recall the exact thing) checkbox in properties. It’ll enable this and you’ll be able to see a new line instead of /n

[_Ahmed ((Mr. Koder)‬‏)]

As @anon92712962 said you can enable html

[AyanDeveloper]

Same problem and

[_Ahmed ((Mr. Koder)‬‏)]

What is the problem
Do you mean in the code section
This is not from my extension code this Is from the API it self

[AyanDeveloper]

@_Ahmed

from the API it self

When I use stream then I got, code and \n problem

It’s parfact with without streaming

[_Ahmed ((Mr. Koder)‬‏)]

I will check it and confirm what I got
And thanks for informing me

[_Ahmed ((Mr. Koder)‬‏)]

Problem solved now and thanks @AyanDeveloper for informing about this problem

[_Ahmed ((Mr. Koder)‬‏)]

@AyanDeveloper

Screenshot_2024-07-02-19-54-32-62_bcf19fdaab5f528e1a140238641ca6331080×2412 162 KB

did you create this app with this UI design ?

[AyanDeveloper]

Yes i made using dynamic

[_Ahmed ((Mr. Koder)‬‏)]

congrats! This is a cool UI design that attracts my attention
what tools that you use to add code viewer in the dynamic component like in this image

@AyanDeveloper