[PAID] LocalAI: Run Large Language Models (LLMs) completely offline ($10 or INR 701)

vknow360 · August 26, 2026, 6:56pm

LocalAI `Extension`

Bring the power of ChatGPT directly to your app - 100% offline! This extension allows you to seamlessly integrate Large Language Models (LLMs) right on the user's Android device. Provide your users with smart, conversational AI without worrying about API limits, cloud hosting costs, or internet connectivity.

Key Features:

Fast Responses: Highly optimized for mobile processors to ensure lightning-fast generation and optimal response times (minimal delay).
Real-time Typewriter UI: Built-in token streaming events let you easily create beautiful, character-by-character typing effects.
100% Free & Private: No API keys, no server costs, and no user data ever leaves the device.
Modern Model Support: Compatible with any standard GGUF model (Llama, Gemma, Qwen, etc.).

1. Overview

Version: 1
Released: August 26, 2026
Last Updated: August 26, 2026
Permissions: android.permission.INTERNET (only required if you choose to download models from the web)

Aix Size: 40.16 KB
(Note: Requires dropping the native liblocalai.so binary file or zip into your project's assets)

2. Blocks

3. Documentation

NativeReady	Event raised when the native AI engine library (liblocalai.so) is successfully loaded and ready.
NativeFailed	Event raised when the native library fails to load. Returns the error message and device ABI info. _{message : text}
ModelReady	Event raised when the GGUF model is fully loaded into memory and ready for text generation.
ModelFailed	Event raised when the GGUF model fails to load (e.g. invalid path, corrupt file, or insufficient memory). _{message : text}
DownloadProgress	Event raised periodically during model download with current progress percentage (0 to 100). _{percent : number}
DownloadCompleted	Event raised when the model file has finished downloading and is ready. Returns the absolute file path to be passed to LoadModel. _{path : text}
DownloadFailed	Event raised when model download or extraction fails. _{message : text}
GotResult	Event raised when text generation is complete. Returns the full generated response text. _{text : text}
TokenGenerated	Event raised in real-time each time a new token is generated. Returns the newest 'token' and the accumulated 'partialText' for building streaming typewriter UI. _{token : text} _{partialText : text}
Failed	Event raised when text generation fails, is cancelled, or encounters an error. _{message : text}
LoadNativeLibrary	Loads the native inference engine (liblocalai.so). Accepts app asset paths ('//liblocalai.so' or '//liblocalai.zip'), absolute paths ('/storage/.../liblocalai.so'), or 'file://' URIs. Zip files are automatically extracted to private storage. Fires NativeReady or NativeFailed. _{path : text}
DownloadModel	Downloads a GGUF model or a .zip containing a GGUF from a remote URL or copies it from local storage into app-specific private directory. Automatically skips download if the file already exists. Fires DownloadProgress, then DownloadCompleted or DownloadFailed. _{url : text}
LoadModel	Loads a GGUF model file into memory from an absolute path (e.g. returned by DownloadCompleted). Requires LoadNativeLibrary to be executed first. Fires ModelReady or ModelFailed. _{path : text}
UnloadModel	Unloads the currently loaded GGUF model from memory to free up RAM. The native library remains loaded so another model can be loaded quickly.
Generate	Starts generating text asynchronously from the given prompt. Fires TokenGenerated in real-time as tokens arrive, GotResult when finished, or Failed if an error occurs. _{prompt : text}
Cancel	Cancels an ongoing text generation or model download task immediately.
PromptMode	Sets the prompt formatting mode. Use 'Chat' (default) to format messages using the model's built-in chat template (System + User prompt), or 'Raw' for direct completion. Accepts: text
SystemPrompt	Sets the system instruction or persona for the AI in Chat mode (e.g. 'You are a helpful assistant.'). Accepts: text
NativeAssetsVersion	Version stamp for native library assets. Increment this value whenever you update the .so or .zip in app assets to force re-extraction. Accepts: text
MaxTokens	Sets the maximum number of tokens (words/sub-words) the model can generate in a single response. Accepts: number
Temperature	Sets the sampling temperature. Lower values (e.g. 0.2) make output focused and deterministic; higher values (e.g. 0.8) make it more creative. Accepts: number
TopP	Sets the Top-P (nucleus sampling) threshold (0.0 to 1.0). Only tokens comprising the top probability mass are considered. Accepts: number
ContextSize	Sets the context window size (number of tokens) allocated in memory. Default is 1024 (recommended for 4GB RAM devices). Accepts: number
Threads	Sets the number of CPU threads used for inference. Set to 0 for automatic detection based on available CPU cores. Accepts: number
IsModelLoaded	Returns true if a GGUF model is currently loaded in memory and ready for text generation. Accepts: boolean
IsNativeLoaded	Returns true if the native AI engine library (liblocalai.so) is loaded and initialized. Accepts: boolean
IsDownloading	Returns true while a model download or extraction is currently in progress. Accepts: boolean

4. Example Usage

Step 1: Initialize the Native Engine Before you can do anything, you must load the native C++ library. Use the LoadNativeLibrary block and pass the asset path to your binary (e.g., //liblocalai.zip or //liblocalai.so). The extension will automatically extract and load the library. Important: Always wait for the NativeReady event to fire before attempting to download or load a model!

Step 2: Download a Model
Once the native engine is ready, you need a model to run. Use the DownloadModel block and provide a direct download URL to a .gguf file. The extension will download it safely to the app's private storage (and skip downloading if it already exists).

Here are some highly recommended, mobile-friendly models you can use for testing:

Qwen 2.5 (0.5B) - Tiny & Lightning Fast: https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct-GGUF/resolve/main/qwen2.5-0.5b-instruct-q4_k_m.gguf
Llama 3.2 (1B) - Smart & Efficient: https://huggingface.co/bartowski/Llama-3.2-1B-Instruct-GGUF/resolve/main/Llama-3.2-1B-Instruct-Q4_K_M.gguf
Phi-3 Mini (3.8B) - High Quality (Requires ~3GB RAM): https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/resolve/main/Phi-3-mini-4k-instruct-q4.gguf

Tip: You can use the DownloadProgress event to show a progress bar.

Step 3: Load the Model into Memory
When the download finishes, the DownloadCompleted event will fire and provide the local path to the file on the device. Pass this path directly into the LoadModel block. Wait for the ModelReady event to confirm the model is successfully loaded into RAM.

Step 4: Generate Text
Simply call the Generate block with your user's query. Customization: Before generating, you can set the SystemPrompt property to give the AI a persona (e.g., "You are a helpful travel assistant"), and adjust properties like Temperature and MaxTokens to control the creativity and length of the response.

Step 5: Show Real-Time Streaming Responses
Use the TokenGenerated event to update your UI in real-time. This event fires every time the AI thinks of a new word. You can simply set your Label's text to the partialText variable to create a beautiful, ChatGPT-style typewriter effect. Once generation is completely finished, the GotResult event will fire.

5. Demo Video

6. Purchase Extension

UPI id: vknow360@apl

Thank you.
Hope it helps!

eAcademy · August 27, 2026, 5:24am

Great extension

Anke · August 27, 2026, 6:44am

Where can I download liblocalai.zip (or liblocalai.so)? Or is this file downloaded/provided automatically by the extension after installation?
Would it be technically feasible to add voice input/output (i.e. real voice-based communication) directly to the extension, without relying on AI2's SpeechRecognizer and TextToSpeech components? Ideally, this would use fully offline/local speech-to-text and text-to-speech models and provide sufficiently natural, low-latency voice interaction.
(Then I would also like to pay $50. )

vknow360 · August 27, 2026, 6:52am

Yes, it is provided after purchase. Size of zip file is 3 mb.

It is definitely possible but would be slow enough and may not work on low end devices.

3dmixer · August 27, 2026, 7:38am

Great extension as always. Does it support tool calling?

Anke · August 27, 2026, 8:16am

Okay, but let's talk about high-end devices (Pixel 11 Pro, ...) ...

Anke · August 27, 2026, 8:56am

Done (via PayPal).

vknow360 · August 27, 2026, 2:49pm

No, that's difficult to implement since I want to keep support for older devices.

3dmixer · August 28, 2026, 8:10am

Still cool extension. Tool calling would be nice tho. Good luck

Anke · August 28, 2026, 8:59am

I strongly advise against this extension — or, more precisely, against using the three LLMs proposed here — as it would be a complete waste of time. I thoroughly tested all three models and received virtually no correct or usable answers, regardless of the topic.

Instead, I regularly received lengthy responses containing long, nonsensical, and irrelevant commentary, even though I had explicitly asked that no comments or remarks be included.

Even translating a simple, single-line sentence from German to English — with instructions to output only the translated text — results in excessive commentary before and after the translation.

Sorry Sunny, I really appreciate your extensions and have been using them for years, but this one is unfortunately a flop.

vknow360 · August 28, 2026, 9:10am

I think there is a small misunderstanding here.

This extension itself is not an LLM. Its purpose is to allow you to run compatible LLMs locally on an Android device.

The quality of the responses, translations, and overall behavior depends primarily on the model being used, rather than the extension itself. The extension provides the interface and runtime needed to run those models completely offline on the device.

For example, a model may not have latest information about AI2 due to it being trained in 2023. The solution to this is adding tool calling feature which will increase response quality by a lot but would defy the purpose of local llm.

That said, I really appreciate you taking the time to test it and share your feedback!

Anke · August 28, 2026, 9:40am

Of course I am aware of that, but you do explicitly recommend these models (for your extension).

Anke · August 28, 2026, 9:48am

and the answer goes on and on...

vknow360 · August 28, 2026, 9:59am

This is the response from SmolLM2 360M.

However, I did optimize original inference engine by huge to shrink output native file. That might have done something with it like hallucination, repeating response, misinterpreting user message etc.

I'll try to fix these issues in v2.
There is definitely scope of improvement.

Anke · August 28, 2026, 10:05am

"The result is" 20 is also a superfluous comment.

davidefa · August 28, 2026, 10:50am

on hugginface you can check/test the models online for example

check gguf and Inference Available ( 314 models out of 200K of gguf available )
then in the page of each model you can make queries

For the other 200K models you have to download them : - (

hug011920×1080 391 KB