Working with Other Components in an Extension

[Kaustubh_Rakhade]

Hello everyone,
I started out with creating extension just few weeks ago so forgive me if I ask anything wrong.

I want to know how to interact with various components in the App Inventor.

Example:
I want to draw a point on the Canvas using a @SimpleFunction in my extension. So, I take the coordinates and the Canvas Component as the inputs.

The Canvas component has a method DrawPoint(x,y).

How can I call this function from within the extension?

---

This is how far I went.

@SimpleFunction(description = "Plots a point")
public void plot(int x, int y, canvas component) {
  component.DrawPoint(x, y);
}

[Kumaraswamy]

@SimpleFunction(description = "Draw something on canvas")
  public void DrawPoint(int xPosition, int yPosition, Component canvas) {
    Canvas canvas1 = (Canvas) canvas; 
    canvas1.DrawPoint(xPosition, yPosition);
  }

Import these things:

import com.google.appinventor.components.runtime.Canvas
import com.google.appinventor.components.runtime.Component

So this should work fine.

[ewpatton]

Note that you can specify any class that implements Component as a type for the argument, you aren't restricted to using just Component. This would eliminate the need to cast within the method body.

[TIMAI2]

How would you return the canvas as a bitmap to an extension:

e.g.

@SimpleFunction(description = "Get Canvas Image")
  public void GetCanvasImage(Component canvas) {
    Canvas canvas1 = (Canvas) canvas;
    Bitmap imageBitmap = canvas1  '??? what goes here ???'  ;

From research there is no getBitmap() function. Most of the guidance on SO seems to want you to create another canvas and set the existing canvas's bitmap to it, thereby getting a bitmap...

[MohamedTamer]

Hello @TIMAI2,
See here how Appinventor retrieves the canvas' bitmap.

[TIMAI2]

Thank you Mohamed.

I see and understand that in the Canvas component.

I do not see or understand how to bring that across from the canvas component to the extension. The variable bitmap that holds the cache does not seem to be available in canvas1. ?

[Kumaraswamy]

Tim, are you trying to get the bitmap out of the canvas component?

[TIMAI2]

Yes I am. Do not want a file, just the bitmap.

[Kumaraswamy]

I couldn't find any method that would allow extensions to get the bitmap (there may be one that I am not aware of) from the canvas component. Here is what I tried and got that working.

There may be a better way of doing this.

@SimpleFunction(
            description = "Transfer data from Canvas to an Image Component!")
    public void TransferToImage(final Canvas canvas, final Image image) throws Exception {
        final View canvasView = canvas.getView();

        final Field bitmapField = canvasView.getClass().
                getDeclaredField("bitmap");
        bitmapField.setAccessible(true);
        final Bitmap bitmap = (Bitmap) bitmapField.get(canvasView);
        ((ImageView) image.getView()).setImageBitmap(bitmap);
    }

Canvas.getView() returns a private subclass of the Canvas that is the class CanvasView at this line here.

They store the bitmap of the android.graphics.Canvas in a global field (private!).

So we could just get the view of the Canvas (CanvasView) and access the private field using reflection.

[Juan_Antonio]

Good idea @Kumaraswamy

@TIMAI2 does not need Component Image, we can try this:

    @SimpleFunction( description = "Canvas to Base 64")
    public void CanvasToBase64(final Canvas canvas) throws Exception {
        final View canvasView = canvas.getView();

        final Field bitmapField = canvasView.getClass().getDeclaredField("bitmap");
        bitmapField.setAccessible(true);
        final Bitmap bitmap = (Bitmap) bitmapField.get(canvasView);

            String encodedString = bitMapToBase64(bitmap);
            AfterPictureBase64(encodedString);
    }



  public String bitMapToBase64(Bitmap bitMap) {
        ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
        bitMap.compress(Bitmap.CompressFormat.PNG, 50, byteArrayOutputStream);
        byte[] byteArray = byteArrayOutputStream.toByteArray();
        return Base64.encodeToString(byteArray, Base64.DEFAULT);
    }

    @SimpleEvent(description = "Called after the picture is taken. The text argument `base64Data` is the base64 string of the image")
    public void AfterPictureBase64(String base64Data) {
        EventDispatcher.dispatchEvent(this, "AfterPictureBase64", base64Data);
    }

[TIMAI2]

Thank you clever people

I will try both and report back.

@Juan_Antonio - how do we indicate which canvas will return its bitmap ? (There could be more than one in the app....)

[Juan_Antonio]

[TIMAI2]

We are nearly there....

My tests will return drawn items (text,circles etc) but neither the background colour or a background image.....(I place the base64 output to another canvas as a background image)

@SimpleFunction(
          description = "Get Canvas Image and returns base64 string")
  public void GetCanvasImageAsBase64(final Canvas canvas) throws Exception {
    final View canvasView = canvas.getView();
    final Field bitmapField = canvasView.getClass().getDeclaredField("bitmap");
    bitmapField.setAccessible(true);
    final Bitmap imageBitmap = (Bitmap) bitmapField.get(canvasView);
    String encodedString = bitMapToBase64(imageBitmap);
    AfterCanvasBase64(encodedString);
  }

[TIMAI2]

Having built and tested I now see a socket is provided

[Juan_Antonio]

borrar_camara.aia (9.9 KB)

Crash when BackgroundBase64.

Edit: It works but without background.

[Kumaraswamy]

I guess the bitmap field which we are accessing is just the top layer for the Canvas.

[Juan_Antonio]

Try this

	 @SimpleFunction( description = "Canvas to Base 64.")
    public void CanvasToBase64(final Canvas canvas) throws Exception {
        final View canvasView = canvas.getView();
		final Bitmap bitmap = loadBitmapFromView(canvasView);
     //   final Field bitmapField = canvasView.getClass().getDeclaredField("bitmap");
     //   bitmapField.setAccessible(true);
    //    final Bitmap bitmap = (Bitmap) bitmapField.get(canvasView);

            String encodedString = bitMapToBase64(bitmap);
            AfterPictureBase64(encodedString);
    }
	
	
public static Bitmap loadBitmapFromView(View v) {
    Bitmap bitmap;
    v.setDrawingCacheEnabled(true);
    bitmap = Bitmap.createBitmap(v.getDrawingCache());
    v.setDrawingCacheEnabled(false);
    return bitmap;
}

Take a screenshot of a whole View

[TIMAI2]

Unfortunately, setDrawingCacheEnabled and getDrawingCache are deprecated after API 28

I have been looking at this example:

public static Bitmap loadBitmapFromView(View v) {
    Bitmap b = Bitmap.createBitmap( v.getLayoutParams().width, v.getLayoutParams().height, Bitmap.Config.ARGB_8888);                
    Canvas c = new Canvas(b);
    v.layout(v.getLeft(), v.getTop(), v.getRight(), v.getBottom());
    v.draw(c);
    return b;
}

or this

public static Bitmap getBitmapFromView(View view) {
    Bitmap returnedBitmap = Bitmap.createBitmap(view.getWidth(), view.getHeight(),Bitmap.Config.ARGB_8888);
    Canvas canvas = new Canvas(returnedBitmap);
    Drawable bgDrawable =view.getBackground();
    if (bgDrawable!=null) 
        bgDrawable.draw(canvas);
    else 
        canvas.drawColor(Color.WHITE);
    view.draw(canvas);
    return returnedBitmap;
}

If we have a second canvas (doesn't look like we can create one)......might cause a blink on screen...but need to link the second canvas in to the function.