Visible Extension Components Are They Possible for Kodular?

Hello Kodular developers and extension developers :waving_hand:

I would like to discuss a feature that could significantly expand what Extension developers can build in Kodular:

True Visible Extension Components

I am not talking about Dynamic Components.

I mean a real Extension Component that behaves similarly to a native Kodular visible component.

For example:

AdvancedButton.aix
↓
Import Extension into Kodular
↓
AdvancedButton appears in the Palette
↓
Drag AdvancedButton into the Screen
↓
AdvancedButton is visible in the Designer Viewer
↓
Properties can be edited in Designer
↓
Blocks and Events are available
↓
Build APK
↓
A real Android View is created at runtime

For example:

AdvancedButton
β”œβ”€β”€ Text
β”œβ”€β”€ TextColor
β”œβ”€β”€ TextSize
β”œβ”€β”€ BackgroundColor
β”œβ”€β”€ CornerRadius
β”œβ”€β”€ Icon
β”œβ”€β”€ Enabled
└── Click


What I discovered

I found that this is not a completely new idea.

There was already a GSoC 2020 project called Visible Component Extension (VCE) for MIT App Inventor.

The project’s goal was specifically to allow visible components to be created as extensions, so that they could look almost identical to built-in visible components.

The project actually demonstrated a β€œSimpleLabel” mock.

The implementation showed that an extension could have a JavaScript mock which created a DOM element and reacted to Designer property changes.

For example, the old proof of concept contained the concept:

class MockSimpleLabel extends MockVisibleExtension {

static TYPE = "SimpleLabel";

constructor(editor) {
    super(editor, MockSimpleLabel.TYPE);

    this.label = document.createElement("p");

    this.initComponent(this.label);
}

onPropertyChange(propertyName, newValue) {

    switch (propertyName) {

        case "Text":
            this.label.textContent = newValue;
            break;
    }
}

static create(editor) {
    return new MockSimpleLabel(editor);
}

}

MockComponentRegistry.register(
MockSimpleLabel.TYPE,
MockSimpleLabel.create
);

The original discussion even shows that the β€œSimpleLabel” mock could be placed in the Designer and its β€œText” property changed successfully.

This is documented in the original VCE discussion:

The relevant implementation discussion and example are around the July/August 2020 updates.


A newer and much more interesting attempt

In 2024, a new GSoC project was proposed:

Mock Preview Support for Extensions

The official discussion:

The project explicitly states that:

Β«Extensions currently cannot have mock previews.Β»

The goal was to allow extension developers to provide their own mock preview.

The proposed system uses a JavaScript API for creating mocks and also discusses security, external CSS, and network access.

This is extremely close to what I believe Kodular could use.


The actual implementation exists as PR #3200

The most important resource I found is the actual App Inventor source-code PR:

PR #3200 β€” Add mock preview support for extensions

Opened:

15 July 2024

The PR currently remains open.

It contains approximately:

+714
-134
35 commits
30 files changed

More importantly, this is not just documentation.

The PR contains actual implementation work including:

  • β€œMockVisibleExtension”
  • mock script loading
  • packaging mock scripts with extensions
  • a test mock for visible extensions
  • Dedicated Worker support
  • Shadow DOM
  • mock sanitization
  • external CSS support
  • initial property handling
  • error handling
  • extension upgrade handling
  • mock script registration
  • mock files stored in extension resources

The GitHub PR can be inspected here:


The architecture is very interesting

The proposed architecture separates the Designer representation from the Android runtime component.

Something like:

             Extension
                 β”‚
      β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
      β”‚                     β”‚
   Runtime                 Mock
      β”‚                     β”‚
 Java / Kotlin         JavaScript
      β”‚                     β”‚
      ↓                     ↓
Android View          Designer View
      β”‚                     β”‚
      ↓                     ↓
     APK              Kodular Viewer

This makes sense because Kodular Creator runs in the browser, while the actual Extension runs inside the Android application.


XML is NOT the missing part

I initially thought FAST’s XML support might solve this.

For example, we can easily create an Android layout:

This can be used by the runtime Android component.

But the Kodular Designer cannot simply load an Android XML β€œButton”.

The Designer needs a browser-side representation.

Therefore we need something like:

Runtime:
Java/Kotlin + Android XML
↓
Real Android View

Designer:
Mock JavaScript
↓
Designer Preview


FAST could still be very useful

FAST is already a powerful Extension build system:

It supports many features useful for a future Visible Extension, including modern Java, Kotlin, Maven/Gradle dependencies, AARs, manifest merging, R8, ProGuard, resource handling, etc.

But I don’t think FAST itself is the missing piece.

The missing piece is:

Kodular Creator
↓
Extension Mock Support
↓
Visible Extension

FAST would then be responsible for building/packaging the runtime extension and potentially its mock resources.


What I would like to know from the Kodular developers

Could the implementation from MIT App Inventor PR #3200 be adapted to Kodular Creator?

Specifically:

Does Kodular Creator already have enough of the Mock Component infrastructure to implement this?

Could β€œMockVisibleExtension” from the App Inventor work be ported to Kodular?

Could a β€œ.aix” contain something like:

AdvancedButton.aix
β”‚
β”œβ”€β”€ Runtime classes
β”‚
β”œβ”€β”€ component metadata
β”‚
β”œβ”€β”€ Android resources
β”‚
└── mock.js

and Kodular Creator automatically load the β€œmock.js” when the extension is imported?

Could the extension metadata contain something like:

visible = true
mock = β€œAdvancedButton.mock.js”

?

Could FAST eventually package this mock file automatically?

For example:

src/
β”œβ”€β”€ AdvancedButton.java
└── AdvancedButton.mock.js

and then:

fast build

produces:

AdvancedButton.aix

with both the runtime and mock included?


Security is obviously an important issue

I understand why this feature is not trivial.

A mock is JavaScript running inside the Designer.

An untrusted extension should not be allowed to freely manipulate the Kodular Creator page, access sensitive data, inject scripts, etc.

Interestingly, the 2024 PR already experimented with several approaches, including:

Dedicated Worker
Shadow DOM
Sanitization
DOMPurify
Linkedom

The PR history specifically includes commits for:

add wrapper for DedicatedWorker
add DOMPurify and linkedom
perform sanitization of mock
render the mock inside Shadow DOM
add external CSS support

This makes PR #3200 particularly interesting as a starting point.


What about containers?

A simple component such as:

AdvancedButton
AdvancedLabel
AdvancedImage
AdvancedSlider

should be relatively straightforward.

But it becomes more interesting for:

AdvancedCard

containing:

Image
Label
Button

The old VCE discussion already identified this issue and introduced the concept of β€œMockContainer” in addition to β€œMockVisibleComponent”.

See the original discussion:


Example of what I want to achieve

Imagine an extension:

AdvancedButton.aix

After importing it:

Kodular Palette
β”‚
β”œβ”€β”€ User Interface
β”œβ”€β”€ Layout
β”œβ”€β”€ Media
└── Extensions
└── AdvancedButton

Drag it into the Viewer:

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ β”‚
β”‚ Advanced Button β”‚
β”‚ β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Change:

Text = β€œHello”

and the Designer mock immediately becomes:

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ Hello β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Then when the APK is built:

Mock JavaScript
↓
NOT used in the Android APK

Java/Kotlin Runtime
↓
Real Android View


Why this would be valuable

It would allow Extension developers to create completely new UI components without having to rely on Dynamic Components or hacks.

Examples:

AdvancedButton
AdvancedCard
CustomSlider
CustomProgressBar
CustomChart
CustomVideoView
CustomCameraView
CustomMapView
CustomInput
CustomRecyclerView

The extension developer would control both:

Designer representation

and:

Android runtime representation

while Kodular would remain responsible for the overall Designer/Blocks architecture.


Important distinction

This proposal is not:

:cross_mark: Dynamic Components

:cross_mark: An extension that creates a Button at runtime

:cross_mark: A WebView pretending to be a component

:cross_mark: A normal non-visible extension controlling an existing Button

It is:

:white_check_mark: A real Extension Component

:white_check_mark: Appears in the Palette

:white_check_mark: Can be dragged into the Designer

:white_check_mark: Has a Designer mock

:white_check_mark: Has Properties

:white_check_mark: Has Blocks/Events

:white_check_mark: Has a real Android implementation at runtime


Main resources

  1. VCE β€” 2020
  1. Mock Preview Support β€” 2024
  1. Actual implementation β€” PR #3200
  1. FAST
  1. Example of the old VCE implementation

Final question

Could the Kodular team/developers investigate whether the work from App Inventor’s Mock Preview Support PR #3200 can be ported or adapted to Kodular?

If someone has already experimented with this, has a fork/branch, or knows which parts of Kodular Creator need to be changed, I would really appreciate any information.

I would also be interested in helping test a proof of concept such as:

AdvancedButton.aix
+
AdvancedButton.mock.js
↓
Kodular Creator
↓
Palette
↓
Designer Viewer
↓
Properties + Blocks
↓
APK
↓
Real Android AdvancedButton

I believe this could open a completely new level of Extension development for Kodular.

@Diego
@Ibrahim_Jamar
@vknow360
@JEWEL
@Taifun
@eAcademy_tech1
Thankss for sharing your opinion on the topic

I think Kodular probably won’t implement it until AppInventor merges it.

But I’ve already put this into FAST. Once AppInventor merges it, you can build a visible component extension with the current version of FAST.

2 Likes

Thanks for the clarification, Jewel! That’s actually great news that FAST already has support for this

I’m really curious about the App Inventor side now so do you happen to know if there has been any progress on the mock preview PR or if the App Inventor team has discussed when it might be merged??
It would be really exciting to finally have proper visible extension components in Kodlar I’d love to experiment with something simple like a custom Button or Label once the required support is available

Ain’t these features already exist in Kodular? Eg [F/OS] ChatKaroUI Extension - Advance Chat with text, images and messages support

1 Like

To get updates from AppInventor, continue the discussion here: Request for Comments – Adding support for Mock Previews in Extensions - Google Summer of Code - MIT App Inventor Community

2 Likes