ADB shell command to perform clicks

Hi everyone:+1:
Can I use ADB shell command directly in my app to perform clicks on screen?

adb shell input tap x y

x and y are the co-ordinates.
you can use various options in place of tap like text, keyevent

adb shell input tap 500 600 // for clicking on x and y coordinates
adb shell input text 'kodular' // for entering or typing kodular
adb shell input keyevent 4 // for sending back button keystroke
1 Like

Hey:upside_down_face:
I have tried to run a demo app with these blocks:

BLOCKS

After 5 seconds ADB command should click on button, but in Companion doesn’t happen.
So I tried to build the apk, but app crashes on start…
I tried also with the command “input tap 500 1000” (so without “ad shell”), but it’s the same.
I sure the coordinates are right because the button fill the parent both to width and height
(so it is very big) and I use this app to calculate them:

These are my AIA and apk:
AIA - DEMO.aia (2.0 KB)
APK - DEMO.apk (5.1 MB)

About Screen Title was empty in your project

Logcat
java.lang.RuntimeException: Unable to start activity ComponentInfo{io.kodular.Panzieri_ale.DEMO/io.kodular.Panzieri_ale.DEMO.Screen1}: /tmp/1632501986606_0.7559402701999748-0/youngandroidproject/../src/io/kodular/Panzieri_ale/DEMO/Screen1.yail:11:55: unbound location null
	at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3311)
	at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3460)
	at android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:83)
	at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:135)
	at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:95)
	at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2047)
	at android.os.Handler.dispatchMessage(Handler.java:107)
	at android.os.Looper.loop(Looper.java:224)
	at android.app.ActivityThread.main(ActivityThread.java:7590)
	at java.lang.reflect.Method.invoke(Native Method)
	at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:539)
	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:950)
Caused by: /tmp/1632501986606_0.7559402701999748-0/youngandroidproject/../src/io/kodular/Panzieri_ale/DEMO/Screen1.yail:11:55: unbound location null
	at gnu.mapping.Location.get(Location.java:67)
	at io.kodular.Panzieri_ale.DEMO.Screen1.lambda3(Screen1.yail:11)
	at io.kodular.Panzieri_ale.DEMO.Screen1$frame.apply0(Screen1.yail:523)
	at gnu.expr.ModuleMethod.apply0(ModuleMethod.java:186)
	at kawa.lang.Promise.force(Promise.java:27)
	at kawa.lang.Promise.force(Promise.java:37)
	at kawa.lib.misc.force(misc.scm:98)
	at io.kodular.Panzieri_ale.DEMO.Screen1.$define(Screen1.yail:10307)
	at com.google.appinventor.components.runtime.Form.onCreateFinish2(SourceFile:503)
	at com.google.appinventor.components.runtime.Form.onCreateFinish(SourceFile:447)
	at com.google.appinventor.components.runtime.Form.onCreate(SourceFile:377)
	at android.app.Activity.performCreate(Activity.java:7893)
	at android.app.Activity.performCreate(Activity.java:7880)
	at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1307)
	at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3286)
	... 11 more

i don’t know if these command will work with shell component. these commands are for adb shell (using USB / TCP). i have never tested it on shell component.

Yes, you are right.
I have added an About Screen Title and app doesn’t crash on start anymore.
Then I have tried to do this experiment, with these blocks:

BLOCKS

When I run app, button text remains default and I cannot press the button manually
(so substantially app is blocked) and after a while app crashes…
If I try with “adb shell input tap 500 1000” command app doesn’t crash and I can press button manually, but button text remain always default…
I suppose the right command is “input tap 500 1000”

These are AIA and apk:
AIA - DEMO.aia (2.2 KB)
APK - DEMO.apk (5.1 MB)

This shell component is a strange trouble​:sweat_smile:

2 Likes