🏃‍♂️ Fast : An Efficient Way to Build Extensions

JEWEL · October 21, 2024, 8:27pm

An update is available, v1.1.6 22.10.24.01.43

FAST now relies on the system's Java compiler.
The Kotlin compiler has been optimized.
A few compilation bugs have been fixed.
The Desugar tool is optimized to fully support JDK8 & 11.
The D8 tool has been optimized.
A few modifications to the logging system.

JEWEL · October 22, 2024, 6:04pm

An update is available, v1.1.7 22.10.24.21.55

The D8 tool is optimized to make it cross-platform compatible.

Today I've did some experiments with the R8 & ProGuard.

Experiments results:

Winner.
Defeated.

Task Name	R8	ProGuard
Version	4.0.63	7.6.0
Faster Compilation
Better Optimization
Better Shrinking
Better Obfuscation
Easy Configuration
Desugaring Features
Enum Obfuscation
HelperClassesObfuscation
GenerateUnreadableCode
Reduce 421KB to	250KB	185KB

Do you think that I should add the R8 tool instead of the ProGuard or as an optional tool?

Horizon · October 22, 2024, 6:40pm

You might have a way to choose an option between both, this will make Fast more fast

Patryk_F · October 23, 2024, 6:07am

If these are the results then I don't think r8 is necessary since proguard works well. After reading this article I thought the r8 was better:

What does it mean?

JEWEL · October 23, 2024, 6:28am

Previously, I made some changes to the compilation process to be compatible with older devices (ex. Windows 7). Also, these changes were required for the D8 tool.

However, I'm planning to add the R8 tool as an optional task in experimental mode. FAST will use R8 instead of ProGuard if you enable R8 from fast.yml. This may be helpful for smaller extension projects.

techxsarthak · October 23, 2024, 9:37am

You can also use Blaze IDE

JEWEL · October 24, 2024, 3:50pm

An update is available, v1.1.8 24.10.24.20.40

Added R8 tool as an optional optimizer. Use -s to run R8. (Experimental)
Changed proguard-rules.pro to be compatible with R8.
Simplified proguard-rules.pro.
A few minor bugs have been fixed.

Patryk_F · October 24, 2024, 4:17pm

What do these warnings mean during compilation and can I get rid of them?

warning: unknown enum constant AnnotationRetention.BINARY
  reason: class file for kotlin.annotation.AnnotationRetention not found
warning: unknown enum constant AnnotationTarget.FUNCTION
  reason: class file for kotlin.annotation.AnnotationTarget not found
warning: unknown enum constant AnnotationTarget.PROPERTY_GETTER
warning: unknown enum constant AnnotationTarget.PROPERTY_SETTER
warning: unknown enum constant AnnotationTarget.VALUE_PARAMETER
warning: unknown enum constant AnnotationTarget.FIELD
warning: unknown enum constant AnnotationTarget.LOCAL_VARIABLE
warning: unknown enum constant AnnotationTarget.ANNOTATION_CLASS
warning: unknown enum constant AnnotationTarget.FILE
warning: unknown enum constant AnnotationRetention.BINARY
warning: unknown enum constant AnnotationTarget.FUNCTION
warning: unknown enum constant AnnotationTarget.PROPERTY_GETTER
warning: unknown enum constant AnnotationTarget.PROPERTY_SETTER
warning: unknown enum constant AnnotationTarget.VALUE_PARAMETER
warning: unknown enum constant AnnotationTarget.FIELD
warning: unknown enum constant AnnotationTarget.LOCAL_VARIABLE
warning: unknown enum constant AnnotationTarget.ANNOTATION_CLASS
warning: unknown enum constant AnnotationTarget.FILE
warning: unknown enum constant AnnotationRetention.BINARY
warning: unknown enum constant AnnotationTarget.CLASS
warning: unknown enum constant AnnotationTarget.PROPERTY
warning: unknown enum constant AnnotationTarget.LOCAL_VARIABLE
warning: unknown enum constant AnnotationTarget.VALUE_PARAMETER
warning: unknown enum constant AnnotationTarget.CONSTRUCTOR
warning: unknown enum constant AnnotationTarget.FUNCTION
warning: unknown enum constant AnnotationTarget.PROPERTY_GETTER
warning: unknown enum constant AnnotationTarget.PROPERTY_SETTER
warning: unknown enum constant AnnotationTarget.FILE
warning: unknown enum constant AnnotationTarget.TYPEALIAS
warning: unknown enum constant AnnotationRetention.BINARY
warning: unknown enum constant AnnotationTarget.FUNCTION
warning: unknown enum constant AnnotationTarget.PROPERTY_GETTER
warning: unknown enum constant AnnotationTarget.PROPERTY_SETTER
warning: unknown enum constant AnnotationTarget.VALUE_PARAMETER
warning: unknown enum constant AnnotationTarget.FIELD
warning: unknown enum constant AnnotationTarget.LOCAL_VARIABLE
warning: unknown enum constant AnnotationTarget.ANNOTATION_CLASS
warning: unknown enum constant AnnotationRetention.BINARY
warning: unknown enum constant AnnotationTarget.FUNCTION
warning: unknown enum constant AnnotationTarget.PROPERTY_GETTER
warning: unknown enum constant AnnotationTarget.PROPERTY_SETTER
warning: unknown enum constant AnnotationTarget.VALUE_PARAMETER
warning: unknown enum constant AnnotationTarget.FIELD
warning: unknown enum constant AnnotationTarget.LOCAL_VARIABLE
warning: unknown enum constant AnnotationTarget.ANNOTATION_CLASS
warning: unknown enum constant AnnotationRetention.BINARY
warning: unknown enum constant AnnotationTarget.ANNOTATION_CLASS
warning: unknown enum constant AnnotationTarget.CLASS
warning: unknown enum constant AnnotationTarget.FUNCTION
warning: unknown enum constant AnnotationTarget.PROPERTY_GETTER
warning: unknown enum constant AnnotationTarget.PROPERTY_SETTER
warning: unknown enum constant AnnotationTarget.CONSTRUCTOR
warning: unknown enum constant AnnotationTarget.FIELD
warning: unknown enum constant AnnotationTarget.FILE
warning: unknown enum constant AnnotationRetention.BINARY
warning: unknown enum constant AnnotationTarget.ANNOTATION_CLASS
warning: unknown enum constant AnnotationTarget.CLASS
warning: unknown enum constant AnnotationTarget.FUNCTION
warning: unknown enum constant AnnotationTarget.PROPERTY_GETTER
warning: unknown enum constant AnnotationTarget.PROPERTY_SETTER
warning: unknown enum constant AnnotationTarget.CONSTRUCTOR
warning: unknown enum constant AnnotationTarget.FIELD
warning: unknown enum constant AnnotationTarget.FILE
warning: unknown enum constant AnnotationTarget.FIELD
warning: unknown enum constant AnnotationTarget.FUNCTION
warning: unknown enum constant AnnotationTarget.PROPERTY_GETTER
warning: unknown enum constant AnnotationTarget.PROPERTY_SETTER
warning: unknown enum constant AnnotationRetention.BINARY
warning: unknown enum constant AnnotationRetention.BINARY
warning: unknown enum constant AnnotationTarget.ANNOTATION_CLASS
warning: unknown enum constant AnnotationRetention.BINARY

I tried R8, but I don't know if it works:

fast build : extension is 11MB
fast build -o : extension is 14MB
fast build -s : extension is 11MB
fast build -r : extension is 4MB

I don't know how optimization works, but it probably works the other way around. Do I need a different rules file to use r8?

Ok I see there are new properties in fast.xml. By the way, why properties in xml file if we use flags -s, -r?

JEWEL · October 24, 2024, 4:40pm

If you’re using ProGuard, so just you need to add -dontwarn option for those enum classes when you're sure that these classes are not required on runtime.

The R8 will utilize the same proguard-rules.pro file. However, this time, I have simplified it to ensure compatibility with R8. Since R8 is an experimental tool, it may encounter unexpected exceptions. One common issue is that the ProGuard tool can handle classes missing issues, while R8 cannot. In such cases, R8 throws a classes missing exception.

To avoid unintentional compilation.

Thank you for your ongoing feedback to improve the FAST.

Patryk_F · October 24, 2024, 4:48pm

These warnings appear during source compilation, before desugaring and proguard. They even appear before warnings about missing descriptions in appinventor methods.

I tried R8 but it stops, some class or method is missing? :

Generating DEX bytecode.
Running the dexing task.
Info in C:\Users\Patryk\AppData\Local\Fast\lib\appinventor\kawa.jar:gnu/kawa/slib/raise$Mnobject$Mnexception.class:
Malformed inner-class attribute:
	outerTypeInternal: gnu/kawa/slib/srfi34
	innerTypeInternal: gnu/kawa/slib/raise$Mnobject$Mnexception
	innerName: Mnexception

Info in C:\Users\Patryk\AppData\Local\Fast\lib\appinventor\kawa.jar:gnu/kawa/slib/srfi34.class:
Malformed inner-class attribute:
	outerTypeInternal: gnu/kawa/slib/srfi34
	innerTypeInternal: gnu/kawa/slib/raise$Mnobject$Mnexception
	innerName: Mnexception

Info in C:\Users\Patryk\AppData\Local\Fast\lib\appinventor\kawa.jar:gnu/kawa/slib/test$Mnrunner.class:
Malformed inner-class attribute:
	outerTypeInternal: gnu/kawa/slib/testing
	innerTypeInternal: gnu/kawa/slib/test$Mnrunner
	innerName: Mnrunner

Info in C:\Users\Patryk\AppData\Local\Fast\lib\appinventor\kawa.jar:gnu/kawa/slib/conditions.class:
Malformed inner-class attribute:
	outerTypeInternal: gnu/kawa/slib/conditions
	innerTypeInternal: gnu/kawa/slib/condition$Mntype
	innerName: Mntype

Info in C:\Users\Patryk\AppData\Local\Fast\lib\appinventor\kawa.jar:gnu/kawa/slib/testing.class:
Malformed inner-class attribute:
	outerTypeInternal: gnu/kawa/slib/testing
	innerTypeInternal: gnu/kawa/slib/test$Mnrunner
	innerName: Mnrunner

Info in C:\Users\Patryk\AppData\Local\Fast\lib\appinventor\kawa.jar:gnu/kawa/slib/conditions$frame.class:
Malformed inner-class attribute:
	outerTypeInternal: gnu/kawa/slib/conditions
	innerTypeInternal: gnu/kawa/slib/condition$Mntype
	innerName: Mntype

Info in C:\Users\Patryk\AppData\Local\Fast\lib\appinventor\kawa.jar:gnu/kawa/slib/condition$Mntype.class:
Malformed inner-class attribute:
	outerTypeInternal: gnu/kawa/slib/conditions
	innerTypeInternal: gnu/kawa/slib/condition$Mntype
	innerName: Mntype

Error: java.lang.NoSuchMethodError: java.util.function.Predicate.not(Ljava/util/function/Predicate;)Ljava/util/function/Predicate;
Compilation failed with an internal error.
Exception in thread "main" java.lang.RuntimeException: com.android.tools.r8.CompilationFailedException: Compilation failed to complete
	at com.android.tools.r8.internal.vk.a(R8_4.0.63_60b6673af9aedf212724e0f5aa211c43280952408a124d03f2973a80bb67dd6b:98)
	at com.android.tools.r8.R8.main(R8_4.0.63_60b6673af9aedf212724e0f5aa211c43280952408a124d03f2973a80bb67dd6b:5)
Caused by: com.android.tools.r8.CompilationFailedException: Compilation failed to complete
	at Version.fakeStackEntry(Version_4.0.63.java:0)
	at com.android.tools.r8.internal.vk.a(R8_4.0.63_60b6673af9aedf212724e0f5aa211c43280952408a124d03f2973a80bb67dd6b:75)
	at com.android.tools.r8.internal.vk.a(R8_4.0.63_60b6673af9aedf212724e0f5aa211c43280952408a124d03f2973a80bb67dd6b:28)
	at com.android.tools.r8.internal.vk.a(R8_4.0.63_60b6673af9aedf212724e0f5aa211c43280952408a124d03f2973a80bb67dd6b:27)
	at com.android.tools.r8.R8.b(R8_4.0.63_60b6673af9aedf212724e0f5aa211c43280952408a124d03f2973a80bb67dd6b:44)
	at com.android.tools.r8.R8.a(R8_4.0.63_60b6673af9aedf212724e0f5aa211c43280952408a124d03f2973a80bb67dd6b:1374)
	at com.android.tools.r8.internal.vk.a(R8_4.0.63_60b6673af9aedf212724e0f5aa211c43280952408a124d03f2973a80bb67dd6b:85)
	... 1 more
Caused by: java.lang.NoSuchMethodError: java.util.function.Predicate.not(Ljava/util/function/Predicate;)Ljava/util/function/Predicate;
	at com.android.tools.r8.internal.bb.a(R8_4.0.63_60b6673af9aedf212724e0f5aa211c43280952408a124d03f2973a80bb67dd6b:57)
	at com.android.tools.r8.internal.bb.a(R8_4.0.63_60b6673af9aedf212724e0f5aa211c43280952408a124d03f2973a80bb67dd6b:18)
	at com.android.tools.r8.internal.c5.a(R8_4.0.63_60b6673af9aedf212724e0f5aa211c43280952408a124d03f2973a80bb67dd6b:4)
	at java.util.LinkedHashMap.forEach(LinkedHashMap.java:684)
	at com.android.tools.r8.internal.c5.a(R8_4.0.63_60b6673af9aedf212724e0f5aa211c43280952408a124d03f2973a80bb67dd6b:3)
	at com.android.tools.r8.internal.bb.a(R8_4.0.63_60b6673af9aedf212724e0f5aa211c43280952408a124d03f2973a80bb67dd6b:10)
	at com.android.tools.r8.internal.hb.a(R8_4.0.63_60b6673af9aedf212724e0f5aa211c43280952408a124d03f2973a80bb67dd6b:72)
	at com.android.tools.r8.internal.hb.a(R8_4.0.63_60b6673af9aedf212724e0f5aa211c43280952408a124d03f2973a80bb67dd6b:104)
	at com.android.tools.r8.internal.no.a(R8_4.0.63_60b6673af9aedf212724e0f5aa211c43280952408a124d03f2973a80bb67dd6b:59)
	at com.android.tools.r8.internal.no.a(R8_4.0.63_60b6673af9aedf212724e0f5aa211c43280952408a124d03f2973a80bb67dd6b:12)
	at com.android.tools.r8.R8.a(R8_4.0.63_60b6673af9aedf212724e0f5aa211c43280952408a124d03f2973a80bb67dd6b:338)
	at com.android.tools.r8.R8.a(R8_4.0.63_60b6673af9aedf212724e0f5aa211c43280952408a124d03f2973a80bb67dd6b:1373)
	at com.android.tools.r8.internal.vk.a(R8_4.0.63_60b6673af9aedf212724e0f5aa211c43280952408a124d03f2973a80bb67dd6b:24)
	... 4 more
R8 dexer is failed to execute!
java.io.IOException: BUILD FAILED!
java.io.IOException: BUILD FAILED!
	at com.jewel.fast.repacked.AgFAKoeZezlMezLLFxcBWVyUsQBaxKDDxvThNDeIOeWETzTLCucNgOJWFpHgBlHx.<init>(Unknown Source)
	at com.jewel.fast.repacked.mRKaQanqzAAYCvMgIoKmGzubeaQpFMJPGnravThpWtntqpXyRrwWWCZLaviTLSPv.<init>(Unknown Source)
	at com.jewel.fast.Fast.main(Unknown Source)

Patryk_F · October 24, 2024, 5:04pm

In the latest version proguard reported a missing class that the previous version did not report.

JEWEL · October 24, 2024, 5:27pm

Could you elaborate more, please?

Patryk_F · October 24, 2024, 5:40pm

I am using the gson library.

Proguard reported the following errors:

Warning: library class com.google.api.client.json.gson.GsonGenerator depends on program class com.google.gson.stream.JsonWriter
Warning: library class com.google.api.client.json.gson.GsonParser depends on program class com.google.gson.stream.JsonReader

Adding the google-http-client-gson.jar library fixed the problem, but it didn't happen before.

JEWEL · October 24, 2024, 5:42pm

Thanks, I'll take a look on it again.

Patryk_F · October 24, 2024, 6:17pm

Does proguard use guava library, does it have it built in? I add guava to extension, removed older versions from Fast folder and did sync. But it seems that proguard still sees older guava version than 33 somewhere, because I have many warnings and I have to use -dontwarn for guava classes. Here are some of them:

Warning: library class com.google.common.util.concurrent.ImmediateFuture$ImmediateFailedCheckedFuture extends or implements program class com.google.common.util.concurrent.ImmediateFuture
Warning: library class com.google.common.util.concurrent.ImmediateFuture$ImmediateSuccessfulCheckedFuture extends or implements program class com.google.common.util.concurrent.ImmediateFuture
Warning: library class com.google.common.util.concurrent.ImmediateFuture$ImmediateSuccessfulFuture extends or implements program class com.google.common.util.concurrent.ImmediateFuture
Warning: library class com.google.common.util.concurrent.Service$State$1 extends or implements program class com.google.common.util.concurrent.Service$State
Warning: library class com.google.common.util.concurrent.Service$State$2 extends or implements program class com.google.common.util.concurrent.Service$State
Warning: library class com.google.common.util.concurrent.Service$State$3 extends or implements program class com.google.common.util.concurrent.Service$State
Warning: library class com.google.common.util.concurrent.Service$State$4 extends or implements program class com.google.common.util.concurrent.Service$State
Warning: library class com.google.common.util.concurrent.Service$State$5 extends or implements program class com.google.common.util.concurrent.Service$State
Warning: library class com.google.common.util.concurrent.Service$State$6 extends or implements program class com.google.common.util.concurrent.Service$State
Warning: library class com.google.common.util.concurrent.Striped$2 extends or implements program class com.google.common.base.Supplier
Warning: library class com.google.common.util.concurrent.Striped$3 extends or implements program class com.google.common.base.Supplier
Warning: library class com.google.common.util.concurrent.Striped$4 extends or implements program class com.google.common.base.Supplier
Warning: library class com.google.common.util.concurrent.Striped$5 extends or implements program class com.google.common.base.Supplier
Warning: library class com.google.common.util.concurrent.Striped$6 extends or implements program class com.google.common.base.Supplier

Class e.g. com.google.common.util.concurrent.Striped$2 does not appear in my version of guava. But it is in guava 27.1 which appinventor uses. But it is present in guava 27.1 which is used by appinventor. But it is nowhere in Fast folders or in my dependencies.

I see that proguard has inside jar guave, gson and many other libraries. Probably in older version than I use in extension and that's where problem is. I don't know if -dontwarn will be safe in this case?

JEWEL · October 26, 2024, 8:40am

An update is available, v1.1.9 25.10.24.11.32

Changes in the printing process on the terminal.

The dontwarn option can be safely used to suppress duplicate class warnings. However, when encountering classes missing issues, it is crucial to ensure that the required classes are present in the dependencies folder. Alternatively, it is necessary to verify that the absence of these classes will not adversely affect runtime behavior.

I've started coding for it and hopefully I'll be able to release it with the next update.

Patryk_F · October 26, 2024, 8:51am

Yes, of course. Only the warnings I showed are not due to missing classes, because my dependencies have those classes...the reason is that proguard has built-in outdated libraries that do not have the appropriate classes.

JEWEL · October 26, 2024, 9:56am

I believe, in such cases you may safely apply the -dontwarn option, if you wish to do so.

mahmoud_hooda · October 26, 2024, 11:02am

hello @JEWEL , i got error while i am using fast build command
error:

 - Error parsing command line: While parsing option --classpath_entry C:\Program: C:\Program is not a valid path: it does not exist.
 - Try --help.
x BUILD FAILED!

edit:
this error just display when desugar_sources: true in fast.xml but when desugar_sources: false there is not errors displayed it is work fine

JEWEL · October 26, 2024, 12:05pm

Your project may have spaces in the directory path. I believe the desugar tool does not accept spaces in the directory path. I will attempt to provide a solution for this issue later.
In the meantime, please move the FAST to a different location where there are no spaces in the path. Then, configure the FAST_HOME variable and the environment path with the new location.

And please, let me know which version of FAST you're using right now?