new package: smallerc

[stsp]

This adds SmallerC C compiler.
It is cross-platform, and under Termux can be used to compile for dosbox. It is also used with dosemu2, which I am also planning to submit.

[stsp]

By the way, can the building recipe
refer directly to git? Not all packages
seem to have archived releases.

[twaik]

By the way, can the building recipe
refer directly to git?

It can.

[stsp]

It can.

In that case I simply don't specify
sha256, correct?

[twaik]

You can check these examples.

[stsp]

Thanks!
Looks like I can use fixed commit,
or even the branch head with
TERMUX_PKG_AUTO_UPDATE=true
to automatically provide new versions.
I guess this is exactly what I am going
to end up with. :)

[stsp]

Strange that it builds for me w/o
any problems... But now I added
TERMUX_PKG_BUILD_IN_SRC=true
which will hopefully work.

[IntinteDAO]

From what I understand it is a compiler for DOS, Linux (...), but only for x86 architecture. Will it work on ARM architecture?

The question is also whether it is needed, because DosBox and DosEmu2 are x86 emulators, so they can use SmallerC for x86

[stsp]

From what I understand it is a compiler for DOS, Linux (...), but only for x86 architecture.
Will it work on ARM architecture?

Yes - as a cross-compiler.
It will work on arm, compile for x86.
There are no other cross-compilers
in termux currently AFAICS. So that
seems very helpful.

The question is also whether it is needed, because DosBox and DosEmu2 are x86 emulators,
so they can use SmallerC for x86

Yes, they can use SmallerC for x86,
running on arm. This is exactly the
case.

[TomJo2000]

Yes - as a cross-compiler.
It will work on arm, compile for x86.
There are no other cross-compilers
in termux currently AFAICS. So that
seems very helpful.

clang, gcc (TUR) and I'm pretty sure also tcc can all cross compile.

All our packages are built on a x86_64, Ubuntu 24.04 docker host container as a matter of fact.
And we use Clang/LLVM for any of them written in C, C++, etc.

[stsp]

From the above gcc spec it doesn't
look like a cross-compiler to me. And
it would also need cross-binutils.
Please let me know if I am wrong on
that, because having cross-binutils is
already helpful.

clang - yes, but not for other OSes.
SmallerC can cross-compile to various
OSes of x86 arch, whereas clang can
cross-compile to other arches for the
same OS only I think, as otherwise
that would require the libraries for
the target OS, and they are not here.
SmallerC comes with the target libs
for all supported OSes.

[stsp]

After lots of experimenting on my
own github repo, I've found the build
problem is here:
CC=x86_64-linux-android-clang
If I use plain clang the build feels
much better.

What does this mean? What should
I change in my build.sh to get the
right compiler? (Note: works perfectly
fine locally, breaks only on github)

[Grimler91]

After lots of experimenting on my own github repo, I've found the build problem is here: CC=x86_64-linux-android-clang If I use plain clang the build feels much better.

If you use /usr/bin/clang then it is building for ubuntu x86_64 host, and not cross-compiling for termux (or well, maybe a mix due to all the env vars we set).

Cross-compilation is fairly complicated, especially for compilers. You need to cross-compile smallerc for the android arches (termux's build script mostly handles this mostly automatically) AND configure smallerc to tell it which arch(es) it should compile for.

In gcc-land one talks about the build platform, the host platform and the target platform (see for example https://stackoverflow.com/questions/5139403/whats-the-difference-of-configure-option-build-host-and-target). In this case build platform is x86_64 (ubuntu docker image), host is termux arch, i.e. one of [arm, aarch64, i686, x86_64], and target is "dosbox" or "dosemu2" (or whatever those targets should be called)

[stsp]

You need to cross-compile smallerc for the android arches (termux's build script mostly handles
this mostly automatically)

Probably this is the culprit, but what
is "android arches"? I made this test:
https://github.com/stsp/termux-packages/actions/runs/12377254025/job/34546321272?pr=2

CC=x86_64-linux-android-clang
+ echo CC=x86_64-linux-android-clang
+ uname -a
Linux 32cb893689b6 6.8.0-1017-azure #20-Ubuntu SMP Tue Oct 22 03:43:13 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
...
+ ldd ./smlrcc
./smlrcc: error while loading shared libraries: /lib/x86_64-linux-gnu/libc.so: invalid ELF header

So I really don't understand what's
going on. From uname -a we can
see that the host arch is x86_64.
The used clang is x86_64-linux-android-clang
so the arches do match.
And yet at the end ldd fails with

`+ ldd ./smlrcc
./smlrcc: error while loading shared libraries: /lib/x86_64-linux-gnu/libc.so: invalid ELF header

Does this all mean that android
uses completely incompatible
(non-ELF?) file format, and the
files built with x86_64-linux-android-clang
can never be run under termux?
If so - why am I compiling with
this clang? I want to compile with
normal termux's clang, so that
the program to work on termux?
Confused.

[Grimler91]

Android uses a different linker and libc than Ubuntu/"standard" linux, you cannot run a termux x86_64 binary in x86_64 "standard" linux and vice versa, see https://wiki.termux.com/wiki/Differences_from_Linux

[stsp]

Android uses a different linker and libc than Ubuntu/"standard" linux

Ahha, so does this mean the github
build farm uses std Ubuntu, and hence
the problem? That would explain why
everything works fine locally: locally
I build under Waydroid. :)
Ok, I am starting to understand.

So... what would be a solution?
Should I build the "bootstrap"
version of smallerc that will be
used during build, and the final
version of it for packaging?
Or can I for example use native
ubuntu's smallerc for the build
process? If I can do that - how to
add the dependency? Am I allowed
to execute "apt-get" in build.sh?
I guess not. :)
Well, looks like a tough problem.

[truboxl]

We mainly use NDK clang running on Ubuntu x86_64 to cross compile for Android.

I see that smallerc makefile do compile to smlrcc and then run that smlrcc to compile libs. This will not work and need to be patched or workaround.

https://github.com/alexfru/SmallerC/blob/master/common.mk

[stsp]

We mainly use NDK clang running on Ubuntu x86_64 to cross compile for Android.

Yep, that's what I've finally realized. :)

I see that smallerc makefile do compile to smlrcc and then run that smlrcc to compile libs.
This will not work and need to be patched or workaround.

So the question for now is: can I use
native ubuntu's smallerc package during
build or not? It has smallerc packaged.
If I can't use that, things are more complicated.

[Grimler91]

If I can do that - how to
add the dependency? Am I allowed
to execute "apt-get" in build.sh?

To test this is fine (or even better, try in docker image locally). If it works then smallerc should be added to the list in setup-ubuntu.sh

[twaik]

Probably adding it to setup-ubuntu.sh is not very wise. It will be better to build it in termux_step_host_build and add it to $PATH.

[Grimler91]

Probably adding it to setup-ubuntu.sh is not very wise. It will be better to build it in termux_step_host_build and add it to $PATH.

Are you thinking from docker-image-size perspective, or smallerc-host-version-vs-version-we-build compatibility perspective?
Good point in any case!

[stsp]

It will be better to build it in termux_step_host_build and add it to $PATH.

Done and seems to work.

[stsp]

I have a few packages that depend
on smallerc. It appears they need a
host version of smallerc to be built
on github. But the PATH is not set
for it, and in general I wonder what
to do in that case. Should I just depend
them on "smallerc" as if nothing happens,
or something special must be done
to signify the host version of it?
And should they set the PATH by hands?

[stsp]

And what's worse, for the dependent
package I need to set PATH to the host
version of smallerc this way:

termux_step_make() {
        local _PREFIX_FOR_BUILD=$TERMUX_TOPDIR/smallerc/host-build/prefix
        export PATH="$PATH:${_PREFIX_FOR_BUILD}/bin"
        make prefix=$PREFIX
}

Which works but looks more than
ridiculous. What would be the better
solution?

[truboxl]

You can:

Download a readily available built smallerc (if the author release the binary) and use it that way

Add the smallerc to scripts/setup-ubuntu.sh (if exist in Ubuntu's repo) which otherwise would need to host build smallerc in every single individual package build.sh that needs it.

Add the host build steps to each build.sh that needs it.

I don't advise:

Putting the same host build path in different build.sh as it may not work. Generally files in host build directory are discarded and not cached. So the next time you try to build, the path may not exist. You can cache them to $TERMUX_PREFIX/opt though and set the PATH to there. A last resort.

[stsp]

Add the smallerc to scripts/setup-ubuntu.sh (if exist in Ubuntu's repo)

It appears to exist only since Oracular (24.10),
which doesn't suit here (termux uses Noble).

You can cache them to $TERMUX_PREFIX/opt though and set the PATH to there. A last resort.

Will try.
But I have to note the following:

• Seems termux-build doesn't natively support this use-case
• Doing so, severely diminishes the use of the smallerc as a package.
  I don't "code" with smallerc, I only use it for build.
  So whether or not the package itself is then needed,
  or just caching it to /opt is enough - is an open question.

[stsp]

which otherwise would need to host build smallerc in every single individual package build.sh that needs it.

I ended up doing it this way.
So here's the result:
stsp@b743df5
Its still quite ugly:

termux_step_host_build() {
	local _TERMUX_PKG_SRCDIR=$TERMUX_TOPDIR/smallerc/src
	local _PREFIX_FOR_BUILD=$TERMUX_TOPDIR/smallerc/host-build/prefix

Am I supposed to access the sources
of another package this way?

[stsp]

Does termux support "host packages"?
The differences from normal packages,
that I can think of:

• built only as a dep, never separately
• never purged
• PATH is adjusted once you depend on it

Is there something like this, or maybe
possible to implement?

[truboxl]

We pretty much do it every time for Qt6 packages

termux-packages/x11-packages/qt6-qtbase/build.sh

Line 72 in da7255b

termux_step_host_build() {

Another example: libwayland-scanner
(subpackage the host build files, then add as build dependency and export PATH in other packages)

termux-packages/packages/libwayland/build.sh

Line 23 in da7255b

termux_step_host_build() {

termux-packages/packages/libwayland/libwayland-cross-scanner.subpackage.sh

Lines 1 to 5 in da7255b

	TERMUX_SUBPKG_DESCRIPTION="wayland-scanner for host (NOT for Termux)"
	TERMUX_SUBPKG_PLATFORM_INDEPENDENT=true
	TERMUX_SUBPKG_INCLUDE="
	opt/${TERMUX_PKG_NAME}/cross/*
	"

termux-packages/x11-packages/wlroots/build.sh

Line 10 in e9908ae

TERMUX_PKG_BUILD_DEPENDS="libglvnd-dev, libwayland-cross-scanner, libwayland-protocols, xwayland"

termux-packages/x11-packages/wlroots/build.sh

Line 26 in e9908ae

export PATH="$TERMUX_PREFIX/opt/libwayland/cross/bin:$PATH"

[stsp]

You seem to mean you do
$TERMUX_PREFIX/opt a lot. But
as you called it a "last resort", I instead
implemented this:
which otherwise would need to host build smallerc in every single individual package build.sh that needs it.
Which requires things like:

	local _TERMUX_PKG_SRCDIR=$TERMUX_TOPDIR/smallerc/src
	local _PREFIX_FOR_BUILD=$TERMUX_TOPDIR/smallerc/host-build/prefix

in other packages.

So should I instead use the /opt way?

[stsp]

Ok, re-done the /opt way.
It indeed looks much nicer than
to do the host build for every
dependent package.

[stsp]

I upstreamed the small makefile patch
within this recipe. But not switching the
recipe to git just for that.

[stsp]

There is now an interesting small
problem here, that smallerc depends
on smallerc-cross to build its libs.
But I don't think I can add that to
build-depends, as the package
probably can't depend on its own
sub-packages.

[truboxl]

There's no need to explicitly declare relationship like that
You will need to do that for other packages that needs host build tools from another package though

[TomJo2000]

Looks good to me.
If there's no further comments I'd like to get this merged tomorrow so it doesn't just sit here.