[Development] Qt 6 buildsystem support requirements

[Thiago Macieira]

Hello

Having spent far too much time trying to figure out why crappy buildsystems
cause failures in distros (like TensorFlow or libvpx - see
https://plus.google.com/+ThiagoMacieira/posts/DqTKdRGfuwR ), I'd like to make
sure this doesn't happen to Qt 6. For that, I'd like to add the following
extra requirements to the buildsystem we choose to use. This is in addition to
the functionality requirements.

These apply to the buildsystem at the time of Qt's the switch to it.

1) Ease of obtention

a) Must be packaged by all major package managers where Qt 6 is expected to be
relevant. That is, it must be available as a package in:
- ArchLinux
- Debian testing
- Gentoo
- Fedora current and previous
- openSUSE current and previous
- Ubuntu LTS released more than 6 months prior
- Homebrew (macOS)

I'll take care of Clear Linux, but we apply the "two major distros" rule, so I
won't be first. It would be nice to have it in vcpkg/nuget/whatever MSVC uses
and FreeBSD ports tree too.

b) Must be easily compiled from source on a standard system installation. All
of its dependencies must come from the system's package manager and there must
be no cyclic dependencies. That is, it cannot require itself to build and it
must not depend on Qt, either in source form or binary.

c) Must not require too-recent version in order to compile Qt. The versions
found in (a) must be sufficient to build Qt 6.0 and this must continue
throughout Qt 6's lifetime.

2) Track record

a) Must be used by roughly a dozen packages in major distros. I'm not saying
all of them have to have a dozen, not even that one of them has a dozen. But
all of the ones listed above must have at least one and there must be a dozen
distinct packages in total.

b) At least one package must have been continuously packaged for two years.
One for an RPM distro and one for a .deb distro.

c) At least one package of comparable complexity to qtbase, packaged by the
three of the six Linux distros I listed. I'm looking for:
- compiling certain files with different compiler flags
- several third-party dependencies, some required, others optional
- lots of configure-time options
- installing several different types of targets (binaries, libraries,
plugins, arch-dependent files, arch-independent files, documentation,
translations, etc.)
- obeying distro-supplied options (compiler & linker flags, compiler
selection [ccache, icecc, clang], debuginfo split, stripping, etc.)

3) Community support
a) Thriving community to ask for help to, whenever issues happen.
b) One packager for .deb and one for RPM that will vouch for the buildsystem.


I don't think I am being unreasonable. I am being strict, though.

I'll put my money where my mouth is: as soon as the requirements are met, I
will begin using it and contributing to it, finding out where there may be
shortcomings and supplying fixes if possible, bug reports if not.

But until then, I will pretend it doesn't exist and will ignore the branch
that uses it to build.
--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center

[Thiago Macieira]

On Friday, 20 July 2018 19:35:48 PDT Thiago Macieira wrote:
> I'll take care of Clear Linux, but we apply the "two major distros" rule, so
> I won't be first.

Fedora and openSUSE already package it, so I will add it to Clear Linux.

However, I am blocked by https://bugreports.qt.io/browse/QTCREATORBUG-20832.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center

[Thiago Macieira]

On Friday, 20 July 2018 22:38:10 PDT Bogdan Vatra via Development wrote:
> > b) Must be easily compiled from source on a standard system installation.
> > All of its dependencies must come from the system's package manager and
> > there must be no cyclic dependencies. That is, it cannot require itself to
> > build and it must not depend on Qt, either in source form or binary.
>
> You really hate QBS don't you ? :)

No, I don't. I've never tried it, not even to bother looking at what a source
file looks like, so how could I hate it?

All I have are requirements and mine are that it should be easy to build and
to fix problems. Because there will be problems. I'm too old to spend time
learning how to debug a new buildsystem. I have better things to do. I'd just
as soon give up and move on to greener pastures than to bash my head against
buildsystem problems again.

> Anyway IMHO is more important to have a clean, nice and easy to use syntax
> and to be tooling friendly than 1.b.

Well, I disagree. If you can't compile the build tool, what use is it to have
the nicest language? It's inaccessible.

Note I didn't say that the tool shouldn't be *part* of qtbase, like qmake is.
I just thought that the objective was to eventually do away with that, to
avoid having to bootstrap a tool at the moment of configure -- a tool that
becomes more complex with each release.

The point on requiring Qt source was that one would need to download and
extract qtbase, putting it alongside the tool, just so it can be built.
Instead of doing that, I suggest copying the files that one needs into the
tool's source tree and packaging those for its release.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center

[Allan Sandfeld Jensen]

On Samstag, 21. Juli 2018 07:38:10 CEST Bogdan Vatra via Development wrote:
> Hi,
>
> În ziua de sâmbătă, 21 iulie 2018, la 05:35:48 EEST, Thiago Macieira a
scris:
> > Hello
> >
> > Having spent far too much time trying to figure out why crappy
> > buildsystems
>
> [...]
>
> > 1) Ease of obtention
> >
> > a) Must be packaged by all major package managers where Qt 6 is expected
> > to
> >
> > be relevant. That is, it must be available as a package in:
> > - ArchLinux
> > - Debian testing
> > - Gentoo
> > - Fedora current and previous
> > - openSUSE current and previous
> > - Ubuntu LTS released more than 6 months prior
> > - Homebrew (macOS)
>
> [...]
>
> > b) Must be easily compiled from source on a standard system installation.
> > All of its dependencies must come from the system's package manager and
> > there must be no cyclic dependencies. That is, it cannot require itself to
> > build and it must not depend on Qt, either in source form or binary.
>
> You really hate QBS don't you ? :)
>
> Anyway IMHO is more important to have a clean, nice and easy to use syntax
> and to be tooling friendly than 1.b.
>
> GN[1] is another example of build system which didn't care too much about
> 1.a,b,c and it still used in quite big projects (e.g. chrome, fuchsia). To
> my huge surprise, they managed to move it into a separate repo and remove
> all chromium dependencies (yep, a few months ago you had to checkout the
> entire chromium repo to build it :) ).
>
Sounds good, but at least for the last year it has also been so unstable that
any older version of GN couldn't build a newer Chromium release, and all the
build logic was heavily tied to how Chromium does this, and it has a
bootstrapper than never worked in any releases ever, so the only way to build
GN was to download GN and build it with the blackbox binary.

`Allan

[Thiago Macieira]

On Saturday, 21 July 2018 00:18:40 PDT Bogdan Vatra via Development wrote:
> I'm not trying to sell GN here, because from my experience, Google folks are
> quite difficult to work with and not very open to changes.

Yes, they are. My email started with relating problems trying to build
TensorFlow. Why do you think it is difficult to build? It uses bazel and
Google loves that too.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center

[Kevin Kofler]

Thiago Macieira wrote:
> https://plus.google.com/+ThiagoMacieira/posts/DqTKdRGfuwR

There is also this classic (more than 9 years old, yet still to the point):
https://spot.livejournal.com/308370.html

In particular:
> * You've written your own build tool for this code [ +100 points of FAIL ]

:-)

Kevin Kofler

[Kevin Kofler]

Bogdan Vatra via Development wrote:
> Anyway IMHO is more important to have a clean, nice and easy to use syntax
> and to be tooling friendly than 1.b.

A custom build system is always a major pain point for distributions. A
circular dependency (what Thiago's 1.b forbids) makes it particularly
painful. How should we bootstrap new architectures or entirely new
distributions if we cannot build Qt due to the circular dependency between
Qt and its build tool? This is a showstopper.

> GN[1] is another example of build system which didn't care too much about
> 1.a,b,c and it still used in quite big projects (e.g. chrome, fuchsia). To
> my huge surprise, they managed to move it into a separate repo and remove
> all chromium dependencies (yep, a few months ago you had to checkout the
> entire chromium repo to build it :) ).

GN (and its predecessor Gyp) is universally hated by distribution packagers
for its non-standardness, weirdness, lack of documentation (including third-
party documentation such as tutorials, an issue inherent to custom build
systems) and lack of flexibility (custom build systems are never as powerful
as widely-used general-purpose build systems).

QtWebEngine is a particular pain to package because it uses TWO custom build
systems (QMake and GN).

The Chromium mess is also what prompted Spot to write the list of FAILs
[https://spot.livejournal.com/308370.html] I have already linked to
elsewhere in this thread.


There is a build system that fulfills all of Thiago's points, and it is
already widely used in the Qt community: CMake.

Kevin Kofler

[Jean-Michaël Celerier]

> There is a build system that fulfills all of Thiago's points, and it is
already widely used in the Qt community: CMake.

+1, I was flabbergasted when the big objection against CMake in Qt 6 boiled
down to "it does not supports all the architectures that Qt supports", so
instead of contributing them - or hell, even forking CMake for those
specific architectures (what are them ? I use cmake for windows, mac,
linux, android, ios and the toolchain file allows for a lot of
customization), what, create a new build system from scratch that splits
the C++ community further ? There would be so much to gain with a better
relationship between Qt and CMake.

Best,
-------
Jean-Michaël Celerier
http://www.jcelerier.name

On Sat, Jul 21, 2018 at 2:42 PM, Kevin Kofler <***@chello.at>
wrote:

> Bogdan Vatra via Development wrote:
> > Anyway IMHO is more important to have a clean, nice and easy to use
> syntax
> > and to be tooling friendly than 1.b.
>
> A custom build system is always a major pain point for distributions. A
> circular dependency (what Thiago's 1.b forbids) makes it particularly
> painful. How should we bootstrap new architectures or entirely new
> distributions if we cannot build Qt due to the circular dependency between
> Qt and its build tool? This is a showstopper.
>
> > GN[1] is another example of build system which didn't care too much about
> > 1.a,b,c and it still used in quite big projects (e.g. chrome, fuchsia).
> To
> > my huge surprise, they managed to move it into a separate repo and remove
> > all chromium dependencies (yep, a few months ago you had to checkout the
> > entire chromium repo to build it :) ).
>
> GN (and its predecessor Gyp) is universally hated by distribution
> packagers
> for its non-standardness, weirdness, lack of documentation (including
> third-
> party documentation such as tutorials, an issue inherent to custom build
> systems) and lack of flexibility (custom build systems are never as
> powerful
> as widely-used general-purpose build systems).
>
> QtWebEngine is a particular pain to package because it uses TWO custom
> build
> systems (QMake and GN).
>
> The Chromium mess is also what prompted Spot to write the list of FAILs
> [https://spot.livejournal.com/308370.html] I have already linked to
> elsewhere in this thread.
>
>
> There is a build system that fulfills all of Thiago's points, and it is
> already widely used in the Qt community: CMake.
>
> Kevin Kofler
>
> _______________________________________________
> Development mailing list
> ***@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development
>

[Ray Donnelly]

As someone who works on a cross platform distribution let me tell you that
cmake is plain terrible. How much custom c++ code does it contains for just
qt? Loads, absolutely tonnes or rubbish.

On Sat, Jul 21, 2018, 1:49 PM Jean-Michaël Celerier <
***@gmail.com> wrote:

> > There is a build system that fulfills all of Thiago's points, and it is
> already widely used in the Qt community: CMake.
>
> +1, I was flabbergasted when the big objection against CMake in Qt 6
> boiled down to "it does not supports all the architectures that Qt
> supports", so instead of contributing them - or hell, even forking CMake
> for those specific architectures (what are them ? I use cmake for windows,
> mac, linux, android, ios and the toolchain file allows for a lot of
> customization), what, create a new build system from scratch that splits
> the C++ community further ? There would be so much to gain with a better
> relationship between Qt and CMake.
>
> Best,
> -------
> Jean-Michaël Celerier
> http://www.jcelerier.name
>
> On Sat, Jul 21, 2018 at 2:42 PM, Kevin Kofler <***@chello.at>
> wrote:
>
>> Bogdan Vatra via Development wrote:
>> > Anyway IMHO is more important to have a clean, nice and easy to use
>> syntax
>> > and to be tooling friendly than 1.b.
>>
>> A custom build system is always a major pain point for distributions. A
>> circular dependency (what Thiago's 1.b forbids) makes it particularly
>> painful. How should we bootstrap new architectures or entirely new
>> distributions if we cannot build Qt due to the circular dependency
>> between
>> Qt and its build tool? This is a showstopper.
>>
>> > GN[1] is another example of build system which didn't care too much
>> about
>> > 1.a,b,c and it still used in quite big projects (e.g. chrome, fuchsia).
>> To
>> > my huge surprise, they managed to move it into a separate repo and
>> remove
>> > all chromium dependencies (yep, a few months ago you had to checkout the
>> > entire chromium repo to build it :) ).
>>
>> GN (and its predecessor Gyp) is universally hated by distribution
>> packagers
>> for its non-standardness, weirdness, lack of documentation (including
>> third-
>> party documentation such as tutorials, an issue inherent to custom build
>> systems) and lack of flexibility (custom build systems are never as
>> powerful
>> as widely-used general-purpose build systems).
>>
>> QtWebEngine is a particular pain to package because it uses TWO custom
>> build
>> systems (QMake and GN).
>>
>> The Chromium mess is also what prompted Spot to write the list of FAILs
>> [https://spot.livejournal.com/308370.html] I have already linked to
>> elsewhere in this thread.
>>
>>
>> There is a build system that fulfills all of Thiago's points, and it is
>> already widely used in the Qt community: CMake.
>>
>> Kevin Kofler
>>
>> _______________________________________________
>> Development mailing list
>> ***@qt-project.org
>> http://lists.qt-project.org/mailman/listinfo/development
>>
>
> _______________________________________________
> Development mailing list
> ***@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development
>

[Jean-Michaël Celerier]

> How much custom c++ code does it contains for just qt?

which build system which supports automatic calling of moc doesn't have
specific code for qt ?




-------
Jean-Michaël Celerier
http://www.jcelerier.name

On Sat, Jul 21, 2018 at 3:48 PM, Ray Donnelly <***@gmail.com>
wrote:

> As someone who works on a cross platform distribution let me tell you that
> cmake is plain terrible. How much custom c++ code does it contains for just
> qt? Loads, absolutely tonnes or rubbish.
>
> On Sat, Jul 21, 2018, 1:49 PM Jean-Michaël Celerier <
> ***@gmail.com> wrote:
>
>> > There is a build system that fulfills all of Thiago's points, and it is
>> already widely used in the Qt community: CMake.
>>
>> +1, I was flabbergasted when the big objection against CMake in Qt 6
>> boiled down to "it does not supports all the architectures that Qt
>> supports", so instead of contributing them - or hell, even forking CMake
>> for those specific architectures (what are them ? I use cmake for windows,
>> mac, linux, android, ios and the toolchain file allows for a lot of
>> customization), what, create a new build system from scratch that splits
>> the C++ community further ? There would be so much to gain with a better
>> relationship between Qt and CMake.
>>
>> Best,
>> -------
>> Jean-Michaël Celerier
>> http://www.jcelerier.name
>>
>> On Sat, Jul 21, 2018 at 2:42 PM, Kevin Kofler <***@chello.at>
>> wrote:
>>
>>> Bogdan Vatra via Development wrote:
>>> > Anyway IMHO is more important to have a clean, nice and easy to use
>>> syntax
>>> > and to be tooling friendly than 1.b.
>>>
>>> A custom build system is always a major pain point for distributions. A
>>> circular dependency (what Thiago's 1.b forbids) makes it particularly
>>> painful. How should we bootstrap new architectures or entirely new
>>> distributions if we cannot build Qt due to the circular dependency
>>> between
>>> Qt and its build tool? This is a showstopper.
>>>
>>> > GN[1] is another example of build system which didn't care too much
>>> about
>>> > 1.a,b,c and it still used in quite big projects (e.g. chrome,
>>> fuchsia). To
>>> > my huge surprise, they managed to move it into a separate repo and
>>> remove
>>> > all chromium dependencies (yep, a few months ago you had to checkout
>>> the
>>> > entire chromium repo to build it :) ).
>>>
>>> GN (and its predecessor Gyp) is universally hated by distribution
>>> packagers
>>> for its non-standardness, weirdness, lack of documentation (including
>>> third-
>>> party documentation such as tutorials, an issue inherent to custom build
>>> systems) and lack of flexibility (custom build systems are never as
>>> powerful
>>> as widely-used general-purpose build systems).
>>>
>>> QtWebEngine is a particular pain to package because it uses TWO custom
>>> build
>>> systems (QMake and GN).
>>>
>>> The Chromium mess is also what prompted Spot to write the list of FAILs
>>> [https://spot.livejournal.com/308370.html] I have already linked to
>>> elsewhere in this thread.
>>>
>>>
>>> There is a build system that fulfills all of Thiago's points, and it is
>>> already widely used in the Qt community: CMake.
>>>
>>> Kevin Kofler
>>>
>>> _______________________________________________
>>> Development mailing list
>>> ***@qt-project.org
>>> http://lists.qt-project.org/mailman/listinfo/development
>>>
>>
>> _______________________________________________
>> Development mailing list
>> ***@qt-project.org
>> http://lists.qt-project.org/mailman/listinfo/development
>>
>

[Ray Donnelly]

CMake has its own 'language' that was retrofitted after the fact, evolving
from a simple definition based system.

So why aren't all these special rules written in that 'language' then? 150
odd k of qt c++ hacks is what we get instead.

On Sat, Jul 21, 2018, 3:26 PM Jean-Michaël Celerier <
***@gmail.com> wrote:

> > How much custom c++ code does it contains for just qt?
>
> which build system which supports automatic calling of moc doesn't have
> specific code for qt ?
>
>
>
>
> -------
> Jean-Michaël Celerier
> http://www.jcelerier.name
>
> On Sat, Jul 21, 2018 at 3:48 PM, Ray Donnelly <***@gmail.com>
> wrote:
>
>> As someone who works on a cross platform distribution let me tell you
>> that cmake is plain terrible. How much custom c++ code does it contains for
>> just qt? Loads, absolutely tonnes or rubbish.
>>
>> On Sat, Jul 21, 2018, 1:49 PM Jean-Michaël Celerier <
>> ***@gmail.com> wrote:
>>
>>> > There is a build system that fulfills all of Thiago's points, and it
>>> is
>>> already widely used in the Qt community: CMake.
>>>
>>> +1, I was flabbergasted when the big objection against CMake in Qt 6
>>> boiled down to "it does not supports all the architectures that Qt
>>> supports", so instead of contributing them - or hell, even forking CMake
>>> for those specific architectures (what are them ? I use cmake for windows,
>>> mac, linux, android, ios and the toolchain file allows for a lot of
>>> customization), what, create a new build system from scratch that splits
>>> the C++ community further ? There would be so much to gain with a better
>>> relationship between Qt and CMake.
>>>
>>> Best,
>>> -------
>>> Jean-Michaël Celerier
>>> http://www.jcelerier.name
>>>
>>> On Sat, Jul 21, 2018 at 2:42 PM, Kevin Kofler <***@chello.at>
>>> wrote:
>>>
>>>> Bogdan Vatra via Development wrote:
>>>> > Anyway IMHO is more important to have a clean, nice and easy to use
>>>> syntax
>>>> > and to be tooling friendly than 1.b.
>>>>
>>>> A custom build system is always a major pain point for distributions. A
>>>> circular dependency (what Thiago's 1.b forbids) makes it particularly
>>>> painful. How should we bootstrap new architectures or entirely new
>>>> distributions if we cannot build Qt due to the circular dependency
>>>> between
>>>> Qt and its build tool? This is a showstopper.
>>>>
>>>> > GN[1] is another example of build system which didn't care too much
>>>> about
>>>> > 1.a,b,c and it still used in quite big projects (e.g. chrome,
>>>> fuchsia). To
>>>> > my huge surprise, they managed to move it into a separate repo and
>>>> remove
>>>> > all chromium dependencies (yep, a few months ago you had to checkout
>>>> the
>>>> > entire chromium repo to build it :) ).
>>>>
>>>> GN (and its predecessor Gyp) is universally hated by distribution
>>>> packagers
>>>> for its non-standardness, weirdness, lack of documentation (including
>>>> third-
>>>> party documentation such as tutorials, an issue inherent to custom
>>>> build
>>>> systems) and lack of flexibility (custom build systems are never as
>>>> powerful
>>>> as widely-used general-purpose build systems).
>>>>
>>>> QtWebEngine is a particular pain to package because it uses TWO custom
>>>> build
>>>> systems (QMake and GN).
>>>>
>>>> The Chromium mess is also what prompted Spot to write the list of FAILs
>>>> [https://spot.livejournal.com/308370.html] I have already linked to
>>>> elsewhere in this thread.
>>>>
>>>>
>>>> There is a build system that fulfills all of Thiago's points, and it is
>>>> already widely used in the Qt community: CMake.
>>>>
>>>> Kevin Kofler
>>>>
>>>> _______________________________________________
>>>> Development mailing list
>>>> ***@qt-project.org
>>>> http://lists.qt-project.org/mailman/listinfo/development
>>>>
>>>
>>> _______________________________________________
>>> Development mailing list
>>> ***@qt-project.org
>>> http://lists.qt-project.org/mailman/listinfo/development
>>>
>>
>

[Jean-Michaël Celerier]

That's fairly disingenuous, if not blatantly false. There are, like, 6 .cpp
files in the CMake repo which provide the Qt-specific commands :
https://github.com/Kitware/CMake/tree/master/Source (grep for cmQt).

Besides... why would it matter that they are implemented in C++ instead of
cmake-lang ? If anything, CMake's automoc is in my experience much faster
to process the whole repo.

Richard: so what's this then ?
https://github.com/qbs/qbs/blob/2d1de8cc84b258174b2dc0a8080f549cd9b59b32/src/lib/corelib/buildgraph/qtmocscanner.cpp



-------
Jean-Michaël Celerier
http://www.jcelerier.name

On Sat, Jul 21, 2018 at 4:39 PM, Ray Donnelly <***@gmail.com>
wrote:

> CMake has its own 'language' that was retrofitted after the fact, evolving
> from a simple definition based system.
>
> So why aren't all these special rules written in that 'language' then? 150
> odd k of qt c++ hacks is what we get instead.
>
> On Sat, Jul 21, 2018, 3:26 PM Jean-Michaël Celerier <
> ***@gmail.com> wrote:
>
>> > How much custom c++ code does it contains for just qt?
>>
>> which build system which supports automatic calling of moc doesn't have
>> specific code for qt ?
>>
>>
>>
>>
>> -------
>> Jean-Michaël Celerier
>> http://www.jcelerier.name
>>
>> On Sat, Jul 21, 2018 at 3:48 PM, Ray Donnelly <***@gmail.com>
>> wrote:
>>
>>> As someone who works on a cross platform distribution let me tell you
>>> that cmake is plain terrible. How much custom c++ code does it contains for
>>> just qt? Loads, absolutely tonnes or rubbish.
>>>
>>> On Sat, Jul 21, 2018, 1:49 PM Jean-Michaël Celerier <
>>> ***@gmail.com> wrote:
>>>
>>>> > There is a build system that fulfills all of Thiago's points, and it
>>>> is
>>>> already widely used in the Qt community: CMake.
>>>>
>>>> +1, I was flabbergasted when the big objection against CMake in Qt 6
>>>> boiled down to "it does not supports all the architectures that Qt
>>>> supports", so instead of contributing them - or hell, even forking CMake
>>>> for those specific architectures (what are them ? I use cmake for windows,
>>>> mac, linux, android, ios and the toolchain file allows for a lot of
>>>> customization), what, create a new build system from scratch that splits
>>>> the C++ community further ? There would be so much to gain with a better
>>>> relationship between Qt and CMake.
>>>>
>>>> Best,
>>>> -------
>>>> Jean-Michaël Celerier
>>>> http://www.jcelerier.name
>>>>
>>>> On Sat, Jul 21, 2018 at 2:42 PM, Kevin Kofler <***@chello.at>
>>>> wrote:
>>>>
>>>>> Bogdan Vatra via Development wrote:
>>>>> > Anyway IMHO is more important to have a clean, nice and easy to use
>>>>> syntax
>>>>> > and to be tooling friendly than 1.b.
>>>>>
>>>>> A custom build system is always a major pain point for distributions.
>>>>> A
>>>>> circular dependency (what Thiago's 1.b forbids) makes it particularly
>>>>> painful. How should we bootstrap new architectures or entirely new
>>>>> distributions if we cannot build Qt due to the circular dependency
>>>>> between
>>>>> Qt and its build tool? This is a showstopper.
>>>>>
>>>>> > GN[1] is another example of build system which didn't care too much
>>>>> about
>>>>> > 1.a,b,c and it still used in quite big projects (e.g. chrome,
>>>>> fuchsia). To
>>>>> > my huge surprise, they managed to move it into a separate repo and
>>>>> remove
>>>>> > all chromium dependencies (yep, a few months ago you had to checkout
>>>>> the
>>>>> > entire chromium repo to build it :) ).
>>>>>
>>>>> GN (and its predecessor Gyp) is universally hated by distribution
>>>>> packagers
>>>>> for its non-standardness, weirdness, lack of documentation (including
>>>>> third-
>>>>> party documentation such as tutorials, an issue inherent to custom
>>>>> build
>>>>> systems) and lack of flexibility (custom build systems are never as
>>>>> powerful
>>>>> as widely-used general-purpose build systems).
>>>>>
>>>>> QtWebEngine is a particular pain to package because it uses TWO custom
>>>>> build
>>>>> systems (QMake and GN).
>>>>>
>>>>> The Chromium mess is also what prompted Spot to write the list of
>>>>> FAILs
>>>>> [https://spot.livejournal.com/308370.html] I have already linked to
>>>>> elsewhere in this thread.
>>>>>
>>>>>
>>>>> There is a build system that fulfills all of Thiago's points, and it
>>>>> is
>>>>> already widely used in the Qt community: CMake.
>>>>>
>>>>> Kevin Kofler
>>>>>
>>>>> _______________________________________________
>>>>> Development mailing list
>>>>> ***@qt-project.org
>>>>> http://lists.qt-project.org/mailman/listinfo/development
>>>>>
>>>>
>>>> _______________________________________________
>>>> Development mailing list
>>>> ***@qt-project.org
>>>> http://lists.qt-project.org/mailman/listinfo/development
>>>>
>>>
>>

[Ray Donnelly]

On Sat, Jul 21, 2018 at 4:42 PM, Jean-Michaël Celerier
<***@gmail.com> wrote:
> That's fairly disingenuous, if not blatantly false. There are, like, 6 .cpp
> files in the CMake repo which provide the Qt-specific commands :
> https://github.com/Kitware/CMake/tree/master/Source (grep for cmQt).
>

What is disingenuous, if not blatantly false? Be specific please.

git clone https://gitlab.kitware.com/cmake/cmake.git

du -csh cmake/Source/cmQt*
8.0K cmake/Source/cmQtAutoGen.cxx
4.0K cmake/Source/cmQtAutoGen.h
52K cmake/Source/cmQtAutoGenInitializer.cxx
4.0K cmake/Source/cmQtAutoGenInitializer.h
24K cmake/Source/cmQtAutoGenerator.cxx
12K cmake/Source/cmQtAutoGenerator.h
64K cmake/Source/cmQtAutoGeneratorMocUic.cxx
12K cmake/Source/cmQtAutoGeneratorMocUic.h
20K cmake/Source/cmQtAutoGeneratorRcc.cxx
4.0K cmake/Source/cmQtAutoGeneratorRcc.h
204K total

I stand corrected, there *used* to be 150K of Qt specific C++ code in
CMake, but since I last checked it's grown to 204K (or perhaps I
counted it less well last time?)

And Giuseppe's devil's advocacy does speak to some of my issues with
CMake (along with it's lack of proper cross compilation support, it
defaulting to looking in *system* folders (which is never what any
non-chrooted distro wants).

Qbs is far better IMHO.

> Besides... why would it matter that they are implemented in C++ instead of
> cmake-lang ? If anything, CMake's automoc is in my experience much faster to
> process the whole repo.
>
> Richard: so what's this then ?
> https://github.com/qbs/qbs/blob/2d1de8cc84b258174b2dc0a8080f549cd9b59b32/src/lib/corelib/buildgraph/qtmocscanner.cpp
>
>
>
> -------
> Jean-Michaël Celerier
> http://www.jcelerier.name
>
> On Sat, Jul 21, 2018 at 4:39 PM, Ray Donnelly <***@gmail.com>
> wrote:
>>
>> CMake has its own 'language' that was retrofitted after the fact, evolving
>> from a simple definition based system.
>>
>> So why aren't all these special rules written in that 'language' then? 150
>> odd k of qt c++ hacks is what we get instead.
>>
>> On Sat, Jul 21, 2018, 3:26 PM Jean-Michaël Celerier
>> <***@gmail.com> wrote:
>>>
>>> > How much custom c++ code does it contains for just qt?
>>>
>>> which build system which supports automatic calling of moc doesn't have
>>> specific code for qt ?
>>>
>>>
>>>
>>>
>>> -------
>>> Jean-Michaël Celerier
>>> http://www.jcelerier.name
>>>
>>> On Sat, Jul 21, 2018 at 3:48 PM, Ray Donnelly <***@gmail.com>
>>> wrote:
>>>>
>>>> As someone who works on a cross platform distribution let me tell you
>>>> that cmake is plain terrible. How much custom c++ code does it contains for
>>>> just qt? Loads, absolutely tonnes or rubbish.
>>>>
>>>> On Sat, Jul 21, 2018, 1:49 PM Jean-Michaël Celerier
>>>> <***@gmail.com> wrote:
>>>>>
>>>>> > There is a build system that fulfills all of Thiago's points, and it
>>>>> > is
>>>>> already widely used in the Qt community: CMake.
>>>>>
>>>>> +1, I was flabbergasted when the big objection against CMake in Qt 6
>>>>> boiled down to "it does not supports all the architectures that Qt
>>>>> supports", so instead of contributing them - or hell, even forking CMake for
>>>>> those specific architectures (what are them ? I use cmake for windows, mac,
>>>>> linux, android, ios and the toolchain file allows for a lot of
>>>>> customization), what, create a new build system from scratch that splits the
>>>>> C++ community further ? There would be so much to gain with a better
>>>>> relationship between Qt and CMake.
>>>>>
>>>>> Best,
>>>>> -------
>>>>> Jean-Michaël Celerier
>>>>> http://www.jcelerier.name
>>>>>
>>>>> On Sat, Jul 21, 2018 at 2:42 PM, Kevin Kofler <***@chello.at>
>>>>> wrote:
>>>>>>
>>>>>> Bogdan Vatra via Development wrote:
>>>>>> > Anyway IMHO is more important to have a clean, nice and easy to use
>>>>>> > syntax
>>>>>> > and to be tooling friendly than 1.b.
>>>>>>
>>>>>> A custom build system is always a major pain point for distributions.
>>>>>> A
>>>>>> circular dependency (what Thiago's 1.b forbids) makes it particularly
>>>>>> painful. How should we bootstrap new architectures or entirely new
>>>>>> distributions if we cannot build Qt due to the circular dependency
>>>>>> between
>>>>>> Qt and its build tool? This is a showstopper.
>>>>>>
>>>>>> > GN[1] is another example of build system which didn't care too much
>>>>>> > about
>>>>>> > 1.a,b,c and it still used in quite big projects (e.g. chrome,
>>>>>> > fuchsia). To
>>>>>> > my huge surprise, they managed to move it into a separate repo and
>>>>>> > remove
>>>>>> > all chromium dependencies (yep, a few months ago you had to checkout
>>>>>> > the
>>>>>> > entire chromium repo to build it :) ).
>>>>>>
>>>>>> GN (and its predecessor Gyp) is universally hated by distribution
>>>>>> packagers
>>>>>> for its non-standardness, weirdness, lack of documentation (including
>>>>>> third-
>>>>>> party documentation such as tutorials, an issue inherent to custom
>>>>>> build
>>>>>> systems) and lack of flexibility (custom build systems are never as
>>>>>> powerful
>>>>>> as widely-used general-purpose build systems).
>>>>>>
>>>>>> QtWebEngine is a particular pain to package because it uses TWO custom
>>>>>> build
>>>>>> systems (QMake and GN).
>>>>>>
>>>>>> The Chromium mess is also what prompted Spot to write the list of
>>>>>> FAILs
>>>>>> [https://spot.livejournal.com/308370.html] I have already linked to
>>>>>> elsewhere in this thread.
>>>>>>
>>>>>>
>>>>>> There is a build system that fulfills all of Thiago's points, and it
>>>>>> is
>>>>>> already widely used in the Qt community: CMake.
>>>>>>
>>>>>> Kevin Kofler
>>>>>>
>>>>>> _______________________________________________
>>>>>> Development mailing list
>>>>>> ***@qt-project.org
>>>>>> http://lists.qt-project.org/mailman/listinfo/development
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Development mailing list
>>>>> ***@qt-project.org
>>>>> http://lists.qt-project.org/mailman/listinfo/development
>>>
>>>
>

[Jean-Michaël Celerier]

My bad, I thought that you were counting in Kloc of code, not in kilobytes.
That's in my opinion a weird metric for measuring code since indenting
with, say, 4 spaces instead of 1 tab would certainly end up giving wild
differences.

cloc cmake/Source/cmQt*
-------------------------------------------------------------------------------
Language files blank comment
code
-------------------------------------------------------------------------------
C++ 5 371 500
4362
C/C++ Header 5 126 173
740
-------------------------------------------------------------------------------
SUM: 10 497 673
5102
-------------------------------------------------------------------------------

I don't know for you, but 5kloc of modern C++ seems to be a fairly small
price to pay to parse all my .cpp automatically for moc macros (no need to
specify headers unlike qmake!), parallel execution of multiple moc /uic /
rcc instances (
https://cmake.org/cmake/help/v3.12/prop_tgt/AUTOGEN_PARALLEL.html), direct
support for multiple files with the same name (e.g. having two files with
Q_OBJECT named Settings.h in different folders won't cause a problem with
cmake, unlike qmake which puts everything in the root build folder by
default and will happilly overwrite a moc_Settings.cpp with another), etc (
https://cmake.org/cmake/help/v3.12/prop_tgt/AUTOMOC.html).

> Playing devil's advocate, please bear with me:

> Because it makes people worry that cmake-lang is too limited, and if you
need to do certain things (*), you need to resort to patch the tool itself.

But, which build system commonly used with Qt doesn't behave like this ?
qmake is cpp + qmake-lang, qbs is cpp + QML. AFAIK meson, waf, scons do not
support automatic moc'ing. At some point, some code has to be injected in
the build tool if you want to add automatic global behaviour - and I'd take
automatic global behaviour everyday over doing stuff like this:
http://mesonbuild.com/Qt5-module.html or
https://scons.org/doc/2.0.1/HTML/scons-user/a8524.html

-------
Jean-Michaël Celerier
http://www.jcelerier.name

On Sat, Jul 21, 2018 at 7:07 PM, Ray Donnelly <***@gmail.com>
wrote:

> On Sat, Jul 21, 2018 at 4:42 PM, Jean-Michaël Celerier
> <***@gmail.com> wrote:
> > That's fairly disingenuous, if not blatantly false. There are, like, 6
> .cpp
> > files in the CMake repo which provide the Qt-specific commands :
> > https://github.com/Kitware/CMake/tree/master/Source (grep for cmQt).
> >
>
> What is disingenuous, if not blatantly false? Be specific please.
>
> git clone https://gitlab.kitware.com/cmake/cmake.git
>
> du -csh cmake/Source/cmQt*
> 8.0K cmake/Source/cmQtAutoGen.cxx
> 4.0K cmake/Source/cmQtAutoGen.h
> 52K cmake/Source/cmQtAutoGenInitializer.cxx
> 4.0K cmake/Source/cmQtAutoGenInitializer.h
> 24K cmake/Source/cmQtAutoGenerator.cxx
> 12K cmake/Source/cmQtAutoGenerator.h
> 64K cmake/Source/cmQtAutoGeneratorMocUic.cxx
> 12K cmake/Source/cmQtAutoGeneratorMocUic.h
> 20K cmake/Source/cmQtAutoGeneratorRcc.cxx
> 4.0K cmake/Source/cmQtAutoGeneratorRcc.h
> 204K total
>
> I stand corrected, there *used* to be 150K of Qt specific C++ code in
> CMake, but since I last checked it's grown to 204K (or perhaps I
> counted it less well last time?)
>
> And Giuseppe's devil's advocacy does speak to some of my issues with
> CMake (along with it's lack of proper cross compilation support, it
> defaulting to looking in *system* folders (which is never what any
> non-chrooted distro wants).
>
> Qbs is far better IMHO.
>
> > Besides... why would it matter that they are implemented in C++ instead
> of
> > cmake-lang ? If anything, CMake's automoc is in my experience much
> faster to
> > process the whole repo.
> >
> > Richard: so what's this then ?
> > https://github.com/qbs/qbs/blob/2d1de8cc84b258174b2dc0a8080f54
> 9cd9b59b32/src/lib/corelib/buildgraph/qtmocscanner.cpp
> >
> >
> >
> > -------
> > Jean-Michaël Celerier
> > http://www.jcelerier.name
> >
> > On Sat, Jul 21, 2018 at 4:39 PM, Ray Donnelly <***@gmail.com>
> > wrote:
> >>
> >> CMake has its own 'language' that was retrofitted after the fact,
> evolving
> >> from a simple definition based system.
> >>
> >> So why aren't all these special rules written in that 'language' then?
> 150
> >> odd k of qt c++ hacks is what we get instead.
> >>
> >> On Sat, Jul 21, 2018, 3:26 PM Jean-Michaël Celerier
> >> <***@gmail.com> wrote:
> >>>
> >>> > How much custom c++ code does it contains for just qt?
> >>>
> >>> which build system which supports automatic calling of moc doesn't have
> >>> specific code for qt ?
> >>>
> >>>
> >>>
> >>>
> >>> -------
> >>> Jean-Michaël Celerier
> >>> http://www.jcelerier.name
> >>>
> >>> On Sat, Jul 21, 2018 at 3:48 PM, Ray Donnelly <***@gmail.com
> >
> >>> wrote:
> >>>>
> >>>> As someone who works on a cross platform distribution let me tell you
> >>>> that cmake is plain terrible. How much custom c++ code does it
> contains for
> >>>> just qt? Loads, absolutely tonnes or rubbish.
> >>>>
> >>>> On Sat, Jul 21, 2018, 1:49 PM Jean-Michaël Celerier
> >>>> <***@gmail.com> wrote:
> >>>>>
> >>>>> > There is a build system that fulfills all of Thiago's points, and
> it
> >>>>> > is
> >>>>> already widely used in the Qt community: CMake.
> >>>>>
> >>>>> +1, I was flabbergasted when the big objection against CMake in Qt 6
> >>>>> boiled down to "it does not supports all the architectures that Qt
> >>>>> supports", so instead of contributing them - or hell, even forking
> CMake for
> >>>>> those specific architectures (what are them ? I use cmake for
> windows, mac,
> >>>>> linux, android, ios and the toolchain file allows for a lot of
> >>>>> customization), what, create a new build system from scratch that
> splits the
> >>>>> C++ community further ? There would be so much to gain with a better
> >>>>> relationship between Qt and CMake.
> >>>>>
> >>>>> Best,
> >>>>> -------
> >>>>> Jean-Michaël Celerier
> >>>>> http://www.jcelerier.name
> >>>>>
> >>>>> On Sat, Jul 21, 2018 at 2:42 PM, Kevin Kofler <
> ***@chello.at>
> >>>>> wrote:
> >>>>>>
> >>>>>> Bogdan Vatra via Development wrote:
> >>>>>> > Anyway IMHO is more important to have a clean, nice and easy to
> use
> >>>>>> > syntax
> >>>>>> > and to be tooling friendly than 1.b.
> >>>>>>
> >>>>>> A custom build system is always a major pain point for
> distributions.
> >>>>>> A
> >>>>>> circular dependency (what Thiago's 1.b forbids) makes it
> particularly
> >>>>>> painful. How should we bootstrap new architectures or entirely new
> >>>>>> distributions if we cannot build Qt due to the circular dependency
> >>>>>> between
> >>>>>> Qt and its build tool? This is a showstopper.
> >>>>>>
> >>>>>> > GN[1] is another example of build system which didn't care too
> much
> >>>>>> > about
> >>>>>> > 1.a,b,c and it still used in quite big projects (e.g. chrome,
> >>>>>> > fuchsia). To
> >>>>>> > my huge surprise, they managed to move it into a separate repo and
> >>>>>> > remove
> >>>>>> > all chromium dependencies (yep, a few months ago you had to
> checkout
> >>>>>> > the
> >>>>>> > entire chromium repo to build it :) ).
> >>>>>>
> >>>>>> GN (and its predecessor Gyp) is universally hated by distribution
> >>>>>> packagers
> >>>>>> for its non-standardness, weirdness, lack of documentation
> (including
> >>>>>> third-
> >>>>>> party documentation such as tutorials, an issue inherent to custom
> >>>>>> build
> >>>>>> systems) and lack of flexibility (custom build systems are never as
> >>>>>> powerful
> >>>>>> as widely-used general-purpose build systems).
> >>>>>>
> >>>>>> QtWebEngine is a particular pain to package because it uses TWO
> custom
> >>>>>> build
> >>>>>> systems (QMake and GN).
> >>>>>>
> >>>>>> The Chromium mess is also what prompted Spot to write the list of
> >>>>>> FAILs
> >>>>>> [https://spot.livejournal.com/308370.html] I have already linked to
> >>>>>> elsewhere in this thread.
> >>>>>>
> >>>>>>
> >>>>>> There is a build system that fulfills all of Thiago's points, and it
> >>>>>> is
> >>>>>> already widely used in the Qt community: CMake.
> >>>>>>
> >>>>>> Kevin Kofler
> >>>>>>
> >>>>>> _______________________________________________
> >>>>>> Development mailing list
> >>>>>> ***@qt-project.org
> >>>>>> http://lists.qt-project.org/mailman/listinfo/development
> >>>>>
> >>>>>
> >>>>> _______________________________________________
> >>>>> Development mailing list
> >>>>> ***@qt-project.org
> >>>>> http://lists.qt-project.org/mailman/listinfo/development
> >>>
> >>>
> >
>

[Alexander Neundorf]

On 2018 M07 21, Sat 18:24:23 CEST Giuseppe D'Angelo via Development wrote:
> Il 21/07/2018 17:42, Jean-Michaël Celerier ha scritto:
> > Besides... why would it matter that they are implemented in C++ instead
> > of cmake-lang ? If anything, CMake's automoc is in my experience much
> > faster to process the whole repo.
>
> Playing devil's advocate, please bear with me:
>
>
> Because it makes people worry that cmake-lang is too limited, and if you
> need to do certain things (*), you need to resort to patch the tool itself.
>
> Which opens other sets of problems, such as:
>
> * what are the things that I can do in cmake-lang and the things I can't
> do?

you can do almost everything in cmake-lang ;-)
By now, in 2018, most build-related challenges should have been taken care of
in CMake.
I think the biggest issues are:
- IDE integration: cmake server mode is still relatively new and and I don't
know exactly how powerful it currently is
- better documentation/tutorial especially for old vs. new-style cmake
- new technology that appaers with new operating system features etc. often
needs a new cmake release

> * Does all of this imply having changes landing in CMake itself, or can
> one have some sort of plugin system so that a project can ship the CMake
> C++ plugins to build it?

there is intentionally no plugin system/public API in CMake, to keep changing
the internals of cmake itself easy.
But really a lot can be done via cmake scripts.

> * If the former, what happens if my users are running a "slightly older"
> CMake that doesn't have my patches in it yet?

The older cmake will error out saying that the project requires a newer
version of CMake.

Alex

[Richard Weickelt]

>> How much custom c++ code does it contains for just qt?
>
> which build system which supports automatic calling of moc doesn't have
> specific code for qt ?
Qbs :-)
At least no C++ code.

[Tobias Hunger]

On Sat, Jul 21, 2018, 14:49 Jean-Michaël Celerier <
***@gmail.com> wrote:

> +1, I was flabbergasted when the big objection against CMake in Qt 6
> boiled down to "it does not supports all the architectures that Qt
> supports",
>

IIRC the bid showstopper was that CMake does not support building host
tools and the target libraries in one go.

That requirement was dropped since then in our side.

Best regards,
Tobias

[Thiago Macieira]

On Saturday, 21 July 2018 05:42:08 PDT Kevin Kofler wrote:
> Bogdan Vatra via Development wrote:
> > Anyway IMHO is more important to have a clean, nice and easy to use syntax
> > and to be tooling friendly than 1.b.
>
> A custom build system is always a major pain point for distributions. A
> circular dependency (what Thiago's 1.b forbids) makes it particularly
> painful. How should we bootstrap new architectures or entirely new
> distributions if we cannot build Qt due to the circular dependency between
> Qt and its build tool? This is a showstopper.

C'mon, this is easy to solve. Just make sure that the build tool can be built
with other tools, at least a minimal version to bootstrap itself. Qbs today
has a qmake-based build. Qmake can be built with a simple Makefile, one that
doesn't even require GNU Make. CMake has a bootstrap mode too.

The problem of the Qt dependency can be solved by copying the necessary files.

Why are we even discussing (1.b)? That's got a technical solution.

I thought that (2) and (3) would be a lot more contentious. Especially (2.c).

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center

[Christian Gagneraud]

On 22 July 2018 at 00:42, Kevin Kofler <***@chello.at> wrote:
> Bogdan Vatra via Development wrote:
>> Anyway IMHO is more important to have a clean, nice and easy to use syntax
>> and to be tooling friendly than 1.b.
>
> A custom build system is always a major pain point for distributions. A
> circular dependency (what Thiago's 1.b forbids) makes it particularly
> painful. How should we bootstrap new architectures or entirely new
> distributions if we cannot build Qt due to the circular dependency between
> Qt and its build tool? This is a showstopper.

How do you build gcc on Debian? Which compiler do you use? How do you
build the afore mentioned compiler in the first place?
Bootstrapping is a natural process in SW dev, and is and has always
been a long meander. Implementation details don't matter.
AFAIK, Debian cannot even be cross-compiled (Debian ports have never
been build this way)....

CMake has a special bootstrapping mode, QMake has one, and any other
build tools (can) have their own, this includes Qbs, and many other..

IMHO, this self-contained, circular dependency discussion is moot.
As mentioned by Thiago himself, this is a non-problem, it can be
easily solved for most build system that are on the table.

Can we stop for a moment about self/circular dependency and bootstrapping?
Let's move on more serious topics.

Chris

[Lisandro Damián Nicanor Pérez Meyer]

El dom., 22 de jul. de 2018 06:40, Christian Gagneraud <***@gmail.com>
escribió:

> On 22 July 2018 at 00:42, Kevin Kofler <***@chello.at> wrote:
> > Bogdan Vatra via Development wrote:
> >> Anyway IMHO is more important to have a clean, nice and easy to use
> syntax
> >> and to be tooling friendly than 1.b.
> >
> > A custom build system is always a major pain point for distributions. A
> > circular dependency (what Thiago's 1.b forbids) makes it particularly
> > painful. How should we bootstrap new architectures or entirely new
> > distributions if we cannot build Qt due to the circular dependency
> between
> > Qt and its build tool? This is a showstopper.
>
> How do you build gcc on Debian? Which compiler do you use? How do you
> build the afore mentioned compiler in the first place?
> Bootstrapping is a natural process in SW dev, and is and has always
> been a long meander. Implementation details don't matter.
>

I think this point needs a clarification (not for Christian, as you can
read below). A self-bootstrapeable tarball/source is enough, like qmake in
qtbase does.

What we packagers do not want is a source A that needs source B to build
but source B needs source A (or the same thing by adding more sources in
the middle).

AFAIK, Debian cannot even be cross-compiled (Debian ports have never
> been build this way)....
>

That's changing quite fast actually, at the point that people are being
able to bootstrap archs by cross compiling.

We can even cross compile sources using Qt and most (if not all) of Qt
itself by using multi-arch.


CMake has a special bootstrapping mode, QMake has one, and any other
> build tools (can) have their own, this includes Qbs, and many other..
>
> IMHO, this self-contained, circular dependency discussion is moot.
> As mentioned by Thiago himself, this is a non-problem, it can be
> easily solved for most build system that are on the table.
>

Right.


Can we stop for a moment about self/circular dependency and bootstrapping?
> Let's move on more serious topics.


Well, the difference between self and circular really does makes a change
for us packagers. Really.

>

[Thiago Macieira]

On Friday, 20 July 2018 19:35:48 PDT Thiago Macieira wrote:
> c) Must not require too-recent version in order to compile Qt. The versions
> found in ( a) must be sufficient to build Qt 6.0 and this must continue
> throughout Qt 6's lifetime.

Clarification: I don't mean the versions found at 6.0's release are the ones
used for 6.18. I meant that throughout 6.x lifetime, we should build with
whatever is packaged in distros relevant for each release, not requiring
updates.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center

[Иван Комиссаров]

<offtop_mode_on>
What you wrote reminds me how government officials spent budget money for the new cars in my country. They write specific tender documentation that fits only one car, say Chevrolet Tahoe (for example, "the car that fits 9 people, has leather interior and has the power of 200hp»). The argumentation is the same - here’s the requirements, who cares that the only one overpriced car fits in them.
<offtop_mode_off>

I have several other requirements for the buildsystem.
1. IDE integration. I really need the IDE know the correct CXX flags for each separate file in my project - that is includes, c++ version and so on. Currently i’m using a generated CMake file at my work and the integration with Qt Creator is awful, i’m using IDE just as a simple text editor - syntax highlighting breaks because IDE can’t find some headers (however, goto definition works for them).
2. Support for cross-compilation and deploying Android/iOS. I’ve used a GYP at work for a QML-based app. I spent a whole week porting Qt application for iOS writing those unreadable gyp files. Not sure if CMake will be much easier, AFAIK, neither QBS nor CMake have full support for iOS and Android deployment in Qt Creator (so i can just press «run» and IDE will do everything for me).

However, i don’t see big problems in implementing this with QBS (i made several small patches and i can say that the codebase is neat). However, I won’t be the guy who volunteer to do anything in CMake. Yes, CMake has much more users than QBS, however, how many of them will volunteer to work with CMake’s Qt Creator integration? 5 people? 10?

We will end with a separate team of Qt guys working on CMake’s integration with Qt and it’s tools. So the question is - is it easier to port QBS to be built without Qt or to use CMake with Qt?

I pay my 2 cents for QBS.

> 21 июля 2018 г., в 5:35, Thiago Macieira <***@intel.com> написал(а):
>
> Hello
>
> Having spent far too much time trying to figure out why crappy buildsystems
> cause failures in distros (like TensorFlow or libvpx - see
> https://plus.google.com/+ThiagoMacieira/posts/DqTKdRGfuwR ), I'd like to make
> sure this doesn't happen to Qt 6. For that, I'd like to add the following
> extra requirements to the buildsystem we choose to use. This is in addition to
> the functionality requirements.
>
> These apply to the buildsystem at the time of Qt's the switch to it.
>
> 1) Ease of obtention
>
> a) Must be packaged by all major package managers where Qt 6 is expected to be
> relevant. That is, it must be available as a package in:
> - ArchLinux
> - Debian testing
> - Gentoo
> - Fedora current and previous
> - openSUSE current and previous
> - Ubuntu LTS released more than 6 months prior
> - Homebrew (macOS)
>
> I'll take care of Clear Linux, but we apply the "two major distros" rule, so I
> won't be first. It would be nice to have it in vcpkg/nuget/whatever MSVC uses
> and FreeBSD ports tree too.
>
> b) Must be easily compiled from source on a standard system installation. All
> of its dependencies must come from the system's package manager and there must
> be no cyclic dependencies. That is, it cannot require itself to build and it
> must not depend on Qt, either in source form or binary.
>
> c) Must not require too-recent version in order to compile Qt. The versions
> found in (a) must be sufficient to build Qt 6.0 and this must continue
> throughout Qt 6's lifetime.
>
> 2) Track record
>
> a) Must be used by roughly a dozen packages in major distros. I'm not saying
> all of them have to have a dozen, not even that one of them has a dozen. But
> all of the ones listed above must have at least one and there must be a dozen
> distinct packages in total.
>
> b) At least one package must have been continuously packaged for two years.
> One for an RPM distro and one for a .deb distro.
>
> c) At least one package of comparable complexity to qtbase, packaged by the
> three of the six Linux distros I listed. I'm looking for:
> - compiling certain files with different compiler flags
> - several third-party dependencies, some required, others optional
> - lots of configure-time options
> - installing several different types of targets (binaries, libraries,
> plugins, arch-dependent files, arch-independent files, documentation,
> translations, etc.)
> - obeying distro-supplied options (compiler & linker flags, compiler
> selection [ccache, icecc, clang], debuginfo split, stripping, etc.)
>
> 3) Community support
> a) Thriving community to ask for help to, whenever issues happen.
> b) One packager for .deb and one for RPM that will vouch for the buildsystem.
>
>
> I don't think I am being unreasonable. I am being strict, though.
>
> I'll put my money where my mouth is: as soon as the requirements are met, I
> will begin using it and contributing to it, finding out where there may be
> shortcomings and supplying fixes if possible, bug reports if not.
>
> But until then, I will pretend it doesn't exist and will ignore the branch
> that uses it to build.
> --
> Thiago Macieira - thiago.macieira (AT) intel.com
> Software Architect - Intel Open Source Technology Center
>
>
>
> _______________________________________________
> Development mailing list
> ***@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development

[Kevin Kofler]

Иван Комиссаров wrote:
> I have several other requirements for the buildsystem.
> 1. IDE integration. I really need the IDE know the correct CXX flags for
> each separate file in my project - that is includes, c++ version and so
> on. Currently i’m using a generated CMake file at my work and the
> integration with Qt Creator is awful, i’m using IDE just as a simple text
> editor - syntax highlighting breaks because IDE can’t find some headers
> (however, goto definition works for them).

https://www.kdevelop.org/
:-)

Kevin Kofler

[Thiago Macieira]

On Saturday, 21 July 2018 10:39:49 PDT Иван Комиссаров wrote:
> 1. IDE integration. I really need the IDE know the correct CXX flags for
> each separate file in my project - that is includes, c++ version and so on.
> Currently i’m using a generated CMake file at my work and the integration
> with Qt Creator is awful, i’m using IDE just as a simple text editor -
> syntax highlighting breaks because IDE can’t find some headers (however,
> goto definition works for them).

That's already more than what we have for qmake, so I wouldn't put this as a
requirement. It's a nice-to-have.

The point is that the tool has to be at least in feature parity with qmake and
that includes cross-compilation and IDE support.

> Yes, CMake has much more users than QBS, however, how many of them will
> volunteer to work with CMake’s Qt Creator integration? 5 people? 10?

5 people would be more than what we currently have for either qmake or qbs
today, so seems ok.

> So the question is - is it easier to port QBS to be built without Qt or to
> use CMake with Qt?

Again, solving the build is easy. Please look into the requirements at (2).
THAT is a lot harder. And all I'm asking is for experience. I don't want Qt to
be the guinea pig.

More to the point: I will not be part of the experimentation. Give me a mature
tool.
--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center

[Иван Комиссаров]

> 21 Оюля 2018 г., в 23:04, Thiago Macieira <***@intel.com> МапОсал(а):
>
> On Saturday, 21 July 2018 10:39:49 PDT ИваМ КПЌОссарПв wrote:
>> 1. IDE integration. I really need the IDE know the correct CXX flags for
>> each separate file in my project - that is includes, c++ version and so on.
>> Currently i’m using a generated CMake file at my work and the integration
>> with Qt Creator is awful, i’m using IDE just as a simple text editor -
>> syntax highlighting breaks because IDE can’t find some headers (however,
>> goto definition works for them).
>
> That's already more than what we have for qmake, so I wouldn't put this as a
> requirement. It's a nice-to-have.

But only qmake works fine with new clang model, other build systems pass incorrect parameters to clang so it can’t compile my files (code model complains about missing std:: classes/functions, however the code compiles so project file is correct)

>
> The point is that the tool has to be at least in feature parity with qmake and
> that includes cross-compilation and IDE support.
>
>> Yes, CMake has much more users than QBS, however, how many of them will
>> volunteer to work with CMake’s Qt Creator integration? 5 people? 10?
>
> 5 people would be more than what we currently have for either qmake or qbs
> today, so seems ok.

That’s true, but despite huge CMake community, Qt Creator’s CMake integration still is not in feature-parity with qmake.
I had a discussion with one of QBS developer about iOS integration in Qt Creator and he told that Qt Creator uses some qmake hacks that are not viable for QBS (yet). Hacking CMake may be much trickier as codebase is not controlled by Qt community.

>
>> So the question is - is it easier to port QBS to be built without Qt or to
>> use CMake with Qt?
>
> Again, solving the build is easy. Please look into the requirements at (2).
> THAT is a lot harder. And all I'm asking is for experience. I don't want Qt to
> be the guinea pig.

I used QBS and CMake for building debian packages in 2015 at one of my previous works. All you need for QBS is to use usual makefile based debian/rules files and replace «make install» with «qbs install».
Here’s the actual code i used to build my tool (except i replace real package/tool names with «projectname») https://pastebin.com/RbPL8GAq <https://pastebin.com/RbPL8GAq>
That’s it. You pointing too much attention to the Linux distros and maintainers, which is very important, but you are using wrong arguments. Give those maintainers a good tool and they will be happy. And the «good tool» is not the same as «tool has been used for at least 2 years». People has been using plain makefiles for decades, people has been using autotools for years. But i guess anyone agrees that autotools sucks.
The same is for CMake - people are using it because they don’t have any alternative. QBS can become that alternative. In case if it will not be buried alive.
Common, how you can compare the tools if you didn’t even used QBS? I’ve been using qmake since 2011 i’ve been using CMake since 2013, i worked both with CMake/QBS to create debian packages, i had to work with GYP in 2016, now i’m using QBS for all of my projects. So i can compare tools not by providing abstract requirements, but from the experience of using all those tools.

>
> More to the point: I will not be part of the experimentation. Give me a mature
> tool.

The autotools are mature:)

PS: BTW, in first letter i forgot to mention that the company i’m working right now ended up with writing it’s own build tool just because CMake didn’t fit all requirements we need from a build tool. We’ve been using CMake, then we wrap all the CMake mess in a set of small macroses (PROGRAM(name), LIBRARY and so on) and then replaced cmake with the tool that understands only those macroses. I don’t know exact reasons why CMake was dropped (when i came to the company, the only thing left from CMake was the project file names - CMakeLists.txt), however it’s true that CMake has a lot of tricky parts when you start using it.

The same is true for Google - why do you think they are experimenting with GYP and GN? CMake just doesn’t fits them very well (neither it will fit Qt or anyone else), but i guess they have no alternative (yet).

> --
> Thiago Macieira - thiago.macieira (AT) intel.com
> Software Architect - Intel Open Source Technology Center
>
>
>
> _______________________________________________
> Development mailing list
> ***@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development

[Jean-Michaël Celerier]

Sounds like you have a configuration problem in Qt Creator. I've been using
qtc with the clang code model and CMake projects for over a year on various
C++11/14/17 projects and everything works fine.

$ echo '#include <string_view>\nint main() { std::string_view s("foo"); }'
> main.cpp ; echo 'project(foo CXX)\nset(CMAKE_CXX_STANDARD
17)\nadd_executable(foo main.cpp)' > CMakeLists.txt

results for me in correct completion of std::string_view in qtc.



-------
Jean-Michaël Celerier
http://www.jcelerier.name

On Sun, Jul 22, 2018 at 12:57 PM, ИваМ КПЌОссарПв <***@gmail.com> wrote:

>
> 21 Оюля 2018 г., в 23:04, Thiago Macieira <***@intel.com>
> МапОсал(а):
>
> On Saturday, 21 July 2018 10:39:49 PDT ИваМ КПЌОссарПв wrote:
>
> 1. IDE integration. I really need the IDE know the correct CXX flags for
> each separate file in my project - that is includes, c++ version and so on.
> Currently i’m using a generated CMake file at my work and the integration
> with Qt Creator is awful, i’m using IDE just as a simple text editor -
> syntax highlighting breaks because IDE can’t find some headers (however,
> goto definition works for them).
>
>
> That's already more than what we have for qmake, so I wouldn't put this as
> a
> requirement. It's a nice-to-have.
>
>
> But only qmake works fine with new clang model, other build systems pass
> incorrect parameters to clang so it can’t compile my files (code model
> complains about missing std:: classes/functions, however the code compiles
> so project file is correct)
>
>
> The point is that the tool has to be at least in feature parity with qmake
> and
> that includes cross-compilation and IDE support.
>
> Yes, CMake has much more users than QBS, however, how many of them will
> volunteer to work with CMake’s Qt Creator integration? 5 people? 10?
>
>
> 5 people would be more than what we currently have for either qmake or qbs
> today, so seems ok.
>
>
> That’s true, but despite huge CMake community, Qt Creator’s CMake
> integration still is not in feature-parity with qmake.
> I had a discussion with one of QBS developer about iOS integration in Qt
> Creator and he told that Qt Creator uses some qmake hacks that are not
> viable for QBS (yet). Hacking CMake may be much trickier as codebase is not
> controlled by Qt community.
>
>
> So the question is - is it easier to port QBS to be built without Qt or to
> use CMake with Qt?
>
>
> Again, solving the build is easy. Please look into the requirements at
> (2).
> THAT is a lot harder. And all I'm asking is for experience. I don't want
> Qt to
> be the guinea pig.
>
>
> I used QBS and CMake for building debian packages in 2015 at one of my
> previous works. All you need for QBS is to use usual makefile based
> debian/rules files and replace «make install» with «qbs install».
> Here’s the actual code i used to build my tool (except i replace real
> package/tool names with «projectname») https://pastebin.com/RbPL8GAq
> That’s it. You pointing too much attention to the Linux distros and
> maintainers, which is very important, but you are using wrong arguments.
> Give those maintainers a good tool and they will be happy. And the «good
> tool» is not the same as «tool has been used for at least 2 years». People
> has been using plain makefiles for decades, people has been using autotools
> for years. But i guess anyone agrees that autotools sucks.
> The same is for CMake - people are using it because they don’t have any
> alternative. QBS can become that alternative. In case if it will not be
> buried alive.
> Common, how you can compare the tools if you didn’t even used QBS? I’ve
> been using qmake since 2011 i’ve been using CMake since 2013, i worked both
> with CMake/QBS to create debian packages, i had to work with GYP in 2016,
> now i’m using QBS for all of my projects. So i can *compare* tools not by
> providing abstract requirements, but from the experience of *using *all
> those tools.
>
>
>
> More to the point: I will not be part of the experimentation. Give me a
> mature
> tool.
>
>
> The autotools are mature:)
>
> PS: BTW, in first letter i forgot to mention that the company i’m working
> right now ended up with writing it’s own build tool just because CMake
> didn’t fit all requirements we need from a build tool. We’ve been using
> CMake, then we wrap all the CMake mess in a set of small macroses
> (PROGRAM(name), LIBRARY and so on) and then replaced cmake with the tool
> that understands only those macroses. I don’t know exact reasons why CMake
> was dropped (when i came to the company, the only thing left from CMake was
> the project file names - CMakeLists.txt), however it’s true that CMake has
> a lot of tricky parts when you start using it.
>
> The same is true for Google - why do you think they are experimenting with
> GYP and GN? CMake just doesn’t fits them very well (neither it will fit Qt
> or anyone else), but i guess they have no alternative (yet).
>
> --
> Thiago Macieira - thiago.macieira (AT) intel.com
> Software Architect - Intel Open Source Technology Center
>
>
>
> _______________________________________________
> Development mailing list
> ***@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development
>
>
>
> _______________________________________________
> Development mailing list
> ***@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development
>
>

[Thiago Macieira]

On Sunday, 22 July 2018 03:57:37 PDT Иван Комиссаров wrote:
> That’s it. You pointing too much attention to the Linux distros and
> maintainers, which is very important, but you are using wrong arguments.
> Give those maintainers a good tool and they will be happy. And the «good
> tool» is not the same as «tool has been used for at least 2 years». People
> has been using plain makefiles for decades, people has been using autotools
> for years. But i guess anyone agrees that autotools sucks.

You're right, "good tool" does not mean "tool has been used for 2 years". Yet
I stand by what I said: I want a good tool, but *need* experience. The point
of 2 years is not to let them fight out to figure out which ones is best. The
point is to make sure the kinks are ironed out and that people have dealt with
them.

Have they figured out how to do debugging when things go wrong? Autoconf
produces a very detailed config.log; qmake does a similar one for qtbase, but
has the -d -d options that almost always explain to me how things were wrong.
CMake is a lot harder and I hate debugging it, but in the worst case I know
there are people I can turn to for help. I want to say the same for the tool
we choose.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center

[Tobias Hunger]

On Sun, Jul 22, 2018, 12:58 ИваМ КПЌОссарПв <***@gmail.com> wrote:

> But only qmake works fine with new clang model, other build systems pass
> incorrect parameters to clang so it can’t compile my files (code model
> complains about missing std:: classes/functions, however the code compiles
> so project file is correct)
>

Please file but reports for those cases then. That *should* work and it
does work with all my test projects.

That’s true, but despite huge CMake community, Qt Creator’s CMake
> integration still is not in feature-parity with qmake.
>

CMake had never been a first class citizen in creator: Noboby inside the Qt
company used cmake in earnest, so nobody cared for CMake support -- once we
had rudimentary functionality in place.

When I spend some month improving CMake support in creator the feedback was
fantastic and there was and still is *lots* of it! There are *LOTS* of
people using qt creator and CMake out there!

But you are right: Some features are not possible with cmake. This includes
most operations that involve modifying CMakeLists.txt files directly,
including e.g. adding files to a target. One can hack something that will
probably work most of the time, but that is as good as it will get.

That is the same situation we also have with qbs by the way: Add some JS
into the wrong place and toolability is gone... Do stuff sensibly and all
is well.

Best regards,
Tobias

[Lisandro Damián Nicanor Pérez Meyer]

El vie., 20 de jul. de 2018 23:36, Thiago Macieira <
***@intel.com> escribió:

> Hello
>
> Having spent far too much time trying to figure out why crappy
> buildsystems
> cause failures in distros (like TensorFlow or libvpx - see
> https://plus.google.com/+ThiagoMacieira/posts/DqTKdRGfuwR ), I'd like to
> make
> sure this doesn't happen to Qt 6. For that, I'd like to add the following
> extra requirements to the buildsystem we choose to use. This is in
> addition to
> the functionality requirements.
>
> These apply to the buildsystem at the time of Qt's the switch to it.
>

Agree to all of them, *specially* 1.b.
I would also argue that API/ABI should be stable enough. I mean: a new
version should not require rewriting the building rules.

[Jason Newton]

I think this is an incredibly important issue! Sorry if this creates
a new thread, I'm hopping on the ML and I don't have any previous
mails to reply-to.

I wanted to mention that this is on my mind alot for a few years days
as a user for a plethora of libraries. My conclusion for the build
system with the brightest future is bazel and here's a few words why.
But first, let me direct to a ticket I filed last year on the subject
of Qt creator support of bazel:
https://bugreports.qt.io/browse/QTCREATORBUG-19085 - I really wish it
was something more trivial to integrate.

There are too many options right now and seeming none of them are
clear winners across the board, though several have shiny features
that as a former cmake purist, made me wish I had some of those
things. I'd most be interested in meson if I chose to have a limited
scope and cmake wins out on "most" libraries supported, which seems to
have taken 5-7 years in my own experience for that shift to happen -
meaning an official or community based FindLibrary script being
available. Cmake has verbosity and limitations though so while it
works, and is much better than a make based solution, when it comes to
jumping out 95% builtin coverage, particularly where it tends to go
more programming language/composition it has problems. I'd support
cmake library users, but I wouldn't put the lion share of my effort in
it. Meson has deterministic builds and a clean language but I can get
that with bazel too - I essentially look at both of these projects as
a very common set of features, but bazel features and capabilities
keep growing where as I'm not sure where meson will go. Both meson
and bazel suffer from few out of the box, batteries included modules
for software libraries, leaving users to scrape github (like cmake in
its earlier years of mass adoption) - this is a problem that will go
away one way or another in time, and an official repo or library with
mass collection of them as well as upstream project support for more
non-trivial libraries (qt is one of these!) would go miles.

Why the qt project may be interested in this in short is:
-Probably the most unbounded future of any existing build system,
development is quite active with a growing feature set - this is not a
simple community effort but fused (although mainly driven by google)
and I think that's why you can count on this remaining true and ahead
of the pack.
-Time tested, running in production environments offering the lowest
barrier of entry to "get going" for bazel based projects being built
by users, even for non-programmer types. As usual YMMV but I can
start a non-software person on a bazel project in a matter of minutes
and have a large project functioning on a machine - stories exist like
this from google and the like.
-Short, python based syntax but much less complicated files than
something like scons, while still allowing real programming when
needed.
-Fast and scaling bazel uses a long lived server technique that hangs
around and lower restart times and combined with solid dependency
analysis and a built in watching for file-system activity (e.g.
inotify) ensures smart rebuilding as well as incremental builds - even
as part of the test runner, which has access to this information such
that only what might have changed is tested. Large projects like qt
build incredibly fast and efficiently - through a variety of
strategies bazel takes. Local build caches, external cache servers -
can greatly speed up CI based testing (meaning developers wait less)
and are integrated in a vastly superior way in contrast to say distcc.
The amount of time that useful work is being performed is greater
than I've observed in large cmake projects, even using ninja (assuming
ninja would work across projects that size, due to frequent make
sensitivities)
-Ability to build external libraries from source or pull in binary
libraries - the former offering alot of power that comes with custom
flags and controlled dependencies for users developing products, and
the later for distribution like software - the choice is always there
and low barrier of entry. One can fuse approaches where it makes
sense.
-Cross-platform builds work, esoteric stuff does too - docker, android
and specialized compilers - not hard to hook in other toolchains
(easier/less error prone than cmake)
-Cross language under a single build system - python, sh, c/c++, go,
obj-c - all have first class support and can "see" eachother in terms
of dependency tracking/targets.

I have built alot of software against bazel - hdf5, boost, and common
simple libraries like tif/png/jpeg, zlib to list a prominent few and
it's taken those projects like a champ - generally being smooth
sailing.

Could it have done better or easier? Yes hdf5 for instance is
trickier than most due to autoconf based configuration - I hope things
will keep getting easier with it (bazel). That happens one project at
a time, but mega projects like qt can be instrumental in making that
happen - both solving the problems unique to qt on bazel (changing
things that needed to be changed) as well as providing the
interface/build rules to correctly use qt6 (official sdk, system
installs etc) - both being amplifiers of usage.

Cross-platform prebuilt binaries are available from here:
https://github.com/bazelbuild/bazel/releases - it can be done fairly
stand alone from distribution (by design) though - it's not too hard
to package as a deb, rpm, or tar, even if you don't use the
facilitated methods of doing that.

-Jason

[Thiago Macieira]

On Saturday, 21 July 2018 16:52:44 PDT Jason Newton wrote:
> -Ability to build external libraries from source or pull in binary
> libraries

This is not a feature. It's a misfeature.

It's EXACTLY the reason our seniormost engineers spent three days trying to
upgrade TensorFlow in Clear Linux. That's not your junior guy who only knows
how to run "npn install". We're talking about people who had been building
stuff on Linux years before you'd ever heard about Linux.

Anyway, you'll see in my original email that I did not name any buildsytem. I
only set forth a set of requirements that any buildsystem worth its salt
should be able to meet. We're more than two years away from switching to a new
buildsystem, so even if a new one sprung up tomorrow, it would have enough
time to meet the (2.b) requirement of one package packaged continuously for 2
years.

So maybe bazel has a shot. Or ninja. Or scons. Who knows.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center

[Christian Gagneraud]

On 22 July 2018 at 14:05, Thiago Macieira <***@intel.com> wrote:
> On Saturday, 21 July 2018 16:52:44 PDT Jason Newton wrote:
>> -Ability to build external libraries from source or pull in binary
>> libraries
>
> This is not a feature. It's a misfeature.
>
> It's EXACTLY the reason our seniormost engineers spent three days trying to
> upgrade TensorFlow in Clear Linux. That's not your junior guy who only knows
> how to run "npn install". We're talking about people who had been building
> stuff on Linux years before you'd ever heard about Linux.

Clear Linux, hum? Are they re-inventing the wheel? And the hammer, the
nail, and the saw, what have you...

"Kata Container with Clear Linux"? What an obscure technology, and
what does it has to do with the Qt project and build system in
general?
So because Intel re-invents the wheel the Qt project shouldn't use
this or that? Honestly, I fail to follow you here.

Chris

[Ray Donnelly]

On Sun, Jul 22, 2018, 10:20 AM Christian Gagneraud <***@gmail.com> wrote:

> On 22 July 2018 at 14:05, Thiago Macieira <***@intel.com>
> wrote:
> > On Saturday, 21 July 2018 16:52:44 PDT Jason Newton wrote:
> >> -Ability to build external libraries from source or pull in binary
> >> libraries
> >
> > This is not a feature. It's a misfeature.
> >
> > It's EXACTLY the reason our seniormost engineers spent three days trying
> to
> > upgrade TensorFlow in Clear Linux. That's not your junior guy who only
> knows
> > how to run "npn install". We're talking about people who had been
> building
> > stuff on Linux years before you'd ever heard about Linux.
>
> Clear Linux, hum? Are they re-inventing the wheel? And the hammer, the
> nail, and the saw, what have you...
>

On the Anaconda Distribution we expend significant ongoing engineer time
battling bazel too. We all detest it.


> "Kata Container with Clear Linux"? What an obscure technology, and
> what does it has to do with the Qt project and build system in
> general?
> So because Intel re-invents the wheel the Qt project shouldn't use
> this or that? Honestly, I fail to follow you here.
>
> Chris
> _______________________________________________
> Development mailing list
> ***@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development
>

[Thiago Macieira]

On Sunday, 22 July 2018 02:20:04 PDT Christian Gagneraud wrote:
> > It's EXACTLY the reason our seniormost engineers spent three days trying
> > to
> > upgrade TensorFlow in Clear Linux. That's not your junior guy who only
> > knows how to run "npn install". We're talking about people who had been
> > building stuff on Linux years before you'd ever heard about Linux.
>
> Clear Linux, hum? Are they re-inventing the wheel? And the hammer, the
> nail, and the saw, what have you...

<off-topic>
We are reinventing the wheel in some things, but in this we're not. We're
simply trying to make an RPM .spec file that builds the latest version of
TensorFlow. There are at least a dozen other distributions who need to do
that, using a 20+-year-old technology (RPM).

Specifically, the problem with TensorFlow is that it wants to download its
dependencies when you compile it, then compile them at the same time. That
can't work because the build servers don't have internet connectivity.

https://build.opensuse.org/package/view_file/science:machinelearning/
tensorflow/tensorflow.spec?expand=1
https://github.com/clearlinux-pkgs/tensorflow/blob/master/tensorflow.spec
</off-topic>

> "Kata Container with Clear Linux"? What an obscure technology, and
> what does it has to do with the Qt project and build system in
> general?

I was replying to the message saying that Bazel would be a good choice by
relating our experience.

You brought Kata Containers up. If you use containers on Linux and you care
about security, you should take a look at it. It's about using the processor's
virtual machine hypervisor functionality to make sure the code in the
container can't break out of the container, without making a full virtual
machine.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center

[Kevin Kofler]

Thiago Macieira wrote:
> Specifically, the problem with TensorFlow is that it wants to download its
> dependencies when you compile it, then compile them at the same time. That
> can't work because the build servers don't have internet connectivity.

+1, this is also a hard, non-negotiable requirement in Fedora. Any build
system that requires downloading dependencies at build time is a complete
non-starter. The builds would just fail with no way to fix them.

Kevin Kofler

[Lisandro Damián Nicanor Pérez Meyer]

El domingo, 22 de julio de 2018 18:54:14 -03 Kevin Kofler escribió:
> Thiago Macieira wrote:
> > Specifically, the problem with TensorFlow is that it wants to download its
> > dependencies when you compile it, then compile them at the same time. That
> > can't work because the build servers don't have internet connectivity.
>
> +1, this is also a hard, non-negotiable requirement in Fedora. Any build
> system that requires downloading dependencies at build time is a complete
> non-starter. The builds would just fail with no way to fix them.

Exactly the same goes for Debian and Ubuntu. And it would not surprise me that
any other derivative out there.

Let me be clear in one thing: if somehow this can be used for Windows instead
of convenience copies but disabled for Linux/builders not wanting auto
download of stuff then we can agree to have this feature.

--
<rata> hmm, el enchufe hace chispas...
<-- rata ha dejado este servidor ("Leaving").
<marga> ouch
Visto en #lugfi, irc.freenode.net

Lisandro Damián Nicanor Pérez Meyer
http://perezmeyer.com.ar/
http://perezmeyer.blogspot.com/

[Kevin Kofler]

Jason Newton wrote:
> My conclusion for the build system with the brightest future is bazel

Looking at this, this build system depends on Python AND Java! They ship
binaries with everything bundled, but that is a huge bloat (and it also does
not help GNU/Linux distributions, which will want to use the system versions
of the languages).

I think a build system should have minimal dependencies, and ideally be
written in C++ as Qt itself is. If you require more stuff to build the build
system than to build Qt, something is clearly wrong.

> -Time tested, running in production environments

Bazel is still fairly new and adoption is low, especially in the Free
Software world. It is being mostly adopted by "hip" projects rather than the
well-known Free Software projects. In particular, it is not widely used in
the Qt community. Compare:
https://github.com/bazelbuild/bazel/wiki/Bazel-Users
vs.:
https://cmake.org/ ("Notable Applications Using CMake" section).

There is also little to no documentation on packaging Bazel-using software
for GNU/Linux distributions. E.g., the Fedora wiki does not contain a single
reference to Bazel. (The only 2 results I get with a full-text search are
about a person with that surname.)

> -Ability to build external libraries from source or pull in binary
> libraries - the former offering alot of power that comes with custom
> flags and controlled dependencies for users developing products, and
> the later for distribution like software - the choice is always there
> and low barrier of entry. One can fuse approaches where it makes
> sense.

GNU/Linux distributions actually want to link against the system libraries,
not pull in binaries (nor library sources) into the package.

Kevin Kofler

[Kevin Kofler]

I wrote:
> Bazel is still fairly new and adoption is low, especially in the Free
> Software world.

PS: In addition, https://bazel.build/ itself describes Bazel as "Beta". In
particular, "Bazel is not yet covered by a deprecation policy, may be
subject to backward-incompatible changes, and is missing some features."

I do not think relying on a beta build system that may change incompatibly
at any time is a good idea.

Kevin Kofler

[Thiago Macieira]

On Sunday, 22 July 2018 03:31:42 PDT Kevin Kofler wrote:
> PS: In addition, https://bazel.build/ itself describes Bazel as "Beta". In
> particular, "Bazel is not yet covered by a deprecation policy, may be
> subject to backward-incompatible changes, and is missing some features."
>
> I do not think relying on a beta build system that may change incompatibly
> at any time is a good idea.

To be fair, it has time to leave that state and provide compatibility.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center

[Jason Newton]

On Sun, Jul 22, 2018 at 3:11 AM Kevin Kofler <***@chello.at> wrote:
>
> Jason Newton wrote:
> > My conclusion for the build system with the brightest future is bazel
>
> Looking at this, this build system depends on Python AND Java! They ship
> binaries with everything bundled, but that is a huge bloat (and it also does
> not help GNU/Linux distributions, which will want to use the system versions
> of the languages).

Yeah, I think java is the only actual curve ball in there but OpenJDK
works just fine and seemingly all they care to use. I think your main
concern as bloat would likely be around ~200 or so megabytes of disk
space used on linux platforms for your distro to pull that in, in
development capable machines (which would pull in copious amounts of
devel packages + doc). Python is in most distributions, and I'm
hesitant to say integral but it frequently seems installed by default,
often relating to package management, and I believe they use the
system install for python one all the time. In practice I really have
to question how much this matters as hard drive space is so cheap and
this really is a maximum of a few hundred megabytes, and they offer
builds with it built-in to the binary image to free the user from
having to install it. Inline history lesson - I believe they started
in python or jython and the GIL or performance caused enough grief
getting things fast/scalable so they rewrote the runtime to address
these and ended up with a python-like interpreter in java. Also the
goto language for things that aren't bazel but are a script are either
shell or python, not sure how well they explicitly spell this out but
that is how it is.

>
> I think a build system should have minimal dependencies, and ideally be
> written in C++ as Qt itself is. If you require more stuff to build the build
> system than to build Qt, something is clearly wrong.
>
> > -Time tested, running in production environments
>
> Bazel is still fairly new and adoption is low, especially in the Free
> Software world. It is being mostly adopted by "hip" projects rather than the
> well-known Free Software projects. In particular, it is not widely used in
> the Qt community. Compare:
> https://github.com/bazelbuild/bazel/wiki/Bazel-Users
> vs.:
> https://cmake.org/ ("Notable Applications Using CMake" section).
>
> There is also little to no documentation on packaging Bazel-using software
> for GNU/Linux distributions. E.g., the Fedora wiki does not contain a single
> reference to Bazel. (The only 2 results I get with a full-text search are
> about a person with that surname.)

Fairly new is interesting in this case because it's core is blaze, and
that's not new - two other build systems were built trying to copy
it. It is a weird case and even towards your later mentioning of
their self declaration of beta, I think it bears in mentioning google
considers that term differently (google maps, google mail anyone?).
However I know they're rewriting c++ code paths under the hood - if
you look at build rules though, you'll see not alot is changing there
- I've used it for ~ 1 year and I haven't experienced breakage in this
way yet, but I would expect it soon as a part of that process - keep
in mind we're looking at a long term shift though, not short term.
Also have to point out cmake 2->3 was not painless and as usual
projects tend stick to a supported range if and when things break and
address it with point releases if possible. Can't deny incompatible
changes wouldn't come up but I think that's possible for just about
any of the options out there.

Not quite sure about what the last section is about - distro-made
binaries for bazel? I don't think it's generally on their radar (yet)
- There are these pages:
https://docs.bazel.build/versions/master/install-redhat.html for the
centralized install guides. If you want to build it, you'll need to
boot-strap it with a prebuilt binary, but building the executable, tar
archives, rpms and debs are all supported - building them oneself is
fairly trivial and here is the file of interest in terms of targets:
https://github.com/bazelbuild/bazel/blob/master/scripts/packages/BUILD
, and owning directory contents. Documentation could be better
though, but that comes with distros being made more aware of it, which
is often a community based thing, or popularity and in terms of
distros acknowledging bazel, yes, I don't think there are any.

>
> > -Ability to build external libraries from source or pull in binary
> > libraries - the former offering alot of power that comes with custom
> > flags and controlled dependencies for users developing products, and
> > the later for distribution like software - the choice is always there
> > and low barrier of entry. One can fuse approaches where it makes
> > sense.
>
> GNU/Linux distributions actually want to link against the system libraries,
> not pull in binaries (nor library sources) into the package.

What I'm saying is you get both and you want to flesh out both, qt
actually is caught in the middle of this (precompiled SDK in
particular) of packages in c/c++ becoming a thing (see conan, jfrog) -
will they have success? I don't know, people desire pip/npm like
capabilities to get libraries close to language, distro agnostic and
it's something even the standards folk are poked with. By the way,
both of those projects are foremost in cmake. Hodgpodge of sources
built JIT and precompiled. Multiple hats and installation types have
to be considered ontop of platforms/architectures, in particular since
Qt tries to work both with FOSS and proprietary software customers.
Also as a python user I frequently felt alot of pain trying to work on
distributions that want to replace pip (poorly), but anaconda rocks
because it is the eject button on when distros can't deal with
dependencies as the developers/users/scientists need to - I bring this
up because of pip, and the qt installation independent of distro,

I think some of the other points I read earlier, for anaconda related
packaging, could be related to third parties not themselves offering -
I am interesting in hearing about this, maybe in a more private space
@Ray Donnelly .

.

>
> Kevin Kofler
>
> _______________________________________________
> Development mailing list
> ***@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development

[Matthew Woehlke]

On 2018-07-21 19:52, Jason Newton wrote:
> I wanted to mention that this is on my mind alot for a few years days
> as a user for a plethora of libraries. My conclusion for the build
> system with the brightest future is bazel [...]

No. Just, *no*.

Persistent build server? Java? No, thanks.

Maybe it's gotten better, but last I knew, bazel had no notion of
"installing" software, which is just a non-starter. It's also heavily
optimized for Google's internal usage, and (partly as a result)
basically does not play at all nicely with software that's intended to
exist in an open source, package-managed ecosystem.

(I'd be interested to know if there is *any* software packaged by any
FLOSS distribution that uses bazel...)

Basically, ask your distributors before even *considering* bazel. My
suspicion is they're going to tell you that they refuse to package
anything using it.

And... seriously, *Java*?! Talk about bloat-ware... As dependencies for
*a build tool* go, that's pretty insane. Especially if you're not
planning to use it to build Java code.

By comparison, CMake+ninja depend on... a C++ compiler. (Which is also
needed for Qt itself, of course, so...) I haven't really looked, but I'd
guess qmake is also pretty light-weight in the dependency department.

> Why the qt project may be interested in this in short is:
> -Fast and scaling bazel uses a long lived server technique that hangs
> around

Yeah... that's *just* the sort of thing we should be forcing on users...
It's even *more* of an anti-feature if some other build systems needs to
build Qt. (Yes, that happens.)

On a related note, "hermetic builds" is pretty ironic. Your *build*
might be hermetic, but bazel itself is *far* from... it's very reliant
on putting all its garbage in "magic locations" in your home directory,
unlike most build tools that only need to write to your build directory.
Again, *not* friendly for anyone that needs to build Qt as an external
dependency.

> The amount of time that useful work is being performed is greater
> than I've observed in large cmake projects, even using ninja (assuming
> ninja would work across projects that size, due to frequent make
> sensitivities)

Um... very few projects don't work with Ninja, and I've seen plenty of
very large projects that *do*.

> -Ability to build external libraries from source

Yeah... no. Building third party libraries yourself is *evil*. And a
great way to piss off package maintainers. No open source project should
*EVER* rely on, or preferably even use by default, this ability.


On 2018-07-22 06:11, Kevin Kofler wrote:
> There is also little to no documentation on packaging Bazel-using software
> for GNU/Linux distributions. E.g., the Fedora wiki does not contain a single
> reference to Bazel.

There's a *reason* why that's so...

--
Matthew

[Denis Shienkov]

Hi guys,

Is there are any list of options from which it is possible to choose?

As for me, is it prefferable variant will be *QBS* (it is best from the
best).

I'm do not like nor CMake, nor QMake, nor Autotools, nor something else.

BR,
Denis

[Thiago Macieira]

On Monday, 30 July 2018 23:11:54 PDT Denis Shienkov wrote:
> As for me, is it prefferable variant will be *QBS* (it is best from the
> best).
>
> I'm do not like nor CMake, nor QMake, nor Autotools, nor something else.

I would say the only two options in the running are qbs and CMake.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center

[Lisandro Damián Nicanor Pérez Meyer]

El martes, 31 de julio de 2018 11:41:15 -03 Thiago Macieira escribió:
> On Monday, 30 July 2018 23:11:54 PDT Denis Shienkov wrote:
> > As for me, is it prefferable variant will be *QBS* (it is best from the
> > best).
> >
> > I'm do not like nor CMake, nor QMake, nor Autotools, nor something else.
>
> I would say the only two options in the running are qbs and CMake.

Just for curiosity I checked qbs' reverse dependencies (packages that require
qbs installed in order to be built) in Debian. We currently only have two of
them: qtcreator and dewalls (a library).

--
Lisandro Damián Nicanor Pérez Meyer
http://perezmeyer.com.ar/
http://perezmeyer.blogspot.com/

[Thiago Macieira]

On Tuesday, 31 July 2018 09:35:42 PDT Lisandro Damián Nicanor Pérez Meyer
wrote:
> > I would say the only two options in the running are qbs and CMake.
>
> Just for curiosity I checked qbs' reverse dependencies (packages that
> require qbs installed in order to be built) in Debian. We currently only
> have two of them: qtcreator and dewalls (a library).

I know. That's why I am challenging the supporters of qbs to meet my requests
in (2). To wit:

a) Must be used by roughly a dozen packages in major distros. I'm not saying
all of them have to have a dozen, not even that one of them has a dozen. But
all of the ones listed above must have at least one and there must be a dozen
distinct packages in total.

b) At least one package must have been continuously packaged for two years.
One for an RPM distro and one for a .deb distro.

c) At least one package of comparable complexity to qtbase, packaged by the
three of the six Linux distros I listed. I'm looking for:
- compiling certain files with different compiler flags
- several third-party dependencies, some required, others optional
- lots of configure-time options
- installing several different types of targets (binaries, libraries,
plugins, arch-dependent files, arch-independent files, documentation,
translations, etc.)
- obeying distro-supplied options (compiler & linker flags, compiler
selection [ccache, icecc, clang], debuginfo split, stripping, etc.)


I know CMake meets these requirements, but it has other problems and the fact
that it currently does not build Qt. On that front, qbs is ahead. But qbs has
a shorter track record. Since we're not switching buildsystems in the next 2
years, there's time for the proponents of qbs to take actions to achieve those
requirements above.

In other words: get others to adopt the buildsystem before we switch Qt. Prove
that it can support Qt.

I know switching Qt would gain it a big critical mass, the same way that KDE
switching to CMake in 2006 gave CMake an important boost [*]. But we're not
switching Qt just yet, so if you're saying the tool is ready, get others to
use it now.


[*] I was there in Akademy 2005 when we decided to use Scons because CMake's
language was too ugly.
--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center

[Ville Voutilainen]

On 31 July 2018 at 20:49, Thiago Macieira <***@intel.com> wrote:
> I know CMake meets these requirements, but it has other problems and the fact
> that it currently does not build Qt. On that front, qbs is ahead. But qbs has
> a shorter track record. Since we're not switching buildsystems in the next 2
> years, there's time for the proponents of qbs to take actions to achieve those
> requirements above.
>
> In other words: get others to adopt the buildsystem before we switch Qt. Prove
> that it can support Qt.
>
> I know switching Qt would gain it a big critical mass, the same way that KDE
> switching to CMake in 2006 gave CMake an important boost [*]. But we're not
> switching Qt just yet, so if you're saying the tool is ready, get others to
> use it now.

This provoked a thought in me, a way to make qbs worth all its effort:
make debugging
a build so staggeringly ridiculously good that it becomes attractive
to use it. I don't know
what debugging builds done with python-based build systems is like,
but debugging make-based
builds is rather horrible.

[Thiago Macieira]

On Tuesday, 31 July 2018 11:15:50 PDT Ville Voutilainen wrote:
> This provoked a thought in me, a way to make qbs worth all its effort:
> make debugging
> a build so staggeringly ridiculously good that it becomes attractive
> to use it. I don't know
> what debugging builds done with python-based build systems is like,
> but debugging make-based
> builds is rather horrible.

Aye.

Debugging qmake builds are actually pretty easy, if you know -d and -d -d
exist. You can easily follow the decisions it made. You'll get to a few corner
cases where things that should be lists aren't or vice-versa, or quirks of the
language where $$ is suppressed, etc. But most of the time you don't need
that. (I'm not talking about debugging the tool itself)

Debugging cmake builds aren't that easy. It writes a lot of logs, but
sometimes magic happens. If the magic doesn't go your way, you're lost.
Compared to qmake, the barrier of where the magic happens is lower, meaning
that it happens more often than it should.

Debugging Makefiles are a PITA. THAT was libvpx's buildsystem and that is what
prompted me to post this thread. Debugging make using the -d option is a last
resort and is still not completely sufficient, since it shows process starts
but not the variable evaluations.

Finally, debugging autoconf is actually not that difficult either. Because
it's a shell script, there are tons of techniques you can use. And it creates
a fairly comprehensive log file, similar to qmake's -d output. Debugging
automake is a different story, but it's a very limited tool. And don't try to
debug libtool (you can live an entire lifetime using it and never debug it).

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center

[Scott Bloom]

-----Original Message-----
From: Development <development-bounces+scott=***@qt-project.org> On Behalf Of Thiago Macieira
Sent: Tuesday, July 31, 2018 11:41
To: ***@qt-project.org
Subject: Re: [Development] Qt 6 buildsystem support requirements

On Tuesday, 31 July 2018 11:15:50 PDT Ville Voutilainen wrote:
> This provoked a thought in me, a way to make qbs worth all its effort:
> make debugging
> a build so staggeringly ridiculously good that it becomes attractive
> to use it. I don't know what debugging builds done with python-based
> build systems is like, but debugging make-based builds is rather
> horrible.

Aye.

Debugging qmake builds are actually pretty easy, if you know -d and -d -d exist. You can easily follow the decisions it made. You'll get to a few corner cases where things that should be lists aren't or vice-versa, or quirks of the language where $$ is suppressed, etc. But most of the time you don't need that. (I'm not talking about debugging the tool itself)

Debugging cmake builds aren't that easy. It writes a lot of logs, but sometimes magic happens. If the magic doesn't go your way, you're lost.
Compared to qmake, the barrier of where the magic happens is lower, meaning that it happens more often than it should.

Debugging Makefiles are a PITA. THAT was libvpx's buildsystem and that is what prompted me to post this thread. Debugging make using the -d option is a last resort and is still not completely sufficient, since it shows process starts but not the variable evaluations.

Finally, debugging autoconf is actually not that difficult either. Because it's a shell script, there are tons of techniques you can use. And it creates a fairly comprehensive log file, similar to qmake's -d output. Debugging automake is a different story, but it's a very limited tool. And don't try to debug libtool (you can live an entire lifetime using it and never debug it).

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center


===============

Just a note.. for CMake, I find the -debug-output -trace and -trace-expand as useful as the -d and -d -d .

One other thing I use all the time for dependency analysis, is the -graphviz switch.

Scott

[Thiago Macieira]

On Tuesday, 31 July 2018 11:43:49 PDT Scott Bloom wrote:
> Just a note.. for CMake, I find the -debug-output -trace and -trace-expand
> as useful as the -d and -d -d .
>
> One other thing I use all the time for dependency analysis, is the -graphviz
> switch.

Thanks, noted. I didn't know about those.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center

[Ville Voutilainen]

On 31 July 2018 at 22:33, Thiago Macieira <***@intel.com> wrote:
> On Tuesday, 31 July 2018 11:43:49 PDT Scott Bloom wrote:
>> Just a note.. for CMake, I find the -debug-output -trace and -trace-expand
>> as useful as the -d and -d -d .
>>
>> One other thing I use all the time for dependency analysis, is the -graphviz
>> switch.
>
> Thanks, noted. I didn't know about those.

Trace-debugging is nice, but I'm actually envisioning launching
Creator, opening a build with
it as a build-project (instead of opening it as a project that builds
what the build builds :P)
and stepping through the build.

[Jedrzej Nowacki]

On Tuesday, July 31, 2018 8:15:50 PM CEST Ville Voutilainen wrote:
> On 31 July 2018 at 20:49, Thiago Macieira <***@intel.com> wrote:
> > I know CMake meets these requirements, but it has other problems and the
> > fact that it currently does not build Qt. On that front, qbs is ahead.
> > But qbs has a shorter track record. Since we're not switching
> > buildsystems in the next 2 years, there's time for the proponents of qbs
> > to take actions to achieve those requirements above.
> >
> > In other words: get others to adopt the buildsystem before we switch Qt.
> > Prove that it can support Qt.
> >
> > I know switching Qt would gain it a big critical mass, the same way that
> > KDE switching to CMake in 2006 gave CMake an important boost [*]. But
> > we're not switching Qt just yet, so if you're saying the tool is ready,
> > get others to use it now.
>
> This provoked a thought in me, a way to make qbs worth all its effort:
> make debugging
> a build so staggeringly ridiculously good that it becomes attractive
> to use it. I don't know
> what debugging builds done with python-based build systems is like,
> but debugging make-based
> builds is rather horrible.
> _______________________________________________
> Development mailing list
> ***@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development

Oh debugging...

<dream world>

I would love a build system that actually thinks about real world in which
sub-processes are crashing, getting locked or became unusually slow. One that
automatically attach debugger and scans operating system for basic issues, for
example I would like to see something like that:

"error: ABC process crashed, here there is backtrace of it (...) at the time
of the crash memory usage was ~ZZ% and avg load of Y, in parallel we were
executing processes: BGD, SFD"
"warning: CBA process seems to not doing anything for last X minutes, the avg
load is 1% for the last 30s, maybe try strace with PID to debug it?"
"warning: performance may be poor because of a heavy swap usage"
"error: you are almost out of disk space (~300 MB) suspending the build"
"warning: process HBG runs already for X minutes, constantly using whole CPU"

A system that is aware of memory constrains and CPU limits. So I'm not
surprised if like with make suddenly -j60 doesn't work as expected because I
have changed to a network without icecream scheduler. A system that
automatically limits number of parallel tasks if swap is getting full to avoid
out of memory kills.

</dream world>

Cheers,
Jędrek

[Jean-Michaël Celerier]

> I would say the only two options in the running are qbs and CMake.

About this, I don't know if C++ community acceptance is a criterion, but :

https://github.com/search?q=cmake

https://github.com/search?q=qbs



On Tue, Jul 31, 2018 at 4:41 PM, Thiago Macieira <***@intel.com>
wrote:

> On Monday, 30 July 2018 23:11:54 PDT Denis Shienkov wrote:
> > As for me, is it prefferable variant will be *QBS* (it is best from the
> > best).
> >
> > I'm do not like nor CMake, nor QMake, nor Autotools, nor something else.
>
> I would say the only two options in the running are qbs and CMake.
>
> --
> Thiago Macieira - thiago.macieira (AT) intel.com
> Software Architect - Intel Open Source Technology Center
>
>
>
> _______________________________________________
> Development mailing list
> ***@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development
>

[Shawn Rutledge]

> On 30 Jul 2018, at 22:30, Matthew Woehlke <***@gmail.com> wrote:
>
> On a related note, "hermetic builds" is pretty ironic. Your *build*
> might be hermetic, but bazel itself is *far* from... it's very reliant
> on putting all its garbage in "magic locations" in your home directory,
> unlike most build tools that only need to write to your build directory.

Speaking of which (and speaking of re-inventing the wheel)… I seem to have a lot of qbs-related files on my system despite the fact that I don’t actively use it. (OK I tried to build some project on github once, one that is already trying to use it.) E.g. under ~/.config/QtProject/qbs/1.10.0 it looks like some meta-info, the kind of stuff that cmake and pkg-config already has:

$ cat ~/.config/QtProject/qbs/1.10.0/profiles/qt-4-8-7/modules/Qt/phonon/phonon.qbs
import qbs 1.0
import '../QtModule.qbs' as QtModule

QtModule {
qtModuleName: "phonon"
Depends { name: "Qt"; submodules: ['core'] }
cpp.defines: ["QT_PHONON_LIB"]
cpp.includePaths: ["/usr/include/qt4", "/usr/include/qt4/Phonon"]
}

I don’t know how it got there, but I prefer to have some understanding of what goes in my .config dir so that I can use a git repo to replicate the important stuff between systems (and ignore the rest, maybe even trash it sometimes). Is there any way that mess can be avoided? I mean really, meta-info about *system* packages going into my home dir, as if qbs could even rely on finding those files there, rather than being installed alongside the system package itself?!?

Maybe it’s better to continue depending on pkg-config, since it’s an old standard that we probably shouldn’t drop support for anyway, rather than duplicating it?

[Иван Комиссаров]

QBS generates Qt Modules file depending on the information that QMake provides about modules installed in the system to make sure you won;t be able to import module that doesn’t exist on your system.

According to the fact that you can have multiple versions of Qt (i have 3 Qt versions on my Linux machine) and they may be installed in you home dir, it is logical to store those files in your home dir (.config).

Of course, you can safely remove the whole dir, it’s just your personal configuration.

Funny that you blaming only QBS about the mess in a home dir:)

What disturbs me more is the folders like ~/.dropbox ~/.subversion ~/.designer in the root of my home dir *sign*

> 31 июля 2018 г., в 10:18, Shawn Rutledge <***@qt.io> написал(а):
>
>
>> On 30 Jul 2018, at 22:30, Matthew Woehlke <***@gmail.com> wrote:
>>
>> On a related note, "hermetic builds" is pretty ironic. Your *build*
>> might be hermetic, but bazel itself is *far* from... it's very reliant
>> on putting all its garbage in "magic locations" in your home directory,
>> unlike most build tools that only need to write to your build directory.
>
> Speaking of which (and speaking of re-inventing the wheel)… I seem to have a lot of qbs-related files on my system despite the fact that I don’t actively use it. (OK I tried to build some project on github once, one that is already trying to use it.) E.g. under ~/.config/QtProject/qbs/1.10.0 it looks like some meta-info, the kind of stuff that cmake and pkg-config already has:
>
> $ cat ~/.config/QtProject/qbs/1.10.0/profiles/qt-4-8-7/modules/Qt/phonon/phonon.qbs
> import qbs 1.0
> import '../QtModule.qbs' as QtModule
>
> QtModule {
> qtModuleName: "phonon"
> Depends { name: "Qt"; submodules: ['core'] }
> cpp.defines: ["QT_PHONON_LIB"]
> cpp.includePaths: ["/usr/include/qt4", "/usr/include/qt4/Phonon"]
> }
>
> I don’t know how it got there, but I prefer to have some understanding of what goes in my .config dir so that I can use a git repo to replicate the important stuff between systems (and ignore the rest, maybe even trash it sometimes). Is there any way that mess can be avoided? I mean really, meta-info about *system* packages going into my home dir, as if qbs could even rely on finding those files there, rather than being installed alongside the system package itself?!?
>
> Maybe it’s better to continue depending on pkg-config, since it’s an old standard that we probably shouldn’t drop support for anyway, rather than duplicating it?
>
> _______________________________________________
> Development mailing list
> ***@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development

[Shawn Rutledge]

> On 31 Jul 2018, at 09:41, Иван Комиссаров <***@gmail.com> wrote:
>
> QBS generates Qt Modules file depending on the information that QMake provides about modules installed in the system to make sure you won;t be able to import module that doesn’t exist on your system.

Files that were dynamically created and that I could delete at any time hardly seem like an authoritative place to check what modules are installed on my system. It’s far more likely that they will be out of date later on when I update or remove a system library.

> According to the fact that you can have multiple versions of Qt (i have 3 Qt versions on my Linux machine) and they may be installed in you home dir, it is logical to store those files in your home dir (.config).

It still looks like needless duplicate info to me. Besides, installing Qt in my home dir is not at all a standard thing to do (it’s only that the Qt installer does it). The standard thing is to use distro-provided packages. And if it becomes necessary for those packages to include qbs-related metainfo, they will end up under /usr somewhere. But maybe it’s better that qbs learns how to use pkg-config, unless there’s a good reason not to?

> Of course, you can safely remove the whole dir, it’s just your personal configuration.
>
> Funny that you blaming only QBS about the mess in a home dir:)

Of course not only qbs. ;-) Browser profiles make a bigger mess for example.

> What disturbs me more is the folders like ~/.dropbox ~/.subversion ~/.designer in the root of my home dir *sign*

yeah, sigh (but dropbox is an easy thing to avoid)

[Jason Newton]

On Mon, Jul 30, 2018 at 1:30 PM Matthew Woehlke
<***@gmail.com> wrote:
>
> On 2018-07-21 19:52, Jason Newton wrote:
> > I wanted to mention that this is on my mind alot for a few years days
> > as a user for a plethora of libraries. My conclusion for the build
> > system with the brightest future is bazel [...]
>
> No. Just, *no*.
>
> Persistent build server? Java? No, thanks.

This is an option, you can keep it in a preference file, or you can
use it as a command line switch to not use this . stat time
performance when monitoring many files, between invocations,
especially if globbing is involved and if you don't have a process
watching with inotify you will need to O(N) check every file for
modification and to hope your platform/hardware makes that check cheap
(windows doesn't,
https://github.com/mesonbuild/meson/blob/master/docs/markdown/FAQ.md#but-i-really-want-to-use-wildcards)
Better to have a plan to deal with the case than not. As for java
in the loop - this is a a build system, how much does it matter with
what it is written in if the implementation is good? I am not saying 0
but you should keep this number in check. Make is made out of
something far more terrible than java. Cmake is built on c++ but is
difficult to modify and the language they made is a mess. Java works
in many situations and provides a write-once run anywhere mode that
works most of the time without issue and is performant enough for most
everyday tasks (better than python, and not always a long shot off
from all kinds of c/c++ depending on the application and exact
algorithms ), and importantly it works in this one without any notable
complaints. OpenJDK has wide coverage for host-build environments:
http://openjdk.java.net - and is packaged by all major distributions,
first class.

>
> Maybe it's gotten better, but last I knew, bazel had no notion of
> "installing" software, which is just a non-starter. It's also heavily
> optimized for Google's internal usage, and (partly as a result)
> basically does not play at all nicely with software that's intended to
> exist in an open source, package-managed ecosystem.

It works well with open source actually, git is a first class data
source, mainly targeting external repositories, frequently github.
Source archive files are too. The convention is that projects using
it embrace that and don't care about the most system packages if they
can get away with it. Streamline switching out pkg_config based
results and package-kit or other kinds of datasources and really this
just works. Discussing this problem upstream will probably get better
capabilities in the box well before expected shift time but your beef
is with closing the gap on package-managed dependencies outside of
bazel, and I'm all for closing it, although even now it is workable.


> (I'd be interested to know if there is *any* software packaged by any
> FLOSS distribution that uses bazel...)
>
> Basically, ask your distributors before even *considering* bazel. My
> suspicion is they're going to tell you that they refuse to package
> anything using it.
>
> And... seriously, *Java*?! Talk about bloat-ware... As dependencies for
> *a build tool* go, that's pretty insane. Especially if you're not
> planning to use it to build Java code.

As I said, ~300-400 megabytes for a JRE (~90MiB DL), in particular
openjdk , prospectively in a development environment where you've got
devel headers and packages installed. The cost of the dependency is
neither bloat nor unbounded, it really is just that size on a machine
that's prepared to build software in 2018. Please explain what issue
this presents.

>
> By comparison, CMake+ninja depend on... a C++ compiler. (Which is also
> needed for Qt itself, of course, so...) I haven't really looked, but I'd
> guess qmake is also pretty light-weight in the dependency department.
>
> > Why the qt project may be interested in this in short is:
> > -Fast and scaling bazel uses a long lived server technique that hangs
> > around
>
> Yeah... that's *just* the sort of thing we should be forcing on users...
> It's even *more* of an anti-feature if some other build systems needs to
> build Qt. (Yes, that happens.)
>
> On a related note, "hermetic builds" is pretty ironic. Your *build*
> might be hermetic, but bazel itself is *far* from... it's very reliant
> on putting all its garbage in "magic locations" in your home directory,
> unlike most build tools that only need to write to your build directory.
> Again, *not* friendly for anyone that needs to build Qt as an external
> dependency.

Actually that's just a default location - the output location you're
talking about is the outputRoot/output_base setting:
https://docs.bazel.build/versions/master/output_directories.html
Again, like gitignore or clang-format, a project can specify it's own
settings and the point of contention is default values with
persecution on capability. I think there's open path ways for change
behaviors that should be changed if there's lingering concern over
features like this that deal with use cases and problems rather than a
build system which does less and has less to worry about, notably the
cost of a build system that's too simple is a build system that's too
limit. One of the things I like about the way bazel keeps this is I
can jump back and forth between an optimized build and a full debug
build (for example) - they don't erase eachother and you can only
achieve that jumping capability, being cheap, if you add a file-level
layer of indirection. The magic locations actually aren't magic at all
and are fully deterministic, hashes like git commits essentially with
1 level of symlinks sitting in the users build directory. You could
easily have all your outputs live adjacent to (at minimum) to your
project/src directory, somewhat like the build convention in
cmake/autotools.

>
> > The amount of time that useful work is being performed is greater
> > than I've observed in large cmake projects, even using ninja (assuming
> > ninja would work across projects that size, due to frequent make
> > sensitivities)
>
> Um... very few projects don't work with Ninja, and I've seen plenty of
> very large projects that *do*.

Yes, this has changed with time but sensitives can exist, I'd still
say it's a crap shoot when pulling down sources.

>
> > -Ability to build external libraries from source
>
> Yeah... no. Building third party libraries yourself is *evil*. And a
> great way to piss off package maintainers. No open source project should
> *EVER* rely on, or preferably even use by default, this ability.
>

It is not evil in any way. It's a different approach that works well
for different concerns to attempt to make a ~constant cost vs a linear
cost in os * distribution * version. What is needed is a way to do
both and make both sides of this happy. Ways to easily lean on a
system install or if you're a dev on a system that doesn't play ball
well (version, timing, whatever), provide gradieted ways to use source
build overlays without tremendous headaches - for getting the latest
stable without going ppa for instance and working in custom build
flags for any sources of interest. Bazel doesn't deal with system
packages in a first class way - there's no direct pkg_config
integration, though there are simple trivial scripts that use it that
are third party that should move to the builtins. I can help make
those kinds of changes, if need be. However I've had no problems
pulling in system libs (pkg-config or otherwise) and most of the time
packing up a library is smooth sailing, when building from source. As
I said you get both and you don't have to pay the penalty of out of
band source builds just because by convention people not on a
distro-packagers side do. And it's nice when you're working on a
software application and want to be distribution agnostic, as is
frequently the case if you are not a packager for a distribution.

Again pointing out sources-JIT package services like vcpkg, buckaroo,
and conan exist and have been gaining traction (users and word of
mouth). The want for these capabilities in c++ is growing and bazel
provides a streamlined mechanism for it, but it is not generally
coupled to the internet for it, and allows gradiantions/mixes of that
JIT-from-sources and the system libraries. On JIT from sources, that
gives you better support for esoteric platforms that receive little to
no TLC from package maintainers, for instance on why that has value
(S390 or arm distros are good examples)

Also wanted to mention a few things I forgot previously:

-Analysis tools for the dependencies, like graphviz, but also memory
usage and timing info. A legitimate profiler, have a look:
https://docs.bazel.build/versions/master/skylark/performance.html
https://www.kchodorow.com/blog/2015/09/18/build-y-u-go-slow/
(admittedly few online examples of this output and this is looks
different now IIRC)
-Does a good job of controlling verbosity while keeping the user
informed on the running targets being processed concurrently while
giving a quick way to hop out, go serial or show commands executed
-Workable, like cmake workable, visual studio generator, and xcode outputs.
-clion has a plugin good direct plugin for it, though more will follow
and they tried to make it easy for IDEs to incorporate (
https://blog.bazel.build/2016/06/10/ide-support.html we can agree it'd
be create if more build systems did this, right?
https://docs.bazel.build/versions/master/ide.html for fuller list )

-Jason

[Edward Welbourne]

Singling out just one tiny corner of all that for a passing remark:

Jason Newton (1 August 2018 10:24)
> One of the things I like about the way bazel keeps this is I can jump
> back and forth between an optimized build and a full debug build (for
> example) - they don't erase eachother

This is easy to do in any build system that supports out-of-source
(a.k.a. "shadow") builds - notably including the existing qmake-based
builds for Qt. I never want to do an in-source build of anything.

> and you can only achieve that jumping capability, being cheap, if you
> add a file-level layer of indirection.

Not sure what you're claiming here, or what system you're claiming lacks
it. If you've got both optimised and debug builds, you're going to have
both sets of files *somewhere* (whether jumbled up in one directory or
kept separate in two out-of-source build trees).

Eddy.

[Thiago Macieira]

On Wednesday, 1 August 2018 02:01:40 PDT Edward Welbourne wrote:
> This is easy to do in any build system that supports out-of-source
> (a.k.a. "shadow") builds - notably including the existing qmake-based
> builds for Qt. I never want to do an in-source build of anything.

And not just that. Building out-of-source is an absolute requirement.

With one set of Qt sources, I build:
- debug
- release
- clang release
- icc release
- i386 release namespace
- MSVC 2017 (source dir is mounted read-only in the VM)

Plus discontinued (used to build, haven't in over 2 years):
- ARM 32-bit
- MIPS 32-bit
- static

Ditto on the Windows machine, where the same set up, I build:
- MSVC 2017 64-bit
- MSVC 2015 64-bit
- MSVC 2013 64-bit (will stop)
- ICC 18 MSVC 2017 64-bit (configure only)
- ICC 18 MSVC 2015 64-bit (only qtbase)

And on the Mac, the same sources builds both frameworks and no-frameworks,
though I think I haven't built the latter in several years (unlike the Mac and
Windows laptops, the Mac machine is from 2012, so it has old stuff).

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center

[Thiago Macieira]

On Wednesday, 1 August 2018 11:58:02 PDT BogDan Vatra via Development wrote:
> Hi,
>
> Did you knew that qbs can build all but windows targets at once from your
> Linux machine?

That's not news. qmake can do that, provided you have the toolchains. You
could compile for Mac and MSVC too, if you had them (for MSVC, via Wine, like
godbolt.org).

> Using mingw you can cross compile Qt for windows, but you
> probably want to run tests and to check if the code compiles with msvc.
> This feature might not seem pretty useful for linux, but is very useful for
> Android where we can bundle multiple platforms in the same package.

Agreed.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center

[Sérgio Martins]

On Wed, Aug 1, 2018 at 8:46 PM, BogDan Vatra via Development
<***@qt-project.org> wrote:
> Hi,
>
> qmake can't compile them all *at once* e.g. $ qmake && make will compile
> only one target at the time not all.
>
> AFAIK QBS and iirc gn too, are the only few that have this cool feature.

And probably do a single moc and uic pass if you want ? The rules
system is pretty flexible.

Regards,
Sergio Martins



>
> On August 1, 2018 10:27:03 PM GMT+03:00, Thiago Macieira
> <***@intel.com> wrote:
>>
>> On Wednesday, 1 August 2018 11:58:02 PDT BogDan Vatra via Development
>> wrote:
>>>
>>> Hi,
>>>
>>> Did you knew that qbs can build all but windows targets at once from
>>> your
>>> Linux machine?
>>
>>
>> That's not news. qmake can do that, provided you have the toolchains. You
>> could compile for Mac and MSVC too, if you had them (for MSVC, via Wine,
>> like
>> godbolt.org).
>>
>>> Using mingw you can cross compile Qt for windows, but you
>>> probably want to run tests and to check if the code compiles with msvc.
>>> This feature might not seem pretty useful for linux, but is very useful
>>> for
>>> Android where we can bundle multiple platforms in the same package.
>>
>>
>> Agreed.
>
>
> --
> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>
> _______________________________________________
> Development mailing list
> ***@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development
>

[Thiago Macieira]

On Wednesday, 1 August 2018 12:46:04 PDT BogDan Vatra via Development wrote:
> Hi,
>
> qmake can't compile them all *at once* e.g. $ qmake && make will compile
> only one target at the time not all.
>
> AFAIK QBS and iirc gn too, are the only few that have this cool feature.

Now that is nice, as we know that the moc, uic, rcc outputs are platform-
independent. That should help reduce the build times on Windows for debug-and-
release builds, as running moc is a significant portion of the build time.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center

[Jason Newton]

Bazel's caching mechanism would also optimize away regeneration of
generated outputs (via genrules) across the different toolchains,
provided the tools/scripts and important environmental variables/files
do not change (which would hash to different keys). Further you could
run a disk local cache or a network one - the network one letting say
spun up VMs to take advantage of out of VM results, or windows/macos.
https://docs.bazel.build/versions/master/remote-caching.html, or
CI/build servers turn around builds very quickly. Deals with branch
changes and build config changes directly. As far as I know the disk
local is concurrent build safe, while the http one is obviously safe -
though usually bazel will utilize your machine (doing useful work, not
something like java garbage collections) such that only one bazel
build at a time makes sense for, but you can do multiple too. I'm not
aware of a better solution to cached build products than bazel's.

While there is a pathway for generating visual studio project files,
you can also use cl.exe (with or without MSVC installed), the state of
direct cl.exe support for most build systems hasn't always been
great... Also since it is on intelij products, that covers the
official android sdk since it uses IDEA for a few years now, though in
this case it understands the build system directly rather than
intermediate project files. Support of android is first class:
https://docs.bazel.build/versions/master/be/android.html and support
for to maven et al package services for jar files.

On downloaded assets, the builtin downloader is parallel and generally
smarter than single tasking wget, as such it tends to outperform
cmake's downloading mechanisms (I believe these are only done
serially?) by quite a bit for pulling down online resources of any
type, provided your project explicitly used that capability.

>> -Analysis tools for the dependencies, like graphviz, but also memory
>> usage and timing info. A legitimate profiler, have a look:
>> https://docs.bazel.build/versions/master/skylark/performance.html
>> https://www.kchodorow.com/blog/2015/09/18/build-y-u-go-slow/
>> (admittedly few online examples of this output and this is looks
>> different now IIRC)
>> -Does a good job of controlling verbosity while keeping the user
>> informed on the running targets being processed concurrently while
>> giving a quick way to hop out, go serial or show commands executed
>> -Workable, like cmake workable, visual studio generator, and xcode outputs.
>> -clion has a plugin good direct plugin for it, though more will follow
>> and they tried to make it easy for IDEs to incorporate (
>> https://blog.bazel.build/2016/06/10/ide-support.html we can agree it'd
>> be create if more build systems did this, right?
>> https://docs.bazel.build/versions/master/ide.html for fuller list )

>Pretty much all of which you *also* have with CMake/ninja...

That's why I listed this section, though the analysis tools are far
better on bazel's outputs than cmakes's you get actual profiler
analysis on the build process with bazel, not just a DAG. Not many
build systems have pathways on visual studio or xcode, in particular
besides cmake. Bazel is afaik the only build system that tried to
make it easy to integrate it into an IDE interactively, short of
perhaps qbs (qmake...?).

-Jason
On Wed, Aug 1, 2018 at 3:31 PM Thiago Macieira
<***@intel.com> wrote:
>
> On Wednesday, 1 August 2018 12:46:04 PDT BogDan Vatra via Development wrote:
> > Hi,
> >
> > qmake can't compile them all *at once* e.g. $ qmake && make will compile
> > only one target at the time not all.
> >
> > AFAIK QBS and iirc gn too, are the only few that have this cool feature.
>
> Now that is nice, as we know that the moc, uic, rcc outputs are platform-
> independent. That should help reduce the build times on Windows for debug-and-
> release builds, as running moc is a significant portion of the build time.
>
> --
> Thiago Macieira - thiago.macieira (AT) intel.com
> Software Architect - Intel Open Source Technology Center
>
>
>
> _______________________________________________
> Development mailing list
> ***@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development

[Simon Hausmann]

Hi,

Given that the output of the moc changes depending on what platform and compiler dependent pre-processor macros are supplied, I would say that the output is not cross-platform.

Simon

>> On 2. Aug 2018, at 00:31, Thiago Macieira <***@intel.com> wrote:
>>
>> On Wednesday, 1 August 2018 12:46:04 PDT BogDan Vatra via Development wrote:
>> Hi,
>>
>> qmake can't compile them all *at once* e.g. $ qmake && make will compile
>> only one target at the time not all.
>>
>> AFAIK QBS and iirc gn too, are the only few that have this cool feature.
>
> Now that is nice, as we know that the moc, uic, rcc outputs are platform-
> independent. That should help reduce the build times on Windows for debug-and-
> release builds, as running moc is a significant portion of the build time.
>
> --
> Thiago Macieira - thiago.macieira (AT) intel.com
> Software Architect - Intel Open Source Technology Center
>
>
>
> _______________________________________________
> Development mailing list
> ***@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development

[Joerg Bornemann]

On 08/02/2018 08:18 AM, Simon Hausmann wrote:

> Given that the output of the moc changes depending on what platform and compiler dependent pre-processor macros are supplied, I would say that the output is not cross-platform.

...which is why qbs does *not* have such a feature.
moc_XXX.cpp and XXX.moc files are generated per build configuration. The
output is not shared.


Joerg

[Lars Knoll]

> On 2 Aug 2018, at 08:38, BogDan Vatra via Development <***@qt-project.org> wrote:
>
> Hi
> În ziua de joi, 2 august 2018, la 09:33:10 EEST, Joerg Bornemann a scris:
>> On 08/02/2018 08:18 AM, Simon Hausmann wrote:
>>> Given that the output of the moc changes depending on what platform and
>>> compiler dependent pre-processor macros are supplied, I would say that
>>> the output is not cross-platform.
>> ...which is why qbs does *not* have such a feature.
>> moc_XXX.cpp and XXX.moc files are generated per build configuration. The
>> output is not shared.
>>
>
> Right, moc files are not cross-platform but I think rcc & uic files are.

They are still build artefacts, and trying to add logic to figure out which of those can be shared between different targets and which can’t sounds like a nice way to introduce complexity that will bite us at some point.

Cheers,
Lars

[Thiago Macieira]

On Wednesday, 1 August 2018 23:40:40 PDT Lars Knoll wrote:
> > Right, moc files are not cross-platform but I think rcc & uic files are.
>
> They are still build artefacts, and trying to add logic to figure out which
> of those can be shared between different targets and which can’t sounds
> like a nice way to introduce complexity that will bite us at some point.

Right, and rcc is definitely not a significant portion of the build. We have
usually at most one rcc run per module. UIC ones are also often limited in
count, as are the other tools.

moc is the only one that in my mind is "significant portion of the build" and
Simon is right that we can't share its output.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center

[Thiago Macieira]

On Wednesday, 1 August 2018 23:18:51 PDT Simon Hausmann wrote:
> Given that the output of the moc changes depending on what platform and
> compiler dependent pre-processor macros are supplied, I would say that the
> output is not cross-platform.

True, even in debug-and-relaese they could change, as QT_NO_DEBUG changes.
Someone could #ifdef on it and add properties, signals or slots based on it.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center

[Lars Knoll]

> On 2 Aug 2018, at 08:13, BogDan Vatra via Development <***@qt-project.org> wrote:
> […]
> It will be nice if this feature is mandatory for Qt 6, so let's add it to
> your requirements list.


Let’s be careful not to add too many requirements upfront or we’ll never get anywhere.

For me the hard requirements are:

1. Better maintainability than with qmake. I don’t want things to become more complicated with the new system.

2. Open source and cross-platform. A given.

3. Easy to install, few external dependencies. Can be compiled form source, no circular dependencies (1b and 1c from Thiago, should also solve 1a implicitly)

4. Usable for packagers. But let’s make sure we’re not only looking at Linux distributions, but also macOS (homebrew), Windows (e.g. vcpkg) and the needs we have for our own binary packages.

5. Simple cross-compilation support. What we don’t need is the mixed mode we currently have where we build host and target binaries at the same time. For Qt 6, I’d like to fully separate that, so that a cross build will use the host tools from a previous host build.

6. Decently documented, and there are places where you can get help

7. Good integration with the world around us. That implies that the build system needs to spit out at the minimum the required files so that our users can seamlessly use both qmake and cmake for their own projects. It should also handle things the other way round so that we can depend on other projects that are build using other build systems.

Cheers,
Lars

[Lars Knoll]

> On 2 Aug 2018, at 09:52, BogDan Vatra <***@kdab.com> wrote:
>
> În ziua de joi, 2 august 2018, la 09:49:48 EEST, Lars Knoll a scris:
>>> On 2 Aug 2018, at 08:13, BogDan Vatra via Development
>>> <***@qt-project.org> wrote:
> […]
>>>
>>> It will be nice if this feature is mandatory for Qt 6, so let's add it to
>>>
>>>
>>> your requirements list.
>>
>>
>>
>> Let’s be careful not to add too many requirements upfront or we’ll never get
>> anywhere.
>
> [...]
>
> Well, if we want to have first class Qt on Android support, this feature is a
> must. Now, if we can use it also for other platforms that's a bonus.

Which feature exactly? Creating binaries for multiple targets in on compile run? Or also only calling rcc once?

The first one I can see, the second one would be just an optimisation when creating multiple binaries.

Cheers,
Lars

[Joerg Bornemann]

On 08/02/2018 10:05 AM, BogDan Vatra via Development wrote:

>>> Well, if we want to have first class Qt on Android support, this feature
>>> is a
> must. Now, if we can use it also for other platforms that's a
>>> bonus.
>>
>> Which feature exactly? Creating binaries for multiple targets in on compile
>> run? Or also only calling rcc once?
>
>> The first one I can see, the second one would be just an optimisation when
>> creating multiple binaries.
>
> First one.

We're now talking about three different things.


1. Build multiple configurations in one go

E.g. I want to build my project on Linux for Linux x86_64 and
cross-compile at the same time with MinGW. The project knows nothing
about this.

This is supported by qbs. [1]


2. Build for multiple platforms in one go

E.g. I want to cross-compile Qt with MinGW on Linux. Host tools (moc,
rcc, qlalr, qwhatnot) which are built for Linux x86_64. Libs are
cross-build for Windows in both, release and debug versions.
The project knows about host tools and "multiplexing".

Other famous use cases are: building for Android and iOS.

This is supported by qbs. [2]


3. Build something via 1 or 2 and share equal generated artifacts

We're calling rcc once, because we somehow know that the output will be
the same for all variants of the build products.

This is not supported by qbs at the moment. [3]


Cheers,

Joerg


[1] http://doc.qt.io/qbs/building-applications.html
[2] http://doc.qt.io/qbs/multiplexing.html
[3]
https://memegenerator.net/img/instances/61770863/im-sorry-do-u-still-love-me-.jpg

[Thiago Macieira]

On Wednesday, 1 August 2018 23:13:13 PDT BogDan Vatra via Development wrote:
> > Now that is nice, as we know that the moc, uic, rcc outputs are platform-
> > independent. That should help reduce the build times on Windows for
> > debug-and- release builds, as running moc is a significant portion of the
> > build time.
>
> It will be nice if this feature is mandatory for Qt 6, so let's add it to
> your requirements list.

No, this is "nice to have", not mandatory. A reduction in build time would be
very welcome, but if we don't have it, the build still works.

> BTW, not only moc, uic & rcc are platform-independent, but also the
> headers (except a few that are generated by configure script which can be
> moved to platform libs dir). What will be cool really is to unify headers,
> examples & demos for the Qt SDKs, this way we'll save a lot of space.

Sorry, I didn't get it. Can you clarify?

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center

[Thiago Macieira]

On Thursday, 2 August 2018 23:42:08 PDT BogDan Vatra via Development wrote:
> Sorry it's kinda off-topic. What I wanted to say is for Qt 6 to have a new
> SDK folders layout. Now each Qt target goes into a separate folder, but the
> bin, include (except a few files), doc, mkspecs, etc. folders are the same.
> The only platform-dependent folders are lib, plugins and qml.
> So, instead of:

We're actually going to get the bin dir shared if we start using the host
tooling to build a cross-compiled Qt and cross-compiled application.

Sharing the includes is possible, but tricky, like you said, because of the
config files that do change across builds. So we'd need two sets of -I flags:
one for cross-platform includes and one for the platform-dependent ones.
Because of that, I don't see us doing this.

> we'll have:
>
> ~/Qt
> /6.0.0
> /bin
> /include
> /lib
> /android_armv7 <-- here goes android armv7 libs
> /include <-- here goes only android specific headers
> /gcc_64 <-- here goes linux amd64 libs
> /include <-- here goes only linux 64 specific headers

Like you're suggesting. But the includes are hardly a disk space hog...

Wait, I take that back. After checking my installed build, it turns out that
the headers are a full fifth of the installed size (I don't build qtwebkit or
qtwebengine).

It might be interesting to explore this, but it would be just a "nice to
have".

Also note that QML files are usually cross-platform too and could be shared.
Any non-cross-platform QML file should be selected by the source via
QFileSelector. Plugins, however, are arch-dependent and should live in an
arch-dependent dir. That would mean splitting the "qml" dir in two:

<prefix>/lib/qml
<prefix>/share/qml

I don't see us spending the effort...

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center

[Kevin Kofler]

Thiago Macieira wrote:
> Also note that QML files are usually cross-platform too and could be
> shared. Any non-cross-platform QML file should be selected by the source
> via QFileSelector. Plugins, however, are arch-dependent and should live in
> an arch-dependent dir. That would mean splitting the "qml" dir in two:
>
> <prefix>/lib/qml
> <prefix>/share/qml
>
> I don't see us spending the effort...

That would also help multilib GNU/Linux distributions though, because right
now we have to treat all QML-only packages as arch-specific just because of
the installation path. Having the QML files themselves in /usr/share would
allow us to make those packages noarch.

Kevin Kofler

[Lisandro Damián Nicanor Pérez Meyer]

El domingo, 26 de agosto de 2018 20:12:51 -03 Kevin Kofler escribió:
> Thiago Macieira wrote:
> > Also note that QML files are usually cross-platform too and could be
> > shared. Any non-cross-platform QML file should be selected by the source
> > via QFileSelector. Plugins, however, are arch-dependent and should live in
> > an arch-dependent dir. That would mean splitting the "qml" dir in two:
> >
> > <prefix>/lib/qml
> > <prefix>/share/qml
> >
> > I don't see us spending the effort...
>
> That would also help multilib GNU/Linux distributions though, because right
> now we have to treat all QML-only packages as arch-specific just because of
> the installation path. Having the QML files themselves in /usr/share would
> allow us to make those packages noarch.

That's right, it will allow less disk usage for sure. But I don't know up to
which point is a "nice to have" feature or something that would really
decrease disk usage.

It is not strange to have more than one arch headers installed if cross
compiling, but QML packages are, and please correct me if I'm wrong, only
needed at run/develop time, but not while building.

But again, it would be nice to have it for sure.


--
I must confess, I was born at a very early age.
-- Groucho Marx

Lisandro Damián Nicanor Pérez Meyer
http://perezmeyer.com.ar/
http://perezmeyer.blogspot.com/

[Oswald Buddenhagen]

On Wed, Aug 01, 2018 at 09:01:40AM +0000, Edward Welbourne wrote:
> Jason Newton (1 August 2018 10:24)
> > One of the things I like about the way bazel keeps this is I can jump
> > back and forth between an optimized build and a full debug build (for
> > example) - they don't erase eachother
>
> This is easy to do in any build system that supports out-of-source
> (a.k.a. "shadow") builds - notably including the existing qmake-based
> builds for Qt. I never want to do an in-source build of anything.
>
indeed, that's why the example was badly chosen.

> > and you can only achieve that jumping capability, being cheap, if you
> > add a file-level layer of indirection.
>
> Not sure what you're claiming here, or what system you're claiming lacks
> it. If you've got both optimised and debug builds, you're going to have
> both sets of files *somewhere* (whether jumbled up in one directory or
> kept separate in two out-of-source build trees).
>
but having fully separate trees for only mildly different builds is
wasteful. a smart developer addresses that by having ccache in the
background, but the solution is more efficient and comprehensive when
the build tool itself implements it.
hence https://bugreports.qt.io/browse/QBS-57

[Matthew Woehlke]

On 2018-08-01 04:24, Jason Newton wrote:
> On Mon, Jul 30, 2018 at 1:30 PM Matthew Woehlke wrote:
>> On 2018-07-21 19:52, Jason Newton wrote:
>>> I wanted to mention that this is on my mind alot for a few years days
>>> as a user for a plethora of libraries. My conclusion for the build
>>> system with the brightest future is bazel [...]
>>
>> No. Just, *no*.
>>
>> Persistent build server? Java? No, thanks.
>
> This is an option, you can keep it in a preference file, or you can
> use it as a command line switch to not use this.

...and how many naïve users that "just want to build Qt" are going to
know about that?

> As for java in the loop - this is a a build system, how much does it
> matter with what it is written in if the implementation is good?

...because Java is an *enormous* added dependency for something that
*ought* to be light-weight.

>> Maybe it's gotten better, but last I knew, bazel had no notion of
>> "installing" software, which is just a non-starter. It's also heavily
>> optimized for Google's internal usage, and (partly as a result)
>> basically does not play at all nicely with software that's intended to
>> exist in an open source, package-managed ecosystem.
>
> The convention is that projects using it embrace that and don't care
> about the most system packages if they can get away with it.
Which is *exactly backwards* from what an open source package should be
doing (but typical of *proprietary* software). In fact, it's the very
antithesis of (L)GPL.

Bazel encourages practices that are *bad* for software that wants to
play nicely in an open source ecosystem.

>> On a related note, "hermetic builds" is pretty ironic. Your *build*
>> might be hermetic, but bazel itself is *far* from... it's very reliant
>> on putting all its garbage in "magic locations" in your home directory,
>> unlike most build tools that only need to write to your build directory.
>> Again, *not* friendly for anyone that needs to build Qt as an external
>> dependency.
>
> Actually that's just a default location - the output location you're
> talking about is the outputRoot/output_base setting:
> https://docs.bazel.build/versions/master/output_directories.html

Again, stuff that users that want it to Just Work aren't going to know
about. And/or that consumers of Qt are going to have to (know to) muck
with. With other build systems, this isn't an issue *in the first place*.

>>> -Ability to build external libraries from source
>>
>> Yeah... no. Building third party libraries yourself is *evil*. And a
>> great way to piss off package maintainers. No open source project should
>> *EVER* rely on, or preferably even use by default, this ability.
>
> It is not evil in any way.

Every disto, ever, disagrees.

> Bazel doesn't deal with system packages in a first class way.
Right. And this is a big problem for an open source package.

> -Analysis tools for the dependencies, like graphviz, but also memory
> usage and timing info. A legitimate profiler, have a look:
> https://docs.bazel.build/versions/master/skylark/performance.html
> https://www.kchodorow.com/blog/2015/09/18/build-y-u-go-slow/
> (admittedly few online examples of this output and this is looks
> different now IIRC)
> -Does a good job of controlling verbosity while keeping the user
> informed on the running targets being processed concurrently while
> giving a quick way to hop out, go serial or show commands executed
> -Workable, like cmake workable, visual studio generator, and xcode outputs.
> -clion has a plugin good direct plugin for it, though more will follow
> and they tried to make it easy for IDEs to incorporate (
> https://blog.bazel.build/2016/06/10/ide-support.html we can agree it'd
> be create if more build systems did this, right?
> https://docs.bazel.build/versions/master/ide.html for fuller list )

Pretty much all of which you *also* have with CMake/ninja...

--
Matthew

[Oswald Buddenhagen]

On Wed, Aug 01, 2018 at 10:33:43AM -0400, Matthew Woehlke wrote:
> On 2018-08-01 04:24, Jason Newton wrote:
> > On Mon, Jul 30, 2018 at 1:30 PM Matthew Woehlke wrote:
> >> On 2018-07-21 19:52, Jason Newton wrote:
> >>> I wanted to mention that this is on my mind alot for a few years days
> >>> as a user for a plethora of libraries. My conclusion for the build
> >>> system with the brightest future is bazel [...]
> >>
> >> No. Just, *no*.
> >>
> >> Persistent build server? Java? No, thanks.
> >
> > This is an option, you can keep it in a preference file, or you can
> > use it as a command line switch to not use this.
>
> ...and how many naïve users that "just want to build Qt" are going to
> know about that?
>
and why exactly would a user who doesn't care ... care?

> > As for java in the loop - this is a a build system, how much does it
> > matter with what it is written in if the implementation is good?
>
> ...because Java is an *enormous* added dependency
>
the actual toolchain and external dependencies play in the same league.
... which is still dwarfed by the size of a single qt debug build.

> for something that *ought* to be light-weight.
>
says *who*?

it's not that i *like* big dependencies, but there is a trade-off to be
made, and bazel is *by far* the most advanced build tool on the market
today when it comes to optimizing rebuilds of massive projects (esp.
when you have a homogeneous IT environment in your intranet). that might
not be the decisive factor when it comes to choosing a build tool for
qt, but blanket statements like yours don't exactly sound like rational
decision making.

if the spectre of bazel scares you, make your contribution to make sure
qbs makes the cut. ;)
https://www.qt.io/open-positions/software-engineer-build-systems for
that matter ...

[Lisandro Damián Nicanor Pérez Meyer]

El jueves, 2 de agosto de 2018 10:03:04 -03 Oswald Buddenhagen escribió:
[snip]
> > > As for java in the loop - this is a a build system, how much does it
> > > matter with what it is written in if the implementation is good?
> >
> > ...because Java is an *enormous* added dependency
>
> the actual toolchain and external dependencies play in the same league.
> ... which is still dwarfed by the size of a single qt debug build.

But would be problematic when building for other archs not that powerful as
"the normal ones".

> > for something that *ought* to be light-weight.
>
> says *who*?

Good point here. But if I may: largely preferred by distro maintainers.

Also I don't see bazel packaged in Debian, is it in other distros?

--
1: Una computadora sirve:
* Para tratar de dominar el mundo, un caso conocido de esto fue el de
Skinet
Damian Nadales
http://mx.grulic.org.ar/lurker/message/20080307.141449.a70fb2fc.es.html

Lisandro Damián Nicanor Pérez Meyer
http://perezmeyer.com.ar/
http://perezmeyer.blogspot.com/

[Lisandro Damián Nicanor Pérez Meyer]

El jueves, 2 de agosto de 2018 10:23:12 -03 Lisandro Damián Nicanor Pérez
Meyer escribió:
> El jueves, 2 de agosto de 2018 10:03:04 -03 Oswald Buddenhagen escribió:
> [snip]
>
> > > > As for java in the loop - this is a a build system, how much does it
> > > > matter with what it is written in if the implementation is good?
> > >
> > > ...because Java is an *enormous* added dependency
> >
> > the actual toolchain and external dependencies play in the same league.
> > ... which is still dwarfed by the size of a single qt debug build.
>
> But would be problematic when building for other archs not that powerful as
> "the normal ones".
>
> > > for something that *ought* to be light-weight.
> >
> > says *who*?
>
> Good point here. But if I may: largely preferred by distro maintainers.
>
> Also I don't see bazel packaged in Debian, is it in other distros?

Looking at:

<https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=782654>

It will take time to get bazel in Debian. And then it will need to fulfill
Thiago's 2.c points. But it still has two years to accomplish it, as qbs does.


--
14: Para acceder y navegar en internet
* Debe tener conexion a Internet
Damian Nadales
http://mx.grulic.org.ar/lurker/message/20080307.141449.a70fb2fc.es.html

Lisandro Damián Nicanor Pérez Meyer
http://perezmeyer.com.ar/
http://perezmeyer.blogspot.com/

[Christian Kandeler]

On Thu, 2 Aug 2018 10:23:12 -0300
Lisandro Damián Nicanor Pérez Meyer <***@gmail.com> wrote:

> El jueves, 2 de agosto de 2018 10:03:04 -03 Oswald Buddenhagen escribió:
> [snip]
> > > > As for java in the loop - this is a a build system, how much does it
> > > > matter with what it is written in if the implementation is good?
> > >
> > > ...because Java is an *enormous* added dependency
> >
> > the actual toolchain and external dependencies play in the same league.
> > ... which is still dwarfed by the size of a single qt debug build.
>
> But would be problematic when building for other archs not that powerful as
> "the normal ones".

You mean *on* other archs? Because the (cross-)compiler does not care how powerful the target architecture is.


Christian

[Konstantin Tokarev]

02.08.2018, 16:58, "Christian Kandeler" <***@qt.io>:
> On Thu, 2 Aug 2018 10:23:12 -0300
> Lisandro Damián Nicanor Pérez Meyer <***@gmail.com> wrote:
>
>>  El jueves, 2 de agosto de 2018 10:03:04 -03 Oswald Buddenhagen escribió:
>>  [snip]
>>  > > > As for java in the loop - this is a a build system, how much does it
>>  > > > matter with what it is written in if the implementation is good?
>>  > >
>>  > > ...because Java is an *enormous* added dependency
>>  >
>>  > the actual toolchain and external dependencies play in the same league.
>>  > ... which is still dwarfed by the size of a single qt debug build.
>>
>>  But would be problematic when building for other archs not that powerful as
>>  "the normal ones".
>
> You mean *on* other archs? Because the (cross-)compiler does not care how powerful the target architecture is.

Unfortunately, Debian does not use cross-compilation for building packages

(Here should be a photo of cluster of low-power SH4 boards which I've seen somewhere)

>
> Christian
> _______________________________________________
> Development mailing list
> ***@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development

--
Regards,
Konstantin

[Thiago Macieira]

On Thursday, 2 August 2018 07:16:18 PDT Konstantin Tokarev wrote:
> Unfortunately, Debian does not use cross-compilation for building packages

Which is unfortunately the case for most (if not all) Linux distributions as a
significant number of frameworks and applications don't cross compile at all.
So they need to compile on target.

Sometimes it's an option to use an emulated environment (qemu), but often
that's even slower than on target. A hybrid approach that runs under qemu and
runs a native cross compiler disguised as a target compiler is also possible,
but not easy to achieve.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center

[Konstantin Tokarev]

02.08.2018, 19:50, "Thiago Macieira" <***@intel.com>:
> On Thursday, 2 August 2018 07:16:18 PDT Konstantin Tokarev wrote:
>>  Unfortunately, Debian does not use cross-compilation for building packages
>
> Which is unfortunately the case for most (if not all) Linux distributions as a
> significant number of frameworks and applications don't cross compile at all.
> So they need to compile on target.

Do you have any prominent examples? I'm working on embedded software for 8 years,
me and my colleagues have dealt with various open source packages, we've never needed
to resort to compilation on target or qemu environment.

There are cross-compiled embedded distros like Buildroot which include thousands of
packages.

>
> Sometimes it's an option to use an emulated environment (qemu), but often
> that's even slower than on target. A hybrid approach that runs under qemu and
> runs a native cross compiler disguised as a target compiler is also possible,
> but not easy to achieve.
>
> --
> Thiago Macieira - thiago.macieira (AT) intel.com
>   Software Architect - Intel Open Source Technology Center
>
> _______________________________________________
> Development mailing list
> ***@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development

--
Regards,
Konstantin

[Lisandro Damián Nicanor Pérez Meyer]

El jueves, 2 de agosto de 2018 14:36:11 -03 Konstantin Tokarev escribió:
> 02.08.2018, 19:50, "Thiago Macieira" <***@intel.com>:
> > On Thursday, 2 August 2018 07:16:18 PDT Konstantin Tokarev wrote:
> >> Unfortunately, Debian does not use cross-compilation for building
> >> packages
> >
> > Which is unfortunately the case for most (if not all) Linux distributions
> > as a significant number of frameworks and applications don't cross
> > compile at all. So they need to compile on target.
>
> Do you have any prominent examples? I'm working on embedded software for 8
> years, me and my colleagues have dealt with various open source packages,
> we've never needed to resort to compilation on target or qemu environment.

Out of the box Qt. You need to specifically build it to cross compile stuff,
which is not what distros look for. What distros need is being able to use the
real-arch binaries to build the target ones.


--
Lisandro Damián Nicanor Pérez Meyer
http://perezmeyer.com.ar/
http://perezmeyer.blogspot.com/

[Thiago Macieira]

On Thursday, 2 August 2018 10:36:11 PDT Konstantin Tokarev wrote:
> Do you have any prominent examples?

No, sorry. We don't need to cross-compile at the company I work for :-)

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center

[Lisandro Damián Nicanor Pérez Meyer]

El jueves, 2 de agosto de 2018 10:57:42 -03 Christian Kandeler escribió:
[snip]
> > But would be problematic when building for other archs not that powerful
> > as
> > "the normal ones".
>
> You mean *on* other archs? Because the (cross-)compiler does not care how
> powerful the target architecture is.

Yes, exactly.

--
Lisandro Damián Nicanor Pérez Meyer
http://perezmeyer.com.ar/
http://perezmeyer.blogspot.com/

[Matthew Woehlke]

On 2018-08-02 09:03, Oswald Buddenhagen wrote:
> On Wed, Aug 01, 2018 at 10:33:43AM -0400, Matthew Woehlke wrote:
>> On 2018-08-01 04:24, Jason Newton wrote:
>>> On Mon, Jul 30, 2018 at 1:30 PM Matthew Woehlke wrote:
>>>> Persistent build server? Java? No, thanks.
>>>
>>> This is an option, you can keep it in a preference file, or you can
>>> use it as a command line switch to not use this.
>>
>> ...and how many naïve users that "just want to build Qt" are going to
>> know about that?
>>
> and why exactly would a user who doesn't care ... care?

...because building Qt just spawned a process that is never going to
terminate, is going to sit around pointlessly monitoring their file
system, and is going to potentially cause who-knows-what issues?

If I want to just download and build some package (i.e. I'm not
*actively developing* that software), doing so shouldn't "infect" my
machine with zombie processes. When the build is done, it should be *done*.

> it's not that i *like* big dependencies, but there is a trade-off to be
> made, and bazel is *by far* the most advanced build tool on the market
> today when it comes to optimizing rebuilds of massive projects

...which is *totally irrelevant* to everyone that is not a Qt developer.
Just like software is written once and read many times, it is developed
by a few and deployed by many. Optimizing for development *at the
expense* of deployment strikes me as... questionable. If the penalty to
deployment cost was minimal, that would be one thing, but with bazel, it
isn't.

--
Matthew

[Jason Newton]

On Thu, Aug 2, 2018 at 9:34 AM Matthew Woehlke <***@gmail.com> wrote:
>
> On 2018-08-02 09:03, Oswald Buddenhagen wrote:
> > On Wed, Aug 01, 2018 at 10:33:43AM -0400, Matthew Woehlke wrote:
> >> On 2018-08-01 04:24, Jason Newton wrote:
> >>> On Mon, Jul 30, 2018 at 1:30 PM Matthew Woehlke wrote:
> >>>> Persistent build server? Java? No, thanks.
> >>>
> >>> This is an option, you can keep it in a preference file, or you can
> >>> use it as a command line switch to not use this.
> >>
> >> ...and how many naïve users that "just want to build Qt" are going to
> >> know about that?
> >>
> > and why exactly would a user who doesn't care ... care?
>
> ...because building Qt just spawned a process that is never going to
> terminate, is going to sit around pointlessly monitoring their file
> system, and is going to potentially cause who-knows-what issues?
>
> If I want to just download and build some package (i.e. I'm not
> *actively developing* that software), doing so shouldn't "infect" my
> machine with zombie processes. When the build is done, it should be *done*.

This is arguing over defaults and for a hypothetical case. The daemon
exits automatically after a preference-set threshold of time of idle.
It was engineered and tested for many years to not have problems as
to what you are describing, such as updates to the bazel installation
itself, but it is possible something goes wrong . The capability
makes sense to everyday developers so the it earns it's place in the
world. There's a flag to opt out and projects can set defaults that
bazel will respect. Users can also disable it permanently in their
home preference files and invoke bazel with the switch to disable. If
that is not sufficient, upstream may listen to behavioral changes such
as opt-out by default and opt-in with flags.


>
> > it's not that i *like* big dependencies, but there is a trade-off to be
> > made, and bazel is *by far* the most advanced build tool on the market
> > today when it comes to optimizing rebuilds of massive projects
>
> ...which is *totally irrelevant* to everyone that is not a Qt developer.
> Just like software is written once and read many times, it is developed
> by a few and deployed by many. Optimizing for development *at the
> expense* of deployment strikes me as... questionable. If the penalty to
> deployment cost was minimal, that would be one thing, but with bazel, it
> isn't.

Perhaps there are strategies to make it reduced by stripping out
things in packages and not depending on a full JRE. Without swing,
some people have reported getting JRE's down to 10 MiB. I'll remind
again it depends on a JRE, not a JDK - there's some semblance to
arguing about python usage, in that case.

-Jason

[Ville Voutilainen]

On 1 August 2018 at 11:24, Jason Newton <***@gmail.com> wrote:
>> And... seriously, *Java*?! Talk about bloat-ware... As dependencies for
>> *a build tool* go, that's pretty insane. Especially if you're not
>> planning to use it to build Java code.
>
> As I said, ~300-400 megabytes for a JRE (~90MiB DL), in particular
> openjdk , prospectively in a development environment where you've got
> devel headers and packages installed. The cost of the dependency is
> neither bloat nor unbounded, it really is just that size on a machine
> that's prepared to build software in 2018. Please explain what issue
> this presents.

The prospect of having to install a JRE to build Qt or eventually Qt
applications
solicits a fairly loud "I think I'll pass" from me. I don't use java
for anything. Having
it around for a supposedly simple and easy-to-adopt build system is a cost
that I am not eager to pay. Do we expect to package a JRE with Qt? Do we
expect a JRE to be a package dependency of Qt for distro vendors?

[Dominik Holland]

Am 01.08.18 um 16:40 schrieb Ville Voutilainen:

> On 1 August 2018 at 11:24, Jason Newton <***@gmail.com> wrote:
>>> And... seriously, *Java*?! Talk about bloat-ware... As dependencies for
>>> *a build tool* go, that's pretty insane. Especially if you're not
>>> planning to use it to build Java code.
>> As I said, ~300-400 megabytes for a JRE (~90MiB DL), in particular
>> openjdk , prospectively in a development environment where you've got
>> devel headers and packages installed. The cost of the dependency is
>> neither bloat nor unbounded, it really is just that size on a machine
>> that's prepared to build software in 2018. Please explain what issue
>> this presents.
> The prospect of having to install a JRE to build Qt or eventually Qt
> applications
> solicits a fairly loud "I think I'll pass" from me. I don't use java
> for anything. Having
> it around for a supposedly simple and easy-to-adopt build system is a cost
> that I am not eager to pay. Do we expect to package a JRE with Qt? Do we
> expect a JRE to be a package dependency of Qt for distro vendors?
That would also have a big inpact on the yocto packages, as that would
require building java itself to compile Qt (yocto usually doesn't use
prepackaged binaries...),
and also compile it again for the SDK architecture...

For me JRE as a Qt build dependency is a no-go

Dominik

[Jason Newton]

On Thu, Aug 2, 2018 at 11:21 PM Dominik Holland
<***@pelagicore.com> wrote:
>
> Am 01.08.18 um 16:40 schrieb Ville Voutilainen:
>
> > On 1 August 2018 at 11:24, Jason Newton <***@gmail.com> wrote:
> >>> And... seriously, *Java*?! Talk about bloat-ware... As dependencies for
> >>> *a build tool* go, that's pretty insane. Especially if you're not
> >>> planning to use it to build Java code.
> >> As I said, ~300-400 megabytes for a JRE (~90MiB DL), in particular
> >> openjdk , prospectively in a development environment where you've got
> >> devel headers and packages installed. The cost of the dependency is
> >> neither bloat nor unbounded, it really is just that size on a machine
> >> that's prepared to build software in 2018. Please explain what issue
> >> this presents.
> > The prospect of having to install a JRE to build Qt or eventually Qt
> > applications
> > solicits a fairly loud "I think I'll pass" from me. I don't use java
> > for anything. Having
> > it around for a supposedly simple and easy-to-adopt build system is a cost
> > that I am not eager to pay. Do we expect to package a JRE with Qt? Do we
> > expect a JRE to be a package dependency of Qt for distro vendors?
> That would also have a big inpact on the yocto packages, as that would
> require building java itself to compile Qt (yocto usually doesn't use
> prepackaged binaries...),
> and also compile it again for the SDK architecture...
>
> For me JRE as a Qt build dependency is a no-go
>

Have a look at this:
http://zlika.github.io/presentations/compile-openjdk8/slides_en.html#/_generation_of_compact_profiles

If the java packager of the distribution puts in some time with the
openjdk packages they support, they can get it down to 13-25 MiB,
before compression, and 50% of those numbers with. Bazel doesn't need
gui stuff.

From OpenSUSE build service, here is the build time for the full
OpenJDK 8 build
https://build.opensuse.org/public/build/Java:Factory/openSUSE_Leap_15.0/x86_64/java-1_8_0-openjdk/_log
- 1976 seconds with what appears to be a single threaded build. It
should be possible to considerably lean this down.

For control, here is qt5's build file
https://build.opensuse.org/public/build/KDE:Qt:5.11/openSUSE_Leap_15.0/x86_64/libqt5-qtbase/_log
clocking in 2455s

-Jason

[Thiago Macieira]

On Thursday, 2 August 2018 23:46:07 PDT Jason Newton wrote:
> From OpenSUSE build service, here is the build time for the full
> OpenJDK 8 build
> https://build.opensuse.org/public/build/Java:Factory/openSUSE_Leap_15.0/x86_
> 64/java-1_8_0-openjdk/_log - 1976 seconds with what appears to be a single
> threaded build. It
> should be possible to considerably lean this down.
>
> For control, here is qt5's build file
> https://build.opensuse.org/public/build/KDE:Qt:5.11/openSUSE_Leap_15.0/x86_6
> 4/libqt5-qtbase/_log clocking in 2455s

You're not going to convince a bunch of C++ developers to use a Java
buildsystem. The solution you're proposing would need to be an order of
magnitude better on everything else for that to be acceptable. And so far we
have found it to be (subjectively) worse than the others.

As for distros, Qt is usually on the critical build path, due to its size.
Doubling the build time by inserting a dependency of equal size is not going
to make you friends.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center

[Tobias Hunger]

Hi Thiago,

On Sat, Jul 21, 2018, 04:36 Thiago Macieira <***@intel.com>
wrote:

> [...] I'd like to add the following
> extra requirements to the buildsystem we choose to use. This is in
> addition to
> the functionality requirements.
>
> [...]
>

1) Ease of obtention


2) Track record
>
> 3) Community support
>

All these are very reasonable to ask for. I personally would like to add
the following point for consideration:

4) Toolable

A) works with IDE(s) -- preferably more than one

B) Should be easy to hook in static and dynamic code analyzer tools

Best Regards,
Tobias

[Thiago Macieira]

On Sunday, 22 July 2018 01:57:59 PDT Tobias Hunger wrote:
> All these are very reasonable to ask for. I personally would like to add
> the following point for consideration:
>
> 4) Toolable
>
> A) works with IDE(s) -- preferably more than one

Hello Tobias

The fact that we have an IDE of ours means this one should be reasonably easy
to address, as we can make that IDE work with the tool. So long as the tool is
actually toolable, of course.

"More than one" would be quite a challenge.

> B) Should be easy to hook in static and dynamic code analyzer tools

Can you elaborate?

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center

[Tobias Hunger]

Hi Thiago,

On Sun, Jul 22, 2018, 17:35 Thiago Macieira <***@intel.com>
wrote:

> > A) works with IDE(s) -- preferably more than one
>
> Hello Tobias
>
> The fact that we have an IDE of ours means this one should be reasonably
> easy
> to address, as we can make that IDE work with the tool. So long as the
> tool is
> actually toolable, of course.


This is a matter of resources and what we want to spend those on.

Creator needs first class support for qmake for years to come. It also
needs first class support for CMake as that is what people out there
actually use. These two are set for the foreseeable future.

When we have a decision on what Qt will use going forward, then that will
be added to the mix.

"More than one" would be quite a challenge.


We can not expect people to drop their preferred set of tools to work on
Qt. That is a sure-fire way to reduce contributions from outside of our
core community.

> B) Should be easy to hook in static and dynamic code analyzer tools
>
> Can you elaborate?
>

Most build tools offer to create a build_commands.json file so that some of
the clang tools can be used easily. I expect that from any build tool
nowadays.

Meson offers a "build with address sanitisation" option (and similar ones
for UBSAN, etc.) that can just turn on for your project. I would love that,
but at the very least compiler-based tools like ASAN and UBSAN should be
easy to enable with good documentation on how to do so.

A build tool with a track record of empowering developers by making new
tools easy to use on existing projects would be welcome.

Best regards,
Tobias

>

[Thiago Macieira]

On Sunday, 22 July 2018 10:34:13 PDT Tobias Hunger wrote:
> > "More than one" would be quite a challenge.
>
> We can not expect people to drop their preferred set of tools to work on
> Qt. That is a sure-fire way to reduce contributions from outside of our
> core community.

Sure, but we compare that to status quo: what is the support out there for
qmake?

The only IDEs that support it natively are Qt Creator and KDevelop. For Visual
Studio and for XCode, qmake itself creates the export, so the tool we choose
could do the same.

> > B) Should be easy to hook in static and dynamic code analyzer tools
> >
> > Can you elaborate?
>
> Most build tools offer to create a build_commands.json file so that some of
> the clang tools can be used easily. I expect that from any build tool
> nowadays.
>
> Meson offers a "build with address sanitisation" option (and similar ones
> for UBSAN, etc.) that can just turn on for your project. I would love that,
> but at the very least compiler-based tools like ASAN and UBSAN should be
> easy to enable with good documentation on how to do so.

Any build tool MUST have a way for the caller to pass override C and C++
flags. Full stop, no exceptions.[*]

Given that, I don't see why the tool must have special support for those
compilation flags. It's a nice-to-have, though. Similarly, it's a very nice to
have a way to specify which C++ language edition you want, whether you want
exceptions and/or RTTI enabled, whether you want warnings turned on and off,
etc.

[*] and a way that isn't unduly difficult to trigger. Counter-example: Qt's
current buildsystem. Look at this build logs:
https://build.opensuse.org/public/build/openSUSE:Factory/standard/x86_64/
libqt5-qtbase/_log
On lines 1056-1059, you see CFLAGS and CXXFLAGS being set. But then if you
scan forward in the actual build, you don't see any of them applied.
--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center

[Tobias Hunger]

On Sat, Jul 21, 2018, 07:38 Bogdan Vatra via Development <
***@qt-project.org> wrote:

> You really hate QBS don't you ? :)


Do you have so few arguments for qbs that you need to appeal to emotions? :)

Anyway IMHO is more important to have a clean, nice and easy to use syntax
> and
> to be tooling friendly than 1.b.
>

Sure you can write toolable build files with qbs, but there is nothing
stopping you writing an unparsable JS mess either.

For QML we needed to define a toolable subset of the language with the UI
files to have any chance at tooling it. For the qbs dialect of QML that has
not been done yet. So I do not consider qbs to be toolable at this point.

Best regards,
Tobias

[Sérgio Martins]

On Sat, Jul 21, 2018 at 3:35 AM, Thiago Macieira
<***@intel.com> wrote:
> Hello
>
> Having spent far too much time trying to figure out why crappy buildsystems
> cause failures in distros (like TensorFlow or libvpx - see
> https://plus.google.com/+ThiagoMacieira/posts/DqTKdRGfuwR ), I'd like to make
> sure this doesn't happen to Qt 6. For that, I'd like to add the following
> extra requirements to the buildsystem we choose to use. This is in addition to
> the functionality requirements.
>
> These apply to the buildsystem at the time of Qt's the switch to it.

Will the switch coincide with 6.0 ? If yes, does it have to or could
the switch happen at Qt > 6.0.0 whenever we have a *good* solution ?


Regards,
Sergio Martins

[Thiago Macieira]

On Wednesday, 1 August 2018 13:14:32 PDT Sérgio Martins wrote:
> On Sat, Jul 21, 2018 at 3:35 AM, Thiago Macieira
>
> <***@intel.com> wrote:
> > Hello
> >
> > Having spent far too much time trying to figure out why crappy
> > buildsystems
> > cause failures in distros (like TensorFlow or libvpx - see
> > https://plus.google.com/+ThiagoMacieira/posts/DqTKdRGfuwR ), I'd like to
> > make sure this doesn't happen to Qt 6. For that, I'd like to add the
> > following extra requirements to the buildsystem we choose to use. This is
> > in addition to the functionality requirements.
> >
> > These apply to the buildsystem at the time of Qt's the switch to it.
>
> Will the switch coincide with 6.0 ? If yes, does it have to or could
> the switch happen at Qt > 6.0.0 whenever we have a *good* solution ?

The plan was to switch for 6.0. It could happen later if we don't get
consensus and is too disruptive earlier, though. But 6.0 is the perfect time,
as we drop old platforms and can reliably require people to upgrade to new
things.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center

[Lars Knoll]

On 2 Aug 2018, at 00:27, Thiago Macieira <***@intel.com<mailto:***@intel.com>> wrote:

On Wednesday, 1 August 2018 13:14:32 PDT Sérgio Martins wrote:
On Sat, Jul 21, 2018 at 3:35 AM, Thiago Macieira

<***@intel.com<mailto:***@intel.com>> wrote:
Hello

Having spent far too much time trying to figure out why crappy
buildsystems
cause failures in distros (like TensorFlow or libvpx - see
https://plus.google.com/+ThiagoMacieira/posts/DqTKdRGfuwR ), I'd like to
make sure this doesn't happen to Qt 6. For that, I'd like to add the
following extra requirements to the buildsystem we choose to use. This is
in addition to the functionality requirements.

These apply to the buildsystem at the time of Qt's the switch to it.

Will the switch coincide with 6.0 ? If yes, does it have to or could
the switch happen at Qt > 6.0.0 whenever we have a *good* solution ?

The plan was to switch for 6.0. It could happen later if we don't get
consensus and is too disruptive earlier, though. But 6.0 is the perfect time,
as we drop old platforms and can reliably require people to upgrade to new
things.

Yes, 6.0 is the time I’d like to move away from qmake for building Qt. It’s a time when we do larger changes anyway, so this fits in. I don’t really see us doing such a switch for a minor release anymore, because I think such a change is difficult to make happen without compromising the 6 months release cycle.

Cheers,
Lars