[Development] Notes on "Qt Build Systems" @ QtCon 2016

[Andrew Knight]

We had a vibrant discussion on Qt Build Systems, hosted by Kai.

tl;dr: Lots of discussion on the merits of which build system (CMake,
Qbs) should replace qmake in building Qt; lots of supporters of CMake
but no volunteers to do the work, many reasons to use Qbs as well. Some
related discussions about how this impacts Qt Creator and the Qt
offering in general.

* Updates from various departments

** qmake
- "Still undead"
- qmake parses c++ now (dependency scanner)
- configure system rewrite drove some qmake changes
-- The changes are "not much" ~1500 lines of code to parse and use
formal configuration
-- These changes do not imply a commitment to qmake as Qt's permanent
build system; the real work is done in configure.json and similar JSON
files and can be ported to another build system
- (Kai) "The message we want to send is: qmake will be deprecated, but
supported for a long time"

** Qbs
- Continues to be developed, mostly small incremental changes/releases
- Has been used to build Creator for a long time (alongside qmake).
Support for building Creator with qmake might eventually be dropped.
- A WIP branch in qtbase (wip/qbs) demonstrates that Qbs can build Qt

** CMake
- (Tobias) Cmake is the "worst" system in Qt Creator because CMake
doesn't give enough feedback to the IDE.
We need first-class support for CMake in Qt Creator, though, so that
is irrelevant to the discussion of which
system we use to build Qt. If we build Qt with Qbs, we have to also
have support in Creator. In other words,
the more build systems we support, the more work Creator folks have
to do to maintain it.


* Quick survey: which build system do you use (raise of hands by ~40 people)
- CMake ~70%
- qmake ~20%
- Qbs ~10%


* What should we use to build Qt in the future (qmake, Qbs, CMake)?

** Pro Qbs:
- (Kai) "We can do better than CMake"
- (Kai) "having our own build system is also about making a more "out of
the box" experience for our users. Qt is more than a C++ library; we
need to ship a good-quality build tool"
- (Rich/Stephen) Qbs might be better at doing host tools and
cross-compiled builds than CMake
- Qbs doesn't have an intermediate step (i.e. makefile generation).
Makefile generation has disadvantages for incremental builds and
dependency tracking
-- (Ossi) "acompletely accurate build dependency tracking system with a
meta build system is very expensive"

** Pro CMake:
- Much larger user base than Qbs, leading to possibly more contributions
-- (Stephen) Most people (in this room) use CMake.
- Qt is not in the business of creating a build tool
- Qbs is still not finished
- CMake is mature and widely used by Qt developers
- Using an external tool tends to benefit both projects better: the
CMake community benefits from Qt's fixes to CMake, while Qt benefits
from the CMake community's improvements to CMake.
- (Milian) CMakeis used by e.g. clang and it works for them
- CMake comes with support for hundreds of modules; Qbs only supports Qt
and pkg-config currently.

** General comments:
- Do we really need to care what "outsiders" think of what we use in Qt?
-- Yes, because it is a statement about what tool is good for Qt
development.
--- (Someone) "If Qbs is not used by Qt, why should I use it for my
project? If Qt switches to Qbs, I will switch to it from CMake. With
CMake, we don't have this problem: a lot of people are using it and it
is evolving"
-- Yes, because it is a statement about what tool is supported by Qt,
and we don't want that tool to go away.
--- (Eddy) "CMake will be there even if we lose interest. By using it in
Qt, we shift the burden to someone else. If qmake's future is uncertain,
it will make users uncomfortable
--- (David) "Either something changes, or it dies"
-- No, because Qt should support all major build systems anyway
- Does Qbs support dependency tracking like CMake does
-- Answer: Yes
- (Stephen) "In reality, rewriting Qt's build system in CMake will
actually be a PITA, and will require changes to CMake to make everything
better"
- (Rich) "Isn't the time better spent improving CMake than build a new
build system?"
- There is work to do to make Qt generate better CMake files
- (Andreas) "I love that Qbs is declarative. I really love how Qbs can
bind dependencies and rules on how I generate artifacts"
- (Someone)"The beauty of Qbs is that it is accessible and it is clear
how the separation between the declarative stops and the imperative begins."
- (Someone) "Qbs doesn't seem very active, and Qt doesn't use it. This
scares us. CMake is the best tool we have right now, but it isn't perfect."
- (Eddy) Why are we using this ancient tool? (Referring to make)
- (Andreas) "Every big project ends up building their own system. Qbs is
a possibility to create a new build system. If it stays in the Qt
Project, less people will use it."

** General sentiment:
- As long as Qbs looks like a part of Qt, it is perceived as a Qt
product, and is less attractive to external users.
- Yet, there remains a conflict: "if Qt doesn't use it, I don't want to
use it" vs. "if it's not outside of Qt, I don't want to use it"

** Summary:
-- We are thinking about switching build systems. We don't know what to
do yet, but we can't decide it here.

** Further notes on the switch:
- Bootstrapping Qbs is required before we can really move forward.
-- Solving this problem isn't rocket science (Ossi), it's just not a
priority.
-- Linux distro maintainers might be more compelled to put a
static-built Qbs in their bootstrap if Qbs is seen as a core tool
- (Stephen) "Go ahead and build a better build system, but make sure
users can still use CMake"
- (Jake) "Sometimes you have to do something different"

[Wolfgang Baron]

I really want to move to QBS, because of its advantages (speed, good
dependency tracking, nice syntax, fexibility). However, as long as
Qt-Creator keeps generating new projects using qmake templates, I am not
so confident in the future of QBS. Also, if QBS is not used for all own
projects, my confidence shrinks even further.

I had a hard time getting around QBS's edges and I do not like to
transform each new project from qmake to QBS and maybe away from QBS
again it it does not stick. So if the Qt-project wants to have QBS, it's
either all in or all out. For me as a developer, I am most interested in
a clear decision, so that I know which build system to use in the future.

Please make this decision a top priority and clearly communicate the result.
Thanks!

[Sune Vuorela]

On 2016-09-05, Andrew Knight <***@intopalo.com> wrote:
> tl;dr: Lots of discussion on the merits of which build system (CMake,
> Qbs) should replace qmake in building Qt; lots of supporters of CMake
> but no volunteers to do the work, many reasons to use Qbs as well. Some

I do think that there is volunteers to get Qt building with cmake if
there is a likly chance of getting it accepted. I think I also
commmunicated that at the session.

/Sune

[NIkolai Marchenko]

Been using QBS for the last 6 months, transformed all projects to it(from
qmake). Never looked back.
It just clicks for me. Most everything seems logical (if poorly explained)
when you understand how to do it.


One thing QBS needs is better documentation, because a lot of things that
are "obvious" to Christian Kandeler and Jake Petroules (sorry if misspeled)
are not ever mentioned in the docs or are in obscure places.
Using QBS effectively is literally impossible without knowledge of
freenode:#qt-qbs

On Mon, Sep 5, 2016 at 1:27 PM, Sune Vuorela <***@vuorela.dk> wrote:

> On 2016-09-05, Andrew Knight <***@intopalo.com> wrote:
> > tl;dr: Lots of discussion on the merits of which build system (CMake,
> > Qbs) should replace qmake in building Qt; lots of supporters of CMake
> > but no volunteers to do the work, many reasons to use Qbs as well. Some
>
> I do think that there is volunteers to get Qt building with cmake if
> there is a likly chance of getting it accepted. I think I also
> commmunicated that at the session.
>
> /Sune
>
> _______________________________________________
> Development mailing list
> ***@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development
>

[Ch'Gans]

On 5 September 2016 at 23:08, NIkolai Marchenko <***@gmail.com> wrote:
> Been using QBS for the last 6 months, transformed all projects to it(from
> qmake). Never looked back.
> It just clicks for me. Most everything seems logical (if poorly explained)
> when you understand how to do it.

I have switched quite a few projects from qmake to Qbs, and me too I
never looked back!
QtCreator has decent support for Qbs (could be improved tho, qmake
support could be improved too and same goes with CMake)
I never wanted to use CMake b/c for me it look like a gross hack
(Reminds me of GNU M4).
Qbs has been designed from the ground up to tackle build management,
and it does it very well in a very elegant and efficient way.
Qbs is a radical change compared to the usual Makefile-generator approach.
Qbs relies on Qt, Qt declarative and Qt JS, which brings a work horse
CMake cannot even dream of.
I would compare CMake and Qbs, the same way Linus Torvald compares SVN and Git:
SVN claims to be CVS but better => Epic fail, Git has been designed to
address devs need using a new and efficient approach.

I've recently contributed to Qbs (with great help from Jake and
Christian), and I've been amazed how easy, simple and straight-forward
the job was.
We're talking 500 lines of raw source code (inc blanks, comments,
autotests, qbs&qmake files, ...) to add a Clang Database generator to
Qbs!
This was that easy because of Qbs architecture but Qt did help a lot
too, generating a JSON Db with Qt was a piece of cake.

I know that lot of project use CMake, but as someone noted, projects
use CMake because they have to, not because they want too.
What are the other cross-platform choices? Autotools, hand-crafted
Makefiles, Scons, Maven, ...
Quite some years ago I was a happy GNU autotools user. Yes I was happy
with it! Why? b/c I didn't have choice back at that time.
Now, each time I have to hack an autotools-based project, I want to
wash my hands 200 times in a raw, yuck!

> One thing QBS needs is better documentation, because a lot of things that
> are "obvious" to Christian Kandeler and Jake Petroules (sorry if misspeled)
> are not ever mentioned in the docs or are in obscure places.
> Using QBS effectively is literally impossible without knowledge of
> freenode:#qt-qbs

Yeah, sort of true. I've learned Qbs by looking at Qbs's own qbs files
and QtCreator's ones too. And of course the documentation, which is
"just" good enough to get started.
It didn't take long to get my projects ported. I now have a
easy-to-read, easy-to-understand, easy-to-extend cross-platform build,
test and packaging (inc. installer) system. Bye bye qmake!

Makefiles are out-dated (no punt intended) and so is CMake and any
other Makefile-based tools.
Makefiles are dead! CMake is ill! (Friendly, easy and provocative argument)

Long live Qbs!

My 2 cents,
Chris

>
> On Mon, Sep 5, 2016 at 1:27 PM, Sune Vuorela <***@vuorela.dk> wrote:
>>
>> On 2016-09-05, Andrew Knight <***@intopalo.com> wrote:
>> > tl;dr: Lots of discussion on the merits of which build system (CMake,
>> > Qbs) should replace qmake in building Qt; lots of supporters of CMake
>> > but no volunteers to do the work, many reasons to use Qbs as well. Some
>>
>> I do think that there is volunteers to get Qt building with cmake if
>> there is a likly chance of getting it accepted. I think I also
>> commmunicated that at the session.
>>
>> /Sune
>>
>> _______________________________________________
>> Development mailing list
>> ***@qt-project.org
>> http://lists.qt-project.org/mailman/listinfo/development
>
>
>
> _______________________________________________
> Development mailing list
> ***@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development
>

[Konstantin Tokarev]

06.09.2016, 18:24, "Jake Petroules" <***@qt.io>:
>> On Sep 6, 2016, at 5:14 PM, Kevin Kofler <***@chello.at> wrote:
>>
>> Ch'Gans wrote:
>>> I never wanted to use CMake b/c for me it look like a gross hack
>>> (Reminds me of GNU M4).
>>
>> The CMake language is much easier to use than m4, and also there is just one
>> layer rather than having autoconf on top of m4, with shell script snippets
>> mixed in.
>>
>> There is a reason CMake is being proposed for Qt and autotools is not.
>>
>>> Makefiles are out-dated (no punt intended) and so is CMake and any
>>> other Makefile-based tools.
>>> Makefiles are dead! CMake is ill! (Friendly, easy and provocative
>>> argument)
>>
>> CMake can generate other build files than makefiles (e.g., the Ninja
>> generator is basically a drop-in replacement).
>
> Again, Ninja has its architectural limitations as well, so this would not be useful. The problem isn't (just) Makefiles, it's the fact that we don't have a build tool that is fundamentally better and more powerful than anything we've ever had before, and we CAN have this. It's like C++ vs Motorola 68k assembler.

Architecture of ninja is not the biggest problem here - I'd rather argue that if your build graph is unknown ahead of time your build process is flawed. Problem is that CMake often makes it hard to do things which can be easily expressed in terms of make or ninja, e.g. it does not have a concept of build rule (instead you have for write foreach cycles), or such trivial thing as building PCH for gcc turns into the page of ugly cmake code [1]

[1] https://gist.github.com/larsch/573926

>
>> I guess somebody could even get CMake to write Qbs files, it would just be
>> one more generator. :-)
>
> Again, useless, because Qbs is more powerful and at a much higher level of abstraction, so a generator would only be useful in the reverse direction. It's like trying to make a compiler to transform Motorola 68k assembler to C++. Only the reverse transformation of that can done in a useful manner.
>
>>        Kevin Kofler
>>
>> _______________________________________________
>> Development mailing list
>> ***@qt-project.org
>> http://lists.qt-project.org/mailman/listinfo/development
>
> --
> Jake Petroules - ***@qt.io
> Consulting Services Engineer - The Qt Company
> Qbs build tool evangelist - qbs.io
> ,
>
> _______________________________________________
> Development mailing list
> ***@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development


--
Regards,
Konstantin

[Cristian Adam]

On Tue, Sep 6, 2016 at 5:14 PM, Kevin Kofler <***@chello.at> wrote:

>
> I guess somebody could even get CMake to write Qbs files, it would just be
> one more generator. :-)
>
>
This was done already
<https://gitlab.kitware.com/cmake/cmake/commit/f85db2f32358e6de921aba7d1cb8ecb81da934c0>,
but it was removed from CMake due to bad feedback from
Qt Creator people.

Cheers,
Cristian.

[Kevin Kofler]

Cristian Adam wrote:
> This was done already
> <https://gitlab.kitware.com/cmake/cmake/commit/f85db2f32358e6de921aba7d1cb8ecb81da934c0>,
> but it was removed from CMake due to bad feedback from
> Qt Creator people.

Interesting, but why was that done on top of existing makefile generators?
I'd expect the Qbs generator to be a new toplevel generator instead.

Kevin Kofler

[André Somers]

Sent from my iPhone

> On 06 Sep 2016, at 17:35, Cristian Adam <***@gmail.com> wrote:
>
>> On Tue, Sep 6, 2016 at 5:14 PM, Kevin Kofler <***@chello.at> wrote:
>>
>> I guess somebody could even get CMake to write Qbs files, it would just be
>> one more generator. :-)
>
> This was done already, but it was removed from CMake due to bad feedback from
> Qt Creator people.

Pitty. That might have made a useful porting tool.

André

>
> Cheers,
> Cristian.
>
> _______________________________________________
> Development mailing list
> ***@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development

[BogDan Vatra]

On marți, 6 septembrie 2016 17:35:03 EEST Cristian Adam wrote:
> On Tue, Sep 6, 2016 at 5:14 PM, Kevin Kofler <***@chello.at> wrote:
> > I guess somebody could even get CMake to write Qbs files, it would just be
> > one more generator. :-)
>
> This was done already
> <https://gitlab.kitware.com/cmake/cmake/commit/f85db2f32358e6de921aba7d1cb8e
> cb81da934c0>, but it was removed from CMake due to bad feedback from
> Qt Creator people.
>

Ha ha ha!
The guy who implemented this didn't know that QBS is not better than cmake
when it comes to give proper information to IDE which is so needed to have
proper syntax highlighting and code completion. For those who don't know yet,
QBS *DOESN'T* provide the necessary info to the IDE:
- no compiler preprocessor defines: https://bugreports.qt.io/browse/QBS-903
even is hallucinating this bug was closed as invalid :) there is a pending
patch https://codereview.qt-project.org/#/c/122000/ which might fix it but
we'll soon celebrate its 2nd anniversary in gerrit :)
- no system includes paths: https://bugreports.qt.io/browse/QBS-904
- no c/cpp flags: https://bugreports.qt.io/browse/QBS-905

Having said that, why on earth to create such a generator when QBS support in
QtCreator is the same (or even worst) than cmake's one?


DISCLAIMER: I was one of the biggest fans of this project, I had so much hopes
for it, but when you have high hopes you'll also have high disappoints :) .
I'll try to summarize my thought on QBS:
- it still has HUGE potential, it has a great easy to use & learn syntax

-it has great features that you can't find in other build systems (e.g. it can
build multiple ABIs/platforms at once).

- personally I don't mind that it depends on Qt, what I do mind is that it
depends on dead Qt modules (e.g. QtScript, it has it's own (outdated?) QML
parser fork). Other cool build systems (e.g. gradle) download half of the
internet before they start, so, a build system that depends on a library like
Qt is not that bad. As I said it has huge potential and in the future Qt will
help to implement cool features like: automatically download/clone/checkout
3rd partly libs, etc.

- QBS was introduced to us as a build system designed with tooling in mind,
sadly that crucial aspect was forgotten (the above bugs prove what I'm
saying).

- QBS developers don't use it in large projects with lots of dependencies,
with situations when you need to build & run tools to generate code, when you
need to build and/or run tools to check dependencies, when you need to test
compiler flags, etc. (apart from QtCreator which has just a few dependencies).
You might think that they started to use QBS to compile Qt to test all these
things, well, think again, that work was started by a brave contributor
(Andrew Knight) who is not a QBS developer! After the work was started, QBS
developers jumped in.

- is QBS finished and ready to replace cmake/qmake/gradle/etc.? IMHO no! There
are not too many remaining features to implement, but if the development
continues at current speed I'm afraid we'll see people walking on Mars before
we'll see QBS finished... I hope that trying to build Qt with QBS will motivate
QBS developers to implement these features faster.


Cheers,
BogDan.

[Konstantin Tokarev]

07.09.2016, 14:17, "BogDan Vatra" <***@kdab.com>:
> On marți, 6 septembrie 2016 17:35:03 EEST Cristian Adam wrote:
>>  On Tue, Sep 6, 2016 at 5:14 PM, Kevin Kofler <***@chello.at> wrote:
>>  > I guess somebody could even get CMake to write Qbs files, it would just be
>>  > one more generator. :-)
>>
>>  This was done already
>>  <https://gitlab.kitware.com/cmake/cmake/commit/f85db2f32358e6de921aba7d1cb8e
>>  cb81da934c0>, but it was removed from CMake due to bad feedback from
>>  Qt Creator people.
>
> Ha ha ha!
> The guy who implemented this didn't know that QBS is not better than cmake
> when it comes to give proper information to IDE which is so needed to have
> proper syntax highlighting and code completion. For those who don't know yet,
> QBS *DOESN'T* provide the necessary info to the IDE:
> - no compiler preprocessor defines: https://bugreports.qt.io/browse/QBS-903
> even is hallucinating this bug was closed as invalid :) there is a pending
> patch https://codereview.qt-project.org/#/c/122000/ which might fix it but
> we'll soon celebrate its 2nd anniversary in gerrit :)
> - no system includes paths: https://bugreports.qt.io/browse/QBS-904
> - no c/cpp flags: https://bugreports.qt.io/browse/QBS-905
>
> Having said that, why on earth to create such a generator when QBS support in
> QtCreator is the same (or even worst) than cmake's one?
>
> DISCLAIMER: I was one of the biggest fans of this project, I had so much hopes
> for it, but when you have high hopes you'll also have high disappoints :) .
> I'll try to summarize my thought on QBS:
> - it still has HUGE potential, it has a great easy to use & learn syntax
>
> -it has great features that you can't find in other build systems (e.g. it can
> build multiple ABIs/platforms at once).

For the record, premake can do it as well.

>
> - personally I don't mind that it depends on Qt, what I do mind is that it
> depends on dead Qt modules (e.g. QtScript, it has it's own (outdated?) QML
> parser fork). Other cool build systems (e.g. gradle) download half of the
> internet before they start, so, a build system that depends on a library like
> Qt is not that bad. As I said it has huge potential and in the future Qt will
> help to implement cool features like: automatically download/clone/checkout
> 3rd partly libs, etc.
>
> - QBS was introduced to us as a build system designed with tooling in mind,
> sadly that crucial aspect was forgotten (the above bugs prove what I'm
> saying).
>
> - QBS developers don't use it in large projects with lots of dependencies,
> with situations when you need to build & run tools to generate code, when you
> need to build and/or run tools to check dependencies, when you need to test
> compiler flags, etc. (apart from QtCreator which has just a few dependencies).
> You might think that they started to use QBS to compile Qt to test all these
> things, well, think again, that work was started by a brave contributor
> (Andrew Knight) who is not a QBS developer! After the work was started, QBS
> developers jumped in.
>
> - is QBS finished and ready to replace cmake/qmake/gradle/etc.? IMHO no! There
> are not too many remaining features to implement, but if the development
> continues at current speed I'm afraid we'll see people walking on Mars before
> we'll see QBS finished... I hope that trying to build Qt with QBS will motivate
> QBS developers to implement these features faster.
>
> Cheers,
> BogDan.
>
> _______________________________________________
> Development mailing list
> ***@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development

--
Regards,
Konstantin

[Stephen Kelly]

Konstantin Tokarev wrote:

>> -Qbs has great features that you can't find in other build systems (e.g.
>> it can build multiple ABIs/platforms at once).
>
> For the record, premake can do it as well.

Can you show me the syntax for this with premake (and with Qbs)? I assume
you're talking about building a tool on the host, then using it to generate
sources, then cross compiling those sources, and having all of that in one
build. My internet searches do not reveal anything about premakes ability
here.

Thanks,

Steve.

[Konstantin Tokarev]

13.09.2016, 23:00, "Stephen Kelly" <***@gmail.com>:
> Konstantin Tokarev wrote:
>
>>>  -Qbs has great features that you can't find in other build systems (e.g.
>>>  it can build multiple ABIs/platforms at once).
>>
>>  For the record, premake can do it as well.
>
> Can you show me the syntax for this with premake (and with Qbs)? I assume
> you're talking about building a tool on the host, then using it to generate
> sources, then cross compiling those sources, and having all of that in one
> build.

No, I meant building different subprojects for different platforms. As premake generates makefile (or project files) for all configurations at once you can run them sequentially, but there was no direct support for described use case when I looked into it last time

> My internet searches do not reveal anything about premakes ability
> here.
>
> Thanks,
>
> Steve.
>
> _______________________________________________
> Development mailing list
> ***@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development

--
Regards,
Konstantin

[Cristian Adam]

On 06-Sep-16 19:59, André Pönitz wrote:
> On Tue, Sep 06, 2016 at 05:35:03PM +0200, Cristian Adam wrote:
>> On Tue, Sep 6, 2016 at 5:14 PM, Kevin Kofler <***@chello.at> wrote:
>>
>>> I guess somebody could even get CMake to write Qbs files, it would just be
>>> one more generator. :-)
>>>
>>>
>> This was done already
>> <https://gitlab.kitware.com/cmake/cmake/commit/f85db2f32358e6de921aba7d1cb8ecb81da934c0>,
>> but it was removed from CMake due to bad feedback from
>> Qt Creator people.
> Is this feedback recorded somewhere?
>
> Andre'

https://git.backbone.ws/cmake/cmake/commit/deec97d8eca4db67be09031757fd11f66c1a037b


Revert "Qbs: Add new 'extra' generator for qbs project files"

This reverts commit f85db2f32358e6de921aba7d1cb8ecb81da934c0.

Discussion by the QtCreator community at

https://bugreports.qt.io/browse/QTCREATORBUG-13695

raises concerns about this particular approach to working with CMake
projects using QtCreator. Also, the functionality and design of the
QBS extra generator was never discussed on the CMake mailing list or
with QtCreator developers. There may be better ways to make the two
tools work together.

In order to avoid committing to long-term support of this generator
prior to such discussion taking place, revert it from CMake for now.
We may restore this or use an alternative design based on results of
such discussion.

Maybe "bad feedback" is strong, but it was non constructive and lead to
the removal of the Qbs generator.

Cheers,
Cristian.

[Kevin Kofler]

Jake Petroules wrote:
> Again, useless, because Qbs is more powerful and at a much higher level of
> abstraction, so a generator would only be useful in the reverse direction.

Of course, the generator would not use all the features of Qbs, just the
minimum subset that is needed to do the work.

> It's like trying to make a compiler to transform Motorola 68k assembler to
> C++. Only the reverse transformation of that can done in a useful manner.

That is not a reasonable comparison. Assembly code is not designed to be
compiled to anything other than the corresponding machine code. Code can be
self-modifying, there are hardware-dependent I/O ports, etc. The CMake
language is designed exactly to perform configuration and then output build
files for all sorts of build systems: make, Ninja, various IDEs, etc.

But of course, all the configury would be done at CMake time and Qbs would
just be tasked to do the actual build, as a make replacement. Will somebody
be interested enough in implementing that? Probably not. But if make is
really the issue, it could be done.

Kevin Kofler

[Cristian Adam]

On Mon, Sep 5, 2016 at 12:27 PM, Sune Vuorela <***@vuorela.dk> wrote:

> On 2016-09-05, Andrew Knight <***@intopalo.com> wrote:
> > tl;dr: Lots of discussion on the merits of which build system (CMake,
> > Qbs) should replace qmake in building Qt; lots of supporters of CMake
> > but no volunteers to do the work, many reasons to use Qbs as well. Some
>
> I do think that there is volunteers to get Qt building with cmake if
> there is a likly chance of getting it accepted. I think I also
> commmunicated that at the session.
>
>
There were volunteers to add CMake support to CopperSpice
<https://github.com/copperspice/copperspice>, the moc-less C++11
Qt fork. I don't see why there won't be any volunteers to add support for
Qt.
Unless it's not welcomed.

That would be the Boost story all over again, where at some point Kitware
volunteered to add manpower to the conversion, but boost people didn't want
it.

Cheers,
Cristian.

[Andrew Knight]

Hi Steve,


On 09/05/16 15:40, Stephen Kelly wrote:
>> - (Stephen) "In reality, rewriting Qt's build system in CMake will
>> actually be a PITA, and will require changes to CMake to make everything
>> better"
>
> I think something was lost in transit on this point. I don’t think it would be a PITA to write a CMake buildsystem for Qt. I recall the above point was in reference to ‘compiling host tools and using them in the build while cross compiling’. The way CMake makes that possible currently(!) is implemented separately to the core of CMake with the ExternalProject module.
>
>

You're right; I remember now that this was referring what you describe
above, not the porting work in general. I actually found another quote
from you in my notes saying "I'm not sure CMake would require a lot of
effort for either CMake or Qt itself". Clearly, I need to work on my
stenography...

Thanks for the clarification!

--
Andrew

[Kevin Kofler]

Andrew Knight wrote:
> - (Tobias) Cmake is the "worst" system in Qt Creator because CMake
> doesn't give enough feedback to the IDE. We need first-class support for
> CMake in Qt Creator, though, so that is irrelevant to the discussion of
> which system we use to build Qt.

Anything you can get into CMake to improve IDE support will also help
KDevelop, and vice-versa. KDevelop dropped their own CMake parser in
KDevelop 5 and now relies on the information CMake gives (using a dedicated
query mode that was added mainly for KDevelop), they know about things
missing there and want them added to CMake.

> * Quick survey: which build system do you use (raise of hands by ~40
> people)
> - CMake ~70%
> - qmake ~20%
> - Qbs ~10%

That basically says it all. :-)

> - (Milian) CMakeis used by e.g. clang and it works for them

… and the whole stack of software from the KDE project, and other large
projects.

> - (Eddy) Why are we using this ancient tool? (Referring to make)

You don't have to use make with CMake, CMake has other generator backends,
e.g., a Ninja generator. And the really nice thing is, the developer of the
application or library (Qt in this case) normally does not have to do
anything to support them, it should just work (and make will still just
work, too, for those who prefer using make).

> - (Andreas) "Every big project ends up building their own system. Qbs is
> a possibility to create a new build system. If it stays in the Qt
> Project, less people will use it."

The fact that many big projects (thankfully not all of them) reinvent their
own square wheel is a bad thing, not something worth copying. This is a
major annoyance when packaging.

That said, what I have also seen in some projects is strange wrapper layers
above CMake, with things like:
* custom Find* files that work differently from the standard ones,
* custom macros that do everything instead of standard commands,
* undocumented hacked-on support for bundled libraries (with custom macros
and custom Find* files) that makes it hard to unbundle them,
* lack of support for important standard CMake variables, e.g., some broken
CMakeLists setups use neither LIB_INSTALL_DIR nor LIB_SUFFIX.
So using CMake is not always enough to avoid being a pain to package.

But an entirely custom build system is always the worst. I don't see why one
would need to do different just for the sake of doing different, it just
makes us packagers' jobs harder (see e.g. Chromium's gyp and/or gn).

Kevin Kofler

[Konstantin Tokarev]

05.09.2016, 16:38, "Kevin Kofler" <***@chello.at>:
> Andrew Knight wrote:
>>  - (Tobias) Cmake is the "worst" system in Qt Creator because CMake
>>  doesn't give enough feedback to the IDE. We need first-class support for
>>  CMake in Qt Creator, though, so that is irrelevant to the discussion of
>>  which system we use to build Qt.
>
> Anything you can get into CMake to improve IDE support will also help
> KDevelop, and vice-versa. KDevelop dropped their own CMake parser in
> KDevelop 5 and now relies on the information CMake gives (using a dedicated
> query mode that was added mainly for KDevelop), they know about things
> missing there and want them added to CMake.
>
>>  * Quick survey: which build system do you use (raise of hands by ~40
>>  people)
>>  - CMake ~70%
>>  - qmake ~20%
>>  - Qbs ~10%
>
> That basically says it all. :-)
>
>>  - (Milian) CMakeis used by e.g. clang and it works for them
>
> … and the whole stack of software from the KDE project, and other large
> projects.
>
>>  - (Eddy) Why are we using this ancient tool? (Referring to make)
>
> You don't have to use make with CMake, CMake has other generator backends,
> e.g., a Ninja generator. And the really nice thing is, the developer of the
> application or library (Qt in this case) normally does not have to do
> anything to support them, it should just work (and make will still just
> work, too, for those who prefer using make).
>
>>  - (Andreas) "Every big project ends up building their own system. Qbs is
>>  a possibility to create a new build system. If it stays in the Qt
>>  Project, less people will use it."
>
> The fact that many big projects (thankfully not all of them) reinvent their
> own square wheel is a bad thing, not something worth copying. This is a
> major annoyance when packaging.

Problem is that square wheel here is CMake. IMNSHO, it doesn't do _anything_ better than competition and it's only workable because of lots and lots of man-hours of pain and suffering, invested in its extension modules.

>
> That said, what I have also seen in some projects is strange wrapper layers
> above CMake, with things like:
> * custom Find* files that work differently from the standard ones,
> * custom macros that do everything instead of standard commands,
> * undocumented hacked-on support for bundled libraries (with custom macros
>   and custom Find* files) that makes it hard to unbundle them,
> * lack of support for important standard CMake variables, e.g., some broken
>   CMakeLists setups use neither LIB_INSTALL_DIR nor LIB_SUFFIX.
> So using CMake is not always enough to avoid being a pain to package.
>
> But an entirely custom build system is always the worst. I don't see why one
> would need to do different just for the sake of doing different, it just
> makes us packagers' jobs harder (see e.g. Chromium's gyp and/or gn).
>
>         Kevin Kofler
>
> _______________________________________________
> Development mailing list
> ***@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development

--
Regards,
Konstantin

[Ch'Gans]

On 6 September 2016 at 01:52, Konstantin Tokarev <***@yandex.ru> wrote:
> 05.09.2016, 16:38, "Kevin Kofler" <***@chello.at>:
>> Andrew Knight wrote:
>>> * Quick survey: which build system do you use (raise of hands by ~40
>>> people)
>>> - CMake ~70%
>>> - qmake ~20%
>>> - Qbs ~10%
>>
>> That basically says it all. :-)

Yes, that basically says it all: 90% of people were certainly unaware
or unfamiliar with Qbs! ;)

Krys

[Thiago Macieira]

Em terça-feira, 6 de setembro de 2016, às 13:40:40 PDT, Ch'Gans escreveu:
> On 6 September 2016 at 01:52, Konstantin Tokarev <***@yandex.ru> wrote:
> > 05.09.2016, 16:38, "Kevin Kofler" <***@chello.at>:
> >> Andrew Knight wrote:
> >>> * Quick survey: which build system do you use (raise of hands by ~40
> >>> people)
> >>> - CMake ~70%
> >>> - qmake ~20%
> >>> - Qbs ~10%
> >>
> >> That basically says it all. :-)
>
> Yes, that basically says it all: 90% of people were certainly unaware
> or unfamiliar with Qbs! ;)

Which is, in itself, an argument: why learn yet another buildsystem?

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center

[Ch'Gans]

On 6 September 2016 at 16:20, Thiago Macieira <***@intel.com> wrote:
> Em terça-feira, 6 de setembro de 2016, às 13:40:40 PDT, Ch'Gans escreveu:
>> On 6 September 2016 at 01:52, Konstantin Tokarev <***@yandex.ru> wrote:
>> > 05.09.2016, 16:38, "Kevin Kofler" <***@chello.at>:
>> >> Andrew Knight wrote:
>> >>> * Quick survey: which build system do you use (raise of hands by ~40
>> >>> people)
>> >>> - CMake ~70%
>> >>> - qmake ~20%
>> >>> - Qbs ~10%
>> >>
>> >> That basically says it all. :-)
>>
>> Yes, that basically says it all: 90% of people were certainly unaware
>> or unfamiliar with Qbs! ;)
>
> Which is, in itself, an argument: why learn yet another buildsystem?

Good question, maybe because it's more powerful, it fits better your
needs, it is more fun, it uses new concepts, ...
Or just out of curiosity!

Why learn yet another programming language?

If I followed this reasoning, I would still be writing my programs in
Motorola assembler...
Luckily, I've learned other languages like C, C++, Python, Lua, JS,
Qml, etc, ...

An average software developer knows about, says 10 to 20 programming
languages, maybe even more depending on the definition of "language".
Why should he/she knows just one build system?

Chris

>
> --
> Thiago Macieira - thiago.macieira (AT) intel.com
> Software Architect - Intel Open Source Technology Center
>
> _______________________________________________
> Development mailing list
> ***@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development

[Thiago Macieira]

Em terça-feira, 6 de setembro de 2016, às 16:52:47 PDT, Ch'Gans escreveu:
> > Which is, in itself, an argument: why learn yet another buildsystem?
>
> Good question, maybe because it's more powerful, it fits better your
> needs, it is more fun, it uses new concepts, ...
> Or just out of curiosity!
>
> Why learn yet another programming language?

I haven't learnt any new programming languages since PHP in the late 90s. That
specifically excludes major languages like Python, C# and even QML itself. I
have yet to write a single QML file (disclaimer: my last GUI application was
qdbusviewer, in 2006, and it was also my first).

I think I'm not doing that bad...

> If I followed this reasoning, I would still be writing my programs in
> Motorola assembler...
> Luckily, I've learned other languages like C, C++, Python, Lua, JS,
> Qml, etc, ...
>
> An average software developer knows about, says 10 to 20 programming
> languages, maybe even more depending on the definition of "language".
> Why should he/she knows just one build system?

There's a difference between "I could read that thing if I needed to" and "I
can write very good code in this language". I can read Python, C# and QML, Go,
Lua, Tcl, LISP, Rust, maybe even Haskell; but I have no interest in becoming
an expert in any of those.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center

[Kevin Kofler]

Ch'Gans wrote:
> If I followed this reasoning, I would still be writing my programs in
> Motorola assembler...

pea msg(%pc)
jbsr puts
addq.l #4,%a7
rts

msg: .asciz "And this would be wrong, why? ;-)"

or if you prefer the more traditional syntax:

pea msg(PC)
jsr puts
addq.l #4,a7
rts

msg: dc.b "And this would be wrong, why? ;-)", 0

:-)

Kevin Kofler

[Viktor Engelmann]

I don't think the joke is racist or sexist.
The protagonists happen to be female, but the gender has nothing to do
with the punchline (at least I don't think so). I have read the same
joke with blond men and with clerks. You could take anyone - and that is
what makes the joke in-offensive for anyone (IMO).

I find it more excluding and off-putting to be called racist/sexist
/*publicly*/ for posting such a harmless joke. If you have any problem,
you could come and talk to me personally first, before pointing fingers.



Am 06.09.2016 um 16:29 schrieb Ulf Hermann:
>> It kind of reminds me of this joke:
>> http://www.ebaumsworld.com/jokes/blond-hole-diggers/80432143/
>
> Do we have a policy about inappropriate content posted to mailing
> lists and similar communication channels? If not, I think we should
> agree on banning at least some basic things like racism or sexism. We
> should be inclusive after all, and not scare off any potential
> contributors.
>
> regards,
> Ulf

--


Viktor Engelmann
Software Engineer

The Qt Company GmbH
Rudower Chaussee 13
D-12489 Berlin
***@qt.io
[phone]
http://qt.io

Geschäftsführer: Mika Pälsi, Juha Varelius, Mika Harjuaho
Sitz der Gesellschaft: Berlin, Registergericht: Amtsgericht
Charlottenburg, HRB 144331 B
<http://qt.io>
<http://www.facebook.com/Qt> <http://www.twitter.com/qtproject>
<https://www.linkedin.com/company/the-qt-company/>
<https://plus.google.com/104580575722059274792>
<https://www.youtube.com/QtStudios>

Qt World Summit 2016 <http://qtworldsummit.com/>
Qt World Summit 2016 | Pier 27, San Francisco, CA
Experience Exponential Potential on October 18-20
www.qtworldsummit.com <http://www.qtworldsummit.com>

[Ch'Gans]

On 7 September 2016 at 01:47, Viktor Engelmann <***@qt.io>
wrote:

>
> Am 06.09.2016 um 06:52 schrieb Ch'Gans:
>
> On 6 September 2016 at 16:20, Thiago Macieira <***@intel.com> <***@intel.com> wrote:
>
> Which is, in itself, an argument: why learn yet another buildsystem?
>
> ...
>
> Why learn yet another programming language?
>
> ...
>
> An average software developer knows about, says 10 to 20 programming
> languages, maybe even more depending on the definition of "language".
> Why should he/she knows just one build system?
>
> Chris
>
>
> My 2 cents: I want to spend my time on programming = being productive -
> Not waste my time fighting against an incredibly stupid build system...
>
> In my opinion, build systems in general haven't matured the way compilers
> have. Using classes and polymorphism, you can easily explain arbitrarily
> complex situations to compilers. I have yet to see a build system that
> understands more complex situations than "FILE1 IS OLDER THAN FILE2".
>

This is the Makefile approach, the whole Makefile thing is based on "FILE1
IS OLDER THAN FILE2" (and so is CMake). I think that Qbs is way more than
this. Maybe have a quick look at the documentation to realise how the
approach is completely different. Qbs understand the concept of classes and
polymorphism. In my projects, I define a "base class" (a Product in Qbs
terminology), and I can derive this product into more specific product, of
course derived products inherit from their base behaviours and properties,
and of course i can overwrite properties (inc, full overwrite, append,
prepend, transform, ...).

Qbs is really different.

See eg. http://doc.qt.io/qbs/language-introduction.html


They don't even have the slightest understanding of the commands they
> execute - they just slap some raw strings together (which YOU have to
> provide) and pass them to a shell. Oh, these raw strings were for a
> different OS? Or even just a different version of the same compiler on the
> same OS? well too bad...
>
> It kind of reminds me of this joke:
> http://www.ebaumsworld.com/jokes/blond-hole-diggers/80432143/
>

Will have to check this one at home, corporate "firewall" tells me: "404:
Access denied - Adult detected" ...

Chris


>
>
> Viktor
>
> --
>
>
> Viktor Engelmann
> Software Engineer
>
> The Qt Company GmbH
> Rudower Chaussee 13
> D-12489 Berlin
> ***@qt.io
> [phone]
> http://qt.io
>
> GeschÀftsfÌhrer: Mika PÀlsi, Juha Varelius, Mika Harjuaho
> Sitz der Gesellschaft: Berlin, Registergericht: Amtsgericht
> Charlottenburg, HRB 144331 B
> <http://qt.io>
> <http://www.facebook.com/Qt> <http://www.twitter.com/qtproject>
> <https://www.linkedin.com/company/the-qt-company/>
> <https://plus.google.com/104580575722059274792>
> <https://www.youtube.com/QtStudios>
> [image: Qt World Summit 2016] <http://qtworldsummit.com/>
> Qt World Summit 2016 | Pier 27, San Francisco, CA
> Experience Exponential Potential on October 18-20
> www.qtworldsummit.com
>
> _______________________________________________
> Development mailing list
> ***@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development
>
>

[Ch'Gans]

[Keeping discussion posted on ML]

On 7 September 2016 at 20:08, Viktor Engelmann <***@qt.io>
wrote:

> Really? I haven't checked out Qbs yet, but that sounds like the build
> system I was looking for.
>
> So maybe I don't have to write my own build tool after all :-D
>
Give it a try, but beware: Qbs is addictive.

Chris

[Jake Petroules]

On Sep 5, 2016, at 3:38 PM, Kevin Kofler <***@chello.at<mailto:***@chello.at>> wrote:

- (Milian) CMakeis used by e.g. clang and it works for them


 and the whole stack of software from the KDE project, and other large
projects.

Keep in mind that "large projects use X, therefore X is great" is a poor argument, because large does not necessarily mean complex. Any build tool is good at handling a large project (including autotools), but few are good at handling complex ones. Moreover, because Y uses X and it works for Y, does not mean X works for Z.

As an incredibly simple example, make is inherently limited in that it cannot even represent a rule with multiple outputs (there are some workarounds, but they are hacky and rather limited in how they can be applied). And ninja is no magic bullet here either, because that still represents a static build graph, whereas the content of Qbs' build graph can actually change during the execution of the build.

Many of you seem to not understand how complex build tools can get and just how simple Qbs can make problems that are incredibly challenging in other systems. Perhaps you should actually try Qbs before complaining about it. Or perhaps we simply need more/better examples to show the community the difference between the Rolls-Royce Trent 900 jet engine that is CMake, and the wet firecrackers that are CMake and qmake. Perhaps both. :)

No one WANTS to use CMake. They use it because they HAVE to, because it's the only thing that exists. Feel free to long for the "good old days" of the stone age, when food was scarce, disease was rampant, and life was short, but we will move forward towards a better future.
--
Jake Petroules - ***@qt.io<mailto:***@qt.io>
Consulting Services Engineer - The Qt Company
Qbs build tool evangelist - qbs.io<http://qbs.io>

[Jake Petroules]

On Sep 5, 2016, at 4:12 PM, Jake Petroules <***@qt.io<mailto:***@qt.io>> wrote:


On Sep 5, 2016, at 3:38 PM, Kevin Kofler <***@chello.at<mailto:***@chello.at>> wrote:

- (Milian) CMakeis used by e.g. clang and it works for them


 and the whole stack of software from the KDE project, and other large
projects.

Keep in mind that "large projects use X, therefore X is great" is a poor argument, because large does not necessarily mean complex. Any build tool is good at handling a large project (including autotools), but few are good at handling complex ones. Moreover, because Y uses X and it works for Y, does not mean X works for Z.

As an incredibly simple example, make is inherently limited in that it cannot even represent a rule with multiple outputs (there are some workarounds, but they are hacky and rather limited in how they can be applied). And ninja is no magic bullet here either, because that still represents a static build graph, whereas the content of Qbs' build graph can actually change during the execution of the build.

Many of you seem to not understand how complex build tools can get and just how simple Qbs can make problems that are incredibly challenging in other systems. Perhaps you should actually try Qbs before complaining about it. Or perhaps we simply need more/better examples to show the community the difference between the Rolls-Royce Trent 900 jet engine that is Qbs, and the wet firecrackers that are CMake and qmake. Perhaps both. :)

No one WANTS to use CMake. They use it because they HAVE to, because it's the only thing that exists. Feel free to long for the "good old days" of the stone age, when food was scarce, disease was rampant, and life was short, but we will move forward towards a better future.
--
Jake Petroules - ***@qt.io<mailto:***@qt.io>
Consulting Services Engineer - The Qt Company
Qbs build tool evangelist - qbs.io<http://qbs.io/>


s/CMake/Qbs/

Fail.
--
Jake Petroules - ***@qt.io<mailto:***@qt.io>
Consulting Services Engineer - The Qt Company
Qbs build tool evangelist - qbs.io<http://qbs.io>

[Milian Wolff]

On Montag, 5. September 2016 14:12:23 CEST Jake Petroules wrote:
> On Sep 5, 2016, at 3:38 PM, Kevin Kofler
> <***@chello.at<mailto:***@chello.at>> wrote:

> - (Milian) CMakeis used by e.g. clang and it works for them
>
> 
 and the whole stack of software from the KDE project, and other large
> projects.
>
> Keep in mind that "large projects use X, therefore X is great" is a poor
> argument, because large does not necessarily mean complex. Any build tool
> is good at handling a large project (including autotools), but few are good
> at handling complex ones. Moreover, because Y uses X and it works for Y,
> does not mean X works for Z.

Currently we only know that QBS can be used for Creator, which is, quite
frankly, far from the complexity of LLVM, all of the KDE stuff, as well as
WebKit and the tons of other big projects that use CMake...

> As an incredibly simple example, make is inherently limited in that it
> cannot even represent a rule with multiple outputs (there are some
> workarounds, but they are hacky and rather limited in how they can be
> applied). And ninja is no magic bullet here either, because that still
> represents a static build graph, whereas the content of Qbs' build graph
> can actually change during the execution of the build.

Can you give an example for why we should care? This may sound flame-baity,
but I'm really truly interested. I simply don't care about my build system, as
long as it gets the job done without too much hassle (and yes, that is the
case for me personally with cmake), and fast, too (which is the case with
ninja). See also below.

> Many of you seem to not understand how complex build tools can get and just
> how simple Qbs can make problems that are incredibly challenging in other
> systems. Perhaps you should actually try Qbs before complaining about it.
> Or perhaps we simply need more/better examples to show the community the
> difference between the Rolls-Royce Trent 900 jet engine that is CMake, and
> the wet firecrackers that are CMake and qmake. Perhaps both. :)

Sure, that can be the case. But do you really think that everyone will rewrite
their working CMake buildsystem in QBS just for the sake of it? What
advantages does it have that would make such an effort worth it?

Also, do note that many people outside the Qt company simply don't get why
such an effort is put into reinventing a build system, when we are in short
supply of developers in other areas, which are arguably more important to keep
Qt relevant. Kai mentioned competing against other languages, I wonder why one
needs to compete against them. If Qt had proper language bindings, it would be
used far more often...

> No one WANTS to use CMake. They use it because they HAVE to, because it's
> the only thing that exists..

I agree. But with the many years I've worked with CMake, I don't get it either
why so many people become so passionate at hating it. Thanks to the new
features in CMake 3 most notably, it's pretty easy to use.

> Feel free to long for the "good old days" of
> the stone age, when food was scarce, disease was rampant, and life was
> short, but we will move forward towards a better future

That is completely besides the point.

Bye
--
Milian Wolff | ***@kdab.com | Software Engineer
KDAB (Deutschland) GmbH&Co KG, a KDAB Group company
Tel: [phone]
KDAB - The Qt Experts

[Bo Thorsen]

Den 05-09-2016 kl. 20:49 skrev Milian Wolff:
>> As an incredibly simple example, make is inherently limited in that it
>> > cannot even represent a rule with multiple outputs (there are some
>> > workarounds, but they are hacky and rather limited in how they can be
>> > applied). And ninja is no magic bullet here either, because that still
>> > represents a static build graph, whereas the content of Qbs' build graph
>> > can actually change during the execution of the build.
> Can you give an example for why we should care? This may sound flame-baity,
> but I'm really truly interested. I simply don't care about my build system, as
> long as it gets the job done without too much hassle (and yes, that is the
> case for me personally with cmake), and fast, too (which is the case with
> ninja). See also below.

I can answer that because I asked for this feature all the way back at
QtCS in Bilbao.

The context I was talking about was code generators. At the time I had
built a code generator that created both the Qt and the PHP side of a
client-server system. It had a single JSON file that described a server
with the available remote methods on it. The output from the C++ code
generator was a .h and .cpp file per method and a single .h and .cpp
that described the server. So on qmake run time you can't know how many
output files you have unless you force the user to run qmake every time
you modify the JSON description.

And to make the problem worse, the customer wanted each of the classes
describing a remote call to be qobjects with a Q_OBJECT so a moc run is
required.

AFAIK this is impossible to do with both qmake and cmake. Not just hard,
actually impossible. And the reason is that you can't know until build
time what files you have to run moc on. (Now as a mental exersize,
imagine having multiple server descriptions in a single json file...)

It's possible to do in pure make by calling the code generator and
creating a sub make file and then calling make on that. But you can't do
that in any way that supports hitting the build button once in QtCreator.

I finally solved this by convincing the customer it was sufficient to
have a qobject base class for the remote calls and live without the meta
object descriptions on those. And yes, this is a very specific case. But
it's an example of something that neither cmake nor qmake can do because
they have a makefile generating step.

Maybe this seems more important to me than others because I'm a huge fan
of custom built code generators for stuff like database connections and
client-server communications.

Bo Thorsen,
Director, Viking Software.

--
Viking Software
Qt and C++ developers for hire
http://www.vikingsoft.eu

[Milian Wolff]

On Donnerstag, 8. September 2016 13:41:21 CEST Bo Thorsen wrote:
> Den 05-09-2016 kl. 20:49 skrev Milian Wolff:
> >> As an incredibly simple example, make is inherently limited in that it
> >>
> >> > cannot even represent a rule with multiple outputs (there are some
> >> > workarounds, but they are hacky and rather limited in how they can be
> >> > applied). And ninja is no magic bullet here either, because that still
> >> > represents a static build graph, whereas the content of Qbs' build
> >> > graph
> >> > can actually change during the execution of the build.
> >
> > Can you give an example for why we should care? This may sound
> > flame-baity,
> > but I'm really truly interested. I simply don't care about my build
> > system, as long as it gets the job done without too much hassle (and yes,
> > that is the case for me personally with cmake), and fast, too (which is
> > the case with ninja). See also below.
>
> I can answer that because I asked for this feature all the way back at
> QtCS in Bilbao.
>
> The context I was talking about was code generators. At the time I had
> built a code generator that created both the Qt and the PHP side of a
> client-server system. It had a single JSON file that described a server
> with the available remote methods on it. The output from the C++ code
> generator was a .h and .cpp file per method and a single .h and .cpp
> that described the server. So on qmake run time you can't know how many
> output files you have unless you force the user to run qmake every time
> you modify the JSON description.
>
> And to make the problem worse, the customer wanted each of the classes
> describing a remote call to be qobjects with a Q_OBJECT so a moc run is
> required.
>
> AFAIK this is impossible to do with both qmake and cmake. Not just hard,
> actually impossible.

At least for CMake, I don't think so - if I'm not misunderstanding the
problem. You add a target that depends on your .json file, and then whenever
that is changed you generate headers. Then you add a dependent target which
runs moc. And then you let your actual target depend on that one?

> And the reason is that you can't know until build
> time what files you have to run moc on. (Now as a mental exersize,
> imagine having multiple server descriptions in a single json file...)

That is not an issue, no? You have one file that you need your targets depend
on - the JSON file. If that one changes, the dependent targets must be
rebuild.

> It's possible to do in pure make by calling the code generator and
> creating a sub make file and then calling make on that. But you can't do
> that in any way that supports hitting the build button once in QtCreator.
>
> I finally solved this by convincing the customer it was sufficient to
> have a qobject base class for the remote calls and live without the meta
> object descriptions on those. And yes, this is a very specific case. But
> it's an example of something that neither cmake nor qmake can do because
> they have a makefile generating step.
>
> Maybe this seems more important to me than others because I'm a huge fan
> of custom built code generators for stuff like database connections and
> client-server communications.

I'm not yet convinced.

Bye

--
Milian Wolff | ***@kdab.com | Software Engineer
KDAB (Deutschland) GmbH&Co KG, a KDAB Group company
Tel: [phone]
KDAB - The Qt Experts

[Bo Thorsen]

Den 08-09-2016 kl. 13:47 skrev Milian Wolff:
> On Donnerstag, 8. September 2016 13:41:21 CEST Bo Thorsen wrote:
>> Den 05-09-2016 kl. 20:49 skrev Milian Wolff:
>>>> As an incredibly simple example, make is inherently limited in that it
>>>>
>>>>> cannot even represent a rule with multiple outputs (there are some
>>>>> workarounds, but they are hacky and rather limited in how they can be
>>>>> applied). And ninja is no magic bullet here either, because that still
>>>>> represents a static build graph, whereas the content of Qbs' build
>>>>> graph
>>>>> can actually change during the execution of the build.
>>>
>>> Can you give an example for why we should care? This may sound
>>> flame-baity,
>>> but I'm really truly interested. I simply don't care about my build
>>> system, as long as it gets the job done without too much hassle (and yes,
>>> that is the case for me personally with cmake), and fast, too (which is
>>> the case with ninja). See also below.
>>
>> I can answer that because I asked for this feature all the way back at
>> QtCS in Bilbao.
>>
>> The context I was talking about was code generators. At the time I had
>> built a code generator that created both the Qt and the PHP side of a
>> client-server system. It had a single JSON file that described a server
>> with the available remote methods on it. The output from the C++ code
>> generator was a .h and .cpp file per method and a single .h and .cpp
>> that described the server. So on qmake run time you can't know how many
>> output files you have unless you force the user to run qmake every time
>> you modify the JSON description.
>>
>> And to make the problem worse, the customer wanted each of the classes
>> describing a remote call to be qobjects with a Q_OBJECT so a moc run is
>> required.
>>
>> AFAIK this is impossible to do with both qmake and cmake. Not just hard,
>> actually impossible.
>
> At least for CMake, I don't think so - if I'm not misunderstanding the
> problem. You add a target that depends on your .json file, and then whenever
> that is changed you generate headers. Then you add a dependent target which
> runs moc. And then you let your actual target depend on that one?
>
>> And the reason is that you can't know until build
>> time what files you have to run moc on. (Now as a mental exersize,
>> imagine having multiple server descriptions in a single json file...)
>
> That is not an issue, no? You have one file that you need your targets depend
> on - the JSON file. If that one changes, the dependent targets must be
> rebuild.
>
>> It's possible to do in pure make by calling the code generator and
>> creating a sub make file and then calling make on that. But you can't do
>> that in any way that supports hitting the build button once in QtCreator.
>>
>> I finally solved this by convincing the customer it was sufficient to
>> have a qobject base class for the remote calls and live without the meta
>> object descriptions on those. And yes, this is a very specific case. But
>> it's an example of something that neither cmake nor qmake can do because
>> they have a makefile generating step.
>>
>> Maybe this seems more important to me than others because I'm a huge fan
>> of custom built code generators for stuff like database connections and
>> client-server communications.
>
> I'm not yet convinced.

Ok, go try it. Create a simple python or perl script that reads a file.
The file just has a single number N inside it. And based on N the script
outputs those files:

server.h
method1.h
method2.h
...
methodN.h

Inside method1.h you write this:

#include <QObject>

class Method1 : public QObject {
Q_OBJECT
};

server.h has this:

#include "method1.h"
...
#include "methodN.h"

class Server {
public:
Method1* call1() { return new Method1; }
...
MethodN* callN() { return new MethodN; }
};

In main.cpp you instantiate Server.

The only problem is that you have to run moc on each of the .h files.

Solution to the problem is only accepted if you can press build one
single time inside both Visual Studio and Qt Creator and it builds this
even when you modify the input file and the main.cpp.

I'm sure there are ways you can make the build call cmake or force a
build to fail and rebuild, and that's what you can currently do. And
those all feels annoying when you work on the project.

You asked why modifying the tree at build time could be necessary. This
is one of the (probably very few) examples of it.

Bo Thorsen,
Director, Viking Software.

--
Viking Software
Qt and C++ developers for hire
http://www.vikingsoft.eu

[Konstantin Tokarev]

08.09.2016, 15:03, "Bo Thorsen" <***@vikingsoft.eu>:
> Den 08-09-2016 kl. 13:47 skrev Milian Wolff:
>>  On Donnerstag, 8. September 2016 13:41:21 CEST Bo Thorsen wrote:
>>>  Den 05-09-2016 kl. 20:49 skrev Milian Wolff:
>>>>>  As an incredibly simple example, make is inherently limited in that it
>>>>>
>>>>>>  cannot even represent a rule with multiple outputs (there are some
>>>>>>  workarounds, but they are hacky and rather limited in how they can be
>>>>>>  applied). And ninja is no magic bullet here either, because that still
>>>>>>  represents a static build graph, whereas the content of Qbs' build
>>>>>>  graph
>>>>>>  can actually change during the execution of the build.
>>>>
>>>>  Can you give an example for why we should care? This may sound
>>>>  flame-baity,
>>>>  but I'm really truly interested. I simply don't care about my build
>>>>  system, as long as it gets the job done without too much hassle (and yes,
>>>>  that is the case for me personally with cmake), and fast, too (which is
>>>>  the case with ninja). See also below.
>>>
>>>  I can answer that because I asked for this feature all the way back at
>>>  QtCS in Bilbao.
>>>
>>>  The context I was talking about was code generators. At the time I had
>>>  built a code generator that created both the Qt and the PHP side of a
>>>  client-server system. It had a single JSON file that described a server
>>>  with the available remote methods on it. The output from the C++ code
>>>  generator was a .h and .cpp file per method and a single .h and .cpp
>>>  that described the server. So on qmake run time you can't know how many
>>>  output files you have unless you force the user to run qmake every time
>>>  you modify the JSON description.
>>>
>>>  And to make the problem worse, the customer wanted each of the classes
>>>  describing a remote call to be qobjects with a Q_OBJECT so a moc run is
>>>  required.
>>>
>>>  AFAIK this is impossible to do with both qmake and cmake. Not just hard,
>>>  actually impossible.
>>
>>  At least for CMake, I don't think so - if I'm not misunderstanding the
>>  problem. You add a target that depends on your .json file, and then whenever
>>  that is changed you generate headers. Then you add a dependent target which
>>  runs moc. And then you let your actual target depend on that one?
>>
>>>  And the reason is that you can't know until build
>>>  time what files you have to run moc on. (Now as a mental exersize,
>>>  imagine having multiple server descriptions in a single json file...)
>>
>>  That is not an issue, no? You have one file that you need your targets depend
>>  on - the JSON file. If that one changes, the dependent targets must be
>>  rebuild.
>>
>>>  It's possible to do in pure make by calling the code generator and
>>>  creating a sub make file and then calling make on that. But you can't do
>>>  that in any way that supports hitting the build button once in QtCreator.
>>>
>>>  I finally solved this by convincing the customer it was sufficient to
>>>  have a qobject base class for the remote calls and live without the meta
>>>  object descriptions on those. And yes, this is a very specific case. But
>>>  it's an example of something that neither cmake nor qmake can do because
>>>  they have a makefile generating step.
>>>
>>>  Maybe this seems more important to me than others because I'm a huge fan
>>>  of custom built code generators for stuff like database connections and
>>>  client-server communications.
>>
>>  I'm not yet convinced.
>
> Ok, go try it. Create a simple python or perl script that reads a file.
> The file just has a single number N inside it. And based on N the script
> outputs those files:
>
> server.h
> method1.h
> method2.h
> ...
> methodN.h
>
> Inside method1.h you write this:
>
> #include <QObject>
>
> class Method1 : public QObject {
>    Q_OBJECT
> };
>
> server.h has this:
>
> #include "method1.h"
> ...
> #include "methodN.h"
>
> class Server {
> public:
>    Method1* call1() { return new Method1; }
>    ...
>    MethodN* callN() { return new MethodN; }
> };
>
> In main.cpp you instantiate Server.
>
> The only problem is that you have to run moc on each of the .h files.

Run moc from inside script when you generate header.

>
> Solution to the problem is only accepted if you can press build one
> single time inside both Visual Studio and Qt Creator and it builds this
> even when you modify the input file and the main.cpp.
>
> I'm sure there are ways you can make the build call cmake or force a
> build to fail and rebuild, and that's what you can currently do. And
> those all feels annoying when you work on the project.
>
> You asked why modifying the tree at build time could be necessary. This
> is one of the (probably very few) examples of it.
>
> Bo Thorsen,
> Director, Viking Software.
>
> --
> Viking Software
> Qt and C++ developers for hire
> http://www.vikingsoft.eu
> _______________________________________________
> Development mailing list
> ***@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development

--
Regards,
Konstantin

[Bo Thorsen]

Den 08-09-2016 kl. 14:19 skrev Konstantin Tokarev:
>> > The only problem is that you have to run moc on each of the .h files.
> Run moc from inside script when you generate header.

Yes, I thought about that at the time as well. While simple enpough,
there are some complications. You would have to run moc exactly like if
it was done by the qmake built makefiles, with exactly the same
environment and arguments. Not impossible, but it does sound brittle.
For example different qmake versions might do things differently.

Bo Thorsen,
Director, Viking Software.

--
Viking Software
Qt and C++ developers for hire
http://www.vikingsoft.eu

[Jake Petroules]

Another thing that's very hard to do in other build systems is building Java code. The class files emitted by a Java compiler actually vary depending on the contents of the Java files themselves.

Imagine you've built a JAR file, and then you add a new anonymous inner class within one of your Java source files. The command line invocation to build the JAR file needs to be updated to contain the new class file that will result. Impossible with qmake/CMake/Makefiles/etc.

Whereas Qbs has sophisticated support exactly for this case: https://github.com/qt-labs/qbs/tree/master/share/qbs/modules/java, made possible by its dynamic build graph.

On Sep 8, 2016, at 6:52 AM, Bo Thorsen <***@vikingsoft.eu<mailto:***@vikingsoft.eu>> wrote:

Den 08-09-2016 kl. 14:19 skrev Konstantin Tokarev:
> The only problem is that you have to run moc on each of the .h files.
Run moc from inside script when you generate header.

Yes, I thought about that at the time as well. While simple enpough, there are some complications. You would have to run moc exactly like if it was done by the qmake built makefiles, with exactly the same environment and arguments. Not impossible, but it does sound brittle. For example different qmake versions might do things differently.

Bo Thorsen,
Director, Viking Software.

--
Viking Software
Qt and C++ developers for hire
http://www.vikingsoft.eu
_______________________________________________
Development mailing list
***@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

--
Jake Petroules - ***@qt.io<mailto:***@qt.io>
Consulting Services Engineer - The Qt Company
Qbs build tool evangelist - qbs.io<http://qbs.io>

[Kevin Kofler]

Jake Petroules wrote:
> Another thing that's very hard to do in other build systems is building
> Java code. The class files emitted by a Java compiler actually vary
> depending on the contents of the Java files themselves.
>
> Imagine you've built a JAR file, and then you add a new anonymous inner
> class within one of your Java source files. The command line invocation to
> build the JAR file needs to be updated to contain the new class file that
> will result. Impossible with qmake/CMake/Makefiles/etc.

Well, what you can do if you have a lot of Java stuff to build is to just
shell out to Ant. That's how Qt Jambi does it, for example. Use the right
tool for the job.

CMake has some support for building Java, but indeed, it does not
automatically figure out the outputs of each .java file for you. Dedicated
Java build systems are much better at that.

Kevin Kofler

[Jake Petroules]

On Sep 9, 2016, at 4:39 AM, Kevin Kofler <***@chello.at<mailto:***@chello.at>> wrote:

Jake Petroules wrote:
Another thing that's very hard to do in other build systems is building
Java code. The class files emitted by a Java compiler actually vary
depending on the contents of the Java files themselves.

Imagine you've built a JAR file, and then you add a new anonymous inner
class within one of your Java source files. The command line invocation to
build the JAR file needs to be updated to contain the new class file that
will result. Impossible with qmake/CMake/Makefiles/etc.

Well, what you can do if you have a lot of Java stuff to build is to just
shell out to Ant. That's how Qt Jambi does it, for example. Use the right
tool for the job.

CMake has some support for building Java, but indeed, it does not
automatically figure out the outputs of each .java file for you. Dedicated
Java build systems are much better at that.

Last I checked, "dedicated" Java build systems (like Ant) don't even handle that case. Maybe I'm wrong but as far as I'm aware, the Java support in Qbs with regard to dependency tracking is superior to anything else out there except perhaps Buck. And no one would call that a "dedicated Java build system".

Kevin Kofler

_______________________________________________
Development mailing list
***@qt-project.org<mailto:***@qt-project.org>
http://lists.qt-project.org/mailman/listinfo/development

--
Jake Petroules - ***@qt.io<mailto:***@qt.io>
Consulting Services Engineer - The Qt Company
Qbs build tool evangelist - qbs.io<http://qbs.io>

[Jake Petroules]

I just found a perfect example of how hard building a JAR file is in qmake for example, compared to qbs:

[BogDan Vatra]

On joi, 8 septembrie 2016 20:22:25 EEST Jake Petroules wrote:
> I just found a perfect example of how hard building a JAR file is in qmake
> for example, compared to qbs:

IMHO will be even easier if instead:
qbs-javac-scan.qbs:
[...]
files: [
"io/qt/qbs/**/*.java"
]
[...]

You'll write:
[...]
dirs: [
"io/qt/qbs/"
]
[...]

As any java developer is used to write in any java build system ;-)

Cheers,
BogDan.

[Christian Kandeler]

On 09/08/2016 02:03 PM, Bo Thorsen wrote:
> Ok, go try it. Create a simple python or perl script that reads a file.
> The file just has a single number N inside it. And based on N the script
> outputs those files:
>
> server.h
> method1.h
> method2.h
> ...
> methodN.h
>
> Inside method1.h you write this:
>
> #include <QObject>
>
> class Method1 : public QObject {
> Q_OBJECT
> };
>
> server.h has this:
>
> #include "method1.h"
> ...
> #include "methodN.h"
>
> class Server {
> public:
> Method1* call1() { return new Method1; }
> ...
> MethodN* callN() { return new MethodN; }
> };
>
> In main.cpp you instantiate Server.
>
> The only problem is that you have to run moc on each of the .h files.
>
> Solution to the problem is only accepted if you can press build one
> single time inside both Visual Studio and Qt Creator and it builds this
> even when you modify the input file and the main.cpp.

In qbs:

Rule {
inputs: ["metadata"]
fileTags: ["hpp"]
outputArtifacts: {
var p = new Process();
try {
p.exec("path_to_script", ["--list", input.filePath]);
var files = p.readStdout.split("\n");
var artifacts = [];
for (var i in files)
artifacts.push({ filePath: files[i],
fileTags: ["hpp"]});
return artifacts:
} finally {
p.close();
}
prepare: {
var cmd = new Command("path_to_script",
["--generate", input.filePath]);
cmd.description = "creating headers";
return [cmd];
}
}

(This is a somewhat more advanced example in that it is not assumed that
we have a priori knowledge about how the content of the input file
relates to the outputs.)


FYI,
Christian

[Edward Welbourne]

For reference, I've turned Andrew's notes (see ur-ancestor post of this thread) into:
https://wiki.qt.io/Qt_build_systems_at_QtCon_2016

Eddy.

[Christian Kandeler]

[Sorry about the formatting, using outlook]

Stephen Kelly wrote:
> Here's the CMake version:

[ ... ]

> execute_process(
> COMMAND python ${CMAKE_CURRENT_SOURCE_DIR}/generator.py --list
> ${CMAKE_CURRENT_BINARY_DIR}/genoutput 5
> OUTPUT_VARIABLE fileList
> )

How do you know where the input file is located?

> However, it is cheating in the same way that the QBS version from
> Christian is cheating - it assumes '--list' exists:

Yes, I was assuming a cooperating tool.

> Christian, can you create a version which does not require --list?

There are two possibilities:
a) The inner workings of the tool are known and "simple enough". That's the case in Bo's example, so there we could just open the input file and derive the output artifacts from the number we find there.
b) Otherwise, our outputArtifacts script has to run the tool in "real mode" (using the "--generate" option). The actual command would be a no-op then. This is icky, both conceptually and for practical reasons, because commands of non-competing rules are run in parallel, whereas the outputArtifacts scripts are not. I think so far we only use this approach for the infamous qdoc tool.


Christian

[Jake Petroules]

On Sep 13, 2016, at 12:10 PM, Christian Kandeler <***@qt.io<mailto:***@qt.io>> wrote:

[Sorry about the formatting, using outlook]

Stephen Kelly wrote:
> Here's the CMake version:

[ ... ]

> execute_process(
> COMMAND python ${CMAKE_CURRENT_SOURCE_DIR}/generator.py --list
> ${CMAKE_CURRENT_BINARY_DIR}/genoutput 5
> OUTPUT_VARIABLE fileList
> )

How do you know where the input file is located?

> However, it is cheating in the same way that the QBS version from
> Christian is cheating - it assumes '--list' exists:

Yes, I was assuming a cooperating tool.

> Christian, can you create a version which does not require --list?

There are two possibilities:
a) The inner workings of the tool are known and "simple enough". That's the case in Bo's example, so there we could just open the input file and derive the output artifacts from the number we find there.
b) Otherwise, our outputArtifacts script has to run the tool in "real mode" (using the "--generate" option). The actual command would be a no-op then. This is icky, both conceptually and for practical reasons, because commands of non-competing rules are run in parallel, whereas the outputArtifacts scripts are not. I think so far we only use this approach for the infamous qdoc tool.

And asset catalogs, XIBs/NIBs/storyboards, Java sources, and TypeScript sources.



Christian
_______________________________________________
Development mailing list
***@qt-project.org<mailto:***@qt-project.org>
http://lists.qt-project.org/mailman/listinfo/development

--
Jake Petroules - ***@qt.io<mailto:***@qt.io>
Consulting Services Engineer - The Qt Company
Qbs build tool evangelist - qbs.io<http://qbs.io>

[Stephen Kelly]

Christian Kandeler wrote:

> [Sorry about the formatting, using outlook]
>
> Stephen Kelly wrote:
>> Here's the CMake version:
>
> [ ... ]
>
>> execute_process(
>> COMMAND python ${CMAKE_CURRENT_SOURCE_DIR}/generator.py --list
>> ${CMAKE_CURRENT_BINARY_DIR}/genoutput 5
>> OUTPUT_VARIABLE fileList
>> )
>
> How do you know where the input file is located?

There is no input file. There is only an input number. The task is from Bo,
who gave it as a simplified example.

>> However, it is cheating in the same way that the QBS version from
>> Christian is cheating - it assumes '--list' exists:
>
> Yes, I was assuming a cooperating tool.

In his real-world example, Bo said that there was a tool which generated
output files, but the output files are not a-priori known:

https://www.mail-archive.com/***@qt-project.org/msg25970.html

Of course, if you have a 'cooperating tool' you can do anything with CMake
too. My understanding is that because Qbs has a dynamic build graph, it can
do things that CMake can't do.

I'm looking for a good example of that so that I can understand. It is the
gap that Qbs is to fill. Jake mentioned examples to show the difference
between the Rolls-Royce Trent 900 jet engine that is Qbs, and the wet
firecrackers that are CMake and qmake. I thought this would be a good one,
but it turns out that in this case, Qbs and CMake have the same requirement
of the tool.

>> Christian, can you create a version which does not require --list?
>
> There are two possibilities:
> a) The inner workings of the tool are known and "simple enough". That's
> the case in Bo's example, so there we could just open the input file and
> derive the output artifacts from the number we find there.

Yes, if we can find out "now" what the tool will generate "later", then
everything is easy.

If we require that of the tool, then we can have a static build graph, so I
don't see how the dynamic build graph is an advantage in that case.

In the case Bo mentioned, he built the tool, so he can ensure it is
cooperative. It seems his mistake and the source of his problem was to use
qmake for the job instead of CMake or Qbs :).

> b) Otherwise,
> our outputArtifacts script has to run the tool in "real mode" (using the
> "--generate" option).

And how do you know what the outputArtifacts are? Do you require the tool to
tell you what files it generated during this step? (ie, again be
cooperative)

> The actual command would be a no-op then. This is
> icky, both conceptually and for practical reasons, because commands of
> non-competing rules are run in parallel, whereas the outputArtifacts
> scripts are not.

Right, assuming the tool is still 'cooperative' and tells you what files it
generated, you would end up with the same thing with a CMake build - you
would generate the files at CMake time, and then ninja would build them in
parallel.

Again, that's no conceptual advantage of the dynamic build graph over the
static one in the case of a somewhat cooperative tool. That's probably why
Milian never saw a need for such a thing :).

So, I'm still trying to find out when the dynamic build graph is actually an
advantage. I have an idea in mind, but it gets quite niche and still has
some assumptions. Maybe you can tell me if I am right:

1) You have an 'uncooperative' tool which can not tell you what it will
generate, and which does not output a list of what it generated. However, it
accepts a command line argument specifying where it should generate its
files.

2) So, you set it up to generate the files in some directory. You assume
that the directory is empty before the tool is run, and you use the
equivalent of running the 'ls' command to determine what the files are.
Rules to run moc and to run the compiler are then added to your dynamic
build graph, which then proceeds and eventually gets around to executing
those rules.



If that is something Qbs is designed to do, then can you post a Qbs file
which does it? You can use the generator.py I posted before. The script
already accepts a directory to generate the files into - just pretend --list
doesn't exist.

Thanks,

Steve.

[Stephen Kelly]

Stephen Kelly wrote:

> Christian Kandeler wrote:
>
>> [Sorry about the formatting, using outlook]
>>
>> Stephen Kelly wrote:
>>> Here's the CMake version:
>>
>> [ ... ]
>>
>>> execute_process(
>>> COMMAND python ${CMAKE_CURRENT_SOURCE_DIR}/generator.py --list
>>> ${CMAKE_CURRENT_BINARY_DIR}/genoutput 5
>>> OUTPUT_VARIABLE fileList
>>> )
>>
>> How do you know where the input file is located?
>
> There is no input file. There is only an input number. The task is from
> Bo, who gave it as a simplified example.

Oops, I'm wrong here. Bo said to read the number from a file.

I don't think that changes anything though regarding dynamic build graph
being an advantage.

Thanks,

Steve.

[Christian Kandeler]

Stephen Kelly wrote:

>> There is no input file. There is only an input number. The task is from
>> Bo, who gave it as a simplified example.
> Oops, I'm wrong here. Bo said to read the number from a file.
> I don't think that changes anything though regarding dynamic build graph
> being an advantage.

Sure? What about the (lack of) need for two rules to agree in advance about the location of a generated file? Also, there could be several layers of indirection, with the second set of generated files also containing meta data etc.


Christian

[Oswald Buddenhagen]

On Wed, Sep 14, 2016 at 12:05:15PM +0200, Stephen Kelly via Development wrote:
> I want to understand Qbs and what it can do with a dynamic build graph
> which CMake can't do.
>
there is no such thing, as after full expansion the graph has to be
static by definition (the output artifacts are expected to be
deterministic, after all). the difference is in when you determine the
final graph. as cmake is a meta build tool, it artificially creates two
phases. as it creates inputs for static build tools, it has to execute
all the dynamic parts itself, which makes the "preparation" phase
actually a partial "execution" phase, and thus slower than it is
supposed to be.

having said that, qbs also has a "meta" mode, to make ide project
generation possible. also, as shown in the bootstrapping discussion,
we'll probably need a simple makefile generator as well.

and as christian already pointed out, in qbs there is also the
fundamental split between preparation and execution phases, only that
they could be scheduled much better for individual artifacts (but as
also pointed out, this isn't taken advantage of yet).

so what it all comes down to is that qbs is, as in pretty much every
other regard, simply more elegant than cmake, not a magic solution for
problems yet to be found.

[Christian Kandeler]

Stephen Kelly wrote:

> My previous guess about Qbs being able to generate unknown files in a
> particular location and then determine them by an 'ls' equivalent, moc
> them and compile everything is not something Qbs would be able to do.

I'm having trouble parsing this, but if you mean that your previous guess was wrong, then I can tell you it was not; that's exactly what we do for "blackbox tools".

> I'm still interested in a Qbs solution to the code/repo I posted before.
> A full and preferably working Qbs solution, instead of a snippet, would
> be good for comparison.

That's more than I'm willing to invest during my vacation, but I might come back to you later there.


Christian

[Matthew Woehlke]

On 2016-09-15 02:57, Oswald Buddenhagen wrote:
> On Wed, Sep 14, 2016 at 12:05:15PM +0200, Stephen Kelly via Development wrote:
>> I want to understand Qbs and what it can do with a dynamic build graph
>> which CMake can't do.
>
> there is no such thing, as after full expansion the graph has to be
> static by definition (the output artifacts are expected to be
> deterministic, after all).

I don't think that's actually true; it just has to *halt*. That is, you
can execute as many steps as you like that generate new build edges, as
long as *at some point* you end up with a static graph.

CMake necessarily imposes that you can run exactly one iteration, but
I'm not aware of any theoretical reason you couldn't have an entire
chain of targets each of which don't know their outputs (which are the
inputs of the next in the chain) until you go to actually build them.

For that matter, you can do that sort of thing with make, by having each
target depend on the previous one, and generate a new Makefile that is
used to build the next one.

--
Matthew

[Konstantin Tokarev]

20.09.2016, 22:21, "Matthew Woehlke" <***@gmail.com>:
> On 2016-09-15 02:57, Oswald Buddenhagen wrote:
>>  On Wed, Sep 14, 2016 at 12:05:15PM +0200, Stephen Kelly via Development wrote:
>>>  I want to understand Qbs and what it can do with a dynamic build graph
>>>  which CMake can't do.
>>
>>  there is no such thing, as after full expansion the graph has to be
>>  static by definition (the output artifacts are expected to be
>>  deterministic, after all).
>
> I don't think that's actually true; it just has to *halt*. That is, you
> can execute as many steps as you like that generate new build edges, as
> long as *at some point* you end up with a static graph.
>
> CMake necessarily imposes that you can run exactly one iteration, but
> I'm not aware of any theoretical reason you couldn't have an entire
> chain of targets each of which don't know their outputs (which are the
> inputs of the next in the chain) until you go to actually build them.
>
> For that matter, you can do that sort of thing with make, by having each
> target depend on the previous one, and generate a new Makefile that is
> used to build the next one.

More popular pattern: include generated makefile (e.g., depfile produced by GCC)
into "master" makefile

--
Regards,
Konstantin

[Milian Wolff]

On Donnerstag, 8. September 2016 14:03:05 CEST Bo Thorsen wrote:
> Den 08-09-2016 kl. 13:47 skrev Milian Wolff:
> > On Donnerstag, 8. September 2016 13:41:21 CEST Bo Thorsen wrote:
> >> Den 05-09-2016 kl. 20:49 skrev Milian Wolff:
> >>>> As an incredibly simple example, make is inherently limited in that it
> >>>>
> >>>>> cannot even represent a rule with multiple outputs (there are some
> >>>>> workarounds, but they are hacky and rather limited in how they can be
> >>>>> applied). And ninja is no magic bullet here either, because that still
> >>>>> represents a static build graph, whereas the content of Qbs' build
> >>>>> graph
> >>>>> can actually change during the execution of the build.
> >>>
> >>> Can you give an example for why we should care? This may sound
> >>> flame-baity,
> >>> but I'm really truly interested. I simply don't care about my build
> >>> system, as long as it gets the job done without too much hassle (and
> >>> yes,
> >>> that is the case for me personally with cmake), and fast, too (which is
> >>> the case with ninja). See also below.
> >>
> >> I can answer that because I asked for this feature all the way back at
> >> QtCS in Bilbao.
> >>
> >> The context I was talking about was code generators. At the time I had
> >> built a code generator that created both the Qt and the PHP side of a
> >> client-server system. It had a single JSON file that described a server
> >> with the available remote methods on it. The output from the C++ code
> >> generator was a .h and .cpp file per method and a single .h and .cpp
> >> that described the server. So on qmake run time you can't know how many
> >> output files you have unless you force the user to run qmake every time
> >> you modify the JSON description.
> >>
> >> And to make the problem worse, the customer wanted each of the classes
> >> describing a remote call to be qobjects with a Q_OBJECT so a moc run is
> >> required.
> >>
> >> AFAIK this is impossible to do with both qmake and cmake. Not just hard,
> >> actually impossible.
> >
> > At least for CMake, I don't think so - if I'm not misunderstanding the
> > problem. You add a target that depends on your .json file, and then
> > whenever that is changed you generate headers. Then you add a dependent
> > target which runs moc. And then you let your actual target depend on that
> > one?
> >
> >> And the reason is that you can't know until build
> >> time what files you have to run moc on. (Now as a mental exersize,
> >> imagine having multiple server descriptions in a single json file...)
> >
> > That is not an issue, no? You have one file that you need your targets
> > depend on - the JSON file. If that one changes, the dependent targets
> > must be rebuild.
> >
> >> It's possible to do in pure make by calling the code generator and
> >> creating a sub make file and then calling make on that. But you can't do
> >> that in any way that supports hitting the build button once in QtCreator.
> >>
> >> I finally solved this by convincing the customer it was sufficient to
> >> have a qobject base class for the remote calls and live without the meta
> >> object descriptions on those. And yes, this is a very specific case. But
> >> it's an example of something that neither cmake nor qmake can do because
> >> they have a makefile generating step.
> >>
> >> Maybe this seems more important to me than others because I'm a huge fan
> >> of custom built code generators for stuff like database connections and
> >> client-server communications.
> >
> > I'm not yet convinced.
>
> Ok, go try it. Create a simple python or perl script that reads a file.
> The file just has a single number N inside it. And based on N the script
> outputs those files:
>
> server.h
> method1.h
> method2.h
> ...
> methodN.h
>
> Inside method1.h you write this:
>
> #include <QObject>
>
> class Method1 : public QObject {
> Q_OBJECT
> };
>
> server.h has this:
>
> #include "method1.h"
> ...
> #include "methodN.h"
>
> class Server {
> public:
> Method1* call1() { return new Method1; }
> ...
> MethodN* callN() { return new MethodN; }
> };
>
> In main.cpp you instantiate Server.
>
> The only problem is that you have to run moc on each of the .h files.
>
> Solution to the problem is only accepted if you can press build one
> single time inside both Visual Studio and Qt Creator and it builds this
> even when you modify the input file and the main.cpp.
>
> I'm sure there are ways you can make the build call cmake or force a
> build to fail and rebuild, and that's what you can currently do. And
> those all feels annoying when you work on the project.
>
> You asked why modifying the tree at build time could be necessary. This
> is one of the (probably very few) examples of it.

Someone else also told me that this is apparently harder then I thought it is
with CMake, when the name of the output files of a code generator is not
known. It is possible, but far from easy esp. when you don't have control over
the generator script (though one can always use a wrapper). I see now that
having support for this in QBS can be advantageous for the cases where one has
such cases. Thankfully, I never ran into this so far, ever.

Cheers

--
Milian Wolff | ***@kdab.com | Software Engineer
KDAB (Deutschland) GmbH&Co KG, a KDAB Group company
Tel: [phone]
KDAB - The Qt Experts

[Kevin Kofler]

Bo Thorsen wrote:
> I'm sure there are ways you can make the build call cmake or force a
> build to fail and rebuild, and that's what you can currently do. And
> those all feels annoying when you work on the project.

Well, that's what CMake does by itself whenever it sees that it's needed (or
at least thinks it is), e.g., if you upgraded some dependency. E.g., I've
seen the CMake-produced makefiles auto-rerun CMake on my projects after
upgrading Qt. Thankfully, CMake is so fast (due to being native C++ code)
that it is barely noticeable.

Kevin Kofler

[Matthew Woehlke]

On 2016-09-08 07:41, Bo Thorsen wrote:
> Den 05-09-2016 kl. 20:49 skrev Milian Wolff:
>>> As an incredibly simple example, make is inherently limited in that it
>>> cannot even represent a rule with multiple outputs (there are some
>>> workarounds, but they are hacky and rather limited in how they can be
>>> applied). And ninja is no magic bullet here either, because that still
>>> represents a static build graph, whereas the content of Qbs' build
>>> graph can actually change during the execution of the build.
>>
>> Can you give an example for why we should care? This may sound
>> flame-baity, but I'm really truly interested. I simply don't care
>> about my build system, as long as it gets the job done without too
>> much hassle (and yes, that is the case for me personally with
>> cmake), and fast, too (which is the case with ninja).
>
> I can answer that because I asked for this feature all the way back at
> QtCS in Bilbao.
>
> The context I was talking about was code generators. At the time I had
> built a code generator that created both the Qt and the PHP side of a
> client-server system. It had a single JSON file that described a server
> with the available remote methods on it. The output from the C++ code
> generator was a .h and .cpp file per method and a single .h and .cpp
> that described the server. So on qmake run time you can't know how many
> output files you have unless you force the user to run qmake every time
> you modify the JSON description.
>
> And to make the problem worse, the customer wanted each of the classes
> describing a remote call to be qobjects with a Q_OBJECT so a moc run is
> required.
>
> AFAIK this is impossible to do with both qmake and cmake. Not just hard,
> actually impossible.

It's not *impossible*... it "just" requires that you be able to
determine the outputs at configure time, e.g. by having the tool run in
a mode that does nothing but report what files will be produced.

Granted, this is arduous, and especially terrible if the complexity of
determining the output files is on the order of producing them in the
first place, but saying it's "actually impossible" is a bit of an
exaggeration.

(This can't exactly work with VS because the *way* it works is to force
CMake to re-run when the file(s) that determine the output files change.
That works with e.g. make/ninja, but not so well with VS, but that's a
VS failing that I don't see how Qbs could overcome, given that VS *is*
the build tool and doesn't AFAIK support dynamic build graphs. Blaming
CMake for VS's shortcomings isn't really fair.)

The project I'm working on currently uses LCM, which has this problem
(we also use CMake, and have a CMake script that parses the input files
in order to determine the names of the output files). So does PySide /
Shiboken, and I'm sure there are other examples out there.

--
Matthew

[Konstantin Tokarev]

20.09.2016, 22:11, "Matthew Woehlke" <***@gmail.com>:
> That works with e.g. make/ninja, but not so well with VS, but that's a
> VS failing that I don't see how Qbs could overcome, given that VS *is*
> the build tool and doesn't AFAIK support dynamic build graphs.

QBS does not use VS as a build tool, it is not a project generator

--
Regards,
Konstantin

[Jake Petroules]

> On Sep 20, 2016, at 12:18 PM, Konstantin Tokarev <***@yandex.ru> wrote:
>
>
>
> 20.09.2016, 22:11, "Matthew Woehlke" <***@gmail.com>:
>> That works with e.g. make/ninja, but not so well with VS, but that's a
>> VS failing that I don't see how Qbs could overcome, given that VS *is*
>> the build tool and doesn't AFAIK support dynamic build graphs.
>
> QBS does not use VS as a build tool, it is not a project generator

Although it can act as one; I recently added support for generating VS projects: https://codereview.qt-project.org/#/c/91353/

Qbs still performs the build entirely on its own though, the VS output is no more than a file listing.

>
> --
> Regards,
> Konstantin
> _______________________________________________
> Development mailing list
> ***@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development

--
Jake Petroules - ***@qt.io
The Qt Company - Silicon Valley
Qbs build tool evangelist - qbs.io

[Matthew Woehlke]

On 2016-09-20 15:18, Konstantin Tokarev wrote:
> 20.09.2016, 22:11, "Matthew Woehlke" <***@gmail.com>:
>> That works with e.g. make/ninja, but not so well with VS, but that's a
>> VS failing that I don't see how Qbs could overcome, given that VS *is*
>> the build tool and doesn't AFAIK support dynamic build graphs.
>
> QBS does not use VS as a build tool, it is not a project generator

...but that's sort of my point; calling CMake deficient because it can't
do something that Qbs can't do either is disingenuous.

Saying "oh, but Qbs just tells VS to invoke Qbs" isn't really better;
there is no reason (besides "no one has implemented it") why CMake
couldn't do the same thing with e.g. ninja as the build driver. (And
either case assumes you don't include the generated files in the VS
project, or both will still have VS complaining about reloading the
project...)

--
Matthew

[Иван Комиссаров]

I was using CMake for several years (and still forced to use it now),
however i moved all my projects to QBS for one reason - CMake is too
complex. It has great documentation, but it can't be used without it; on
the contrary, qbs is very intuitive. I don't have to remember tons of
variables like CMAKE_CURRENT_SOURCE_DIR, CMAKE_SOURCE_DIR (there is also
PROJECT_SOURCE_DIR!) with qbs, property names are simple. Every complex
task lead to tons of code with CMake, it's hard to maintain those scripts.
I don't want to do that complex job maintaining buildsystem, i want to
write code.
Yes, if a project will use both CMake and QBS build systems, all the
variables and logic will be almost the same; however qbs is much clearer
and easier.
I tried to create a simple project with typical usecases - libraries,
tests, app bundles that uses both CMake and QBS, but i didn't have time to
finish the cmake part https://github.com/ABBAPOH/qbsfish (not sure if it
even works).
For now, cmake scripts looks much easier, but adding new features is rather
painful - i have to use the docs all the time.

[Thiago Macieira]

Em segunda-feira, 5 de setembro de 2016, às 14:12:23 PDT, Jake Petroules
escreveu:
> Many of you seem to not understand how complex build tools can get and just
> how simple Qbs can make problems that are incredibly challenging in other
> systems. Perhaps you should actually try Qbs before complaining about it.
> Or perhaps we simply need more/better examples to show the community the
> difference between the Rolls-Royce Trent 900 jet engine that is CMake, and
> the wet firecrackers that are CMake and qmake. Perhaps both. :)

I'm interested in seeing that.

Please port one of the complex KF5 libraries to qbs. One that has dozens of
configuration decisions and dependencies, both mandatory and optional, few of
which can be detected with pkg-config.

Then we can compare a real-world case of CMake vs qbs.

Qt Creator is not a good example, because it has exactly one external
dependency (the LLVM libs) and it barely tries to detect it. You have to
inform the build with qmake that LLVM is available. Botan is included as a
bundled 3rdparty, instead of being detected, etc.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center

[Thiago Macieira]

Em segunda-feira, 5 de setembro de 2016, às 12:49:03 PDT, Andrew Knight
escreveu:
> ** General sentiment:
> - As long as Qbs looks like a part of Qt, it is perceived as a Qt
> product, and is less attractive to external users.
> - Yet, there remains a conflict: "if Qt doesn't use it, I don't want to
> use it" vs. "if it's not outside of Qt, I don't want to use it"

Sounds like the way to go for qbs is to decouple it, make it a separate
project, one that doesn't release in lockstep with Qt.

That puts an extra burden in Qbs development: it has to be ahead of Qt's own
development by at least two releases. Building a library should not require a
change in the buildsystem tool: the tool should already support it by the time
we get to that problem. That's unlike qmake, for which we make changes as we
need them in Qt.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center

[Andrew Knight]

On 09/06/16 03:08, Thiago Macieira wrote:
> Em segunda-feira, 5 de setembro de 2016, às 12:49:03 PDT, Andrew Knight
> escreveu:
>> ** General sentiment:
>> - As long as Qbs looks like a part of Qt, it is perceived as a Qt
>> product, and is less attractive to external users.
>> - Yet, there remains a conflict: "if Qt doesn't use it, I don't want to
>> use it" vs. "if it's not outside of Qt, I don't want to use it"
> Sounds like the way to go for qbs is to decouple it, make it a separate
> project, one that doesn't release in lockstep with Qt.

That's already the case. Apart from having a Qt dependency, and being
somewhat influenced by Qt Creator development, it does not release in
lockstep with either.

>
> That puts an extra burden in Qbs development: it has to be ahead of Qt's own
> development by at least two releases. Building a library should not require a
> change in the buildsystem tool: the tool should already support it by the time
> we get to that problem. That's unlike qmake, for which we make changes as we
> need them in Qt.
>

That might become a burden if Qbs development starts using new Qt
features, but that hasn't really been an issue so far. AFAIK Qbs still
builds against Qt 5.2.

Over the weekend we had a number of discussions about bootstrapping Qbs
(or even removing the Qt dependency altogether). So, I think this
requirement is already used in practice and hasn't greatly burdened the
project. The Qbs developers are interested in it being small, fast, and
portable more than relying on any recent innovations in Qt.

--
Andrew

[Thiago Macieira]

Em segunda-feira, 5 de setembro de 2016, às 12:40:54 PDT, Stephen Kelly via
Development escreveu:
> I think something was lost in transit on this point. I don’t think it would
> be a PITA to write a CMake buildsystem for Qt. I recall the above point was
> in reference to ‘compiling host tools and using them in the build while
> cross compiling’. The way CMake makes that possible currently(!) is
> implemented separately to the core of CMake with the ExternalProject
> module.

That's how it should be. Every single project out there, except for Qt and
possibly GCC itself, builds for one single target. Building something for one
architecture so that it can be run to build another is ungainly and
unexpected.

Whenever you cross-compile Qt, you end up with tools that can only be run on
the host. So Qt's cross-compilation mechanism can't be used to build tools
that can be run on the target platform.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center

[Thiago Macieira]

Em terça-feira, 6 de setembro de 2016, às 11:46:12 PDT, Stephen Kelly via
Development escreveu:
> > Whenever you cross-compile Qt, you end up with tools that can only be run
> > on the host. So Qt's cross-compilation mechanism can't be used to build
> > tools that can be run on the target platform.
>
> Yes, I have had problems with that in the past too. However, there's
> nothing preventing building the tools for both the host and the target.
> I think that would be cleaner (This is independent of buildsystem tool -
> I would also be happy if the current qmake build did this).

The biggest offender of this is qmake itself. So if we move away from qmake to
something else, I'd expect that one could just use any host pre-built set of
tools (moc, uic, rcc, etc.) to build your target Qt. Any mismatch of
qobjectsdefs.h and moc is already an #error.

Building the host tools while cross-compiling is a convenience and I think we
can keep it, but I don't think we should simply have different "bin" dirs.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center

[Thiago Macieira]

Em terça-feira, 6 de setembro de 2016, às 06:20:34 PDT, Thiago Macieira
escreveu:
> Building the host tools while cross-compiling is a convenience and I think
> we can keep it, but I don't think we should simply have different "bin"
> dirs.

Fail in sentence rewrite. I wanted to write "don't think we should mix" and
then rewrote to "think we should simply have different 'bin' dirs" -- except I
forgot to remove the "don't".

I don't have a cross-compiled build of Qt handy, but I think we put both host
and target builds in the same "bin" dir during the build. It should be easy to
split and then always build target tools too.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center

[Thiago Macieira]

Em terça-feira, 6 de setembro de 2016, às 14:00:55 PDT, Laszlo Agocs escreveu:
> Using one "bin" is the default behavior, yes, but one can pass -hostprefix
> to separate them upon install.
>
>
> http://doc-snapshots.qt.io/qt5-5.8/embedded-linux.html#configuring-a-specifi
> c-device

That's for installation. I was thinking during the build too, so that moc is
built twice when the "convenience host tools" are built.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center

[Thiago Macieira]

Em terça-feira, 6 de setembro de 2016, às 16:29:38 PDT, Ulf Hermann escreveu:
> Do we have a policy about inappropriate content posted to mailing lists and
> similar communication channels?

No, but there was a side-discussion at QCS that we should write and adopt a
Code of Conduct.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center

[Jake Petroules]

On Sep 6, 2016, at 4:34 PM, Thiago Macieira <***@intel.com<mailto:***@intel.com>> wrote:

Em terça-feira, 6 de setembro de 2016, às 16:29:38 PDT, Ulf Hermann escreveu:
Do we have a policy about inappropriate content posted to mailing lists and
similar communication channels?

No, but there was a side-discussion at QCS that we should write and adopt a
Code of Conduct.

Let's start a new thread for that, then, if someone is interested (keep this one on topic of build systems).

--
Thiago Macieira - thiago.macieira (AT) intel.com<http://intel.com>
Software Architect - Intel Open Source Technology Center

_______________________________________________
Development mailing list
***@qt-project.org<mailto:***@qt-project.org>
http://lists.qt-project.org/mailman/listinfo/development

--
Jake Petroules - ***@qt.io<mailto:***@qt.io>
Consulting Services Engineer - The Qt Company
Qbs build tool evangelist - qbs.io<http://qbs.io>