Discussion:
pygnulib: progress and some questions (func_emit_autoconf_snippets)
Dmitry Selyutin
2017-12-27 21:51:35 UTC
Permalink
Hello,

this is a kind reminder that python branch is still alive, though it's quite difficult to find a time
for coding. :-) Anyway, I think I'll finish the "import" section in about a month: right now I'm
slowly processing gnulib-comp.m4 generation part of the code. I have a question regarding a
function named func_emit_autoconf_snippets.

TL;DR: feel free to skip to ===QUESTIONS=== section if this is a way too long introduction to
the topic. :-) The questions mainly target Bruno as the original author, but I'd be very grateful
for any help.


Could you please explain in details how func_emit_autoconf_snippets works? The most
important for me is to understand how to map it into the API I've recently implemented:
currently pygnulib can create a special object, "module database", which allows to iterate over
modules and dependencies very quickly. Basically it is a hash table where each entry contains
yet another hash table.


Given a dependency DEP, it is really easy to iterate over each demander DEM and condition
COND like this:

for (dem, cond) in db.demanders(dep):
print(dem, cond)

One can also iterate over all dependencies of the demander DEM, acquiring each dependency
DEP and the condition COND:

for (dep, cond) in db.dependencies(dem):
print(dep, cond)

So given a set of modules MODULES, I can hopefully obtain all the required information to
implement func_emit_autoconf_snippets analogue. However, I can't quite understand how the
modules are processed. The questions are followed; they all are relevant only for --
conditional-dependencies switch since otherwise the algorithm is trivial (just treat each
module as unconditional).


===QUESTIONS===
1. At line 4260, what's the place where $modules variable gets assigned? The nearest explicit
assignment I've found is at line 5298; is it correct that this is the one whose result is used at
4260?
2. What's the logic for dependencies processing, starting at line 4298?
3. Do you have an idea how could the algorithm mapped to the described API, given that I
have module database, main modules, tests modules and final modules (and hopefully every
other bit of information)? There's no need to check whether module exists, since all the
mentioned collections (database and modules lists) contain not strings, but a real module
instances.


Thank you for your help!
--
With best regards,
Dmitry Selyutin
Bruno Haible
2017-12-28 01:35:04 UTC
Permalink
Hi Dmitry,
Post by Dmitry Selyutin
1. At line 4260, what's the place where $modules variable gets assigned?
Indeed the comments for function func_emit_autoconf_snippets are misleading:
This functions receives two module lists:
$modules (implicit),
$1 (explicit: first argument).

Where does $modules come from?
- For the invocation in func_import, line 5640 and 5659
it is set at line 5298. But I'm not sure this was intended this way, maybe
another list should be used instead of "$main_modules"?
- For the invocation in func_create_testdir, line 6288 and 6290 and 6402 and 6404 and 6417
it does not appear to be set in a consistent and correct way.
(It is set differently if $single_configure and ! $single_configure.)
This is probably a bug.
The set of modules that _ought_ to be used here is the set of modules
being used for the current configure file. That's my current understanding.
But I'd really need to look at a couple of test cases in order to
firmly answer the question.
Post by Dmitry Selyutin
2. What's the logic for dependencies processing, starting at line 4298?
First the code intersects $deps with $modules. This matters for two cases:
- when --avoid is used: we don't want to invoke the initialization code for
a module that is not present because the user has required it to be avoided.
- when ! $single_configure and we are generating the configure file for the tests
directory: we may have a dependency from a module in $tests_modules to a module
in $main_modules, but since the $main_modules are in a different directory
they should be all treated like --avoid'ed modules here.

Then the test
if func_cond_module_p "$dep"; then
is based on the consideration that the unconditional modules have their
initialization code already emitted (unconditionally) by lines 4261..4271.

Then comes the conditional invocation of the initialization code
echo " if $condition; then"
echo " $shellfunc"
echo ' fi'
with a trivial optimization for the (frequent) case that $condition = "true".
Post by Dmitry Selyutin
3. Do you have an idea how could the algorithm mapped to the described API
You need db.dependencies($module) here. db.demanders(...) is not used here.
db.demanders(...) will be useful, in order to implement a new command-line
option --extract-reverse-dependencies.

Bruno
Bruno Haible
2017-12-28 23:32:23 UTC
Permalink
Hi Dmitry,
Post by Bruno Haible
Post by Dmitry Selyutin
1. At line 4260, what's the place where $modules variable gets assigned?
$modules (implicit),
$1 (explicit: first argument).
Where does $modules come from?
- For the invocation in func_import, line 5640 and 5659
it is set at line 5298. But I'm not sure this was intended this way, maybe
another list should be used instead of "$main_modules"?
- For the invocation in func_create_testdir, line 6288 and 6290 and 6402 and 6404 and 6417
it does not appear to be set in a consistent and correct way.
(It is set differently if $single_configure and ! $single_configure.)
This is probably a bug.
The set of modules that _ought_ to be used here is the set of modules
being used for the current configure file. That's my current understanding.
But I'd really need to look at a couple of test cases in order to
firmly answer the question.
I've understood it now: the code in func_create_testdir was not really a bug,
because --conditional-dependencies was not allowed in this case anyway.

Here's the patch that implements --conditional-dependencies also with
--with-tests.

Bruno
Dmitry Selyutin
2017-12-29 07:13:28 UTC
Permalink
Hi Bruno,

thank you for the detailed answer. I think I still have to do some
debugging to understand why the Python version of the algorithm for
autoconf snippets generation works incorrectly, but I hope I understood how
it should work.
Post by Bruno Haible
Hi Dmitry,
Post by Bruno Haible
Post by Dmitry Selyutin
1. At line 4260, what's the place where $modules variable gets
assigned?
Post by Bruno Haible
Indeed the comments for function func_emit_autoconf_snippets are
$modules (implicit),
$1 (explicit: first argument).
Where does $modules come from?
- For the invocation in func_import, line 5640 and 5659
it is set at line 5298. But I'm not sure this was intended this way,
maybe
Post by Bruno Haible
another list should be used instead of "$main_modules"?
- For the invocation in func_create_testdir, line 6288 and 6290 and
6402 and 6404 and 6417
Post by Bruno Haible
it does not appear to be set in a consistent and correct way.
(It is set differently if $single_configure and ! $single_configure.)
This is probably a bug.
The set of modules that _ought_ to be used here is the set of modules
being used for the current configure file. That's my current
understanding.
Post by Bruno Haible
But I'd really need to look at a couple of test cases in order to
firmly answer the question.
I've understood it now: the code in func_create_testdir was not really a bug,
because --conditional-dependencies was not allowed in this case anyway.
Here's the patch that implements --conditional-dependencies also with
--with-tests.
Bruno
Loading...