Discussion:
Multiple versions of a non-exported dependency
cowwoc
2016-08-31 17:56:25 UTC
Permalink
Hi,

I recently became aware of the fact that the Jigsaw specification declared
"version-selection" as a non-goal. While I understand how we ended up here,
I am hoping that you were able to support the following (very common)
use-case:

* Module "HelloWorld" depends on modules "Guava" and "JSoup".
* Module "Guava" depends on module slf4j version 1 (requires but does not
export it).
* Module "JSoup" depends on module slf4j version 2 (requires but does not
export it).
* slf4j version 2 and is not backwards-compatible with version 1.

What happens at runtime? Will Jigsaw (out of the box, without 3rd-party
tools like Maven or OSGI) be smart enough to provide different versions of
slf4j to "Guava" and "JSoup"?

I hope that you did not drop version numbers altogether when dropping
version-selection. Jigsaw should be able to map fixed version numbers
(string literals) to different JAR files without having to understand the
underlying versioning rules behind each string literal. Maven/OSGI could use
their proprietary version-selection algorithms to select a specific version,
generate a module-info.java containing that fixed version, and pass that to
the JDK at build time. Would that not work?

If "requires" no longer takes a version number, I fear that you will never
be able to add versioning support in the future...

Thanks,
Gili



--
View this message in context: http://jigsaw-dev.1059479.n5.nabble.com/Multiple-versions-of-a-non-exported-dependency-tp5713364.html
Sent from the jigsaw-dev mailing list archive at Nabble.com.
Alex Buckley
2016-08-31 18:29:40 UTC
Permalink
Post by cowwoc
I recently became aware of the fact that the Jigsaw specification declared
"version-selection" as a non-goal. While I understand how we ended up here,
I am hoping that you were able to support the following (very common)
* Module "HelloWorld" depends on modules "Guava" and "JSoup".
* Module "Guava" depends on module slf4j version 1 (requires but does not
export it).
* Module "JSoup" depends on module slf4j version 2 (requires but does not
export it).
* slf4j version 2 and is not backwards-compatible with version 1.
What happens at runtime? Will Jigsaw (out of the box, without 3rd-party
tools like Maven or OSGI) be smart enough to provide different versions of
slf4j to "Guava" and "JSoup"?
(You mean Guava/JSoup requires slf4j version 1/2 and does not
"re-export" it a.k.a. 'requires public'.)

This use case isn't possible on JDK 8 for JARs on the classpath, and
it's not supported on JDK 9 for modular JARs on the modulepath:

- If you have two versions of a modular JAR slf4j.jar in different
directories on the modulepath, then the first one to be found will
dominate, and that's what will be resolved for both Guava and JSoup.

- If you have two modular JARs slf4j_v1.jar and slf4j_v2.jar on the
modulepath, and Guava requires slf4j_v1 and JSoup requires slf4j_v2,
then launching 'java -m HelloWorld' will fail. The boot layer will
refuse to map the "same" packages from different slf4j_v* modules to the
application class loader.

The use case _is_ supported on JDK 9 for modular JARs loaded into custom
loaders of custom layers. That is, the Java Platform Module System is
perfectly capable of supporting the use case -- please see any of my
"Jigsaw: Under The Hood" presentations. The use case just isn't
supported "out of the box" by the 'java' launcher for JARs on the
modulepath.

Alex
Neil Bartlett
2016-08-31 18:54:44 UTC
Permalink
Gili,

As Alex points out: your use-case can be supported in Java 9 but only with the addition of custom ClassLoaders, or by using an existing ClassLoader-based module system such as OSGi.

The same is also true of Java 8, and Java 7, etc.

Regards,
Neil
Post by cowwoc
I recently became aware of the fact that the Jigsaw specification declared
"version-selection" as a non-goal. While I understand how we ended up here,
I am hoping that you were able to support the following (very common)
* Module "HelloWorld" depends on modules "Guava" and "JSoup".
* Module "Guava" depends on module slf4j version 1 (requires but does not
export it).
* Module "JSoup" depends on module slf4j version 2 (requires but does not
export it).
* slf4j version 2 and is not backwards-compatible with version 1.
What happens at runtime? Will Jigsaw (out of the box, without 3rd-party
tools like Maven or OSGI) be smart enough to provide different versions of
slf4j to "Guava" and "JSoup"?
(You mean Guava/JSoup requires slf4j version 1/2 and does not "re-export" it a.k.a. 'requires public'.)
- If you have two versions of a modular JAR slf4j.jar in different directories on the modulepath, then the first one to be found will dominate, and that's what will be resolved for both Guava and JSoup.
- If you have two modular JARs slf4j_v1.jar and slf4j_v2.jar on the modulepath, and Guava requires slf4j_v1 and JSoup requires slf4j_v2, then launching 'java -m HelloWorld' will fail. The boot layer will refuse to map the "same" packages from different slf4j_v* modules to the application class loader.
The use case _is_ supported on JDK 9 for modular JARs loaded into custom loaders of custom layers. That is, the Java Platform Module System is perfectly capable of supporting the use case -- please see any of my "Jigsaw: Under The Hood" presentations. The use case just isn't supported "out of the box" by the 'java' launcher for JARs on the modulepath.
Alex
cowwoc
2016-08-31 19:21:49 UTC
Permalink
Well, this is unfortunate. As I stated earlier, I fail to see how
depending on constant version numbers (not version ranges) fall under
the scope of "version selection". Was this case considered/discussed in
depth?

Not everyone is sold on version ranges (e.g. the vast majority of Maven
artifacts I've seen depend on constant versions) and I think this would
go a long way towards solving the original "classpath hell" problem.

Gili
Post by Neil Bartlett
Gili,
As Alex points out: your use-case can be supported in Java 9 but only
with the addition of custom ClassLoaders, or by using an existing
ClassLoader-based module system such as OSGi.
The same is also true of Java 8, and Java 7, etc.
Regards,
Neil
On 31 Aug 2016, at 19:29, Alex Buckley <[hidden email]
Post by cowwoc
I recently became aware of the fact that the Jigsaw specification
declared
Post by cowwoc
"version-selection" as a non-goal. While I understand how we ended
up here,
Post by cowwoc
I am hoping that you were able to support the following (very common)
* Module "HelloWorld" depends on modules "Guava" and "JSoup".
* Module "Guava" depends on module slf4j version 1 (requires but
does not
Post by cowwoc
export it).
* Module "JSoup" depends on module slf4j version 2 (requires but
does not
Post by cowwoc
export it).
* slf4j version 2 and is not backwards-compatible with version 1.
What happens at runtime? Will Jigsaw (out of the box, without
3rd-party
Post by cowwoc
tools like Maven or OSGI) be smart enough to provide different
versions of
Post by cowwoc
slf4j to "Guava" and "JSoup"?
(You mean Guava/JSoup requires slf4j version 1/2 and does not
"re-export" it a.k.a. 'requires public'.)
This use case isn't possible on JDK 8 for JARs on the classpath, and
- If you have two versions of a modular JAR slf4j.jar in different
directories on the modulepath, then the first one to be found will
dominate, and that's what will be resolved for both Guava and JSoup.
- If you have two modular JARs slf4j_v1.jar and slf4j_v2.jar on the
modulepath, and Guava requires slf4j_v1 and JSoup requires slf4j_v2,
then launching 'java -m HelloWorld' will fail. The boot layer will
refuse to map the "same" packages from different slf4j_v* modules to
the application class loader.
The use case _is_ supported on JDK 9 for modular JARs loaded into
custom loaders of custom layers. That is, the Java Platform Module
System is perfectly capable of supporting the use case -- please see
any of my "Jigsaw: Under The Hood" presentations. The use case just
isn't supported "out of the box" by the 'java' launcher for JARs on
the modulepath.
Alex
------------------------------------------------------------------------
If you reply to this email, your message will be added to the
http://jigsaw-dev.1059479.n5.nabble.com/Multiple-versions-of-a-non-exported-dependency-tp5713364p5713366.html
To unsubscribe from Multiple versions of a non-exported dependency,
click here
<http://jigsaw-dev.1059479.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=5713364&code=Y293d29jQGJicy5kYXJrdGVjaC5vcmd8NTcxMzM2NHwxNTc0MzIxMjQ3>.
NAML
<http://jigsaw-dev.1059479.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
--
View this message in context: http://jigsaw-dev.1059479.n5.nabble.com/Multiple-versions-of-a-non-exported-dependency-tp5713364p5713367.html
Sent from the jigsaw-dev mailing list archive at Nabble.com.
Paul Benedict
2016-08-31 19:25:55 UTC
Permalink
Neil, but doesn't the prohibition of duplicate packages continue to rear
its head?

Cheers,
Paul
Post by cowwoc
Well, this is unfortunate. As I stated earlier, I fail to see how
depending on constant version numbers (not version ranges) fall under
the scope of "version selection". Was this case considered/discussed in
depth?
Not everyone is sold on version ranges (e.g. the vast majority of Maven
artifacts I've seen depend on constant versions) and I think this would
go a long way towards solving the original "classpath hell" problem.
Gili
Post by Neil Bartlett
Gili,
As Alex points out: your use-case can be supported in Java 9 but only
with the addition of custom ClassLoaders, or by using an existing
ClassLoader-based module system such as OSGi.
The same is also true of Java 8, and Java 7, etc.
Regards,
Neil
On 31 Aug 2016, at 19:29, Alex Buckley <[hidden email]
Post by cowwoc
I recently became aware of the fact that the Jigsaw specification
declared
Post by cowwoc
"version-selection" as a non-goal. While I understand how we ended
up here,
Post by cowwoc
I am hoping that you were able to support the following (very common)
* Module "HelloWorld" depends on modules "Guava" and "JSoup".
* Module "Guava" depends on module slf4j version 1 (requires but
does not
Post by cowwoc
export it).
* Module "JSoup" depends on module slf4j version 2 (requires but
does not
Post by cowwoc
export it).
* slf4j version 2 and is not backwards-compatible with version 1.
What happens at runtime? Will Jigsaw (out of the box, without
3rd-party
Post by cowwoc
tools like Maven or OSGI) be smart enough to provide different
versions of
Post by cowwoc
slf4j to "Guava" and "JSoup"?
(You mean Guava/JSoup requires slf4j version 1/2 and does not
"re-export" it a.k.a. 'requires public'.)
This use case isn't possible on JDK 8 for JARs on the classpath, and
- If you have two versions of a modular JAR slf4j.jar in different
directories on the modulepath, then the first one to be found will
dominate, and that's what will be resolved for both Guava and JSoup.
- If you have two modular JARs slf4j_v1.jar and slf4j_v2.jar on the
modulepath, and Guava requires slf4j_v1 and JSoup requires slf4j_v2,
then launching 'java -m HelloWorld' will fail. The boot layer will
refuse to map the "same" packages from different slf4j_v* modules to
the application class loader.
The use case _is_ supported on JDK 9 for modular JARs loaded into
custom loaders of custom layers. That is, the Java Platform Module
System is perfectly capable of supporting the use case -- please see
any of my "Jigsaw: Under The Hood" presentations. The use case just
isn't supported "out of the box" by the 'java' launcher for JARs on
the modulepath.
Alex
------------------------------------------------------------------------
If you reply to this email, your message will be added to the
http://jigsaw-dev.1059479.n5.nabble.com/Multiple-versions-
of-a-non-exported-dependency-tp5713364p5713366.html
Post by Neil Bartlett
To unsubscribe from Multiple versions of a non-exported dependency,
click here
<http://jigsaw-dev.1059479.n5.nabble.com/template/NamlServlet.jtp?macro=
unsubscribe_by_code&node=5713364&code=Y293d29jQGJicy5kYXJrdGVjaC5vcm
d8NTcxMzM2NHwxNTc0MzIxMjQ3>.
Post by Neil Bartlett
NAML
<http://jigsaw-dev.1059479.n5.nabble.com/template/
NamlServlet.jtp?macro=macro_viewer&id=instant_html%
21nabble%3Aemail.naml&base=nabble.naml.namespaces.
BasicNamespace-nabble.view.web.template.NabbleNamespace-
nabble.naml.namespaces.BasicNamespace-nabble.view.
web.template.NabbleNamespace-nabble.naml.namespaces.
BasicNamespace-nabble.view.web.template.NabbleNamespace-
nabble.naml.namespaces.BasicNamespace-nabble.view.
web.template.NabbleNamespace-nabble.naml.namespaces.
BasicNamespace-nabble.view.web.template.NabbleNamespace-
nabble.naml.namespaces.BasicNamespace-nabble.view.
web.template.NabbleNamespace-nabble.view.web.template.
NodeNamespace&breadcrumbs=notify_subscribers%21nabble%
3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_
instant_email%21nabble%3Aemail.naml>
--
View this message in context: http://jigsaw-dev.1059479.n5.
nabble.com/Multiple-versions-of-a-non-exported-dependency-
tp5713364p5713367.html
Sent from the jigsaw-dev mailing list archive at Nabble.com.
--
Cheers,
Paul
Neil Bartlett
2016-09-01 07:28:48 UTC
Permalink
Hi Paul,
Post by Paul Benedict
Neil, but doesn't the prohibition of duplicate packages continue to rear
its head?
I’m sorry I lost track of the context of this question! If you’re asking whether OSGi is prohibited from loading duplicate packages on Java 9, the answer is no.

In OSGi on Java 9 (or any earlier Java version) we use ClassLoaders to achieve isolation between modules. Therefore we explicitly support overlapping private packages, and even multiple modules with overlapping exported packages (NB the latter feature does require OSGi Release 4 or above, which was released in 2004).

In contrast, Jigsaw/JPMS implements isolation using access control rather than ClassLoader visibility. All modules on the modulepath are loaded into the same ClassLoader, and a single ClassLoader can have at most one definition of each class. The implication is that two modules can conflict purely based on their private internals. If any tool attempts to construct valid module sets for use with the modulepath, it will not be sufficient for that tool to examine only the declared module metadata in module-info.class.

As Alex points out, you can work around this restriction by creating a Layer and mapping modules within that layer to separate ClassLoaders. If you do this then you will still need to manage the visibility of classes and delegation between ClassLoaders. In other words you will have started down the path of implementing your own ClassLoader-based module system comparable to where OSGi was in 1998.

Regards,
Neil
Post by Paul Benedict
Cheers,
Paul
Post by cowwoc
Well, this is unfortunate. As I stated earlier, I fail to see how
depending on constant version numbers (not version ranges) fall under
the scope of "version selection". Was this case considered/discussed in
depth?
Not everyone is sold on version ranges (e.g. the vast majority of Maven
artifacts I've seen depend on constant versions) and I think this would
go a long way towards solving the original "classpath hell" problem.
Gili
Post by Neil Bartlett
Gili,
As Alex points out: your use-case can be supported in Java 9 but only
with the addition of custom ClassLoaders, or by using an existing
ClassLoader-based module system such as OSGi.
The same is also true of Java 8, and Java 7, etc.
Regards,
Neil
On 31 Aug 2016, at 19:29, Alex Buckley <[hidden email]
Post by cowwoc
I recently became aware of the fact that the Jigsaw specification
declared
Post by cowwoc
"version-selection" as a non-goal. While I understand how we ended
up here,
Post by cowwoc
I am hoping that you were able to support the following (very common)
* Module "HelloWorld" depends on modules "Guava" and "JSoup".
* Module "Guava" depends on module slf4j version 1 (requires but
does not
Post by cowwoc
export it).
* Module "JSoup" depends on module slf4j version 2 (requires but
does not
Post by cowwoc
export it).
* slf4j version 2 and is not backwards-compatible with version 1.
What happens at runtime? Will Jigsaw (out of the box, without
3rd-party
Post by cowwoc
tools like Maven or OSGI) be smart enough to provide different
versions of
Post by cowwoc
slf4j to "Guava" and "JSoup"?
(You mean Guava/JSoup requires slf4j version 1/2 and does not
"re-export" it a.k.a. 'requires public'.)
This use case isn't possible on JDK 8 for JARs on the classpath, and
- If you have two versions of a modular JAR slf4j.jar in different
directories on the modulepath, then the first one to be found will
dominate, and that's what will be resolved for both Guava and JSoup.
- If you have two modular JARs slf4j_v1.jar and slf4j_v2.jar on the
modulepath, and Guava requires slf4j_v1 and JSoup requires slf4j_v2,
then launching 'java -m HelloWorld' will fail. The boot layer will
refuse to map the "same" packages from different slf4j_v* modules to
the application class loader.
The use case _is_ supported on JDK 9 for modular JARs loaded into
custom loaders of custom layers. That is, the Java Platform Module
System is perfectly capable of supporting the use case -- please see
any of my "Jigsaw: Under The Hood" presentations. The use case just
isn't supported "out of the box" by the 'java' launcher for JARs on
the modulepath.
Alex
------------------------------------------------------------------------
If you reply to this email, your message will be added to the
http://jigsaw-dev.1059479.n5.nabble.com/Multiple-versions-
of-a-non-exported-dependency-tp5713364p5713366.html
Post by Neil Bartlett
To unsubscribe from Multiple versions of a non-exported dependency,
click here
<http://jigsaw-dev.1059479.n5.nabble.com/template/NamlServlet.jtp?macro=
unsubscribe_by_code&node=5713364&code=Y293d29jQGJicy5kYXJrdGVjaC5vcm
d8NTcxMzM2NHwxNTc0MzIxMjQ3>.
Post by Neil Bartlett
NAML
<http://jigsaw-dev.1059479.n5.nabble.com/template/
NamlServlet.jtp?macro=macro_viewer&id=instant_html%
21nabble%3Aemail.naml&base=nabble.naml.namespaces.
BasicNamespace-nabble.view.web.template.NabbleNamespace-
nabble.naml.namespaces.BasicNamespace-nabble.view.
web.template.NabbleNamespace-nabble.naml.namespaces.
BasicNamespace-nabble.view.web.template.NabbleNamespace-
nabble.naml.namespaces.BasicNamespace-nabble.view.
web.template.NabbleNamespace-nabble.naml.namespaces.
BasicNamespace-nabble.view.web.template.NabbleNamespace-
nabble.naml.namespaces.BasicNamespace-nabble.view.
web.template.NabbleNamespace-nabble.view.web.template.
NodeNamespace&breadcrumbs=notify_subscribers%21nabble%
3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_
instant_email%21nabble%3Aemail.naml>
--
View this message in context: http://jigsaw-dev.1059479.n5.
nabble.com/Multiple-versions-of-a-non-exported-dependency-
tp5713364p5713367.html
Sent from the jigsaw-dev mailing list archive at Nabble.com.
--
Cheers,
Paul
Alan Bateman
2016-09-01 07:59:00 UTC
Permalink
Post by Neil Bartlett
In contrast, Jigsaw/JPMS implements isolation using access control rather than ClassLoader visibility. All modules on the modulepath are loaded into the same ClassLoader, and a single ClassLoader can have at most one definition of each class. The implication is that two modules can conflict purely based on their private internals. If any tool attempts to construct valid module sets for use with the modulepath, it will not be sufficient for that tool to examine only the declared module metadata in module-info.class.
Tools or containers that are looking at module declarations will need to
be concerned with the non-exported packages, even if each module is
ultimately mapped to its own class loader (easy to do with Layer API, no
need to get into creating custom class loaders). Specifically, it is an
error if:

"Two or more modules in the configuration export the same package to a
module that reads both. This includes the case where a module M
containing package p reads another module that exports p to M."

Once you move beyond the command line and application module path then
there is no issue if two modules have the same non-exported package.

-Alan
David M. Lloyd
2016-08-31 19:29:44 UTC
Permalink
There are two dimensions to the version range issue: build
reproducibility and future-proofing.

In the former category, single versions are generally used for build
because that ensures that the build will not change over time as new
versions of things become available. In other words, I build version X
of my module on Monday or on Friday and I get the same bits in the end
(more or less). Depending on a version range means that a newer version
of a thing becoming available to the build environment can potentially
cause a change in the final artifact, so version ranges are clearly not
good for the build phase.

In the latter category, and this is really the primary motivator for
having version ranges in the first place, there is a desire for the
developer to stipulate that a module will work with a range of versions
of another module. Thus specifying a range of such versions is
desirable. However this is really a test phase concern: you really only
get an idea of compatibility with some version of a dependency after it
has been tested or after some other process wherein such a compatibility
can be reliably asserted. Furthermore, the range of compatibility may
be found to have changed over the lifetime of a single version of a
given artifact, such that one may wish to add or remove versions from
the known-to-be-compatible set. Thus this kind of information cannot
and should not be part and parcel with the artifact itself; rather it
belongs to the repository or distribution which hosts the artifact, and
to which (for example) a CI system may refer.
Post by cowwoc
Well, this is unfortunate. As I stated earlier, I fail to see how
depending on constant version numbers (not version ranges) fall under
the scope of "version selection". Was this case considered/discussed in
depth?
Not everyone is sold on version ranges (e.g. the vast majority of Maven
artifacts I've seen depend on constant versions) and I think this would
go a long way towards solving the original "classpath hell" problem.
Gili
Post by Neil Bartlett
Gili,
As Alex points out: your use-case can be supported in Java 9 but only
with the addition of custom ClassLoaders, or by using an existing
ClassLoader-based module system such as OSGi.
The same is also true of Java 8, and Java 7, etc.
Regards,
Neil
On 31 Aug 2016, at 19:29, Alex Buckley <[hidden email]
Post by cowwoc
I recently became aware of the fact that the Jigsaw specification
declared
Post by cowwoc
"version-selection" as a non-goal. While I understand how we ended
up here,
Post by cowwoc
I am hoping that you were able to support the following (very common)
* Module "HelloWorld" depends on modules "Guava" and "JSoup".
* Module "Guava" depends on module slf4j version 1 (requires but
does not
Post by cowwoc
export it).
* Module "JSoup" depends on module slf4j version 2 (requires but
does not
Post by cowwoc
export it).
* slf4j version 2 and is not backwards-compatible with version 1.
What happens at runtime? Will Jigsaw (out of the box, without
3rd-party
Post by cowwoc
tools like Maven or OSGI) be smart enough to provide different
versions of
Post by cowwoc
slf4j to "Guava" and "JSoup"?
(You mean Guava/JSoup requires slf4j version 1/2 and does not
"re-export" it a.k.a. 'requires public'.)
This use case isn't possible on JDK 8 for JARs on the classpath, and
- If you have two versions of a modular JAR slf4j.jar in different
directories on the modulepath, then the first one to be found will
dominate, and that's what will be resolved for both Guava and JSoup.
- If you have two modular JARs slf4j_v1.jar and slf4j_v2.jar on the
modulepath, and Guava requires slf4j_v1 and JSoup requires slf4j_v2,
then launching 'java -m HelloWorld' will fail. The boot layer will
refuse to map the "same" packages from different slf4j_v* modules to
the application class loader.
The use case _is_ supported on JDK 9 for modular JARs loaded into
custom loaders of custom layers. That is, the Java Platform Module
System is perfectly capable of supporting the use case -- please see
any of my "Jigsaw: Under The Hood" presentations. The use case just
isn't supported "out of the box" by the 'java' launcher for JARs on
the modulepath.
Alex
------------------------------------------------------------------------
If you reply to this email, your message will be added to the
http://jigsaw-dev.1059479.n5.nabble.com/Multiple-versions-of-a-non-exported-dependency-tp5713364p5713366.html
To unsubscribe from Multiple versions of a non-exported dependency,
click here
<http://jigsaw-dev.1059479.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=5713364&code=Y293d29jQGJicy5kYXJrdGVjaC5vcmd8NTcxMzM2NHwxNTc0MzIxMjQ3>.
NAML
<http://jigsaw-dev.1059479.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
--
View this message in context: http://jigsaw-dev.1059479.n5.nabble.com/Multiple-versions-of-a-non-exported-dependency-tp5713364p5713367.html
Sent from the jigsaw-dev mailing list archive at Nabble.com.
--
- DML
cowwoc
2016-08-31 19:39:43 UTC
Permalink
I don't disagree.

By requiring developers to provide constant version numbers we kill two
birds with one stone:

* We solve part of the "classpath hell" problem out-of-the-box.
* We future-proof the specification so version handling can be
extended in the future.

If "requires" does not take a version number today, adding it tomorrow
might very well be impossible.

I don't even care if you keep the current restrictions in place (allow
only one version per layer) so long as the specification collects
version numbers from day one. This will allow you to ease the
1-version-per-layer restriction in the future without breaking backwards
compatibility for existing deployments.

Gili
Post by David M. Lloyd
There are two dimensions to the version range issue: build
reproducibility and future-proofing.
In the former category, single versions are generally used for build
because that ensures that the build will not change over time as new
versions of things become available. In other words, I build version X
of my module on Monday or on Friday and I get the same bits in the end
(more or less). Depending on a version range means that a newer version
of a thing becoming available to the build environment can potentially
cause a change in the final artifact, so version ranges are clearly not
good for the build phase.
In the latter category, and this is really the primary motivator for
having version ranges in the first place, there is a desire for the
developer to stipulate that a module will work with a range of versions
of another module. Thus specifying a range of such versions is
desirable. However this is really a test phase concern: you really only
get an idea of compatibility with some version of a dependency after it
has been tested or after some other process wherein such a compatibility
can be reliably asserted. Furthermore, the range of compatibility may
be found to have changed over the lifetime of a single version of a
given artifact, such that one may wish to add or remove versions from
the known-to-be-compatible set. Thus this kind of information cannot
and should not be part and parcel with the artifact itself; rather it
belongs to the repository or distribution which hosts the artifact, and
to which (for example) a CI system may refer.
Post by cowwoc
Well, this is unfortunate. As I stated earlier, I fail to see how
depending on constant version numbers (not version ranges) fall under
the scope of "version selection". Was this case considered/discussed in
depth?
Not everyone is sold on version ranges (e.g. the vast majority of Maven
artifacts I've seen depend on constant versions) and I think this would
go a long way towards solving the original "classpath hell" problem.
Gili
Post by Neil Bartlett
Gili,
As Alex points out: your use-case can be supported in Java 9 but only
with the addition of custom ClassLoaders, or by using an existing
ClassLoader-based module system such as OSGi.
The same is also true of Java 8, and Java 7, etc.
Regards,
Neil
On 31 Aug 2016, at 19:29, Alex Buckley <[hidden email]
Post by cowwoc
I recently became aware of the fact that the Jigsaw specification
declared
Post by cowwoc
"version-selection" as a non-goal. While I understand how we ended
up here,
Post by cowwoc
I am hoping that you were able to support the following (very
common)
Post by cowwoc
Post by Neil Bartlett
Post by cowwoc
* Module "HelloWorld" depends on modules "Guava" and "JSoup".
* Module "Guava" depends on module slf4j version 1 (requires but
does not
Post by cowwoc
export it).
* Module "JSoup" depends on module slf4j version 2 (requires but
does not
Post by cowwoc
export it).
* slf4j version 2 and is not backwards-compatible with version 1.
What happens at runtime? Will Jigsaw (out of the box, without
3rd-party
Post by cowwoc
tools like Maven or OSGI) be smart enough to provide different
versions of
Post by cowwoc
slf4j to "Guava" and "JSoup"?
(You mean Guava/JSoup requires slf4j version 1/2 and does not
"re-export" it a.k.a. 'requires public'.)
This use case isn't possible on JDK 8 for JARs on the classpath, and
- If you have two versions of a modular JAR slf4j.jar in different
directories on the modulepath, then the first one to be found will
dominate, and that's what will be resolved for both Guava and JSoup.
- If you have two modular JARs slf4j_v1.jar and slf4j_v2.jar on the
modulepath, and Guava requires slf4j_v1 and JSoup requires slf4j_v2,
then launching 'java -m HelloWorld' will fail. The boot layer will
refuse to map the "same" packages from different slf4j_v* modules to
the application class loader.
The use case _is_ supported on JDK 9 for modular JARs loaded into
custom loaders of custom layers. That is, the Java Platform Module
System is perfectly capable of supporting the use case -- please see
any of my "Jigsaw: Under The Hood" presentations. The use case just
isn't supported "out of the box" by the 'java' launcher for JARs on
the modulepath.
Alex
------------------------------------------------------------------------
Post by cowwoc
Post by Neil Bartlett
If you reply to this email, your message will be added to the
http://jigsaw-dev.1059479.n5.nabble.com/Multiple-versions-of-a-non-exported-dependency-tp5713364p5713366.html
Post by cowwoc
Post by Neil Bartlett
To unsubscribe from Multiple versions of a non-exported dependency,
click here
<
NAML
<http://jigsaw-dev.1059479.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
Post by cowwoc
--
http://jigsaw-dev.1059479.n5.nabble.com/Multiple-versions-of-a-non-exported-dependency-tp5713364p5713367.html
Post by cowwoc
Sent from the jigsaw-dev mailing list archive at Nabble.com.
--
- DML
------------------------------------------------------------------------
If you reply to this email, your message will be added to the
http://jigsaw-dev.1059479.n5.nabble.com/Multiple-versions-of-a-non-exported-dependency-tp5713364p5713370.html
To unsubscribe from Multiple versions of a non-exported dependency,
click here
<http://jigsaw-dev.1059479.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=5713364&code=Y293d29jQGJicy5kYXJrdGVjaC5vcmd8NTcxMzM2NHwxNTc0MzIxMjQ3>.
NAML
<http://jigsaw-dev.1059479.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
--
View this message in context: http://jigsaw-dev.1059479.n5.nabble.com/Multiple-versions-of-a-non-exported-dependency-tp5713364p5713373.html
Sent from the jigsaw-dev mailing list archive at Nabble.com.
Alex Buckley
2016-08-31 19:50:44 UTC
Permalink
Post by cowwoc
I don't even care if you keep the current restrictions in place (allow
only one version per layer)
There is no such restriction in the Java Platform Module System --
please look at java.lang.reflect.Layer. The restriction to a single
version of each module resides in the 'java' launcher [*] because we
don't want to change the class loading behavior of regular Java
applications.

Alex

[*] That's not the full story, as the boot layer with the application
loader has already been created by the time the launcher runs, but from
the user's point of view it's all "the 'java' launcher".
cowwoc
2016-08-31 20:07:01 UTC
Permalink
Alex,

I get this point but if "requires" does not take a version number today
then the Java launcher can never add this functionality in the future.
On the flip side, if you take a version string today and decide to drop
it tomorrow ("this value is ignored in Java 12 and higher") there is
minimal harm.

What's the downside here? How much work is involved in taking in this
string?

Gili
Post by Alex Buckley
Post by cowwoc
I don't even care if you keep the current restrictions in place (allow
only one version per layer)
There is no such restriction in the Java Platform Module System --
please look at java.lang.reflect.Layer. The restriction to a single
version of each module resides in the 'java' launcher [*] because we
don't want to change the class loading behavior of regular Java
applications.
Alex
[*] That's not the full story, as the boot layer with the application
loader has already been created by the time the launcher runs, but from
the user's point of view it's all "the 'java' launcher".
------------------------------------------------------------------------
If you reply to this email, your message will be added to the
http://jigsaw-dev.1059479.n5.nabble.com/Multiple-versions-of-a-non-exported-dependency-tp5713364p5713374.html
To unsubscribe from Multiple versions of a non-exported dependency,
click here
<http://jigsaw-dev.1059479.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=5713364&code=Y293d29jQGJicy5kYXJrdGVjaC5vcmd8NTcxMzM2NHwxNTc0MzIxMjQ3>.
NAML
<http://jigsaw-dev.1059479.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
--
View this message in context: http://jigsaw-dev.1059479.n5.nabble.com/Multiple-versions-of-a-non-exported-dependency-tp5713364p5713376.html
Sent from the jigsaw-dev mailing list archive at Nabble.com.
Alex Buckley
2016-08-31 19:38:14 UTC
Permalink
In the JDK 8 java launcher, classpath hell is having two [incompatible]
versions of the "same" JAR on the classpath and getting types from both
of them.

In the JDK 9 java launcher, we avoid modulepath hell -- two
[incompatible] versions of the "same" module on the modulepath -- by
uniquifying each module on the modulepath. If someone tries to work
around that by giving different names to the different versions of the
"same" module, then we still avoid modulepath hell by refusing to load
their [incompatible] packages into the same loader. The java launcher is
not becoming a mini container that spins loaders for multiple versions
of the same module -- code defined by the application loader on JDK 8 is
defined by the application loader on JDK 9.

Alex
Post by cowwoc
Well, this is unfortunate. As I stated earlier, I fail to see how
depending on constant version numbers (not version ranges) fall under
the scope of "version selection". Was this case considered/discussed in
depth?
Not everyone is sold on version ranges (e.g. the vast majority of Maven
artifacts I've seen depend on constant versions) and I think this would
go a long way towards solving the original "classpath hell" problem.
Gili
Post by Neil Bartlett
Gili,
As Alex points out: your use-case can be supported in Java 9 but only
with the addition of custom ClassLoaders, or by using an existing
ClassLoader-based module system such as OSGi.
The same is also true of Java 8, and Java 7, etc.
Regards,
Neil
On 31 Aug 2016, at 19:29, Alex Buckley <[hidden email]
Post by cowwoc
I recently became aware of the fact that the Jigsaw specification
declared
Post by cowwoc
"version-selection" as a non-goal. While I understand how we ended
up here,
Post by cowwoc
I am hoping that you were able to support the following (very common)
* Module "HelloWorld" depends on modules "Guava" and "JSoup".
* Module "Guava" depends on module slf4j version 1 (requires but
does not
Post by cowwoc
export it).
* Module "JSoup" depends on module slf4j version 2 (requires but
does not
Post by cowwoc
export it).
* slf4j version 2 and is not backwards-compatible with version 1.
What happens at runtime? Will Jigsaw (out of the box, without
3rd-party
Post by cowwoc
tools like Maven or OSGI) be smart enough to provide different
versions of
Post by cowwoc
slf4j to "Guava" and "JSoup"?
(You mean Guava/JSoup requires slf4j version 1/2 and does not
"re-export" it a.k.a. 'requires public'.)
This use case isn't possible on JDK 8 for JARs on the classpath, and
- If you have two versions of a modular JAR slf4j.jar in different
directories on the modulepath, then the first one to be found will
dominate, and that's what will be resolved for both Guava and JSoup.
- If you have two modular JARs slf4j_v1.jar and slf4j_v2.jar on the
modulepath, and Guava requires slf4j_v1 and JSoup requires slf4j_v2,
then launching 'java -m HelloWorld' will fail. The boot layer will
refuse to map the "same" packages from different slf4j_v* modules to
the application class loader.
The use case _is_ supported on JDK 9 for modular JARs loaded into
custom loaders of custom layers. That is, the Java Platform Module
System is perfectly capable of supporting the use case -- please see
any of my "Jigsaw: Under The Hood" presentations. The use case just
isn't supported "out of the box" by the 'java' launcher for JARs on
the modulepath.
Alex
------------------------------------------------------------------------
If you reply to this email, your message will be added to the
http://jigsaw-dev.1059479.n5.nabble.com/Multiple-versions-of-a-non-exported-dependency-tp5713364p5713366.html
To unsubscribe from Multiple versions of a non-exported dependency,
click here
<http://jigsaw-dev.1059479.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=5713364&code=Y293d29jQGJicy5kYXJrdGVjaC5vcmd8NTcxMzM2NHwxNTc0MzIxMjQ3>.
NAML
<http://jigsaw-dev.1059479.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
--
View this message in context: http://jigsaw-dev.1059479.n5.nabble.com/Multiple-versions-of-a-non-exported-dependency-tp5713364p5713367.html
Sent from the jigsaw-dev mailing list archive at Nabble.com.
cowwoc
2016-08-31 19:51:46 UTC
Permalink
Alex,

I agree that the situation is better, but not by much. Developers
routinely run across transitive dependencies that are incompatible with
each other. You seem to be under the impression that this a rare
occurrence or only occurs in the context of web containers, but this is
simply not the case.

The current jigsaw implementation converts "classpath hell" failures
from runtime to compile-time. While this is certainly an improvement I
think we can do much better at a minimal implementation cost.

I propose having developers pass constant version numbers to "requires".
You can keep all other restrictions in place (e.g. one version of each
library) and hopefully we can loosen these restrictions in the future.
If you're feeling extra generous, I would ask you to expose the version
number through reflection (but, again this is not required up-front). I
just want to lay the groundwork for future incremental improvement.

My fear is that if "requires" does not take a version number today, you
will never be able to add it in future releases.

Gili
Post by Alex Buckley
In the JDK 8 java launcher, classpath hell is having two [incompatible]
versions of the "same" JAR on the classpath and getting types from both
of them.
In the JDK 9 java launcher, we avoid modulepath hell -- two
[incompatible] versions of the "same" module on the modulepath -- by
uniquifying each module on the modulepath. If someone tries to work
around that by giving different names to the different versions of the
"same" module, then we still avoid modulepath hell by refusing to load
their [incompatible] packages into the same loader. The java launcher is
not becoming a mini container that spins loaders for multiple versions
of the same module -- code defined by the application loader on JDK 8 is
defined by the application loader on JDK 9.
Alex
Post by cowwoc
Well, this is unfortunate. As I stated earlier, I fail to see how
depending on constant version numbers (not version ranges) fall under
the scope of "version selection". Was this case considered/discussed in
depth?
Not everyone is sold on version ranges (e.g. the vast majority of Maven
artifacts I've seen depend on constant versions) and I think this would
go a long way towards solving the original "classpath hell" problem.
Gili
Post by Neil Bartlett
Gili,
As Alex points out: your use-case can be supported in Java 9 but only
with the addition of custom ClassLoaders, or by using an existing
ClassLoader-based module system such as OSGi.
The same is also true of Java 8, and Java 7, etc.
Regards,
Neil
On 31 Aug 2016, at 19:29, Alex Buckley <[hidden email]
Post by cowwoc
I recently became aware of the fact that the Jigsaw specification
declared
Post by cowwoc
"version-selection" as a non-goal. While I understand how we ended
up here,
Post by cowwoc
I am hoping that you were able to support the following (very
common)
Post by cowwoc
Post by Neil Bartlett
Post by cowwoc
* Module "HelloWorld" depends on modules "Guava" and "JSoup".
* Module "Guava" depends on module slf4j version 1 (requires but
does not
Post by cowwoc
export it).
* Module "JSoup" depends on module slf4j version 2 (requires but
does not
Post by cowwoc
export it).
* slf4j version 2 and is not backwards-compatible with version 1.
What happens at runtime? Will Jigsaw (out of the box, without
3rd-party
Post by cowwoc
tools like Maven or OSGI) be smart enough to provide different
versions of
Post by cowwoc
slf4j to "Guava" and "JSoup"?
(You mean Guava/JSoup requires slf4j version 1/2 and does not
"re-export" it a.k.a. 'requires public'.)
This use case isn't possible on JDK 8 for JARs on the classpath, and
- If you have two versions of a modular JAR slf4j.jar in different
directories on the modulepath, then the first one to be found will
dominate, and that's what will be resolved for both Guava and JSoup.
- If you have two modular JARs slf4j_v1.jar and slf4j_v2.jar on the
modulepath, and Guava requires slf4j_v1 and JSoup requires slf4j_v2,
then launching 'java -m HelloWorld' will fail. The boot layer will
refuse to map the "same" packages from different slf4j_v* modules to
the application class loader.
The use case _is_ supported on JDK 9 for modular JARs loaded into
custom loaders of custom layers. That is, the Java Platform Module
System is perfectly capable of supporting the use case -- please see
any of my "Jigsaw: Under The Hood" presentations. The use case just
isn't supported "out of the box" by the 'java' launcher for JARs on
the modulepath.
Alex
------------------------------------------------------------------------
Post by cowwoc
Post by Neil Bartlett
If you reply to this email, your message will be added to the
http://jigsaw-dev.1059479.n5.nabble.com/Multiple-versions-of-a-non-exported-dependency-tp5713364p5713366.html
Post by cowwoc
Post by Neil Bartlett
To unsubscribe from Multiple versions of a non-exported dependency,
click here
<
NAML
<http://jigsaw-dev.1059479.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
Post by cowwoc
--
http://jigsaw-dev.1059479.n5.nabble.com/Multiple-versions-of-a-non-exported-dependency-tp5713364p5713367.html
Post by cowwoc
Sent from the jigsaw-dev mailing list archive at Nabble.com.
------------------------------------------------------------------------
If you reply to this email, your message will be added to the
http://jigsaw-dev.1059479.n5.nabble.com/Multiple-versions-of-a-non-exported-dependency-tp5713364p5713372.html
To unsubscribe from Multiple versions of a non-exported dependency,
click here
<http://jigsaw-dev.1059479.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=5713364&code=Y293d29jQGJicy5kYXJrdGVjaC5vcmd8NTcxMzM2NHwxNTc0MzIxMjQ3>.
NAML
<http://jigsaw-dev.1059479.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
--
View this message in context: http://jigsaw-dev.1059479.n5.nabble.com/Multiple-versions-of-a-non-exported-dependency-tp5713364p5713375.html
Sent from the jigsaw-dev mailing list archive at Nabble.com.
Alex Buckley
2016-08-31 20:07:59 UTC
Permalink
Post by cowwoc
I agree that the situation is better, but not by much. Developers
routinely run across transitive dependencies that are incompatible with
each other. You seem to be under the impression that this a rare
occurrence or only occurs in the context of web containers, but this is
simply not the case.
We're not under this impression at all. The problem is that the cost of
solving incompatible indirect dependencies is high -- it would mean
introducing first-class versions and changing the class loading
characteristics of regular ('java'-launched) applications. Project
Jigsaw chose a long time ago to let tools handle the version selection
problem while it pursues other areas, namely strong encapsulation and
"modules all the way down" (the modular JDK). I see no reason why a
future project that wanted to introduce first-class versions couldn't do
it on top of JDK 9.

Alex
cowwoc
2016-08-31 20:15:50 UTC
Permalink
Alex,

Thank you for the clarification.

I am a bit confused by your assertion... If you wanted to introduce
first-class versions in JDK 10, how would you do so (without breaking
backwards compatibility) in light of this format?

module com.foo.bar {
*requires* org.baz.qux;
}

Gili
Post by Alex Buckley
Post by cowwoc
I agree that the situation is better, but not by much. Developers
routinely run across transitive dependencies that are incompatible with
each other. You seem to be under the impression that this a rare
occurrence or only occurs in the context of web containers, but this is
simply not the case.
We're not under this impression at all. The problem is that the cost of
solving incompatible indirect dependencies is high -- it would mean
introducing first-class versions and changing the class loading
characteristics of regular ('java'-launched) applications. Project
Jigsaw chose a long time ago to let tools handle the version selection
problem while it pursues other areas, namely strong encapsulation and
"modules all the way down" (the modular JDK). I see no reason why a
future project that wanted to introduce first-class versions couldn't do
it on top of JDK 9.
Alex
------------------------------------------------------------------------
If you reply to this email, your message will be added to the
http://jigsaw-dev.1059479.n5.nabble.com/Multiple-versions-of-a-non-exported-dependency-tp5713364p5713377.html
To unsubscribe from Multiple versions of a non-exported dependency,
click here
<http://jigsaw-dev.1059479.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=5713364&code=Y293d29jQGJicy5kYXJrdGVjaC5vcmd8NTcxMzM2NHwxNTc0MzIxMjQ3>.
NAML
<http://jigsaw-dev.1059479.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
--
View this message in context: http://jigsaw-dev.1059479.n5.nabble.com/Multiple-versions-of-a-non-exported-dependency-tp5713364p5713378.html
Sent from the jigsaw-dev mailing list archive at Nabble.com.
Alex Buckley
2016-08-31 20:28:59 UTC
Permalink
This has been a good conversation but I'm unwilling to enter into
unbounded discussion of hypothetical future features. Here are some
practical things you can try with JDK 9 right now:

- Have you run your JDK 8 application on a JDK 9 EA build?
- Have you jlinked an image consisting of a handful of JDK 9 modules,
and run your application on top of that?
- Have you declared modules for your application and put supporting
legacy JARs on the modulepath as automatic modules?

Alex
Post by cowwoc
Alex,
Thank you for the clarification.
I am a bit confused by your assertion... If you wanted to introduce
first-class versions in JDK 10, how would you do so (without breaking
backwards compatibility) in light of this format?
module com.foo.bar {
*requires* org.baz.qux;
}
Gili
cowwoc
2016-08-31 21:54:43 UTC
Permalink
Alex,
Post by Alex Buckley
I see no reason why a
future project that wanted to introduce first-class versions couldn't do
it on top of JDK 9.
When I asked you to clarify how it could do so you responded with a
non-answer. This doesn't inspire much confidence.

As far as I can tell, the current syntax makes this impossible.

Gili
Post by Alex Buckley
This has been a good conversation but I'm unwilling to enter into
unbounded discussion of hypothetical future features. Here are some
- Have you run your JDK 8 application on a JDK 9 EA build?
- Have you jlinked an image consisting of a handful of JDK 9 modules,
and run your application on top of that?
- Have you declared modules for your application and put supporting
legacy JARs on the modulepath as automatic modules?
Alex
Post by cowwoc
Alex,
Thank you for the clarification.
I am a bit confused by your assertion... If you wanted to introduce
first-class versions in JDK 10, how would you do so (without breaking
backwards compatibility) in light of this format?
module com.foo.bar {
*requires* org.baz.qux;
}
Gili
------------------------------------------------------------------------
If you reply to this email, your message will be added to the
http://jigsaw-dev.1059479.n5.nabble.com/Multiple-versions-of-a-non-exported-dependency-tp5713364p5713379.html
To unsubscribe from Multiple versions of a non-exported dependency,
click here
<http://jigsaw-dev.1059479.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=5713364&code=Y293d29jQGJicy5kYXJrdGVjaC5vcmd8NTcxMzM2NHwxNTc0MzIxMjQ3>.
NAML
<http://jigsaw-dev.1059479.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
--
View this message in context: http://jigsaw-dev.1059479.n5.nabble.com/Multiple-versions-of-a-non-exported-dependency-tp5713364p5713382.html
Sent from the jigsaw-dev mailing list archive at Nabble.com.
Remi Forax
2016-08-31 21:34:36 UTC
Permalink
I think Alex is too gentle,
I don't want the JDK dictate to me how to resolve incompatible versions,
JBoss has it's own way to resolve versions, OSGI has another way, etc,
I don't want a n+1 way to resolve incompatible versions.

https://xkcd.com/927/

Rémi

----- Mail original -----
Envoyé: Mercredi 31 Août 2016 22:07:59
Objet: Re: Multiple versions of a non-exported dependency
Post by cowwoc
I agree that the situation is better, but not by much. Developers
routinely run across transitive dependencies that are incompatible with
each other. You seem to be under the impression that this a rare
occurrence or only occurs in the context of web containers, but this is
simply not the case.
We're not under this impression at all. The problem is that the cost of
solving incompatible indirect dependencies is high -- it would mean
introducing first-class versions and changing the class loading
characteristics of regular ('java'-launched) applications. Project
Jigsaw chose a long time ago to let tools handle the version selection
problem while it pursues other areas, namely strong encapsulation and
"modules all the way down" (the modular JDK). I see no reason why a
future project that wanted to introduce first-class versions couldn't do
it on top of JDK 9.
Alex
cowwoc
2016-08-31 21:48:04 UTC
Permalink
Remi,

I don't understand what you mean by "resolve incompatible versions". All
I am asking the JDK to do is note the version number associated with a
dependency and expose it using reflection for tools to use. Which part
of this causes you problems?

Gili
Post by Remi Forax
I think Alex is too gentle,
I don't want the JDK dictate to me how to resolve incompatible versions,
JBoss has it's own way to resolve versions, OSGI has another way, etc,
I don't want a n+1 way to resolve incompatible versions.
https://xkcd.com/927/
Rémi
----- Mail original -----
De: "Alex Buckley" <[hidden email]
</user/SendEmail.jtp?type=node&node=5713380&i=0>>
À: [hidden email] </user/SendEmail.jtp?type=node&node=5713380&i=1>
Envoyé: Mercredi 31 Août 2016 22:07:59
Objet: Re: Multiple versions of a non-exported dependency
Post by cowwoc
I agree that the situation is better, but not by much. Developers
routinely run across transitive dependencies that are incompatible
with
Post by cowwoc
each other. You seem to be under the impression that this a rare
occurrence or only occurs in the context of web containers, but
this is
Post by cowwoc
simply not the case.
We're not under this impression at all. The problem is that the cost of
solving incompatible indirect dependencies is high -- it would mean
introducing first-class versions and changing the class loading
characteristics of regular ('java'-launched) applications. Project
Jigsaw chose a long time ago to let tools handle the version selection
problem while it pursues other areas, namely strong encapsulation and
"modules all the way down" (the modular JDK). I see no reason why a
future project that wanted to introduce first-class versions
couldn't do
it on top of JDK 9.
Alex
------------------------------------------------------------------------
If you reply to this email, your message will be added to the
http://jigsaw-dev.1059479.n5.nabble.com/Multiple-versions-of-a-non-exported-dependency-tp5713364p5713380.html
To unsubscribe from Multiple versions of a non-exported dependency,
click here
<http://jigsaw-dev.1059479.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=5713364&code=Y293d29jQGJicy5kYXJrdGVjaC5vcmd8NTcxMzM2NHwxNTc0MzIxMjQ3>.
NAML
<http://jigsaw-dev.1059479.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
--
View this message in context: http://jigsaw-dev.1059479.n5.nabble.com/Multiple-versions-of-a-non-exported-dependency-tp5713364p5713381.html
Sent from the jigsaw-dev mailing list archive at Nabble.com.
Remi Forax
2016-08-31 19:28:50 UTC
Permalink
The other solution is to statically link the right version of slf4j inside guava and jsoup.
A tool like jarjar can be updated to merge two modular jars (merge two module-info).

cheers,
Rémi

----- Mail original -----
Envoyé: Mercredi 31 Août 2016 20:54:44
Objet: Re: Multiple versions of a non-exported dependency
Gili,
As Alex points out: your use-case can be supported in Java 9 but only with the
addition of custom ClassLoaders, or by using an existing ClassLoader-based
module system such as OSGi.
The same is also true of Java 8, and Java 7, etc.
Regards,
Neil
Post by cowwoc
I recently became aware of the fact that the Jigsaw specification declared
"version-selection" as a non-goal. While I understand how we ended up here,
I am hoping that you were able to support the following (very common)
* Module "HelloWorld" depends on modules "Guava" and "JSoup".
* Module "Guava" depends on module slf4j version 1 (requires but does not
export it).
* Module "JSoup" depends on module slf4j version 2 (requires but does not
export it).
* slf4j version 2 and is not backwards-compatible with version 1.
What happens at runtime? Will Jigsaw (out of the box, without 3rd-party
tools like Maven or OSGI) be smart enough to provide different versions of
slf4j to "Guava" and "JSoup"?
(You mean Guava/JSoup requires slf4j version 1/2 and does not "re-export" it
a.k.a. 'requires public'.)
This use case isn't possible on JDK 8 for JARs on the classpath, and it's not
- If you have two versions of a modular JAR slf4j.jar in different directories
on the modulepath, then the first one to be found will dominate, and that's
what will be resolved for both Guava and JSoup.
- If you have two modular JARs slf4j_v1.jar and slf4j_v2.jar on the modulepath,
and Guava requires slf4j_v1 and JSoup requires slf4j_v2, then launching 'java
-m HelloWorld' will fail. The boot layer will refuse to map the "same" packages
from different slf4j_v* modules to the application class loader.
The use case _is_ supported on JDK 9 for modular JARs loaded into custom loaders
of custom layers. That is, the Java Platform Module System is perfectly capable
of supporting the use case -- please see any of my "Jigsaw: Under The Hood"
presentations. The use case just isn't supported "out of the box" by the 'java'
launcher for JARs on the modulepath.
Alex
cowwoc
2016-08-31 19:35:07 UTC
Permalink
I am aware of this approach, but this is precisely the kind of mess that
I expected Jigsaw to replace.

As far as I know, the main reason JarJar came into existence was to work
around "classpath hell" and the main goal behind Jigsaw was to solve it.
For all of Jigsaw's accomplishments, it feels like it fails to solve the
very problem it was designed to solve (maybe not Oracle's mind but
certainly in the mind of most developers).

Gili
Post by Remi Forax
The other solution is to statically link the right version of slf4j inside guava and jsoup.
A tool like jarjar can be updated to merge two modular jars (merge two module-info).
cheers,
Rémi
----- Mail original -----
De: "Neil Bartlett" <[hidden email]
</user/SendEmail.jtp?type=node&node=5713369&i=0>>
À: [hidden email] </user/SendEmail.jtp?type=node&node=5713369&i=1>,
"Alex Buckley" <[hidden email]
</user/SendEmail.jtp?type=node&node=5713369&i=2>>
Cc: "ZML-OpenJDK-Jigsaw-Developers" <[hidden email]
</user/SendEmail.jtp?type=node&node=5713369&i=3>>
Envoyé: Mercredi 31 Août 2016 20:54:44
Objet: Re: Multiple versions of a non-exported dependency
Gili,
As Alex points out: your use-case can be supported in Java 9 but
only with the
addition of custom ClassLoaders, or by using an existing
ClassLoader-based
module system such as OSGi.
The same is also true of Java 8, and Java 7, etc.
Regards,
Neil
On 31 Aug 2016, at 19:29, Alex Buckley <[hidden email]
Post by cowwoc
I recently became aware of the fact that the Jigsaw specification
declared
Post by cowwoc
"version-selection" as a non-goal. While I understand how we ended
up here,
Post by cowwoc
I am hoping that you were able to support the following (very common)
* Module "HelloWorld" depends on modules "Guava" and "JSoup".
* Module "Guava" depends on module slf4j version 1 (requires but
does not
Post by cowwoc
export it).
* Module "JSoup" depends on module slf4j version 2 (requires but
does not
Post by cowwoc
export it).
* slf4j version 2 and is not backwards-compatible with version 1.
What happens at runtime? Will Jigsaw (out of the box, without
3rd-party
Post by cowwoc
tools like Maven or OSGI) be smart enough to provide different
versions of
Post by cowwoc
slf4j to "Guava" and "JSoup"?
(You mean Guava/JSoup requires slf4j version 1/2 and does not
"re-export" it
a.k.a. 'requires public'.)
This use case isn't possible on JDK 8 for JARs on the classpath,
and it's not
- If you have two versions of a modular JAR slf4j.jar in different
directories
on the modulepath, then the first one to be found will dominate,
and that's
what will be resolved for both Guava and JSoup.
- If you have two modular JARs slf4j_v1.jar and slf4j_v2.jar on the
modulepath,
and Guava requires slf4j_v1 and JSoup requires slf4j_v2, then
launching 'java
-m HelloWorld' will fail. The boot layer will refuse to map the
"same" packages
from different slf4j_v* modules to the application class loader.
The use case _is_ supported on JDK 9 for modular JARs loaded into
custom loaders
of custom layers. That is, the Java Platform Module System is
perfectly capable
of supporting the use case -- please see any of my "Jigsaw: Under
The Hood"
presentations. The use case just isn't supported "out of the box"
by the 'java'
launcher for JARs on the modulepath.
Alex
------------------------------------------------------------------------
If you reply to this email, your message will be added to the
http://jigsaw-dev.1059479.n5.nabble.com/Multiple-versions-of-a-non-exported-dependency-tp5713364p5713369.html
To unsubscribe from Multiple versions of a non-exported dependency,
click here
<http://jigsaw-dev.1059479.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=5713364&code=Y293d29jQGJicy5kYXJrdGVjaC5vcmd8NTcxMzM2NHwxNTc0MzIxMjQ3>.
NAML
<http://jigsaw-dev.1059479.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
--
View this message in context: http://jigsaw-dev.1059479.n5.nabble.com/Multiple-versions-of-a-non-exported-dependency-tp5713364p5713371.html
Sent from the jigsaw-dev mailing list archive at Nabble.com.
Neil Bartlett
2016-09-01 00:45:17 UTC
Permalink
Remi,

Actually I don’t think that statically linking will work. This would produce modules that have overlapping private (non-exported) packages, and such modules also cannot be used in Java 9 on the modulepath.

I tested this in build 9-ea+126-jigsaw-nightly-h5280-20160713 by creating two modules both containing a private package org.example.util. The following exception resulted: java.lang.reflect.LayerInstantiationException: Package org.example.util in both module a and module b.

Again this could be “solved” by using custom ClassLoaders or a ClassLoader-based module system like OSGi on Java 9.

Neil
Post by Remi Forax
The other solution is to statically link the right version of slf4j inside guava and jsoup.
A tool like jarjar can be updated to merge two modular jars (merge two module-info).
cheers,
Rémi
----- Mail original -----
Envoyé: Mercredi 31 Août 2016 20:54:44
Objet: Re: Multiple versions of a non-exported dependency
Gili,
As Alex points out: your use-case can be supported in Java 9 but only with the
addition of custom ClassLoaders, or by using an existing ClassLoader-based
module system such as OSGi.
The same is also true of Java 8, and Java 7, etc.
Regards,
Neil
Post by cowwoc
I recently became aware of the fact that the Jigsaw specification declared
"version-selection" as a non-goal. While I understand how we ended up here,
I am hoping that you were able to support the following (very common)
* Module "HelloWorld" depends on modules "Guava" and "JSoup".
* Module "Guava" depends on module slf4j version 1 (requires but does not
export it).
* Module "JSoup" depends on module slf4j version 2 (requires but does not
export it).
* slf4j version 2 and is not backwards-compatible with version 1.
What happens at runtime? Will Jigsaw (out of the box, without 3rd-party
tools like Maven or OSGI) be smart enough to provide different versions of
slf4j to "Guava" and "JSoup"?
(You mean Guava/JSoup requires slf4j version 1/2 and does not "re-export" it
a.k.a. 'requires public'.)
This use case isn't possible on JDK 8 for JARs on the classpath, and it's not
- If you have two versions of a modular JAR slf4j.jar in different directories
on the modulepath, then the first one to be found will dominate, and that's
what will be resolved for both Guava and JSoup.
- If you have two modular JARs slf4j_v1.jar and slf4j_v2.jar on the modulepath,
and Guava requires slf4j_v1 and JSoup requires slf4j_v2, then launching 'java
-m HelloWorld' will fail. The boot layer will refuse to map the "same" packages
from different slf4j_v* modules to the application class loader.
The use case _is_ supported on JDK 9 for modular JARs loaded into custom loaders
of custom layers. That is, the Java Platform Module System is perfectly capable
of supporting the use case -- please see any of my "Jigsaw: Under The Hood"
presentations. The use case just isn't supported "out of the box" by the 'java'
launcher for JARs on the modulepath.
Alex
David M. Lloyd
2016-09-01 12:29:04 UTC
Permalink
It seems like there is no good reason why the application modules aren't
loaded with classloader-per-module now. The platform stuff could all be
in one, but the application stuff? Problems like this are going to come
up a lot otherwise; let's consider making that change.
Post by Neil Bartlett
Remi,
Actually I don’t think that statically linking will work. This would produce modules that have overlapping private (non-exported) packages, and such modules also cannot be used in Java 9 on the modulepath.
I tested this in build 9-ea+126-jigsaw-nightly-h5280-20160713 by creating two modules both containing a private package org.example.util. The following exception resulted: java.lang.reflect.LayerInstantiationException: Package org.example.util in both module a and module b.
Again this could be “solved” by using custom ClassLoaders or a ClassLoader-based module system like OSGi on Java 9.
Neil
Post by Remi Forax
The other solution is to statically link the right version of slf4j inside guava and jsoup.
A tool like jarjar can be updated to merge two modular jars (merge two module-info).
cheers,
Rémi
----- Mail original -----
Envoyé: Mercredi 31 Août 2016 20:54:44
Objet: Re: Multiple versions of a non-exported dependency
Gili,
As Alex points out: your use-case can be supported in Java 9 but only with the
addition of custom ClassLoaders, or by using an existing ClassLoader-based
module system such as OSGi.
The same is also true of Java 8, and Java 7, etc.
Regards,
Neil
Post by cowwoc
I recently became aware of the fact that the Jigsaw specification declared
"version-selection" as a non-goal. While I understand how we ended up here,
I am hoping that you were able to support the following (very common)
* Module "HelloWorld" depends on modules "Guava" and "JSoup".
* Module "Guava" depends on module slf4j version 1 (requires but does not
export it).
* Module "JSoup" depends on module slf4j version 2 (requires but does not
export it).
* slf4j version 2 and is not backwards-compatible with version 1.
What happens at runtime? Will Jigsaw (out of the box, without 3rd-party
tools like Maven or OSGI) be smart enough to provide different versions of
slf4j to "Guava" and "JSoup"?
(You mean Guava/JSoup requires slf4j version 1/2 and does not "re-export" it
a.k.a. 'requires public'.)
This use case isn't possible on JDK 8 for JARs on the classpath, and it's not
- If you have two versions of a modular JAR slf4j.jar in different directories
on the modulepath, then the first one to be found will dominate, and that's
what will be resolved for both Guava and JSoup.
- If you have two modular JARs slf4j_v1.jar and slf4j_v2.jar on the modulepath,
and Guava requires slf4j_v1 and JSoup requires slf4j_v2, then launching 'java
-m HelloWorld' will fail. The boot layer will refuse to map the "same" packages
from different slf4j_v* modules to the application class loader.
The use case _is_ supported on JDK 9 for modular JARs loaded into custom loaders
of custom layers. That is, the Java Platform Module System is perfectly capable
of supporting the use case -- please see any of my "Jigsaw: Under The Hood"
presentations. The use case just isn't supported "out of the box" by the 'java'
launcher for JARs on the modulepath.
Alex
--
- DML
Alan Bateman
2016-09-01 13:59:27 UTC
Permalink
Post by David M. Lloyd
It seems like there is no good reason why the application modules
aren't loaded with classloader-per-module now. The platform stuff
could all be in one, but the application stuff? Problems like this
are going to come up a lot otherwise; let's consider making that change.
If we were just dealing with a graph of explicit modules they it may be
an option, assuming you get over all the issues that arise when
arranging visibility this way. You might remember "module mode" in the
original exploratory phase of Project Jigsaw for example.

The issue is of course that there is lot more in picture, esp. when you
have the unnamed module (= class path) reading all modules, also
automatic modules that bridge to the class path (and so read the unnamed
module). Then add upgradable modules into the picture, ... and you will
quickly start to see there is a lot more to this, not to mind the risk
of circular delegation.

So I think what we have ended up with sane and not difficult to explain.
It favors migration and good interop over green field. Sure, there will
be periodic complaints when people try to deploy modules with
overlapping packages on the application module path. Anyone using Maven
Shade Plugin and the like can continue to do this. Finally, it's not
hard to create your own "launcher" that instantiates the configuration
with Layer.defineModulesWithManyLoaders if you really want.

-Alan
David M. Lloyd
2016-09-01 14:35:01 UTC
Permalink
Post by Alan Bateman
Post by David M. Lloyd
It seems like there is no good reason why the application modules
aren't loaded with classloader-per-module now. The platform stuff
could all be in one, but the application stuff? Problems like this
are going to come up a lot otherwise; let's consider making that change.
If we were just dealing with a graph of explicit modules they it may be
an option, assuming you get over all the issues that arise when
arranging visibility this way. You might remember "module mode" in the
original exploratory phase of Project Jigsaw for example.
The issue is of course that there is lot more in picture, esp. when you
have the unnamed module (= class path) reading all modules, also
automatic modules that bridge to the class path (and so read the unnamed
module). Then add upgradable modules into the picture, ... and you will
quickly start to see there is a lot more to this, not to mind the risk
of circular delegation.
Risk? If the modules don't have circular delegation then the class
loaders won't; but anyway I don't understand *at all* why circular
delegation in class loaders is a problem (we do this today and it has
worked great since Java 7, and there's no particular magic necessary to
do so). And in any event circular delegation in modules *should* be
allowed, full stop; there's no good reason not to allow it (especially
given the very-super-eager-loading behavior of module layers). Again we
do this today and it's fine.
Post by Alan Bateman
So I think what we have ended up with sane and not difficult to explain.
It favors migration and good interop over green field.
Yeah having the class path remain on the legacy application class loader
is demonstrably better for interop. But new modules? Does that make sense?
Post by Alan Bateman
Sure, there will
be periodic complaints when people try to deploy modules with
overlapping packages on the application module path. Anyone using Maven
Shade Plugin and the like can continue to do this. Finally, it's not
hard to create your own "launcher" that instantiates the configuration
with Layer.defineModulesWithManyLoaders if you really want.
I think many people will do this. The benefits of modules are
diminishing if you don't actually get this level of isolation by
default. And I think that it's much harder to imagine a real interop
problem that could arise from it than it is to imagine a real problem
that will occur from not doing it.

Anyway using interoperability as an argument is very weak as long as
"export dynamic *" or any variation thereof is considered to be an
acceptable solution to #ReflectiveAccessToNonExportedTypes. In order to
have proper interoperability for any reflection-using or reflected
module, you have to do this, which defeats the primary security measure
of modules. Isn't this a much more likely interop problem than putting
modules (something that never existed before) into their own class loaders?
--
- DML
Alan Bateman
2016-09-01 17:11:16 UTC
Permalink
Post by David M. Lloyd
Yeah having the class path remain on the legacy application class
loader is demonstrably better for interop. But new modules? Does
that make sense?
Yes, specifically automatic modules where a JAR file is moved from the
class path to module path without any changes. Also any library that is
migrated to an explicit module. If the library is used to being on the
class path today, and it's not in a world of hurt that is split packages
or cycles, then the effort to make it an explicit module might be minimal.
Post by David M. Lloyd
Anyway using interoperability as an argument is very weak as long as
"export dynamic *" or any variation thereof is considered to be an
acceptable solution to #ReflectiveAccessToNonExportedTypes. In order
to have proper interoperability for any reflection-using or reflected
module, you have to do this, which defeats the primary security
measure of modules. Isn't this a much more likely interop problem
than putting modules (something that never existed before) into their
own class loaders?
Modules might be new but a lot existing code that will be migrated. I
don't wish to engage on the #ReflectiveAccessToNonExportedTypes topic in
this thread, mostly because that topic is still open on the JSR list and
there are several overlapping threads already.

-Alan
Stephen Felts
2016-09-09 14:30:41 UTC
Permalink
We have an application that is running into a problem with a utility program.  Below is a standalone reproducer.

 

The program does not import the SPI package sun.nio.ch - it isn't aware of

it, and SocketChannel.isConnected() is a public method of a public type. In

short, it does not break any law of encapsulation, so call

setAccessible(true) should be OK.

 

import java.lang.reflect.Method;

 

 

public class JDK9Nio {

  public static void main(String args[]) throws Exception {

    call();

  }

 

  public static void call() throws Exception {

    Class clzz = Class.forName("java.nio.channels.SocketChannel");

    Method open = clzz.getMethod("open");

    Object obj = open.invoke(null);

    Method isConn = obj.getClass().getMethod("isConnected");

    isConn.setAccessible(true); // OK with JDK8, fail with JDK9

    System.out.println(isConn.invoke(obj));

  }

}

 

 

java JDK9Nio

Exception in thread "main" java.lang.reflect.InaccessibleObjectException: Unable to make member of class sun.nio.ch.SocketChannelImpl accessible:  module java.base does not export sun.nio.ch to unnamed module @3857f613

        at jdk.internal.reflect.Reflection.throwInaccessibleObjectException(***@9-ea/Reflection.java:414)

        at java.lang.reflect.AccessibleObject.checkCanSetAccessible(***@9-ea/AccessibleObject.java:174)

        at java.lang.reflect.Method.checkCanSetAccessible(***@9-ea/Method.java:192)

        at java.lang.reflect.Method.setAccessible(***@9-ea/Method.java:186)

        at JDK9Nio.call(JDK9Nio.java:14)

        at JDK9Nio.main(JDK9Nio.java:6)

 

 

 

It's easy to say that the program should be re-written and the setAccessible is not necessary but this is a utility program over which the application has no control (a jython script).

What's ugly is that the internal implementation is showing through to the application.

 

Many people especially tool makers have this problem:

https://bugs.eclipse.org/bugs/show_bug.cgi?id=482318

https://netbeans.org/bugzilla/show_bug.cgi?id=258952

https://netbeans.org/bugzilla/show_bug.cgi?id=262765

http://mail.openjdk.java.net/pipermail/quality-discuss/2015-November/000468.html

https://community.oracle.com/thread/3937249

 

 
Stephen Felts
2016-09-09 15:31:16 UTC
Permalink
It’s common in enterprise applications to have multiple frameworks running in the same JVM.  For example, and application server might have Spring or Birt used in the application.  There are even cases where part of one application server might be used as a library running in the same JVM as another application server (e.g., for foreign JMS queues).

 

In JDK 8 and earlier, merging two frameworks together into the same JVM meant putting the appropriate jar files in a single classpath.  Some products even add their jar files to the CLASSPATH environment variable transparently.  So although there may be ordering problems, the problem is pretty well understood.

 

In the case of JDK 9, there are a lot of new options related to Jigsaw.  The likely scenario is that frameworks will try to hide these options using the new argument file feature.  There are a number of problems related to doing this.

 

1. Argument files support relative pathnames for things like module path, upgrade path, and patch files.  However, they are relative to the working directory.  You can’t use environment variables.  That means that the argument file needs to be programmatically generated (just unzipping an argument file won’t work).   Supporting path names relative to the containing file is the common practice of many system like ant, gradle, etc.

 

2. With the new argument syntax in JDK 9 build 132, the launcher arguments now match the command line arguments.  That’s a big step in the right direction.  However, duplicates are a fatal error.  So doing

export _JAVA_OPTIONS=-XX:VMOptionsFile=/mydir/jdk9args
java @/mydir/jdk9args something

fails. 

If one framework needs java.xml.bind and another framework uses the same module, that should just work.  If one needs bind and another needs jdk.rmic, I shouldn’t have to concatenate the two framework argument files together and manually merge the add-modules arguments together.  It seems conceptually simple to add the required lists together in java or javac.  The same is true for other things like exports, module path, upgrade module path, patch module, etc.

Assuming that all legacy usages are fixed, exports, patch module, and upgrade module path go away leaving module path.  Unlike CLASSPATH, there is no equivalent MODULEPATH to get the option off of the command line.

 

In general, it’s not clear how to manage JDK 9 options for multiple frameworks in the same JVM.  At this point, our project has banned the use of all JDK 9 options including module path.

 

I understand that some of these things were discussed two years ago.  That doesn’t fix the problem that there isn’t a good way to merge options for our customers.  Requiring a customer to cat a bunch of argument files together and then merge them manually isn’t a good solution.

 
Alan Bateman
2016-09-09 16:04:35 UTC
Permalink
2. With the new argument syntax in JDK 9 build 132, the launcher arguments now match the command line arguments. That’s a big step in the right direction. However, duplicates are a fatal error. So doing
export _JAVA_OPTIONS=-XX:VMOptionsFile=/mydir/jdk9args
fails.
If one framework needs java.xml.bind and another framework uses the same module, that should just work. If one needs bind and another needs jdk.rmic, I shouldn’t have to concatenate the two framework argument files together and manually merge the add-modules arguments together. It seems conceptually simple to add the required lists together in java or javac. The same is true for other things like exports, module path, upgrade module path, patch module, etc.
In the case of --add-modules then Mandy and Harold have changes in
review on hotspot-runtime-dev and core-libs-dev to change this into a
repeating option (it's currently last one wins). There will be follow-on
work needed in javac and other tools to get things aligned. There is
related patch coming that will propose to update the exiting repeating
options (specifically --add-exports and --add-reads) to tolerate dups.
Again, the complete solution requires updates to javac, maybe jlink, to
get everything consistent. Assuming there aren't any issues when I would
expect JEP 261 be to updated to reflect these proposals.

-Alan
Peter Levart
2016-09-09 15:53:51 UTC
Permalink
Hi Stephen,
We have an application that is running into a problem with a utility program. Below is a standalone reproducer.
The program does not import the SPI package sun.nio.ch - it isn't aware of
it, and SocketChannel.isConnected() is a public method of a public type. In
short, it does not break any law of encapsulation, so call
setAccessible(true) should be OK.
Ok, but...
import java.lang.reflect.Method;
public class JDK9Nio {
public static void main(String args[]) throws Exception {
call();
}
public static void call() throws Exception {
Class clzz = Class.forName("java.nio.channels.SocketChannel");
Method open = clzz.getMethod("open");
Object obj = open.invoke(null);
Method isConn = obj.getClass().getMethod("isConnected");
...This is a classical reflection anti-pattern. What program should be
doing to call the public and exported SocketChannel.isConnected() method
is the following:

Method isConn = SocketChannel.class.getMethod("isConnected");

obj.getClass().getMethod(...) is rarely what is desired. What you get
back is a Method object for method declared in a package-private class.
That's why setAccessible() was needed. And now in JDK 9, this class is
also in a non-exported package, so setAccessible() does not help any
more. But I see your point. It worked before and is not working any more...
isConn.setAccessible(true); // OK with JDK8, fail with JDK9
System.out.println(isConn.invoke(obj));
}
}
java JDK9Nio
at JDK9Nio.call(JDK9Nio.java:14)
at JDK9Nio.main(JDK9Nio.java:6)
It's easy to say that the program should be re-written and the setAccessible is not necessary but this is a utility program over which the application has no control (a jython script).
What's ugly is that the internal implementation is showing through to the application.
Maybe there could be a solution in supporting such sloppy programming
though. If the reflective invocation is performed to a virtual method,
then the JVM virtual dispatch chooses the method declared in the most
specific class regardless of what the Method object used is telling
about the method's declaring class. So if there exists at least one
matching method declared in the hierarchy of types comprising the
runtime type of the object upon which the method is being invoked, and
such method is accessible to the invoker, such invocation could be
allowed. The rationale is simple: if the invocation dispatches to the
same method for distinct Method objects, it should also succeed or not
succeed consistently regardless of which Method object was used to
perform the invocation.

But such access check would surely be much slower. It's better to just
fix the program (if you can :-( ).

Regards, Peter
https://bugs.eclipse.org/bugs/show_bug.cgi?id=482318
https://netbeans.org/bugzilla/show_bug.cgi?id=258952
https://netbeans.org/bugzilla/show_bug.cgi?id=262765
http://mail.openjdk.java.net/pipermail/quality-discuss/2015-November/000468.html
https://community.oracle.com/thread/3937249
Stephen Felts
2016-09-09 17:32:17 UTC
Permalink
This is a general problem with utility programs.  This isn't, unlike the sample, something where I can just re-code it to use the public interface.  The utility program isn't checking to see what is public and what is not.  It looks up the class and invokes on it, creating or opening the object.  Then the object is used for further operations.  The class is public.  We grep'ed the entire code base looking for an import or dynamic reference to the internal API but obviously didn't find it.

This is an interpreted language where obj = clzz.open();  obj.method2() is invoked in a general purpose way.

 

Class clzz = Class.forName("clzz");

Method m = clzz.getMethod("open");

Object obj = m.invoke(null);

Method m2 = obj.getClass().getMethod("method2");

m.setAccessible(true);

m2.invoke(obj);

 

If this problem isn't fixed in the JDK, then I might have code that works in JDK 9 and is broken when someone decides to re-implement something directly using an internal package in JDK 10.

 

 

 

 

-----Original Message-----
From: Peter Levart [mailto:***@gmail.com]
Sent: Friday, September 09, 2016 11:54 AM
To: Stephen Felts; jigsaw-***@openjdk.java.net
Subject: Re: JDK9 encapsulation problem

 

Hi Stephen,

 

 
Post by Stephen Felts
We have an application that is running into a problem with a utility program.  Below is a standalone reproducer.
 
  
 
The program does not import the SPI package sun.nio.ch - it isn't
aware of
 
it, and SocketChannel.isConnected() is a public method of a public
type. In
 
short, it does not break any law of encapsulation, so call
 
setAccessible(true) should be OK.
 

Ok, but...

 
Post by Stephen Felts
 
  
 
import java.lang.reflect.Method;
 
  
 
  
 
public class JDK9Nio {
 
    public static void main(String args[]) throws Exception {
 
      call();
 
    }
 
  
 
    public static void call() throws Exception {
 
      Class clzz = Class.forName("java.nio.channels.SocketChannel");
 
      Method open = clzz.getMethod("open");
 
      Object obj = open.invoke(null);
 
      Method isConn = obj.getClass().getMethod("isConnected");
 

...This is a classical reflection anti-pattern. What program should be doing to call the public and exported SocketChannel.isConnected() method is the following:

 

Method isConn = SocketChannel.class.getMethod("isConnected");

 

obj.getClass().getMethod(...) is rarely what is desired. What you get back is a Method object for method declared in a package-private class.

That's why setAccessible() was needed. And now in JDK 9, this class is also in a non-exported package, so setAccessible() does not help any more. But I see your point. It worked before and is not working any more...

 
Post by Stephen Felts
 
      isConn.setAccessible(true); // OK with JDK8, fail with JDK9
 
      System.out.println(isConn.invoke(obj));
 
    }
 
}
 
  
 
  
 
java JDK9Nio
 
Exception in thread "main"
java.lang.reflect.InaccessibleObjectException: Unable to make member
of class sun.nio.ch.SocketChannelImpl accessible:  module java.base
 
          at
jdk.internal.reflect.Reflection.throwInaccessibleObjectException(java.
 
          at
a/AccessibleObject.java:174)
 
          at
ava:192)
 
          at
 
          at JDK9Nio.call(JDK9Nio.java:14)
 
          at JDK9Nio.main(JDK9Nio.java:6)
 
  
 
  
 
  
 
It's easy to say that the program should be re-written and the setAccessible is not necessary but this is a utility program over which the application has no control (a jython script).
 
What's ugly is that the internal implementation is showing through to the application.
 

Maybe there could be a solution in supporting such sloppy programming though. If the reflective invocation is performed to a virtual method, then the JVM virtual dispatch chooses the method declared in the most specific class regardless of what the Method object used is telling about the method's declaring class. So if there exists at least one matching method declared in the hierarchy of types comprising the runtime type of the object upon which the method is being invoked, and such method is accessible to the invoker, such invocation could be allowed. The rationale is simple: if the invocation dispatches to the same method for distinct Method objects, it should also succeed or not succeed consistently regardless of which Method object was used to perform the invocation.

 

But such access check would surely be much slower. It's better to just fix the program (if you can :-( ).

 

Regards, Peter

 
Post by Stephen Felts
 
  
 
 
https://bugs.eclipse.org/bugs/show_bug.cgi?id=482318
 
https://netbeans.org/bugzilla/show_bug.cgi?id=258952
 
https://netbeans.org/bugzilla/show_bug.cgi?id=262765
 
http://mail.openjdk.java.net/pipermail/quality-discuss/2015-November/0
00468.html
 
https://community.oracle.com/thread/3937249
 
  
 
  
 

 
Peter Levart
2016-09-09 20:00:23 UTC
Permalink
Hi Stephen,

I see your problem...
This is a general problem with utility programs. This isn't, unlike
the sample, something where I can just re-code it to use the public
interface. The utility program isn't checking to see what is public
and what is not. It looks up the class and invokes on it, creating or
opening the object. Then the object is used for further operations.
The class is public. We grep'ed the entire code base looking for an
import or dynamic reference to the internal API but obviously didn't
find it.
This is an interpreted language where obj = clzz.open();
obj.method2() is invoked in a general purpose way.
Class clzz = Class.forName("clzz");
Method m = clzz.getMethod("open");
Object obj = m.invoke(null);
Method m2 = obj.getClass().getMethod("method2");
m.setAccessible(true);
m2.invoke(obj);
If this problem isn't fixed in the JDK, then I might have code that
works in JDK 9 and is broken when someone decides to re-implement
something directly using an internal package in JDK 10.
If you are in a position to change the implementation of the interpreted
language, then there might be a general way to reflectively invoke
methods on objects of unknown types that will always be the right way to
invoke them. When looking up a static method in some class 'clzz' then
the above code is ok: clzz.getMethod(name, parameterTypes...), but when
looking up an instance method to be called upon an instance of some
object, then something like the following could be used:

import java.lang.reflect.Method;
import java.lang.reflect.Modifier;
import java.util.Arrays;

public class Test {

public static void main(String[] args) throws Exception {

Class clzz = Class.forName("java.nio.channels.SocketChannel");
Method open = clzz.getMethod("open");
Object obj = open.invoke(null);

Method isConn = getAccessibleMethod(Test.class, obj.getClass(),
"isConnected");
// no need to invoke setAccessible
System.out.println(isConn.invoke(obj));
}

/**
* Returns a public method in a public class or interface in a package
* exported to the module of the caller class.
*
* @param caller the caller class that will invoke the method
* @param clazz the runtime class of the object upon which
the method
* will be called
* @param name the name of the method
* @param parameterTypes the types of method parameters
* @return a public method that is accessible to the caller class or
* null if no such method exists
*/
public static Method getAccessibleMethod(Class<?> caller, Class<?>
clazz,
String name, Class<?>...
parameterTypes)
throws NoSuchMethodException {
Method res;

// 1st lookup declared method if the class or interface is public
// and its package is exported to the caller module
if (Modifier.isPublic(clazz.getModifiers()) &&
clazz.getModule().isExported(clazz.getPackageName(),
caller.getModule()) &&
(res = findPublicMethod(clazz.getDeclaredMethods(),
name, parameterTypes)) != null) {
return res;
}

// 2nd search the superclass recursively if there is one
Class<?> superClass = clazz.getSuperclass();
if (superClass != null &&
(res = getAccessibleMethod(caller, superClass,
name, parameterTypes)) != null) {
return res;
}

// finally search the directly implemented interfaces
for (Class<?> intf : clazz.getInterfaces()) {
if ((res = getAccessibleMethod(caller, intf,
name, parameterTypes)) !=
null) {
return res;
}
}

// no luck
return null;
}

private static Method findPublicMethod(Method[] methods, String name,
Class<?>... parameterTypes) {
Method res = null;
for (Method m : methods) {
if (Modifier.isPublic(m.getModifiers()) &&
m.getName().equals(name) &&
Arrays.equals(m.getParameterTypes(), parameterTypes) &&
(res == null ||
res.getReturnType().isAssignableFrom(m.getReturnType()))) {
res = m;
}
}
return res;
}
}



This is similar logic as is used in Class::getMethod() but skips methods
that are not accessible.

Regards, Peter
-----Original Message-----
Sent: Friday, September 09, 2016 11:54 AM
Subject: Re: JDK9 encapsulation problem
Hi Stephen,
Post by Stephen Felts
We have an application that is running into a problem with a utility
program. Below is a standalone reproducer.
Post by Stephen Felts
The program does not import the SPI package sun.nio.ch - it isn't
aware of
it, and SocketChannel.isConnected() is a public method of a public
type. In
short, it does not break any law of encapsulation, so call
setAccessible(true) should be OK.
Ok, but...
Post by Stephen Felts
import java.lang.reflect.Method;
public class JDK9Nio {
public static void main(String args[]) throws Exception {
call();
}
public static void call() throws Exception {
Class clzz = Class.forName("java.nio.channels.SocketChannel");
Method open = clzz.getMethod("open");
Object obj = open.invoke(null);
Method isConn = obj.getClass().getMethod("isConnected");
...This is a classical reflection anti-pattern. What program should be
doing to call the public and exported SocketChannel.isConnected()
Method isConn = SocketChannel.class.getMethod("isConnected");
obj.getClass().getMethod(...) is rarely what is desired. What you get
back is a Method object for method declared in a package-private class.
That's why setAccessible() was needed. And now in JDK 9, this class is
also in a non-exported package, so setAccessible() does not help any
more. But I see your point. It worked before and is not working any more...
Post by Stephen Felts
isConn.setAccessible(true); // OK with JDK8, fail with JDK9
System.out.println(isConn.invoke(obj));
}
}
java JDK9Nio
Exception in thread "main"
java.lang.reflect.InaccessibleObjectException: Unable to make member
of class sun.nio.ch.SocketChannelImpl accessible: module java.base
at
jdk.internal.reflect.Reflection.throwInaccessibleObjectException(java.
at
a/AccessibleObject.java:174)
at
ava:192)
at
at JDK9Nio.call(JDK9Nio.java:14)
at JDK9Nio.main(JDK9Nio.java:6)
It's easy to say that the program should be re-written and the
setAccessible is not necessary but this is a utility program over
which the application has no control (a jython script).
Post by Stephen Felts
What's ugly is that the internal implementation is showing through
to the application.
Maybe there could be a solution in supporting such sloppy programming
though. If the reflective invocation is performed to a virtual method,
then the JVM virtual dispatch chooses the method declared in the most
specific class regardless of what the Method object used is telling
about the method's declaring class. So if there exists at least one
matching method declared in the hierarchy of types comprising the
runtime type of the object upon which the method is being invoked, and
such method is accessible to the invoker, such invocation could be
allowed. The rationale is simple: if the invocation dispatches to the
same method for distinct Method objects, it should also succeed or not
succeed consistently regardless of which Method object was used to
perform the invocation.
But such access check would surely be much slower. It's better to just
fix the program (if you can :-( ).
Regards, Peter
Post by Stephen Felts
https://bugs.eclipse.org/bugs/show_bug.cgi?id=482318
https://netbeans.org/bugzilla/show_bug.cgi?id=258952
https://netbeans.org/bugzilla/show_bug.cgi?id=262765
http://mail.openjdk.java.net/pipermail/quality-discuss/2015-November/0
00468.html
https://community.oracle.com/thread/3937249
Richard Opalka
2016-09-01 16:34:45 UTC
Permalink
Post by Alan Bateman
The issue is of course that there is lot more in picture, esp. when
you have the unnamed module (= class path) reading all modules, also
automatic modules that bridge to the class path (and so read the
unnamed module). Then add upgradable modules into the picture, ... and
you will quickly start to see there is a lot more to this, not to mind
the risk of circular delegation.
Trying not to hijack this thread (and yes I am aware it is maybe too
late for Jigsaw project to change it) but I'd like to share one idea.

What if Jigsaw would work like this?
- [A] Non-platform explicit modules (specified on --module-path) would
support versions with loaders-per-module
- [B] Version should be optional literal in module-info.java when
declaring module dependencies for such modules
- [C] UNNAMED classpath module shouldn't see non-platform explicit
modules by default
(users might use -XaddExports to export them explicitly with risk
for split-package issue and other issues)

Further I can't see the real benefit of automatic modules (they read
UNNAMED module(s) and all other explicit modules).
I am aware of two real world usecases it might solve:
1) to workaround licensing issues of dead java projects (where
consumers are disallowed to change them in any way)
2) automatic module placed on --upgrade-module-path (to allow smooth
migration for EE APIs without need to define module-info.class for them)

Considering 1) (i.e. dead wrong licensed projects should die) and 2)
here's final proposal point:
- [D] automatic modules would be supported only in
--upgrade-module-path universe without possibility to read UNNAMED modules.
Since upgradeable modules replace platform modules UNNAMED
classpath module would have to read/see them.

Richard
Alan Bateman
2016-09-01 16:58:32 UTC
Permalink
Post by Richard Opalka
- [C] UNNAMED classpath module shouldn't see non-platform explicit
modules by default
(users might use -XaddExports to export them explicitly with risk
for split-package issue and other issues)
That is how it works, except you use --add-modules to ensure that any
needed modules on the module path are resolved.
Post by Richard Opalka
Further I can't see the real benefit of automatic modules (they read
UNNAMED module(s) and all other explicit modules).
1) to workaround licensing issues of dead java projects (where
consumers are disallowed to change them in any way)
2) automatic module placed on --upgrade-module-path (to allow smooth
migration for EE APIs without need to define module-info.class for them)
Automatic modules facilitate top-level migration, you can migrate to
modules without waiting for everything that you transitively depend to
migrate. They also allow bridging to the class path - say where you move
just your direct dependences while leaving the rest on the class path.
The topic is covered in the STOMS [1] and also in the Advanced
Modularity talks at JavaOne and Devoxx last year [2].

-Alan

[1] http://openjdk.java.net/projects/jigsaw/spec/sotms/
[1] http://openjdk.java.net/projects/jigsaw/talks/
Richard Opalka
2016-09-01 19:33:45 UTC
Permalink
Post by Richard Opalka
Further I can't see the real benefit of automatic modules (they read
UNNAMED module(s) and all other explicit modules).
Post by Richard Opalka
1) to workaround licensing issues of dead java projects (where
consumers are disallowed to change them in any way)
2) automatic module placed on --upgrade-module-path (to allow smooth
migration for EE APIs without need to define module-info.class for them)
Automatic modules facilitate top-level migration, you can migrate to
modules without waiting for everything that you transitively depend to
migrate. They also allow bridging to the class path - say where you
move just your direct dependences while leaving the rest on the class
path. The topic is covered in the STOMS [1] and also in the Advanced
Modularity talks at JavaOne and Devoxx last year [2].
-Alan
[1] http://openjdk.java.net/projects/jigsaw/spec/sotms/
[1] http://openjdk.java.net/projects/jigsaw/talks/
Yes, I'm familiar and aware of these. What I meant is:

Is the benefit of incremental migration (automatic modules provide) that
valuable?
What's bad with "modularize all or nothing" kind of migration?
There would be no need for bridges to the classpath if automatic modules
would disappear.

Richard
cowwoc
2016-09-01 20:04:13 UTC
Permalink
Another possibility (not saying it's better, just putting it out there)
is to do the following:

1. Provide a tool like "javah" that would generate module-info.java for
non-modularized JAR files.
2. Provide a mechanism to "glue" the generated module-info files to the
original non-modularized JAR files without modification (as if the
files were inside the JAR file, but they aren't).
3. Developers could use the generated templates as-is or customize them
further after generation.

This way everything would be a real module and you'd get extra
customization that is currently not available with automatic modules.
This process moves the "glue" from runtime to package-time.

Gili
Post by Richard Opalka
Post by Richard Opalka
Further I can't see the real benefit of automatic modules (they read
UNNAMED module(s) and all other explicit modules).
Post by Richard Opalka
1) to workaround licensing issues of dead java projects (where
consumers are disallowed to change them in any way)
2) automatic module placed on --upgrade-module-path (to allow smooth
migration for EE APIs without need to define module-info.class for
them)
Post by Richard Opalka
Automatic modules facilitate top-level migration, you can migrate to
modules without waiting for everything that you transitively depend to
migrate. They also allow bridging to the class path - say where you
move just your direct dependences while leaving the rest on the class
path. The topic is covered in the STOMS [1] and also in the Advanced
Modularity talks at JavaOne and Devoxx last year [2].
-Alan
[1] http://openjdk.java.net/projects/jigsaw/spec/sotms/
[1] http://openjdk.java.net/projects/jigsaw/talks/
Is the benefit of incremental migration (automatic modules provide) that
valuable?
What's bad with "modularize all or nothing" kind of migration?
There would be no need for bridges to the classpath if automatic modules
would disappear.
Richard
------------------------------------------------------------------------
If you reply to this email, your message will be added to the
http://jigsaw-dev.1059479.n5.nabble.com/Multiple-versions-of-a-non-exported-dependency-tp5713364p5713397.html
To unsubscribe from Multiple versions of a non-exported dependency,
click here
<http://jigsaw-dev.1059479.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=5713364&code=Y293d29jQGJicy5kYXJrdGVjaC5vcmd8NTcxMzM2NHwxNTc0MzIxMjQ3>.
NAML
<http://jigsaw-dev.1059479.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
--
View this message in context: http://jigsaw-dev.1059479.n5.nabble.com/Multiple-versions-of-a-non-exported-dependency-tp5713364p5713398.html
Sent from the jigsaw-dev mailing list archive at Nabble.com.
Alan Bateman
2016-09-02 08:27:26 UTC
Permalink
Post by cowwoc
Another possibility (not saying it's better, just putting it out there)
1. Provide a tool like "javah" that would generate module-info.java for
non-modularized JAR files.
Look at `jdeps --gen-module-info ..`. It's a starting point to create a
source file that is easy to edit (see the slide decks).

It's not difficult to write your own ModuleFinder that reads the module
declaration from elsewhere if you really want. If you are willing to
write a module declaration for each of the 100 JAR files on your class
path then more power to you. To be honest, the feedback over the years
has almost always been that developers don't want to take responsibility
for libraries maintained by others. This usually means they won't want
to write/maintain the module declaration for libraries that they won't
own, or they don't know enough about the code in these components to be
confident that the module declaration is correct.

-Alan
Richard Opalka
2016-09-02 09:35:15 UTC
Permalink
Post by Alan Bateman
Post by cowwoc
Another possibility (not saying it's better, just putting it out there)
1. Provide a tool like "javah" that would generate module-info.java for
non-modularized JAR files.
Look at `jdeps --gen-module-info ..`. It's a starting point to create
a source file that is easy to edit (see the slide decks).
It's not difficult to write your own ModuleFinder that reads the
module declaration from elsewhere if you really want. If you are
willing to write a module declaration for each of the 100 JAR files on
your class path then more power to you. To be honest, the feedback
over the years has almost always been that developers don't want to
take responsibility for libraries maintained by others. This usually
means they won't want to write/maintain the module declaration for
libraries that they won't own, or they don't know enough about the
code in these components to be confident that the module declaration
is correct.
-Alan
Seems the main reason why automatic modules need to see the classpath
UNNAMED module
is there might be conflicting packages on that classpath which
modularization would be non trivial
(would require custom ModuleFinders, Layers and ClassLoader factories).
Yes, automatic modules help in such scenarios.

Will Java support classpath forever or there are plans to remove it?
Asking because if classpath would be removed some time in the future
such compromise (automatic modules) is just short term win affecting
many key design decisions.

Richard
Alan Bateman
2016-09-02 09:54:11 UTC
Permalink
Post by Richard Opalka
Will Java support classpath forever or there are plans to remove it?
Asking because if classpath would be removed some time in the future
such compromise (automatic modules) is just short term win affecting
many key design decisions.
I'm not aware of any proposal to drop the class path, it's just too
entrenched. Remember that we are retrofitting modules to a mature
platform and so migration, compatibility, and interoperability have to
be at the fore.

-Alan.
Richard Opalka
2016-09-01 21:02:04 UTC
Permalink
Post by Richard Opalka
Post by Alan Bateman
The issue is of course that there is lot more in picture, esp. when
you have the unnamed module (= class path) reading all modules, also
automatic modules that bridge to the class path (and so read the
unnamed module). Then add upgradable modules into the picture, ...
and you will quickly start to see there is a lot more to this, not to
mind the risk of circular delegation.
Trying not to hijack this thread (and yes I am aware it is maybe too
late for Jigsaw project to change it) but I'd like to share one idea.
What if Jigsaw would work like this?
- [A] Non-platform explicit modules (specified on --module-path)
would support versions with loaders-per-module
- [B] Version should be optional literal in module-info.java when
declaring module dependencies for such modules
- [C] UNNAMED classpath module shouldn't see non-platform explicit
modules by default
(users might use -XaddExports to export them explicitly with risk
for split-package issue and other issues)
Further I can't see the real benefit of automatic modules (they read
UNNAMED module(s) and all other explicit modules).
1) to workaround licensing issues of dead java projects (where
consumers are disallowed to change them in any way)
2) automatic module placed on --upgrade-module-path (to allow smooth
migration for EE APIs without need to define module-info.class for them)
Considering 1) (i.e. dead wrong licensed projects should die) and 2)
- [D] automatic modules would be supported only in
--upgrade-module-path universe without possibility to read UNNAMED modules.
Since upgradeable modules replace platform modules UNNAMED
classpath module would have to read/see them.
Richard
In order for Jigsaw to support multiple module versions

(not saying it should do so, just reasoning about it)

the following steps would be necessary:

* Disconnect Modularized world from Classpath world (eliminate
automatic modules)

* All platform provided modules would be "unversioned" and unique
(they'd have implicit platform version, e.g. 9-ea)

* Separate graphs of "user provided explicit modules universe"
(provided via --module-path option) from "platform modules universe"
(provided in JDK image)

* User provided explicit modules would support multiple versions via
loader-per-module (including explicit module version string in
module-info.java)

* Update module-info.java format to allow both to specify explicit
module version and to allow dependencies on explicit module versions
(e.g. "requires module.foo 1.0)

* "requires public" would be disallowed for explicitly versioned modules

* Update module graph mapping to the VM to take module version string
into account


Richard
Gregg Wonderly
2016-09-01 13:23:36 UTC
Permalink
The important detail for me, is that ClassLoader per module, with the current Class resolution scheme (this ClassLoader and whatever I can find in the parent), provides a lot of issues. The “custom ClassLoaders” or “containers like OSGi” remarks point at the “us and them” attitude that is pretty prevalent in this conversation. The majority of developers are looking for a module system that is not an “us or them” proposition. These “all or nothing” compromises are what create the “hell” that dominates conversations here. What we all want to be able to do, is write software once, target it to “THE Java platform”, and be done.

What Sun and now Oracle are continuing to do, is create more stuff that is nothing like what everyone else is doing with modularity and instead create something that is orthogonal to most peoples problem spaces and in the end creates tremendously more “work” for nothing more than compatibility with the new “JVM” environment.

The real goal here needs to be making all of the other module and container systems obsolete. Those systems should “want” to provide support for the awesome, new module system that will make in unnecessary for them to roll their own details any longer.

Yes, that is a long road and a tall measure for success. But frankly, even the lack of any visibility of the style of modules that Netbeans has used for decades makes it clear that this groups view at Oracle is extremely narrow and perhaps even more uninformed about what the community actually needs.

Gregg
It seems like there is no good reason why the application modules aren't loaded with classloader-per-module now. The platform stuff could all be in one, but the application stuff? Problems like this are going to come up a lot otherwise; let's consider making that change.
Post by Neil Bartlett
Remi,
Actually I don’t think that statically linking will work. This would produce modules that have overlapping private (non-exported) packages, and such modules also cannot be used in Java 9 on the modulepath.
I tested this in build 9-ea+126-jigsaw-nightly-h5280-20160713 by creating two modules both containing a private package org.example.util. The following exception resulted: java.lang.reflect.LayerInstantiationException: Package org.example.util in both module a and module b.
Again this could be “solved” by using custom ClassLoaders or a ClassLoader-based module system like OSGi on Java 9.
Neil
Post by Remi Forax
The other solution is to statically link the right version of slf4j inside guava and jsoup.
A tool like jarjar can be updated to merge two modular jars (merge two module-info).
cheers,
Rémi
----- Mail original -----
Envoyé: Mercredi 31 Août 2016 20:54:44
Objet: Re: Multiple versions of a non-exported dependency
Gili,
As Alex points out: your use-case can be supported in Java 9 but only with the
addition of custom ClassLoaders, or by using an existing ClassLoader-based
module system such as OSGi.
The same is also true of Java 8, and Java 7, etc.
Regards,
Neil
Post by cowwoc
I recently became aware of the fact that the Jigsaw specification declared
"version-selection" as a non-goal. While I understand how we ended up here,
I am hoping that you were able to support the following (very common)
* Module "HelloWorld" depends on modules "Guava" and "JSoup".
* Module "Guava" depends on module slf4j version 1 (requires but does not
export it).
* Module "JSoup" depends on module slf4j version 2 (requires but does not
export it).
* slf4j version 2 and is not backwards-compatible with version 1.
What happens at runtime? Will Jigsaw (out of the box, without 3rd-party
tools like Maven or OSGI) be smart enough to provide different versions of
slf4j to "Guava" and "JSoup"?
(You mean Guava/JSoup requires slf4j version 1/2 and does not "re-export" it
a.k.a. 'requires public'.)
This use case isn't possible on JDK 8 for JARs on the classpath, and it's not
- If you have two versions of a modular JAR slf4j.jar in different directories
on the modulepath, then the first one to be found will dominate, and that's
what will be resolved for both Guava and JSoup.
- If you have two modular JARs slf4j_v1.jar and slf4j_v2.jar on the modulepath,
and Guava requires slf4j_v1 and JSoup requires slf4j_v2, then launching 'java
-m HelloWorld' will fail. The boot layer will refuse to map the "same" packages
from different slf4j_v* modules to the application class loader.
The use case _is_ supported on JDK 9 for modular JARs loaded into custom loaders
of custom layers. That is, the Java Platform Module System is perfectly capable
of supporting the use case -- please see any of my "Jigsaw: Under The Hood"
presentations. The use case just isn't supported "out of the box" by the 'java'
launcher for JARs on the modulepath.
Alex
--
- DML
David M. Lloyd
2016-09-01 13:37:07 UTC
Permalink
You've missed the point I'm afraid. I'm just talking about having the
launcher use (the equivalent of)
java.lang.reflect.Layer#defineModulesWithManyLoaders instead of (the
equivalent of) java.lang.reflect.Layer#defineModulesWithOneLoader. (The
launcher actually uses the slightly lower level defineModules() method I
think, but really what I'm suggesting is to update the function to
assign new class loaders for named modules instead of reusing the same one.)
Post by Gregg Wonderly
The important detail for me, is that ClassLoader per module, with the current Class resolution scheme (this ClassLoader and whatever I can find in the parent), provides a lot of issues. The “custom ClassLoaders” or “containers like OSGi” remarks point at the “us and them” attitude that is pretty prevalent in this conversation. The majority of developers are looking for a module system that is not an “us or them” proposition. These “all or nothing” compromises are what create the “hell” that dominates conversations here. What we all want to be able to do, is write software once, target it to “THE Java platform”, and be done.
What Sun and now Oracle are continuing to do, is create more stuff that is nothing like what everyone else is doing with modularity and instead create something that is orthogonal to most peoples problem spaces and in the end creates tremendously more “work” for nothing more than compatibility with the new “JVM” environment.
The real goal here needs to be making all of the other module and container systems obsolete. Those systems should “want” to provide support for the awesome, new module system that will make in unnecessary for them to roll their own details any longer.
Yes, that is a long road and a tall measure for success. But frankly, even the lack of any visibility of the style of modules that Netbeans has used for decades makes it clear that this groups view at Oracle is extremely narrow and perhaps even more uninformed about what the community actually needs.
Gregg
It seems like there is no good reason why the application modules aren't loaded with classloader-per-module now. The platform stuff could all be in one, but the application stuff? Problems like this are going to come up a lot otherwise; let's consider making that change.
Post by Neil Bartlett
Remi,
Actually I don’t think that statically linking will work. This would produce modules that have overlapping private (non-exported) packages, and such modules also cannot be used in Java 9 on the modulepath.
I tested this in build 9-ea+126-jigsaw-nightly-h5280-20160713 by creating two modules both containing a private package org.example.util. The following exception resulted: java.lang.reflect.LayerInstantiationException: Package org.example.util in both module a and module b.
Again this could be “solved” by using custom ClassLoaders or a ClassLoader-based module system like OSGi on Java 9.
Neil
Post by Remi Forax
The other solution is to statically link the right version of slf4j inside guava and jsoup.
A tool like jarjar can be updated to merge two modular jars (merge two module-info).
cheers,
Rémi
----- Mail original -----
Envoyé: Mercredi 31 Août 2016 20:54:44
Objet: Re: Multiple versions of a non-exported dependency
Gili,
As Alex points out: your use-case can be supported in Java 9 but only with the
addition of custom ClassLoaders, or by using an existing ClassLoader-based
module system such as OSGi.
The same is also true of Java 8, and Java 7, etc.
Regards,
Neil
Post by cowwoc
I recently became aware of the fact that the Jigsaw specification declared
"version-selection" as a non-goal. While I understand how we ended up here,
I am hoping that you were able to support the following (very common)
* Module "HelloWorld" depends on modules "Guava" and "JSoup".
* Module "Guava" depends on module slf4j version 1 (requires but does not
export it).
* Module "JSoup" depends on module slf4j version 2 (requires but does not
export it).
* slf4j version 2 and is not backwards-compatible with version 1.
What happens at runtime? Will Jigsaw (out of the box, without 3rd-party
tools like Maven or OSGI) be smart enough to provide different versions of
slf4j to "Guava" and "JSoup"?
(You mean Guava/JSoup requires slf4j version 1/2 and does not "re-export" it
a.k.a. 'requires public'.)
This use case isn't possible on JDK 8 for JARs on the classpath, and it's not
- If you have two versions of a modular JAR slf4j.jar in different directories
on the modulepath, then the first one to be found will dominate, and that's
what will be resolved for both Guava and JSoup.
- If you have two modular JARs slf4j_v1.jar and slf4j_v2.jar on the modulepath,
and Guava requires slf4j_v1 and JSoup requires slf4j_v2, then launching 'java
-m HelloWorld' will fail. The boot layer will refuse to map the "same" packages
from different slf4j_v* modules to the application class loader.
The use case _is_ supported on JDK 9 for modular JARs loaded into custom loaders
of custom layers. That is, the Java Platform Module System is perfectly capable
of supporting the use case -- please see any of my "Jigsaw: Under The Hood"
presentations. The use case just isn't supported "out of the box" by the 'java'
launcher for JARs on the modulepath.
Alex
--
- DML
--
- DML
cowwoc
2016-09-01 14:28:56 UTC
Permalink
Post by Gregg Wonderly
What Sun and now Oracle are continuing to do, is create more stuff
that is nothing like what everyone else is doing with modularity and
instead create something that is orthogonal to most peoples problem
spaces and in the end creates tremendously more “work” for nothing
more than compatibility with the new “JVM” environment.
A big +1. The cost/benefit of Jigsaw without version-awareness is very
poor for end-users.

Our long-term goal should be to "import" the best ideas from existing
module systems in the Java platform over time. No one is expecting you
to do this from day one, but the fact that "requires" does not take a
constant version number from day one prevents this kind of evolution
from ever taking place.
Post by Gregg Wonderly
So I think what we have ended up with sane and not difficult to explain.
It favors migration and good interop over green field. Sure, there will
be periodic complaints when people try to deploy modules with
overlapping packages on the application module path. Anyone using Maven
Shade Plugin and the like can continue to do this. Finally, it's not
hard to create your own "launcher" that instantiates the configuration
with Layer.defineModulesWithManyLoaders if you really want.
It would help if you could publish a sample "launcher" to show people
how easy it really is. This won't resolve my outstanding problem with
the specification (lack of a version number in "requires") but at least
then we can move the discussion to whether a (simple) concrete
implementation is able to do what we want.

Gili
Post by Gregg Wonderly
The important detail for me, is that ClassLoader per module, with the
current Class resolution scheme (this ClassLoader and whatever I can
find in the parent), provides a lot of issues. The “custom
ClassLoaders” or “containers like OSGi” remarks point at the “us and
them” attitude that is pretty prevalent in this conversation. The
majority of developers are looking for a module system that is not an
“us or them” proposition. These “all or nothing” compromises are
what create the “hell” that dominates conversations here. What we all
want to be able to do, is write software once, target it to “THE Java
platform”, and be done.
What Sun and now Oracle are continuing to do, is create more stuff
that is nothing like what everyone else is doing with modularity and
instead create something that is orthogonal to most peoples problem
spaces and in the end creates tremendously more “work” for nothing
more than compatibility with the new “JVM” environment.
The real goal here needs to be making all of the other module and
container systems obsolete. Those systems should “want” to provide
support for the awesome, new module system that will make in
unnecessary for them to roll their own details any longer.
Yes, that is a long road and a tall measure for success. But frankly,
even the lack of any visibility of the style of modules that Netbeans
has used for decades makes it clear that this groups view at Oracle is
extremely narrow and perhaps even more uninformed about what the
community actually needs.
Gregg
On Sep 1, 2016, at 7:29 AM, David M. Lloyd <[hidden email]
It seems like there is no good reason why the application modules
aren't loaded with classloader-per-module now. The platform stuff
could all be in one, but the application stuff? Problems like this
are going to come up a lot otherwise; let's consider making that change.
Post by Neil Bartlett
Remi,
Actually I don’t think that statically linking will work. This
would produce modules that have overlapping private (non-exported)
packages, and such modules also cannot be used in Java 9 on the
modulepath.
Post by Neil Bartlett
I tested this in build 9-ea+126-jigsaw-nightly-h5280-20160713 by
creating two modules both containing a private package
java.lang.reflect.LayerInstantiationException: Package
org.example.util in both module a and module b.
Post by Neil Bartlett
Again this could be “solved” by using custom ClassLoaders or a
ClassLoader-based module system like OSGi on Java 9.
Post by Neil Bartlett
Neil
On 31 Aug 2016, at 20:28, Remi Forax <[hidden email]
The other solution is to statically link the right version of
slf4j inside guava and jsoup.
Post by Neil Bartlett
A tool like jarjar can be updated to merge two modular jars (merge
two module-info).
Post by Neil Bartlett
cheers,
Rémi
----- Mail original -----
De: "Neil Bartlett" <[hidden email]
</user/SendEmail.jtp?type=node&node=5713387&i=2>>
Post by Neil Bartlett
À: [hidden email]
</user/SendEmail.jtp?type=node&node=5713387&i=3>, "Alex Buckley"
<[hidden email] </user/SendEmail.jtp?type=node&node=5713387&i=4>>
Post by Neil Bartlett
Cc: "ZML-OpenJDK-Jigsaw-Developers" <[hidden email]
</user/SendEmail.jtp?type=node&node=5713387&i=5>>
Post by Neil Bartlett
Envoyé: Mercredi 31 Août 2016 20:54:44
Objet: Re: Multiple versions of a non-exported dependency
Gili,
As Alex points out: your use-case can be supported in Java 9 but
only with the
Post by Neil Bartlett
addition of custom ClassLoaders, or by using an existing
ClassLoader-based
Post by Neil Bartlett
module system such as OSGi.
The same is also true of Java 8, and Java 7, etc.
Regards,
Neil
On 31 Aug 2016, at 19:29, Alex Buckley <[hidden email]
Post by cowwoc
I recently became aware of the fact that the Jigsaw
specification declared
Post by Neil Bartlett
Post by cowwoc
"version-selection" as a non-goal. While I understand how we
ended up here,
Post by Neil Bartlett
Post by cowwoc
I am hoping that you were able to support the following (very
common)
Post by Neil Bartlett
Post by cowwoc
* Module "HelloWorld" depends on modules "Guava" and "JSoup".
* Module "Guava" depends on module slf4j version 1 (requires
but does not
Post by Neil Bartlett
Post by cowwoc
export it).
* Module "JSoup" depends on module slf4j version 2 (requires
but does not
Post by Neil Bartlett
Post by cowwoc
export it).
* slf4j version 2 and is not backwards-compatible with version 1.
What happens at runtime? Will Jigsaw (out of the box, without
3rd-party
Post by Neil Bartlett
Post by cowwoc
tools like Maven or OSGI) be smart enough to provide different
versions of
Post by Neil Bartlett
Post by cowwoc
slf4j to "Guava" and "JSoup"?
(You mean Guava/JSoup requires slf4j version 1/2 and does not
"re-export" it
Post by Neil Bartlett
a.k.a. 'requires public'.)
This use case isn't possible on JDK 8 for JARs on the classpath,
and it's not
Post by Neil Bartlett
- If you have two versions of a modular JAR slf4j.jar in
different directories
Post by Neil Bartlett
on the modulepath, then the first one to be found will dominate,
and that's
Post by Neil Bartlett
what will be resolved for both Guava and JSoup.
- If you have two modular JARs slf4j_v1.jar and slf4j_v2.jar on
the modulepath,
Post by Neil Bartlett
and Guava requires slf4j_v1 and JSoup requires slf4j_v2, then
launching 'java
Post by Neil Bartlett
-m HelloWorld' will fail. The boot layer will refuse to map the
"same" packages
Post by Neil Bartlett
from different slf4j_v* modules to the application class loader.
The use case _is_ supported on JDK 9 for modular JARs loaded
into custom loaders
Post by Neil Bartlett
of custom layers. That is, the Java Platform Module System is
perfectly capable
Under The Hood"
Post by Neil Bartlett
presentations. The use case just isn't supported "out of the
box" by the 'java'
Post by Neil Bartlett
launcher for JARs on the modulepath.
Alex
--
- DML
------------------------------------------------------------------------
If you reply to this email, your message will be added to the
http://jigsaw-dev.1059479.n5.nabble.com/Multiple-versions-of-a-non-exported-dependency-tp5713364p5713387.html
To unsubscribe from Multiple versions of a non-exported dependency,
click here
<http://jigsaw-dev.1059479.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=5713364&code=Y293d29jQGJicy5kYXJrdGVjaC5vcmd8NTcxMzM2NHwxNTc0MzIxMjQ3>.
NAML
<http://jigsaw-dev.1059479.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
--
View this message in context: http://jigsaw-dev.1059479.n5.nabble.com/Multiple-versions-of-a-non-exported-dependency-tp5713364p5713390.html
Sent from the jigsaw-dev mailing list archive at Nabble.com.
Gregg Wonderly
2016-09-01 15:12:08 UTC
Permalink
Why I was referring to is how will modules find classes from other modules? How will the different version of the same package namespace that Gili was talking about be hidden? In other words, you can only make them all visible by using a single class loader, unless there is a DAG of dependencies that naturally falls out, because all we have is ClassLoaders parent as the pointer to other interesting detail.

How will definedModulesWithManyLoaders make it possible for all of the correct details to be visible?

Some explicit example detail:

module A uses l4j logging in module B

module C uses l4j logging in module D

module E is a standalone module

A and C exchange data objects defined in E

+— B — A
|
+ E
|
+— D — C

This graph you can draw today with ClassLoader parent references. But, there are more complicated graphs that fall out from less simple needs. It’s this specific issue of still not trying to provide support for versions and for arranging the module dependencies more explicitly through the use of version as part of the package name namespace which Gili is trying to speak to I feel.

Since B and D are the same package name, you have to hide them from each other in separate class loaders, obviously. But the Graph from C to E and A to E is not always direct and can in some cases not be possible with a single instance of the jar. That breaks static class data designs if you create two copies in separate loaders.

Gregg
You've missed the point I'm afraid. I'm just talking about having the launcher use (the equivalent of) java.lang.reflect.Layer#defineModulesWithManyLoaders instead of (the equivalent of) java.lang.reflect.Layer#defineModulesWithOneLoader. (The launcher actually uses the slightly lower level defineModules() method I think, but really what I'm suggesting is to update the function to assign new class loaders for named modules instead of reusing the same one.)
Post by Gregg Wonderly
The important detail for me, is that ClassLoader per module, with the current Class resolution scheme (this ClassLoader and whatever I can find in the parent), provides a lot of issues. The “custom ClassLoaders” or “containers like OSGi” remarks point at the “us and them” attitude that is pretty prevalent in this conversation. The majority of developers are looking for a module system that is not an “us or them” proposition. These “all or nothing” compromises are what create the “hell” that dominates conversations here. What we all want to be able to do, is write software once, target it to “THE Java platform”, and be done.
What Sun and now Oracle are continuing to do, is create more stuff that is nothing like what everyone else is doing with modularity and instead create something that is orthogonal to most peoples problem spaces and in the end creates tremendously more “work” for nothing more than compatibility with the new “JVM” environment.
The real goal here needs to be making all of the other module and container systems obsolete. Those systems should “want” to provide support for the awesome, new module system that will make in unnecessary for them to roll their own details any longer.
Yes, that is a long road and a tall measure for success. But frankly, even the lack of any visibility of the style of modules that Netbeans has used for decades makes it clear that this groups view at Oracle is extremely narrow and perhaps even more uninformed about what the community actually needs.
Gregg
It seems like there is no good reason why the application modules aren't loaded with classloader-per-module now. The platform stuff could all be in one, but the application stuff? Problems like this are going to come up a lot otherwise; let's consider making that change.
Post by Neil Bartlett
Remi,
Actually I don’t think that statically linking will work. This would produce modules that have overlapping private (non-exported) packages, and such modules also cannot be used in Java 9 on the modulepath.
I tested this in build 9-ea+126-jigsaw-nightly-h5280-20160713 by creating two modules both containing a private package org.example.util. The following exception resulted: java.lang.reflect.LayerInstantiationException: Package org.example.util in both module a and module b.
Again this could be “solved” by using custom ClassLoaders or a ClassLoader-based module system like OSGi on Java 9.
Neil
Post by Remi Forax
The other solution is to statically link the right version of slf4j inside guava and jsoup.
A tool like jarjar can be updated to merge two modular jars (merge two module-info).
cheers,
Rémi
----- Mail original -----
Envoyé: Mercredi 31 Août 2016 20:54:44
Objet: Re: Multiple versions of a non-exported dependency
Gili,
As Alex points out: your use-case can be supported in Java 9 but only with the
addition of custom ClassLoaders, or by using an existing ClassLoader-based
module system such as OSGi.
The same is also true of Java 8, and Java 7, etc.
Regards,
Neil
Post by cowwoc
I recently became aware of the fact that the Jigsaw specification declared
"version-selection" as a non-goal. While I understand how we ended up here,
I am hoping that you were able to support the following (very common)
* Module "HelloWorld" depends on modules "Guava" and "JSoup".
* Module "Guava" depends on module slf4j version 1 (requires but does not
export it).
* Module "JSoup" depends on module slf4j version 2 (requires but does not
export it).
* slf4j version 2 and is not backwards-compatible with version 1.
What happens at runtime? Will Jigsaw (out of the box, without 3rd-party
tools like Maven or OSGI) be smart enough to provide different versions of
slf4j to "Guava" and "JSoup"?
(You mean Guava/JSoup requires slf4j version 1/2 and does not "re-export" it
a.k.a. 'requires public'.)
This use case isn't possible on JDK 8 for JARs on the classpath, and it's not
- If you have two versions of a modular JAR slf4j.jar in different directories
on the modulepath, then the first one to be found will dominate, and that's
what will be resolved for both Guava and JSoup.
- If you have two modular JARs slf4j_v1.jar and slf4j_v2.jar on the modulepath,
and Guava requires slf4j_v1 and JSoup requires slf4j_v2, then launching 'java
-m HelloWorld' will fail. The boot layer will refuse to map the "same" packages
from different slf4j_v* modules to the application class loader.
The use case _is_ supported on JDK 9 for modular JARs loaded into custom loaders
of custom layers. That is, the Java Platform Module System is perfectly capable
of supporting the use case -- please see any of my "Jigsaw: Under The Hood"
presentations. The use case just isn't supported "out of the box" by the 'java'
launcher for JARs on the modulepath.
Alex
--
- DML
--
- DML
Gregg Wonderly
2016-09-01 15:34:04 UTC
Permalink
Trying not to distract this conversation away from the details… In the end, the concept of “micro services” and things like “Jini” discovery can allow smaller distribution of systems in single class loaders without conflicts. But still, there is the static data design issue. I have a Jini discovery based desktop application that I’ve used for more than a decade to “find” services on my network, download their proxy and associated UI, and use them. Those applications are all separate class loaders and do not interact inside of my desktop application. If they need to interact, they do so via their services which interact with other services through discovery of that service, download of the proxy and calls out through the jointly know interface/class name which was used in the lookup criteria. All of this has explicit version management because the Jar file names are well known and include version numbers in them.

In Jini, there is a lot of class loading happening with downloaded code, but it is a well formed tree structure inside of my desktop application because each application is using a specific version of jars in its setting of the exported class details.

Jigsaw is orthogonal to what I am doing with Jini because Jigsaw is about starting the application, not extending the application as the Jini discovery and downloads do. However, I am trying to point out how the direct references to version details in the design and operation of the system allow for multiple versions to coexist trivially and be managed explicitly to help the system continue to function correctly.

If I have an application deployed twice, with two different sets of jars because of version changes, and my desktop application picks one of them at random to use, it won’t matter if I in fact try to open both instances by explicitly knowing how to do that because the class loader design keeps them separated and the exported class loader detail identifies the specifics need to make each function as a separate application within my desktop container.

I don’t know what base of users and what paradigms of deployments the Jigsaw team considered. It’s clear that they focused first on modularity of the JVM. It’s not obvious that how other module systems unrelated to pluggable functionality that the JVM is trying to separate were considered. The separation of detail that is not referenced is a simple modularization detail. The isolation of detail that should not be referenced is a different modularization detail which just doesn’t seem to be completely considered for any detail other than “exposure” as opposed to “compatibility” which is what “Version” points to.

Gregg
Post by Gregg Wonderly
Why I was referring to is how will modules find classes from other modules? How will the different version of the same package namespace that Gili was talking about be hidden? In other words, you can only make them all visible by using a single class loader, unless there is a DAG of dependencies that naturally falls out, because all we have is ClassLoaders parent as the pointer to other interesting detail.
How will definedModulesWithManyLoaders make it possible for all of the correct details to be visible?
module A uses l4j logging in module B
module C uses l4j logging in module D
module E is a standalone module
A and C exchange data objects defined in E
+— B — A
|
+ E
|
+— D — C
This graph you can draw today with ClassLoader parent references. But, there are more complicated graphs that fall out from less simple needs. It’s this specific issue of still not trying to provide support for versions and for arranging the module dependencies more explicitly through the use of version as part of the package name namespace which Gili is trying to speak to I feel.
Since B and D are the same package name, you have to hide them from each other in separate class loaders, obviously. But the Graph from C to E and A to E is not always direct and can in some cases not be possible with a single instance of the jar. That breaks static class data designs if you create two copies in separate loaders.
Gregg
You've missed the point I'm afraid. I'm just talking about having the launcher use (the equivalent of) java.lang.reflect.Layer#defineModulesWithManyLoaders instead of (the equivalent of) java.lang.reflect.Layer#defineModulesWithOneLoader. (The launcher actually uses the slightly lower level defineModules() method I think, but really what I'm suggesting is to update the function to assign new class loaders for named modules instead of reusing the same one.)
Post by Gregg Wonderly
The important detail for me, is that ClassLoader per module, with the current Class resolution scheme (this ClassLoader and whatever I can find in the parent), provides a lot of issues. The “custom ClassLoaders” or “containers like OSGi” remarks point at the “us and them” attitude that is pretty prevalent in this conversation. The majority of developers are looking for a module system that is not an “us or them” proposition. These “all or nothing” compromises are what create the “hell” that dominates conversations here. What we all want to be able to do, is write software once, target it to “THE Java platform”, and be done.
What Sun and now Oracle are continuing to do, is create more stuff that is nothing like what everyone else is doing with modularity and instead create something that is orthogonal to most peoples problem spaces and in the end creates tremendously more “work” for nothing more than compatibility with the new “JVM” environment.
The real goal here needs to be making all of the other module and container systems obsolete. Those systems should “want” to provide support for the awesome, new module system that will make in unnecessary for them to roll their own details any longer.
Yes, that is a long road and a tall measure for success. But frankly, even the lack of any visibility of the style of modules that Netbeans has used for decades makes it clear that this groups view at Oracle is extremely narrow and perhaps even more uninformed about what the community actually needs.
Gregg
It seems like there is no good reason why the application modules aren't loaded with classloader-per-module now. The platform stuff could all be in one, but the application stuff? Problems like this are going to come up a lot otherwise; let's consider making that change.
Post by Neil Bartlett
Remi,
Actually I don’t think that statically linking will work. This would produce modules that have overlapping private (non-exported) packages, and such modules also cannot be used in Java 9 on the modulepath.
I tested this in build 9-ea+126-jigsaw-nightly-h5280-20160713 by creating two modules both containing a private package org.example.util. The following exception resulted: java.lang.reflect.LayerInstantiationException: Package org.example.util in both module a and module b.
Again this could be “solved” by using custom ClassLoaders or a ClassLoader-based module system like OSGi on Java 9.
Neil
Post by Remi Forax
The other solution is to statically link the right version of slf4j inside guava and jsoup.
A tool like jarjar can be updated to merge two modular jars (merge two module-info).
cheers,
Rémi
----- Mail original -----
Envoyé: Mercredi 31 Août 2016 20:54:44
Objet: Re: Multiple versions of a non-exported dependency
Gili,
As Alex points out: your use-case can be supported in Java 9 but only with the
addition of custom ClassLoaders, or by using an existing ClassLoader-based
module system such as OSGi.
The same is also true of Java 8, and Java 7, etc.
Regards,
Neil
Post by cowwoc
I recently became aware of the fact that the Jigsaw specification declared
"version-selection" as a non-goal. While I understand how we ended up here,
I am hoping that you were able to support the following (very common)
* Module "HelloWorld" depends on modules "Guava" and "JSoup".
* Module "Guava" depends on module slf4j version 1 (requires but does not
export it).
* Module "JSoup" depends on module slf4j version 2 (requires but does not
export it).
* slf4j version 2 and is not backwards-compatible with version 1.
What happens at runtime? Will Jigsaw (out of the box, without 3rd-party
tools like Maven or OSGI) be smart enough to provide different versions of
slf4j to "Guava" and "JSoup"?
(You mean Guava/JSoup requires slf4j version 1/2 and does not "re-export" it
a.k.a. 'requires public'.)
This use case isn't possible on JDK 8 for JARs on the classpath, and it's not
- If you have two versions of a modular JAR slf4j.jar in different directories
on the modulepath, then the first one to be found will dominate, and that's
what will be resolved for both Guava and JSoup.
- If you have two modular JARs slf4j_v1.jar and slf4j_v2.jar on the modulepath,
and Guava requires slf4j_v1 and JSoup requires slf4j_v2, then launching 'java
-m HelloWorld' will fail. The boot layer will refuse to map the "same" packages
from different slf4j_v* modules to the application class loader.
The use case _is_ supported on JDK 9 for modular JARs loaded into custom loaders
of custom layers. That is, the Java Platform Module System is perfectly capable
of supporting the use case -- please see any of my "Jigsaw: Under The Hood"
presentations. The use case just isn't supported "out of the box" by the 'java'
launcher for JARs on the modulepath.
Alex
--
- DML
--
- DML
Loading...