[PATCH 3/4] lint: Export 'run-checkers'.

[Alex Kost]

* guix/scripts/lint.scm (run-checkers): Export. Make 'checkers'
argument optional.
---
guix/scripts/lint.scm | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/guix/scripts/lint.scm b/guix/scripts/lint.scm
index 3b4ff72..b1707ad 100644
--- a/guix/scripts/lint.scm
+++ b/guix/scripts/lint.scm
@@ -62,6 +62,7 @@
check-source-file-name
check-license
check-formatting
+ run-checkers

%checkers
lint-checker
@@ -709,8 +710,8 @@ or a list thereof")
(description "Look for formatting issues in the source")
(check check-formatting))))

-(define (run-checkers package checkers)
- ;; Run the given CHECKERS on PACKAGE.
+(define* (run-checkers package #:optional (checkers %checkers))
+ "Run the given CHECKERS on PACKAGE."
(let ((tty? (isatty? (current-error-port)))
(name (package-full-name package)))
(for-each (lambda (checker)

[Alex Kost]

* emacs/guix-devel.el (guix-devel-with-definition): New macro.
(guix-devel-build-package-definition): Use it.
(guix-devel-font-lock-keywords): New variable.
---
emacs/guix-devel.el | 22 +++++++++++++++++++---
1 file changed, 19 insertions(+), 3 deletions(-)

diff --git a/emacs/guix-devel.el b/emacs/guix-devel.el
index 8a6fc1e..ee7629f 100644
--- a/emacs/guix-devel.el
+++ b/emacs/guix-devel.el
@@ -88,12 +88,21 @@ Interactively, use the module defined by the current scheme file."
(guix-devel-setup-repl repl)
(push process guix-devel-repl-processes))))

+(defmacro guix-devel-with-definition (def-var &rest body)
+ "Run BODY with the current guile definition bound to DEF-VAR.
+Bind DEF-VAR variable to the name of the current top-level
+definition, setup the current REPL, use the current module, and
+run BODY."
+ (declare (indent 1) (debug (symbolp body)))
+ `(let ((,def-var (guix-guile-current-definition)))
+ (guix-devel-setup-repl-maybe)
+ (guix-devel-use-modules (guix-guile-current-module))
+ ,@body))
+
(defun guix-devel-build-package-definition ()
"Build a package defined by the current top-level variable definition."
(interactive)
- (let ((def (guix-guile-current-definition)))
- (guix-devel-setup-repl-maybe)
- (guix-devel-use-modules (guix-guile-current-module))
+ (guix-devel-with-definition def
(when (or (not guix-operation-confirm)
(guix-operation-prompt (format "Build '%s'?" def)))
(guix-geiser-eval-in-repl
@@ -184,6 +193,13 @@ bindings:
(when guix-devel-activate-mode
(guix-devel-mode)))

+
+(defvar guix-devel-font-lock-keywords
+ (eval-when-compile
+ `((,(rx "(" (group "guix-devel-with-definition") symbol-end) . 1))))
+
+(font-lock-add-keywords 'emacs-lisp-mode guix-devel-font-lock-keywords)
+
(provide 'guix-devel)

;;; guix-devel.el ends here

[Ludovic Courtès]

Post by Alex Kost
* emacs/guix-devel.el (guix-devel-with-definition): New macro.
(guix-devel-build-package-definition): Use it.
(guix-devel-font-lock-keywords): New variable.

LGTM, thanks.

Ludo’.

[Alex Kost]

Suggested by Ludovic Courtès <***@gnu.org>.

* emacs/guix-devel.el (guix-devel-setup-repl): Add a workaround for
'guix-warning-port'.
(guix-devel-lint-package): New command.
(guix-devel-keys-map): Add key binding for it.
* doc/emacs.texi (Emacs Development): Document it.
---
doc/emacs.texi | 4 ++++
emacs/guix-devel.el | 21 ++++++++++++++++++---
2 files changed, 22 insertions(+), 3 deletions(-)

diff --git a/doc/emacs.texi b/doc/emacs.texi
index 00f86f3..18009fa 100644
--- a/doc/emacs.texi
+++ b/doc/emacs.texi
@@ -667,6 +667,10 @@ this command---for example, with @kbd{C-M-x} (@pxref{To eval or not to
eval,,, geiser, Geiser User Manual})
(@code{guix-devel-build-package-definition}).

+@item C-c . l
+Lint (check) a package defined by the current variable definition
+(@pxref{Invoking guix lint}) (@code{guix-devel-lint-package}).
+
@item C-c . s
Download a source of the package defined by the current variable
definition and print its SHA256 hash. This command has the same meaning
diff --git a/emacs/guix-devel.el b/emacs/guix-devel.el
index a8167a0..4d380ce 100644
--- a/emacs/guix-devel.el
+++ b/emacs/guix-devel.el
@@ -71,11 +71,14 @@ Interactively, use the module defined by the current scheme file."
(guix-devel-use-modules "(guix monad-repl)"
"(guix packages)"
"(guix scripts)"
- "(guix store)")
- ;; Without this workaround, the build output disappears. See
+ "(guix store)"
+ "(guix ui)")
+ ;; Without this workaround, the warning/build output disappears. See
;; <https://github.com/jaor/geiser/issues/83> for details.
(guix-geiser-eval-in-repl
- "(current-build-output-port (current-error-port))"
+ "(begin
+ (guix-warning-port (current-warning-port))
+ (current-build-output-port (current-error-port)))"
repl 'no-history 'no-display))

(defvar guix-devel-repl-processes nil
@@ -126,6 +129,17 @@ Use this function to compute SHA256 hash of the package source."
(format "(guix-download (origin-uri (package-source %s)))"
def)))))

+(defun guix-devel-lint-package ()
+ "Check the current package.
+See Info node `(guix) Invoking guix lint' for details."
+ (interactive)
+ (guix-devel-with-definition def
+ (guix-devel-use-modules "(guix scripts lint)")
+ (when (or (not guix-operation-confirm)
+ (y-or-n-p (format "Lint '%s' package?" def)))
+ (guix-geiser-eval-in-repl
+ (format "(run-checkers %s)" def)))))
+

;;; Font-lock

@@ -164,6 +178,7 @@ to find 'modify-phases' keywords."
(defvar guix-devel-keys-map
(let ((map (make-sparse-keymap)))
(define-key map (kbd "b") 'guix-devel-build-package-definition)
+ (define-key map (kbd "l") 'guix-devel-lint-package)
(define-key map (kbd "s") 'guix-devel-download-package-source)
(define-key map (kbd "k") 'guix-devel-copy-module-as-kill)
(define-key map (kbd "u") 'guix-devel-use-module)

[Ludovic Courtès]

Post by Alex Kost
* emacs/guix-devel.el (guix-devel-setup-repl): Add a workaround for
'guix-warning-port'.
(guix-devel-lint-package): New command.
(guix-devel-keys-map): Add key binding for it.
* doc/emacs.texi (Emacs Development): Document it.

OK!

Ludo'.

[Alex Kost]

* emacs/guix-devel.el (guix-devel-setup-repl): Use (guix packages) module.
(guix-devel-download-package-source): New command.
(guix-devel-keys-map): Add key binding for it.
* doc/emacs.texi (Emacs Development): Document it.
---
doc/emacs.texi | 6 ++++++
emacs/guix-devel.el | 14 ++++++++++++++
2 files changed, 20 insertions(+)

diff --git a/doc/emacs.texi b/doc/emacs.texi
index b6f2701..00f86f3 100644
--- a/doc/emacs.texi
+++ b/doc/emacs.texi
@@ -667,6 +667,12 @@ this command---for example, with @kbd{C-M-x} (@pxref{To eval or not to
eval,,, geiser, Geiser User Manual})
(@code{guix-devel-build-package-definition}).

+@item C-c . s
+Download a source of the package defined by the current variable
+definition and print its SHA256 hash. This command has the same meaning
+as running @code{guix download} on the package source (@pxref{Invoking
+guix download}) (@code{guix-devel-download-package-source}).
+
@end table

Unluckily, there is a limitation related to long-running REPL commands.
diff --git a/emacs/guix-devel.el b/emacs/guix-devel.el
index ee7629f..a8167a0 100644
--- a/emacs/guix-devel.el
+++ b/emacs/guix-devel.el
@@ -69,6 +69,7 @@ Interactively, use the module defined by the current scheme file."
(defun guix-devel-setup-repl (&optional repl)
"Setup REPL for using `guix-devel-...' commands."
(guix-devel-use-modules "(guix monad-repl)"
+ "(guix packages)"
"(guix scripts)"
"(guix store)")
;; Without this workaround, the build output disappears. See
@@ -113,6 +114,18 @@ run BODY."
guix-use-substitutes)
"#:dry-run?" (guix-guile-boolean guix-dry-run)))))))

+(defun guix-devel-download-package-source ()
+ "Download the source of the current package.
+Use this function to compute SHA256 hash of the package source."
+ (interactive)
+ (guix-devel-with-definition def
+ (guix-devel-use-modules "(guix scripts download)")
+ (when (or (not guix-operation-confirm)
+ (y-or-n-p (format "Download '%s' package source?" def)))
+ (guix-geiser-eval-in-repl
+ (format "(guix-download (origin-uri (package-source %s)))"
+ def)))))
+

;;; Font-lock

@@ -151,6 +164,7 @@ to find 'modify-phases' keywords."
(defvar guix-devel-keys-map
(let ((map (make-sparse-keymap)))
(define-key map (kbd "b") 'guix-devel-build-package-definition)
+ (define-key map (kbd "s") 'guix-devel-download-package-source)
(define-key map (kbd "k") 'guix-devel-copy-module-as-kill)
(define-key map (kbd "u") 'guix-devel-use-module)
map)

[Ludovic Courtès]

Post by Alex Kost
* emacs/guix-devel.el (guix-devel-setup-repl): Use (guix packages) module.
(guix-devel-download-package-source): New command.
(guix-devel-keys-map): Add key binding for it.
* doc/emacs.texi (Emacs Development): Document it.

[...]

Post by Alex Kost
+(defun guix-devel-download-package-source ()
+ "Download the source of the current package.
+Use this function to compute SHA256 hash of the package source."
+ (interactive)
+ (guix-devel-with-definition def
+ (guix-devel-use-modules "(guix scripts download)")
+ (when (or (not guix-operation-confirm)
+ (y-or-n-p (format "Download '%s' package source?" def)))
+ (guix-geiser-eval-in-repl
+ (format "(guix-download (origin-uri (package-source %s)))"
+ def)))))

What about instead building the ‘package-source-derivation’ of the
package? That way, that would do the exact same thing as ‘guix build
-S’ and would work not only with ‘url-fetch’ but also with the other
things.

WDYT?

Thanks,
Ludo’.

[Alex Kost]

Post by Ludovic Courtès

Post by Alex Kost
* emacs/guix-devel.el (guix-devel-setup-repl): Use (guix packages) module.
(guix-devel-download-package-source): New command.
(guix-devel-keys-map): Add key binding for it.
* doc/emacs.texi (Emacs Development): Document it.

[...]

Post by Alex Kost
+(defun guix-devel-download-package-source ()
+ "Download the source of the current package.
+Use this function to compute SHA256 hash of the package source."
+ (interactive)
+ (guix-devel-with-definition def
+ (guix-devel-use-modules "(guix scripts download)")
+ (when (or (not guix-operation-confirm)
+ (y-or-n-p (format "Download '%s' package source?" def)))
+ (guix-geiser-eval-in-repl
+ (format "(guix-download (origin-uri (package-source %s)))"
+ def)))))

What about instead building the ‘package-source-derivation’ of the
package? That way, that would do the exact same thing as ‘guix build
-S’ and would work not only with ‘url-fetch’ but also with the other
things.
WDYT?

The goal of this command is to display a hash. At first I also thought
about building a package source and then to calculate the hash of the
store file, but this way will lead to the wrong hashes for "snippet"-ed
origins. Or do I miss anything?

[Ludovic Courtès]

Post by Alex Kost

Post by Ludovic Courtès

Post by Alex Kost
* emacs/guix-devel.el (guix-devel-setup-repl): Use (guix packages) module.
(guix-devel-download-package-source): New command.
(guix-devel-keys-map): Add key binding for it.
* doc/emacs.texi (Emacs Development): Document it.

[...]

Post by Alex Kost
+(defun guix-devel-download-package-source ()
+ "Download the source of the current package.
+Use this function to compute SHA256 hash of the package source."
+ (interactive)
+ (guix-devel-with-definition def
+ (guix-devel-use-modules "(guix scripts download)")
+ (when (or (not guix-operation-confirm)
+ (y-or-n-p (format "Download '%s' package source?" def)))
+ (guix-geiser-eval-in-repl
+ (format "(guix-download (origin-uri (package-source %s)))"
+ def)))))

What about instead building the ‘package-source-derivation’ of the
package? That way, that would do the exact same thing as ‘guix build
-S’ and would work not only with ‘url-fetch’ but also with the other
things.
WDYT?

The goal of this command is to display a hash.

So this would be something one would use as they write a package
definition, to fill in the ‘sha256’ field, right?

In that case, I would suggest something based on the URL at point rather
than the origin at point.

However, if this is “too convenient”, I’m afraid this would give an
incentive to not check OpenPGP signatures when they are available.

Post by Alex Kost
At first I also thought about building a package source and then to
calculate the hash of the store file, but this way will lead to the
wrong hashes for "snippet"-ed origins. Or do I miss anything?

Well, I think bindings for ‘package-source-derivation’ would also be
useful, but IIUC this is not what you had in mind.

Thanks,
Ludo’.

[Alex Kost]

Post by Ludovic Courtès

Post by Alex Kost

Post by Ludovic Courtès

Post by Alex Kost
* emacs/guix-devel.el (guix-devel-setup-repl): Use (guix packages) module.
(guix-devel-download-package-source): New command.
(guix-devel-keys-map): Add key binding for it.
* doc/emacs.texi (Emacs Development): Document it.

[...]

Post by Alex Kost
+(defun guix-devel-download-package-source ()
+ "Download the source of the current package.
+Use this function to compute SHA256 hash of the package source."
+ (interactive)
+ (guix-devel-with-definition def
+ (guix-devel-use-modules "(guix scripts download)")
+ (when (or (not guix-operation-confirm)
+ (y-or-n-p (format "Download '%s' package source?" def)))
+ (guix-geiser-eval-in-repl
+ (format "(guix-download (origin-uri (package-source %s)))"
+ def)))))

What about instead building the ‘package-source-derivation’ of the
package? That way, that would do the exact same thing as ‘guix build
-S’ and would work not only with ‘url-fetch’ but also with the other
things.
WDYT?

The goal of this command is to display a hash.

So this would be something one would use as they write a package
definition, to fill in the ‘sha256’ field, right?

Exactly. For example, here is how I update a package using Geiser:

- I modify its 'version' field,
- "C-M-x" to reevaluate the package definition,
- "C-c . s" to download the new source and show its hash,
- replace the old hash with the new one,
- and finally "C-c . b" to build the new version of package.

Post by Ludovic Courtès
In that case, I would suggest something based on the URL at point rather
than the origin at point.

It's neither URL at point, nor origin at point: it's like "C-c . b" but
to download the source of the current package definition instead of
building it.

Also I don't understand how it can be based on the URL at point as
instead of the full URL, we have something like this:

(uri (string-append "http://.../foo-" version ".tar.xz"))

So currently to use "guix download", at first you need to manually
construct the full URL. I find this inconvenient, that's why I want to
have a command to download a source of the current package and to
display its hash.

Post by Ludovic Courtès
However, if this is “too convenient”, I’m afraid this would give an
incentive to not check OpenPGP signatures when they are available.

Sorry, I have no idea what it means :-(

Post by Ludovic Courtès

Post by Alex Kost
At first I also thought about building a package source and then to
calculate the hash of the store file, but this way will lead to the
wrong hashes for "snippet"-ed origins. Or do I miss anything?

Well, I think bindings for ‘package-source-derivation’ would also be
useful, but IIUC this is not what you had in mind.

If you mean to make a command to build the current package source, it
can be done, although I don't see how it can be useful.

[Ludovic Courtès]

Post by Alex Kost

Post by Ludovic Courtès

Post by Alex Kost

Post by Ludovic Courtès

Post by Alex Kost
* emacs/guix-devel.el (guix-devel-setup-repl): Use (guix packages) module.
(guix-devel-download-package-source): New command.
(guix-devel-keys-map): Add key binding for it.
* doc/emacs.texi (Emacs Development): Document it.

[...]

Post by Alex Kost
+(defun guix-devel-download-package-source ()
+ "Download the source of the current package.
+Use this function to compute SHA256 hash of the package source."
+ (interactive)
+ (guix-devel-with-definition def
+ (guix-devel-use-modules "(guix scripts download)")
+ (when (or (not guix-operation-confirm)
+ (y-or-n-p (format "Download '%s' package source?" def)))
+ (guix-geiser-eval-in-repl
+ (format "(guix-download (origin-uri (package-source %s)))"
+ def)))))

What about instead building the ‘package-source-derivation’ of the
package? That way, that would do the exact same thing as ‘guix build
-S’ and would work not only with ‘url-fetch’ but also with the other
things.
WDYT?

The goal of this command is to display a hash.

So this would be something one would use as they write a package
definition, to fill in the ‘sha256’ field, right?

- I modify its 'version' field,
- "C-M-x" to reevaluate the package definition,
- "C-c . s" to download the new source and show its hash,
- replace the old hash with the new one,
- and finally "C-c . b" to build the new version of package.

OK, nice.

Post by Alex Kost

Post by Ludovic Courtès
In that case, I would suggest something based on the URL at point rather
than the origin at point.

It's neither URL at point, nor origin at point: it's like "C-c . b" but
to download the source of the current package definition instead of
building it.
Also I don't understand how it can be based on the URL at point as
(uri (string-append "http://.../foo-" version ".tar.xz"))
So currently to use "guix download", at first you need to manually
construct the full URL. I find this inconvenient, that's why I want to
have a command to download a source of the current package and to
display its hash.

Yes, that make sense.

The problem I had in mind was that, when writing a new package
definition from scratch, you don’t know the SHA256 yet, but ‘sha256’ is
a required field of ‘origin’. So one would have to write a fake
‘sha256’ value just for the sake of being able to use that feature,
which seems odd to me.

See what I mean? How would you handle such a case?

Post by Alex Kost

Post by Ludovic Courtès
However, if this is “too convenient”, I’m afraid this would give an
incentive to not check OpenPGP signatures when they are available.

Sorry, I have no idea what it means :-(

When upstream digitally signs its source code tarballs, packagers should
check those signatures to authenticate the code they have.

If the tool makes it too easy to fill out the ‘sha256’ field without
going through the trouble of downloading the ‘.sig’ file and checking
it, then people will have an incentive not to check those signatures.

Post by Alex Kost

Post by Ludovic Courtès

Post by Alex Kost
At first I also thought about building a package source and then to
calculate the hash of the store file, but this way will lead to the
wrong hashes for "snippet"-ed origins. Or do I miss anything?

Well, I think bindings for ‘package-source-derivation’ would also be
useful, but IIUC this is not what you had in mind.

If you mean to make a command to build the current package source, it
can be done, although I don't see how it can be useful.

It’s occasionally useful, similar to ‘guix build -S’ or the “Show”
button in package info buffers, except that it would operate directly on
the package at point.

WDYT?

Thanks,
Ludo’.

[Alex Kost]

[...]

Post by Ludovic Courtès

Post by Alex Kost

Post by Ludovic Courtès
In that case, I would suggest something based on the URL at point rather
than the origin at point.

It's neither URL at point, nor origin at point: it's like "C-c . b" but
to download the source of the current package definition instead of
building it.
Also I don't understand how it can be based on the URL at point as
(uri (string-append "http://.../foo-" version ".tar.xz"))
So currently to use "guix download", at first you need to manually
construct the full URL. I find this inconvenient, that's why I want to
have a command to download a source of the current package and to
display its hash.

Yes, that make sense.
The problem I had in mind was that, when writing a new package
definition from scratch, you don’t know the SHA256 yet, but ‘sha256’ is
a required field of ‘origin’. So one would have to write a fake
‘sha256’ value just for the sake of being able to use that feature,
which seems odd to me.
See what I mean? How would you handle such a case?

I don't see a problem here, since a fake sha256 may be any string, for
example "" (an empty string). Also I believe people begin to write a
new package from some template, so you have a working skeleton of future
package with all required fields from the very beginning. Then after
filling an origin 'uri', you could "C-c . s" to download the source and
get its hash.

Post by Ludovic Courtès

Post by Alex Kost

Post by Ludovic Courtès
However, if this is “too convenient”, I’m afraid this would give an
incentive to not check OpenPGP signatures when they are available.

Sorry, I have no idea what it means :-(

When upstream digitally signs its source code tarballs, packagers should
check those signatures to authenticate the code they have.
If the tool makes it too easy to fill out the ‘sha256’ field without
going through the trouble of downloading the ‘.sig’ file and checking
it, then people will have an incentive not to check those signatures.

Oh, now I see what you mean. Well, I don't know, I think if a user has
a habbit to check a signature, he will check it anyway; and if not, then
not. Besides, at first a packager needs to find an URL of the source
tarball, so he will meet a signature anyway, if it exists. So it's up
to him if he checks it or not.

(I confess that I never check signatures)

Post by Ludovic Courtès

Post by Alex Kost

Post by Ludovic Courtès

Post by Alex Kost
At first I also thought about building a package source and then to
calculate the hash of the store file, but this way will lead to the
wrong hashes for "snippet"-ed origins. Or do I miss anything?

Well, I think bindings for ‘package-source-derivation’ would also be
useful, but IIUC this is not what you had in mind.

If you mean to make a command to build the current package source, it
can be done, although I don't see how it can be useful.

It’s occasionally useful, similar to ‘guix build -S’ or the “Show”
button in package info buffers, except that it would operate directly on
the package at point.
WDYT?

Indeed, I agree. So if you don't mind the “download” command, then
since there will be 2 commands for working with the source of the
current package, what about the following key bindings for them:

"C-c . s d" - to download the source (to know its hash)
"C-c . s b" - to build it

[Mark H Weaver]

Post by Alex Kost

Post by Ludovic Courtès

Post by Alex Kost

Post by Ludovic Courtès
However, if this is “too convenient”, I’m afraid this would give an
incentive to not check OpenPGP signatures when they are available.

Sorry, I have no idea what it means :-(

When upstream digitally signs its source code tarballs, packagers should
check those signatures to authenticate the code they have.
If the tool makes it too easy to fill out the ‘sha256’ field without
going through the trouble of downloading the ‘.sig’ file and checking
it, then people will have an incentive not to check those signatures.

Oh, now I see what you mean. Well, I don't know, I think if a user has
a habbit to check a signature, he will check it anyway; and if not, then
not.

I share Ludovic's concern. It is a serious problem if packagers fail to
check signatures. We should not provide mechanisms that encourage such
behavior. It jeopardizes the security of every user of those packages.

IMO, we should rather be going in the other direction, to formalize and
automate the checking of signatures. IMO, our 'origin' objects should
include a set of fingerprints of acceptable GPG signing keys for that
package, as well as information on how to find the signature (in cases
where it cannot be guessed).

This would have several beneficial effects:

* If the packager downloaded a key belonging to a man-in-the-middle
(quite possible given that we rarely have a validated chain of trust
to the developer), then that bad key will be stored in our git repo
for all to see, allowing someone to notice that it's the wrong key.

* When the package is later updated, it will not be possible for a new
man-in-the-middle attack to be made on us. If a new signing key is
used, we cannot fail to notice it. It will raise a red flag and we
can investigate.

* It would strongly encourage packagers to do these checks, and make it
obvious to reviewers or users when the packager failed to do so. It
would also make it easy to find unsigned packages, so that we can
encourage upstream to start signing the packages, at least for the
most important ones.

Also, our linter should download and check the signature, so that it's
easy for others to independently check the verification done by the
original packager.

What do you think?

Mark

[Christopher Allan Webber]

Post by Mark H Weaver

Post by Alex Kost

Post by Ludovic Courtès

Post by Alex Kost

Post by Ludovic Courtès
However, if this is “too convenient”, I’m afraid this would give an
incentive to not check OpenPGP signatures when they are available.

Sorry, I have no idea what it means :-(

When upstream digitally signs its source code tarballs, packagers should
check those signatures to authenticate the code they have.
If the tool makes it too easy to fill out the ‘sha256’ field without
going through the trouble of downloading the ‘.sig’ file and checking
it, then people will have an incentive not to check those signatures.

Oh, now I see what you mean. Well, I don't know, I think if a user has
a habbit to check a signature, he will check it anyway; and if not, then
not.

I share Ludovic's concern. It is a serious problem if packagers fail to
check signatures. We should not provide mechanisms that encourage such
behavior. It jeopardizes the security of every user of those packages.
IMO, we should rather be going in the other direction, to formalize and
automate the checking of signatures. IMO, our 'origin' objects should
include a set of fingerprints of acceptable GPG signing keys for that
package, as well as information on how to find the signature (in cases
where it cannot be guessed).
* If the packager downloaded a key belonging to a man-in-the-middle
(quite possible given that we rarely have a validated chain of trust
to the developer), then that bad key will be stored in our git repo
for all to see, allowing someone to notice that it's the wrong key.
* When the package is later updated, it will not be possible for a new
man-in-the-middle attack to be made on us. If a new signing key is
used, we cannot fail to notice it. It will raise a red flag and we
can investigate.
* It would strongly encourage packagers to do these checks, and make it
obvious to reviewers or users when the packager failed to do so. It
would also make it easy to find unsigned packages, so that we can
encourage upstream to start signing the packages, at least for the
most important ones.
Also, our linter should download and check the signature, so that it's
easy for others to independently check the verification done by the
original packager.
What do you think?
Mark

This sounds great to me!

[Andreas Enge]

This sounds all very good. In practice, the difference would unfortunately be
only slight: Most packages have no signature, mainly the gnu packages do. But
it would be useful for the cases where signatures exist, and show our
commitment to security.

Andreas

[Ludovic Courtès]

Post by Mark H Weaver
IMO, we should rather be going in the other direction, to formalize and
automate the checking of signatures. IMO, our 'origin' objects should
include a set of fingerprints of acceptable GPG signing keys for that
package, as well as information on how to find the signature (in cases
where it cannot be guessed).

I thought about it (it used to be in TODO), but this design has several
problems. One of them is that a keyring is needed if we are to verify
signatures; where do we get it?

Another one is that, inherently, the daemon already handles integrity
checks for fixed-output derivations, and authentication should really be
made beforehand by the packager.

Most of the time the authentication model is trust-on-first-download:
The packager fetches upstream’s public key when they first download a
tarball (so this particular phase is subject to MiTM), and subsequent
downloads are checked against the key that’s already in the packager’s
keyring.

Post by Mark H Weaver
* If the packager downloaded a key belonging to a man-in-the-middle
(quite possible given that we rarely have a validated chain of trust
to the developer), then that bad key will be stored in our git repo
for all to see, allowing someone to notice that it's the wrong key.

So you’re suggesting to put the keyring under version control in a way,
right?

It sounds like a good idea, provided we at least the Git commits that
add/modify the keyring are signed. There’s a couple of issues: unless
it can be stored in a plain-text format, it’ll be hard to audit changes
that are made; it would quickly end up containing hundreds of public
keys, so there’s a scalability issue: how do we keep it up-to-date? how
do we decide who’s authorized to update it, etc.?

First step would be to be able to do something meaningful with the GNU
keyring. It’s at ftp.gnu.org/gnu/gnu-keyring.gpg, but it’s currently
unsigned; previous attempts to do something about it haven’t gone
anywhere, but we could try again.

Post by Mark H Weaver
* When the package is later updated, it will not be possible for a new
man-in-the-middle attack to be made on us. If a new signing key is
used, we cannot fail to notice it. It will raise a red flag and we
can investigate.
* It would strongly encourage packagers to do these checks, and make it
obvious to reviewers or users when the packager failed to do so. It
would also make it easy to find unsigned packages, so that we can
encourage upstream to start signing the packages, at least for the
most important ones.

As Andreas notes, outside of gnu.org, savannah.gnu.org, kernel.org, and
a few others, it’s very frequent for packages to not be signed at all.

Post by Mark H Weaver
Also, our linter should download and check the signature, so that it's
easy for others to independently check the verification done by the
original packager.

‘guix import gnu’ and ‘guix refresh’ already do that, but only for GNU
packages.

I have been thinking that ‘guix download’ could also automatically look
for .sig, .sign, and .asc files. That’s easily done, and already an
improvement.

Thoughts?

Ludo’.

[Mark H Weaver]

Post by Ludovic Courtès

Post by Mark H Weaver
IMO, we should rather be going in the other direction, to formalize and
automate the checking of signatures. IMO, our 'origin' objects should
include a set of fingerprints of acceptable GPG signing keys for that
package, as well as information on how to find the signature (in cases
where it cannot be guessed).

I thought about it (it used to be in TODO), but this design has several
problems. One of them is that a keyring is needed if we are to verify
signatures; where do we get it?

We can download the keys from a public keyserver. However, checking
these signatures would be optional. The sha256 hashes would continue to
be the authentication method used by default. My goal is to improve
security by making it easy for others to independently verify the
signatures, and to keep an institutional knowledge of which keys are
authorized to sign for which packages.

Post by Ludovic Courtès
Another one is that, inherently, the daemon already handles integrity
checks for fixed-output derivations, and authentication should really be
made beforehand by the packager.

Agreed. My proposal would not change that.

Post by Ludovic Courtès
The packager fetches upstream’s public key when they first download a
tarball (so this particular phase is subject to MiTM), and subsequent
downloads are checked against the key that’s already in the packager’s
keyring.

Right, and every time the package is updated, that's another opportunity
for a MiTM attack. My proposal would fix that problem. It would also
allow MiTM attacks to be detected later, because the bad key would be
recorded in our git repository for all to see.

Post by Ludovic Courtès

Post by Mark H Weaver
* If the packager downloaded a key belonging to a man-in-the-middle
(quite possible given that we rarely have a validated chain of trust
to the developer), then that bad key will be stored in our git repo
for all to see, allowing someone to notice that it's the wrong key.

So you’re suggesting to put the keyring under version control in a way,
right?

No, I suggest storing only the key fingerprints.

Post by Ludovic Courtès
It sounds like a good idea, provided we at least the Git commits that
add/modify the keyring are signed. There’s a couple of issues: unless
it can be stored in a plain-text format, it’ll be hard to audit changes
that are made; it would quickly end up containing hundreds of public
keys, so there’s a scalability issue: how do we keep it up-to-date? how
do we decide who’s authorized to update it, etc.?

Indeed, storing the keyring would have many thorny issues, which is why
I didn't suggest it.

Post by Ludovic Courtès
First step would be to be able to do something meaningful with the GNU
keyring. It’s at ftp.gnu.org/gnu/gnu-keyring.gpg, but it’s currently
unsigned; previous attempts to do something about it haven’t gone
anywhere, but we could try again.

The GNU keyring could be useful but is not sufficient by itself, because
unless we record which keys are authorized to sign which packages, and
use that information during checking, the result is that _any_ key can
sign _any_ package, and the security of the entire system is reduced to
that of the weakest key on the GNU keyring. This is not good.

Post by Ludovic Courtès

Post by Mark H Weaver
* When the package is later updated, it will not be possible for a new
man-in-the-middle attack to be made on us. If a new signing key is
used, we cannot fail to notice it. It will raise a red flag and we
can investigate.
* It would strongly encourage packagers to do these checks, and make it
obvious to reviewers or users when the packager failed to do so. It
would also make it easy to find unsigned packages, so that we can
encourage upstream to start signing the packages, at least for the
most important ones.

As Andreas notes, outside of gnu.org, savannah.gnu.org, kernel.org, and
a few others, it’s very frequent for packages to not be signed at all.

That's true, but I don't see why this argues against my proposal.

Post by Ludovic Courtès

Post by Mark H Weaver
Also, our linter should download and check the signature, so that it's
easy for others to independently check the verification done by the
original packager.

‘guix import gnu’ and ‘guix refresh’ already do that, but only for GNU
packages.

The current mechanism is very weak. _Any_ key on your keyring could
sign the package, and it will be happy with that, thus reducing the
security of the check to that of the weakest key on your keyring. Also,
each person who performs the check downloads the key from the keyserver
independently, typically without any verification that we haven't been
MiTM'd.

Thanks,
Mark

[Leo Famulari]

Post by Mark H Weaver

Post by Ludovic Courtès
The packager fetches upstream’s public key when they first download a
tarball (so this particular phase is subject to MiTM), and subsequent
downloads are checked against the key that’s already in the packager’s
keyring.

Right, and every time the package is updated, that's another opportunity
for a MiTM attack. My proposal would fix that problem. It would also
allow MiTM attacks to be detected later, because the bad key would be
recorded in our git repository for all to see.

I have been wondering about this issue as I created package and I share
Mark's concern. The current system relies on packagers to get it right
for every update.

[Ludovic Courtès]

Post by Mark H Weaver

Post by Ludovic Courtès

Post by Mark H Weaver
IMO, we should rather be going in the other direction, to formalize and
automate the checking of signatures. IMO, our 'origin' objects should
include a set of fingerprints of acceptable GPG signing keys for that
package, as well as information on how to find the signature (in cases
where it cannot be guessed).

I thought about it (it used to be in TODO), but this design has several
problems. One of them is that a keyring is needed if we are to verify
signatures; where do we get it?

We can download the keys from a public keyserver. However, checking
these signatures would be optional. The sha256 hashes would continue to
be the authentication method used by default. My goal is to improve
security by making it easy for others to independently verify the
signatures, and to keep an institutional knowledge of which keys are
authorized to sign for which packages.

I agree with the goals. I’m just wondering how this can be achieved in
practice in a way that scales.

I would suggest starting with GNU, because it’s easier. If we manage to
get something that works and scales to all of GNU, then let’s extend it.

Post by Mark H Weaver

Post by Ludovic Courtès
The packager fetches upstream’s public key when they first download a
tarball (so this particular phase is subject to MiTM), and subsequent
downloads are checked against the key that’s already in the packager’s
keyring.

Right, and every time the package is updated, that's another opportunity
for a MiTM attack.

Not if the package is updated by the same person that retrieved the key
initially (similar to TOFU with SSH.)

Post by Mark H Weaver

Post by Ludovic Courtès

Post by Mark H Weaver
* If the packager downloaded a key belonging to a man-in-the-middle
(quite possible given that we rarely have a validated chain of trust
to the developer), then that bad key will be stored in our git repo
for all to see, allowing someone to notice that it's the wrong key.

So you’re suggesting to put the keyring under version control in a way,
right?

No, I suggest storing only the key fingerprints.

How do we know which key is authorized for which package?

Even for GNU, we’d have to ask the FSF, and obviously the set of
authorized keys for each package keeps changing. So we’d need the FSF
to provide us with a database/server to answer questions such as “which
public keys could sign for GNU Foo at this date?” in a secure way.

Post by Mark H Weaver

Post by Ludovic Courtès

Post by Mark H Weaver
Also, our linter should download and check the signature, so that it's
easy for others to independently check the verification done by the
original packager.

‘guix import gnu’ and ‘guix refresh’ already do that, but only for GNU
packages.

The current mechanism is very weak. _Any_ key on your keyring could
sign the package, and it will be happy with that, thus reducing the
security of the check to that of the weakest key on your keyring. Also,
each person who performs the check downloads the key from the keyserver
independently, typically without any verification that we haven't been
MiTM'd.

Sure, I agree. But that’s what we have.

When I download a package, the best I can do is to download its .sig and
check it, optionally adding the corresponding public key to my keyring
if it’s missing. And that’s it.

There’s a handful of packages for which I know personally the person who
uploads tarballs and their associated key. For the rest, I just have to
trust the .sig to originate from an “authorized person”.

What you suggest would be perfect but, if I understand it correctly,
it’s far from reality. There’s not a single project I know of that
publishes the list of public keys authorized to sign its tarballs. Even
if they did, we’d need a way to authenticate that list.

Does that make sense? Sounds very tricky to me!

Thanks,
Ludo’.

[Ludovic Courtès]

Post by Ludovic Courtès
Even for GNU, we’d have to ask the FSF, and obviously the set of
authorized keys for each package keeps changing. So we’d need the FSF
to provide us with a database/server to answer questions such as “which
public keys could sign for GNU Foo at this date?” in a secure way.

Actually I see that GSRC already maintains per-package keyrings.

How is this maintained, Brandon? That is, where do you get information
on which keys to put in the keyring, etc.?

Thanks,
Ludo’.

PS: For context, see the thread starting at
<https://lists.gnu.org/archive/html/guix-devel/2015-10/msg00115.html>.

[Brandon Invergo]

Hi everyone,

Post by Ludovic Courtès
Actually I see that GSRC already maintains per-package keyrings.
How is this maintained, Brandon? That is, where do you get information
on which keys to put in the keyring, etc.?

Admittedly, it's not ideal. When we first add a package, we make a
keyring for it based on whatever information is available to us.
Sometimes the public key is listed in the release announcement. Other
times, we just have to grab the public key of whatever we see the
package was signed with. Obviously, that's not very secure since it
could have been signed by an attacker. However usually this process is
only performed when adding a new (to GNU) package. Then, if the
signature-checking process ever fails on future releases, I actually
look into it. Sometimes, no public key is available in any of the key
servers as far as I can tell. In those cases, we ignore the signature.

As I said, this isn't ideal and I would welcome any ideas for a unified
solution for both GSRC and Guix. I could swear that previously a
keyring of the GNU maintainers was made available by the FSF somewhere
but I cannot find it. One minimal thing that I can do is to send out a
request to all maintainers to make the public key (or at least its id)
available on the package's home page...we still probably wouldn't have
100% coverage, but it's a start.

-brandon

[Brandon Invergo]

I could swear that previously a keyring of the GNU maintainers was
made available by the FSF somewhere but I cannot find it.

http://ftp.gnu.org/gnu/gnu-keyring.gpg

Of course, this doesn't help for those GNU packages that are not made
available on ftp.gnu.org.

-brandon

[Ludovic Courtès]

Post by Brandon Invergo

I could swear that previously a keyring of the GNU maintainers was
made available by the FSF somewhere but I cannot find it.

http://ftp.gnu.org/gnu/gnu-keyring.gpg

The main issue is that this file is not signed (that would have to be
done by the person responsible for FTP uploads, presumably an FSF
employee.)

A second issue, as Mark wrote, is that it is coarse-grain: it does not
tell exactly which package a given key corresponds to.

However, this package → keys mapping necessarily exists somewhere. I
think we should ask the FSF to publish it and provide a way to
authenticate it.

WDYT?

Ludo’.

[Brandon Invergo]

However, this package → keys mapping necessarily exists somewhere.  I
think we should ask the FSF to publish it and provide a way to
authenticate it.
WDYT?

If they would be willing to publish it, I think it would be a very good
idea.

-brandon

[Ludovic Courtès]

Post by Brandon Invergo

However, this package → keys mapping necessarily exists somewhere.  I
think we should ask the FSF to publish it and provide a way to
authenticate it.
WDYT?

If they would be willing to publish it, I think it would be a very good
idea.

We would need the help and support of someone from the GNU Advisory
Committee, I guess…

:-)

Ludo’.

[Brandon Invergo]

Post by Ludovic Courtès
We would need the help and support of someone from the GNU Advisory
Committee, I guess

:-)

OK I think I know just the guy for the job. :)

I'll start the discussion.  Feel free to ping me to check on the
progress!

-brandon

[Ludovic Courtès]

Post by Brandon Invergo

Post by Ludovic Courtès
We would need the help and support of someone from the GNU Advisory
Committee, I guess…
:-)

OK I think I know just the guy for the job. :)
I'll start the discussion.  Feel free to ping me to check on the
progress!

Excellent, thank you!

Ludo’.

[Ludovic Courtès]

Post by Brandon Invergo
Hi everyone,

Post by Ludovic Courtès
Actually I see that GSRC already maintains per-package keyrings.
How is this maintained, Brandon? That is, where do you get information
on which keys to put in the keyring, etc.?

Admittedly, it's not ideal. When we first add a package, we make a
keyring for it based on whatever information is available to us.
Sometimes the public key is listed in the release announcement. Other
times, we just have to grab the public key of whatever we see the
package was signed with. Obviously, that's not very secure since it
could have been signed by an attacker. However usually this process is
only performed when adding a new (to GNU) package. Then, if the
signature-checking process ever fails on future releases, I actually
look into it. Sometimes, no public key is available in any of the key
servers as far as I can tell. In those cases, we ignore the signature.

OK. That’s roughly what Mark suggests that we do in Guix, an
improvement over the current situation.

Thanks for your feedback!

Ludo’.

[Christopher Allan Webber]

Post by Ludovic Courtès

Post by Brandon Invergo
Hi everyone,

Post by Ludovic Courtès
Actually I see that GSRC already maintains per-package keyrings.
How is this maintained, Brandon? That is, where do you get information
on which keys to put in the keyring, etc.?

Admittedly, it's not ideal. When we first add a package, we make a
keyring for it based on whatever information is available to us.
Sometimes the public key is listed in the release announcement. Other
times, we just have to grab the public key of whatever we see the
package was signed with. Obviously, that's not very secure since it
could have been signed by an attacker. However usually this process is
only performed when adding a new (to GNU) package. Then, if the
signature-checking process ever fails on future releases, I actually
look into it. Sometimes, no public key is available in any of the key
servers as far as I can tell. In those cases, we ignore the signature.

OK. That’s roughly what Mark suggests that we do in Guix, an
improvement over the current situation.
Thanks for your feedback!
Ludo’.

Extra reasons to want to do signature based verification:
http://www.zdnet.com/article/hacker-hundreds-were-tricked-into-installing-linux-mint-backdoor/

... be careful out there!
- Chris

[Alex Vong]

Post by Ludovic Courtès
What you suggest would be perfect but, if I understand it correctly,
it’s far from reality. There’s not a single project I know of that
publishes the list of public keys authorized to sign its tarballs. Even
if they did, we’d need a way to authenticate that list.

I think <https://www.kernel.org/signature.html> has listed all their
public keys used to sign their releases. This seems to be quite a neat
way of doing things. But you're right that there is no easy way to
authenticate that list.

[Mark H Weaver]

Post by Ludovic Courtès
What you suggest would be perfect but, if I understand it correctly,
it’s far from reality.

What exactly is far from reality? I did not speak about what _is_, but
rather about what we _should_ do to improve things.

Post by Ludovic Courtès
There’s not a single project I know of that publishes the list of
public keys authorized to sign its tarballs. Even if they did, we’d
need a way to authenticate that list.

Tor publishes a list, but I agree that it's rare. So, in practice, we
would populate the list of authorized signing keys from the *.sig files
we find. So, we'd replace the current practice of "trust on first file
download" with "trust on first key download for each new signing key".
It's obviously not perfect, but it's better than what we have now:

* There would be fewer opportunities for MiTM attacks, because typically
signing keys change less frequently than new upstream releases are
made.

* We have better tools and practices for verifying the authenticity of
GPG key fingerprints, e.g. key signing parties, the web of trust, key
fingerprints printed on business cards, etc.

* I expect that people will be more motivated to make an effort to
verify the set of authorized signing key fingerprints. Speaking for
myself, I would consider it well worth my time to walk up to an
upstream developer at a conference and ask them which keys are
authorized to sign their releases, and to get copies of the key
fingerprints. However, if I asked them instead for the SHA256 hash of
their latest release, they'd probably look at me funny. They'd be
unlikely to have that information handy, and frankly it would be a bad
approach because we'd have to do it all over again for their next
release.

It seems to me that you're rejecting this proposal because you see that
it's not yet practical to do the job perfectly. In my view, it is
enough that it would be a significant improvement over what we have now.

Post by Ludovic Courtès
* If the packager downloaded a key belonging to a man-in-the-middle
(quite possible given that we rarely have a validated chain of trust
to the developer), then that bad key will be stored in our git repo
for all to see, allowing someone to notice that it's the wrong key.
* When the package is later updated, it will not be possible for a new
man-in-the-middle attack to be made on us. If a new signing key is
used, we cannot fail to notice it. It will raise a red flag and we
can investigate.
* It would strongly encourage packagers to do these checks, and make it
obvious to reviewers or users when the packager failed to do so. It
would also make it easy to find unsigned packages, so that we can
encourage upstream to start signing the packages, at least for the
most important ones.

Do you disagree that my proposal would have these benefits?

Thanks,
Mark

[Ludovic Courtès]

Post by Mark H Weaver

Post by Ludovic Courtès
What you suggest would be perfect but, if I understand it correctly,
it’s far from reality.

What exactly is far from reality? I did not speak about what _is_, but
rather about what we _should_ do to improve things.

Sorry, that was poorly worded. What I meant to say is that the
practices of upstream developers are on average far more “sloppy” than
we, as downstream, would end up setting up.

A particular problem I see, is that we would end up maintaining a list
of authorized fingerprints when 90% of upstream developers do not
actually provide that information.

This is not to say we shouldn’t try. Rather, what I’m saying is that we
should make sure we don’t over-engineer a solution whose benefits would,
in effect, be significantly limited by what upstream actually does.

[...]

Post by Mark H Weaver
It seems to me that you're rejecting this proposal because you see that
it's not yet practical to do the job perfectly.

I’m not rejecting the proposal. My main concern is the implementation
of the proposal; I think it’s easy to over-engineer it, or end up with
something that doesn’t scale, or provides wrong assurance.

Post by Mark H Weaver
In my view, it is enough that it would be a significant improvement
over what we have now. In my first message in this thread, I listed

Post by Ludovic Courtès
* If the packager downloaded a key belonging to a man-in-the-middle
(quite possible given that we rarely have a validated chain of trust
to the developer), then that bad key will be stored in our git repo
for all to see, allowing someone to notice that it's the wrong key.

I agree that’s an improvement, because it provides an audit trail.

In practice it may still be hard to determine whether this is the
“wrong” key, because only upstream can tell.

Post by Mark H Weaver

Post by Ludovic Courtès
* When the package is later updated, it will not be possible for a new
man-in-the-middle attack to be made on us. If a new signing key is
used, we cannot fail to notice it. It will raise a red flag and we
can investigate.

Agreed.

Post by Mark H Weaver

Post by Ludovic Courtès
* It would strongly encourage packagers to do these checks, and make it
obvious to reviewers or users when the packager failed to do so. It
would also make it easy to find unsigned packages, so that we can
encourage upstream to start signing the packages, at least for the
most important ones.

Agreed.

Post by Mark H Weaver
Do you disagree that my proposal would have these benefits?

No!

So, how do we do this? :-)

Something like
<http://git.savannah.gnu.org/cgit/guix.git/tree/TODO?id=3476ded934dc0beab1801d7fcdcc37b5c17bbf01#n52>,
where the signature fields are optional?

Do we force signature checks as part of the derivation builds, or do we
make it off-line (say, in ‘guix lint’), or both?

How do we handle projects that maintain a list of authorized public
keys, like GNU, Tor, and kernel.org?

Thanks,
Ludo’.

[Rastus Vernon]

When the code comes from a Git repository, it's possible for the source
tarballs not to be signed (or not to exist at all), but for the tags in
the repository to be signed at each release. In these cases, there is
no signature file, but this is still a way for packagers to verify the
authenticity of the source code.

Post by Ludovic Courtès
When I download a package, the best I can do is to download its .sig
and check it, optionally adding the corresponding public key to my
keyring if it’s missing. And that’s it.

A small improvement is to download the signature from another location
(for example a public library, or using a proxy or Tor) and compare the
two to verify that they are the same. This makes a MiTM attack between
the server and the computer the signature is downloaded to nearly
impossible. The server could still be compromised, so this is not as
good as having a trusted keyring, but it's a significant improvement.

[Mark H Weaver]

Post by Rastus Vernon
When the code comes from a Git repository, it's possible for the source
tarballs not to be signed (or not to exist at all), but for the tags in
the repository to be signed at each release. In these cases, there is
no signature file, but this is still a way for packagers to verify the
authenticity of the source code.

Good point!

Post by Rastus Vernon

Post by Ludovic Courtès
When I download a package, the best I can do is to download its .sig
and check it, optionally adding the corresponding public key to my
keyring if it’s missing. And that’s it.

A small improvement is to download the signature from another location
(for example a public library, or using a proxy or Tor) and compare the
two to verify that they are the same. This makes a MiTM attack between
the server and the computer the signature is downloaded to nearly
impossible.

It is indeed "a small improvement", but I strongly disagree that it's
"nearly impossible". A compromised router or switch near the server
could successfully perform MiTM attacks even if Tor is used. In most
cases this is well within the capabilities of the NSA or GCHQ, as is
breaking into the server itself.

Mark

[Alex Kost]

Post by Mark H Weaver

Post by Alex Kost

Post by Ludovic Courtès

Post by Alex Kost

Post by Ludovic Courtès
However, if this is “too convenient”, I’m afraid this would give an
incentive to not check OpenPGP signatures when they are available.

Sorry, I have no idea what it means :-(

When upstream digitally signs its source code tarballs, packagers should
check those signatures to authenticate the code they have.
If the tool makes it too easy to fill out the ‘sha256’ field without
going through the trouble of downloading the ‘.sig’ file and checking
it, then people will have an incentive not to check those signatures.

Oh, now I see what you mean. Well, I don't know, I think if a user has
a habbit to check a signature, he will check it anyway; and if not, then
not.

I share Ludovic's concern. It is a serious problem if packagers fail to
check signatures. We should not provide mechanisms that encourage such
behavior. It jeopardizes the security of every user of those packages.

OK, apparently I underestimate security issues, thanks.

[Ludovic Courtès]

Post by Alex Kost
[...]

Post by Ludovic Courtès

Post by Alex Kost

Post by Ludovic Courtès
In that case, I would suggest something based on the URL at point rather
than the origin at point.

It's neither URL at point, nor origin at point: it's like "C-c . b" but
to download the source of the current package definition instead of
building it.
Also I don't understand how it can be based on the URL at point as
(uri (string-append "http://.../foo-" version ".tar.xz"))
So currently to use "guix download", at first you need to manually
construct the full URL. I find this inconvenient, that's why I want to
have a command to download a source of the current package and to
display its hash.

Yes, that make sense.
The problem I had in mind was that, when writing a new package
definition from scratch, you don’t know the SHA256 yet, but ‘sha256’ is
a required field of ‘origin’. So one would have to write a fake
‘sha256’ value just for the sake of being able to use that feature,
which seems odd to me.
See what I mean? How would you handle such a case?

I don't see a problem here, since a fake sha256 may be any string,

Not really, since ‘base32’ is a macro that checks its argument at
expansion time. So in practice one cannot C-M-x a package with a random
base32 string.

Post by Alex Kost
for example "" (an empty string). Also I believe people begin to
write a new package from some template, so you have a working skeleton
of future package with all required fields from the very beginning.
Then after filling an origin 'uri', you could "C-c . s" to download
the source and get its hash.

Hmm. I’m skeptical. :-)

What about, instead, providing an interactive function that would prompt
for a URL, run ‘guix download’ on that, and emit an ‘origin’ template at
point with all the info?

Post by Alex Kost
Oh, now I see what you mean. Well, I don't know, I think if a user has
a habbit to check a signature, he will check it anyway; and if not, then
not. Besides, at first a packager needs to find an URL of the source
tarball, so he will meet a signature anyway, if it exists. So it's up
to him if he checks it or not.

(Him or her.)

I think we really want to give packagers a strong incentive to check
signatures. Tools should make it easy to do that.

Post by Alex Kost

Post by Ludovic Courtès

Post by Alex Kost

Post by Ludovic Courtès

Post by Alex Kost
At first I also thought about building a package source and then to
calculate the hash of the store file, but this way will lead to the
wrong hashes for "snippet"-ed origins. Or do I miss anything?

Well, I think bindings for ‘package-source-derivation’ would also be
useful, but IIUC this is not what you had in mind.

If you mean to make a command to build the current package source, it
can be done, although I don't see how it can be useful.

It’s occasionally useful, similar to ‘guix build -S’ or the “Show”
button in package info buffers, except that it would operate directly on
the package at point.
WDYT?

Indeed, I agree. So if you don't mind the “download” command, then
since there will be 2 commands for working with the source of the
"C-c . s d" - to download the source (to know its hash)
"C-c . s b" - to build it

Sounds good!

Thanks,
Ludo’.

[Alex Kost]

[...]

Post by Ludovic Courtès

Post by Alex Kost
I don't see a problem here, since a fake sha256 may be any string,

Not really, since ‘base32’ is a macro that checks its argument at
expansion time. So in practice one cannot C-M-x a package with a random
base32 string.

Ah, indeed, it can't be any string, but it can be an empty string
(perhaps it's a bug in ‘base32’?)

Post by Ludovic Courtès

Post by Alex Kost
for example "" (an empty string). Also I believe people begin to
write a new package from some template, so you have a working skeleton
of future package with all required fields from the very beginning.
Then after filling an origin 'uri', you could "C-c . s" to download
the source and get its hash.

Hmm. I’m skeptical. :-)

Sorry, I didn't get it. Skeptical that people start from a template?
Or that one can download a source after filling an origin 'uri'? If the
latter, I definitely did it.

Post by Ludovic Courtès
What about, instead, providing an interactive function that would prompt
for a URL, run ‘guix download’ on that, and emit an ‘origin’ template at
point with all the info?

I see several problems here, but the main is: this sounds like it should
be synchronous: you give an URL, wait until the source is downloaded and
finally get the template at point. But downloading can take a VERY long
time, so I don't think it will be a usable command.

Post by Ludovic Courtès

Post by Alex Kost
Oh, now I see what you mean. Well, I don't know, I think if a user has
a habbit to check a signature, he will check it anyway; and if not, then
not. Besides, at first a packager needs to find an URL of the source
tarball, so he will meet a signature anyway, if it exists. So it's up
to him if he checks it or not.

(Him or her.)

Yes, I just always say/write "he", "him", etc.

Post by Ludovic Courtès
I think we really want to give packagers a strong incentive to check
signatures. Tools should make it easy to do that.

OK, I understand.

[Ian Denhardt]

Quoting Alex Kost (2015-10-07 13:25:44)

Post by Alex Kost

Post by Ludovic Courtès
What about, instead, providing an interactive function that would prompt
for a URL, run ‘guix download’ on that, and emit an ‘origin’ template at
point with all the info?

I see several problems here, but the main is: this sounds like it should
be synchronous: you give an URL, wait until the source is downloaded and
finally get the template at point. But downloading can take a VERY long
time, so I don't think it will be a usable command.

What about just having command line (non-emacs) tool? I'm thinking
something inspired by Archlinux's `makepkg -g`, which downloads the
sources based on the current build script and outputs something that can
just be copied into it. I don't feel I have the full picture re: how
that interacts with the macro's checking, but we could fall back on the
user supplying URLs.

-Ian

[Alex Kost]

Post by Ian Denhardt
What about just having command line (non-emacs) tool? I'm thinking
something inspired by Archlinux's `makepkg -g`, which downloads the
sources based on the current build script and outputs something that can
just be copied into it. I don't feel I have the full picture re: how
that interacts with the macro's checking, but we could fall back on the
user supplying URLs.

IIUC such a tool would also have the same problem (an incentive not to
check signatures).

[Ludovic Courtès]

Post by Alex Kost
[...]

Post by Ludovic Courtès

Post by Alex Kost
I don't see a problem here, since a fake sha256 may be any string,

Not really, since ‘base32’ is a macro that checks its argument at
expansion time. So in practice one cannot C-M-x a package with a random
base32 string.

Ah, indeed, it can't be any string, but it can be an empty string
(perhaps it's a bug in ‘base32’?)

Oh right. It’s not really a bug, since the empty string is a valid
base32 representation of the empty bytevector.

Post by Alex Kost

Post by Ludovic Courtès

Post by Alex Kost
for example "" (an empty string). Also I believe people begin to
write a new package from some template, so you have a working skeleton
of future package with all required fields from the very beginning.
Then after filling an origin 'uri', you could "C-c . s" to download
the source and get its hash.

Hmm. I’m skeptical. :-)

Sorry, I didn't get it. Skeptical that people start from a template?

Yes, it feels weird to me, the idea that an <origin> object with bogus
values would be created just for the sake of satisfying the download
tool.

Wouldn’t the explanation in the manual of how to use this be relatively
complex? That’s a good benchmark.

Post by Alex Kost

Post by Ludovic Courtès
What about, instead, providing an interactive function that would prompt
for a URL, run ‘guix download’ on that, and emit an ‘origin’ template at
point with all the info?

I see several problems here, but the main is: this sounds like it should
be synchronous: you give an URL, wait until the source is downloaded and
finally get the template at point. But downloading can take a VERY long
time, so I don't think it will be a usable command.

Good point. But there’s the same problem with what you propose no? The
user somehow has to wait for the download to complete?

Post by Alex Kost

Post by Ludovic Courtès
(Him or her.)

Yes, I just always say/write "he", "him", etc.

Yeah, but I think it’s best to try and avoid this language bias (see
<https://en.wikipedia.org/wiki/Gender-neutral_language>.) We want to
welcome all human beings to our Guix party, don’t we? :-)

Thanks,
Ludo’.

[Alex Kost]

Ludovic Courtès (2015-10-08 01:10 +0300) wrote:

[...]

Post by Ludovic Courtès

Post by Alex Kost

Post by Ludovic Courtès

Post by Alex Kost
for example "" (an empty string). Also I believe people begin to
write a new package from some template, so you have a working skeleton
of future package with all required fields from the very beginning.
Then after filling an origin 'uri', you could "C-c . s" to download
the source and get its hash.

Hmm. I’m skeptical. :-)

Sorry, I didn't get it. Skeptical that people start from a template?

Yes, it feels weird to me, the idea that an <origin> object with bogus
values would be created just for the sake of satisfying the download
tool.

Hm, I have an opposite opinion: for me it is weird to write a package
from scratch. I usually start from a package template that has all
required fields (including <origin>) or even I just copy an existing
package and then modify the fields I need.

I wonder, do you start to write a package from scratch?

Post by Ludovic Courtès
Wouldn’t the explanation in the manual of how to use this be relatively
complex? That’s a good benchmark.

As I see it, it is not complex:

«Download a source of the package defined by the current variable
definition and print its SHA256 hash. This command has the same meaning
as running @code{guix download} on the package source (@pxref{Invoking
guix download})»

Anyway, after all I'm not going to add this command to not give people
an incentive not to check signatures.

Post by Ludovic Courtès

Post by Alex Kost

Post by Ludovic Courtès
What about, instead, providing an interactive function that would prompt
for a URL, run ‘guix download’ on that, and emit an ‘origin’ template at
point with all the info?

I see several problems here, but the main is: this sounds like it should
be synchronous: you give an URL, wait until the source is downloaded and
finally get the template at point. But downloading can take a VERY long
time, so I don't think it will be a usable command.

Good point. But there’s the same problem with what you propose no? The
user somehow has to wait for the download to complete?

Well, sure you also need to wait for the download, but your Emacs won't
be freezed for all this time, so you can do something else while it is
being downloaded in the REPL.

Post by Ludovic Courtès

Post by Alex Kost

Post by Ludovic Courtès
(Him or her.)

Yes, I just always say/write "he", "him", etc.

Yeah, but I think it’s best to try and avoid this language bias (see
<https://en.wikipedia.org/wiki/Gender-neutral_language>.) We want to
welcome all human beings to our Guix party, don’t we? :-)

Wow, I didn't realize it could be counted as an offence. In Russian (my
native language) a phrase "A user does something, because he ..." is
absolutely neutral. Thanks for the info!

[Ludovic Courtès]

Post by Alex Kost
[...]

Post by Ludovic Courtès

Post by Alex Kost

Post by Ludovic Courtès

Post by Alex Kost
for example "" (an empty string). Also I believe people begin to
write a new package from some template, so you have a working skeleton
of future package with all required fields from the very beginning.
Then after filling an origin 'uri', you could "C-c . s" to download
the source and get its hash.

Hmm. I’m skeptical. :-)

Sorry, I didn't get it. Skeptical that people start from a template?

Yes, it feels weird to me, the idea that an <origin> object with bogus
values would be created just for the sake of satisfying the download
tool.

Hm, I have an opposite opinion: for me it is weird to write a package
from scratch. I usually start from a package template that has all
required fields (including <origin>) or even I just copy an existing
package and then modify the fields I need.
I wonder, do you start to write a package from scratch?

Sometimes I write from scratch, sometimes I use ‘guix import’, sometimes
I copy/paste some other definition.

Post by Alex Kost

Post by Ludovic Courtès
Wouldn’t the explanation in the manual of how to use this be relatively
complex? That’s a good benchmark.

«Download a source of the package defined by the current variable
definition and print its SHA256 hash. This command has the same meaning
guix download})»

Hmm OK. (Is it really “current variable definition”, or rather
“‘origin’ form at point”?)

Post by Alex Kost
Anyway, after all I'm not going to add this command to not give people
an incentive not to check signatures.

Yeah, maybe we can see where the other discussion goes.

The command to call ‘package-source-derivation’ for the package at point
is still welcome, though. :-)

Post by Alex Kost

Post by Ludovic Courtès

Post by Alex Kost

Post by Ludovic Courtès
What about, instead, providing an interactive function that would prompt
for a URL, run ‘guix download’ on that, and emit an ‘origin’ template at
point with all the info?

I see several problems here, but the main is: this sounds like it should
be synchronous: you give an URL, wait until the source is downloaded and
finally get the template at point. But downloading can take a VERY long
time, so I don't think it will be a usable command.

Good point. But there’s the same problem with what you propose no? The
user somehow has to wait for the download to complete?

Well, sure you also need to wait for the download, but your Emacs won't
be freezed for all this time, so you can do something else while it is
being downloaded in the REPL.

Right.

Thanks,
Ludo’.

[Alex Kost]

Ludovic CourtÚs (2015-10-08 14:46 +0300) wrote:

[...]

The command to call ‘package-source-derivation’ for the package at point
is still welcome, though. :-)

Thanks, here it is.

‘build-package-source’ procedure (as I see it) is very similar to
‘build-package’, so maybe it is worth to make a macro or something,
WDYT?

[Ludovic Courtès]

From d2f72a9375fe592ebf5fe5ee84ad6db614ea2c03 Mon Sep 17 00:00:00 2001
Date: Fri, 9 Oct 2015 16:45:24 +0300
Subject: [PATCH] emacs: Add 'guix-devel-build-package-source'.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
* guix/scripts.scm (build-package-source): New procedure.
* emacs/guix-devel.el (guix-devel-build-package-source): New command.
(guix-devel-keys-map): Add key binding for it.
* doc/emacs.texi (Emacs Development): Document it.

LGTM, thanks!

Ludo’.

[Alex Kost]

Post by Ludovic Courtès

Post by Alex Kost
[...]

Post by Ludovic Courtès
Wouldn’t the explanation in the manual of how to use this be relatively
complex? That’s a good benchmark.

«Download a source of the package defined by the current variable
definition and print its SHA256 hash. This command has the same meaning
guix download})»

Hmm OK. (Is it really “current variable definition”, or rather
“‘origin’ form at point”?)

As I said before it's the same as "C-c . b" is used for downloading the
package source, i.e. it is performed "on the current variable
definition". It is not possible to make it "on origin at point",
because most of the times it has a reference to 'version' ('name'):

(origin
...
(uri (string-append "http://..." version ".tar.gz")))

and 'version' is known only for the whole package, not for the 'origin'.

Post by Ludovic Courtès

Post by Alex Kost
Anyway, after all I'm not going to add this command to not give people
an incentive not to check signatures.

Yeah, maybe we can see where the other discussion goes.
The command to call ‘package-source-derivation’ for the package at point
is still welcome, though. :-)

OK.

[Ludovic Courtès]

Post by Alex Kost

Post by Ludovic Courtès

Post by Alex Kost
[...]

Post by Ludovic Courtès
Wouldn’t the explanation in the manual of how to use this be relatively
complex? That’s a good benchmark.

«Download a source of the package defined by the current variable
definition and print its SHA256 hash. This command has the same meaning
guix download})»

Hmm OK. (Is it really “current variable definition”, or rather
“‘origin’ form at point”?)

As I said before it's the same as "C-c . b" is used for downloading the
package source, i.e. it is performed "on the current variable
definition". It is not possible to make it "on origin at point",
(origin
...
(uri (string-append "http://..." version ".tar.gz")))
and 'version' is known only for the whole package, not for the 'origin'.

Oh right. Thanks for bearing with me!

Ludo’.

[Christopher Allan Webber]

Post by Alex Kost
[...]

Post by Ludovic Courtès

Post by Alex Kost

Post by Ludovic Courtès

Post by Alex Kost
for example "" (an empty string). Also I believe people begin to
write a new package from some template, so you have a working skeleton
of future package with all required fields from the very beginning.
Then after filling an origin 'uri', you could "C-c . s" to download
the source and get its hash.

Hmm. I’m skeptical. :-)

Sorry, I didn't get it. Skeptical that people start from a template?

Yes, it feels weird to me, the idea that an <origin> object with bogus
values would be created just for the sake of satisfying the download
tool.

Hm, I have an opposite opinion: for me it is weird to write a package
from scratch. I usually start from a package template that has all
required fields (including <origin>) or even I just copy an existing
package and then modify the fields I need.
I wonder, do you start to write a package from scratch?

I think helpers to get people started are useful. I cannot remember
fields from memory, not just in Guix, but in pretty much all code. Thus
I ususally look at other code examples to remember how things work.
Sometimes I set up yasnippet expansion templates in emacs.

So, just a vote for: helpers are helpful!

[Ludovic Courtès]

Post by Christopher Allan Webber

Post by Alex Kost
[...]

Post by Ludovic Courtès

Post by Alex Kost

Post by Ludovic Courtès

Post by Alex Kost
for example "" (an empty string). Also I believe people begin to
write a new package from some template, so you have a working skeleton
of future package with all required fields from the very beginning.
Then after filling an origin 'uri', you could "C-c . s" to download
the source and get its hash.

Hmm. I’m skeptical. :-)

Sorry, I didn't get it. Skeptical that people start from a template?

Yes, it feels weird to me, the idea that an <origin> object with bogus
values would be created just for the sake of satisfying the download
tool.

Hm, I have an opposite opinion: for me it is weird to write a package
from scratch. I usually start from a package template that has all
required fields (including <origin>) or even I just copy an existing
package and then modify the fields I need.
I wonder, do you start to write a package from scratch?

I think helpers to get people started are useful. I cannot remember
fields from memory, not just in Guix, but in pretty much all code. Thus
I ususally look at other code examples to remember how things work.
Sometimes I set up yasnippet expansion templates in emacs.

Yes I think it’s good to have something that generates a template.

I’ve always had on my to-do list some sort of a catch-all importer:
you’d give it a tarball URL, and it’d produce a template with the right
build system, maybe some dependencies inferred from configure.ac or
CMakeLists.txt or setup.py, etc.

Ludo’.

[Ludovic Courtès]

Post by Alex Kost
* guix/scripts/lint.scm (run-checkers): Export. Make 'checkers'
argument optional.

OK!

Ludo'.