Commit Graph

24 Commits

Author SHA1 Message Date
Ludovic Courtès
01e5d63c87
serialization: 'read-byte-string' makes a single read(2) call.
On "guix build libreoffice -nd", this reduces the number of read(2)
system calls from 10,434 to 8092.

* guix/serialization.scm (sub-bytevector): New procedure.
(read-byte-string): Make a single 'get-bytevector-n*' call and use
'sub-bytevector'.
2020-01-24 23:56:43 +01:00
Ludovic Courtès
0aa6b38695
serialize: Export 'dump-port*'.
* guix/serialization.scm (dump): Export as 'dump-port*'.
* guix/scripts/challenge.scm (dump-port*): Remove.
2020-01-15 23:48:33 +01:00
Ludovic Courtès
1d9a4456a8
serialization: Remove unused procedure.
* guix/serialization.scm (write-contents): Remove.
2019-12-12 17:56:58 +01:00
Ludovic Courtès
12c1afcdbd
serialization: Add 'fold-archive'.
* guix/serialization.scm (read-contents): Remove.
(read-file-type, fold-archive): New procedures.
(restore-file): Rewrite in terms of 'fold-archive'.
* tests/nar.scm ("write-file-tree + fold-archive")
("write-file-tree + fold-archive, flat file"): New tests.
2019-12-12 17:56:57 +01:00
Ludovic Courtès
f9e8a12379
store: Rename '&nix-error' to '&store-error'.
* guix/store.scm (&nix-error): Rename to...
(&store-error): ... this, and adjust users.
(&nix-connection-error): Rename to...
(&store-connection-error): ... this, and adjust users.
(&nix-protocol-error): Rename to...
(&store-protocol-error): ... this, adjust users.
(&nix-error, &nix-connection-error, &nix-protocol-error): Define these
condition types and their getters as deprecrated aliases.
* build-aux/run-system-tests.scm, guix/derivations.scm,
guix/grafts.scm, guix/scripts/challenge.scm,
guix/scripts/graph.scm, guix/scripts/lint.scm,
guix/scripts/offload.scm, guix/serialization.scm,
guix/ssh.scm, guix/tests.scm, guix/ui.scm,
tests/derivations.scm, tests/gexp.scm, tests/guix-daemon.sh,
tests/packages.scm, tests/store.scm, doc/guix.texi: Adjust to use the
new names.
2019-01-21 23:09:55 +01:00
Ludovic Courtès
9fe3f11398
serialization: 'restore-file' errors out upon non-convertible file names.
Fixes <https://bugs.gnu.org/33603>.
Reported by Maxim Cournoyer <maxim.cournoyer@gmail.com>.

* guix/serialization.scm (port-conversion-strategy): New variable.
(restore-file): Parameterize it.
* tests/nar.scm ("restore-file with non-UTF8 locale"): New test.
2019-01-18 17:51:34 +01:00
Ludovic Courtès
c122a2e509
serialization: Remove redundancy in 'write-file'.
* guix/serialization.scm (write-file): Remove redundant 'member' call.
2018-09-23 23:34:16 +02:00
Ludovic Courtès
b94b698d4e
serialization: Add 'write-file-tree'.
* guix/serialization.scm (write-contents-from-port): New procedure.
(write-contents): Write in terms of 'write-contents-from-port'.
(filter/sort-directory-entries, write-file-tree): New procedures.
(write-file): Rewrite in terms of 'write-file-tree'.
* tests/nar.scm ("write-file-tree + restore-file"): New test.
2018-07-19 11:48:04 +02:00
Ludovic Courtès
39d1e9654c
store: Fix potential over-reads in 'import-paths'.
Previously 'process-stderr' would always pass a bytevector of MAX-LEN to
then daemon in the %stderr-read case (i.e., 'import-paths'), instead of
LEN (where LEN <= MAX-LEN).

In practice the extra bytes didn't cause a protocol violation or
anything because they happen at the end of the stream, which typically
contains the canonical sexp of the signature, and the extra zeros were
just ignored.

* guix/serialization.scm (write-bytevector): Add optional 'l' parameter
and honor it.
* guix/store.scm (process-stderr): Pass LEN to 'write-bytevector'.
2018-01-11 00:00:02 +01:00
Ludovic Courtès
e16c823f80
serialization: Remove Guile < 2.0.9 workaround.
* guix/serialization.scm (write-contents): Assume 'sendfile' is always
defined.
2017-04-17 23:31:28 +02:00
Ludovic Courtès
36626c556e
build: Require Guile >= 2.0.9.
* configure.ac: Bump requirement to 2.0.9.
* doc/guix.texi (Requirements): Adjust accordingly.
* README (Requirements): Likewise.
* build-aux/download.scm: Remove workaround for <http://bugs.gnu.org/13095>.
* guix/build/download.scm: Likewise.
(http-fetch)[post-2.0.7?]: Remove.  Remove conditional code for not
POST-2.0.7?.
* guix/http-client.scm:  Remove workaround for <http://bugs.gnu.org/13095>.
(http-fetch)[post-2.0.7?]: Remove.  Remove conditional code for not
POST-2.0.7?.
* guix/serialization.scm (read-latin1-string): Remove mention of 2.0.9.
* tests/nar.scm: Use (ice-9 control).
(let/ec): Remove.
2017-03-18 00:39:49 +01:00
Ludovic Courtès
0d268c5d70
store: Add 'add-data-to-store'.
* guix/serialization.scm (write-bytevector): New procedure.
(write-string): Rewrite in terms of 'write-bytevector'.
* guix/store.scm (write-arg): Add 'bytevector' case.
(add-data-to-store): New procedure, from former 'add-text-to-store'.
(add-text-to-store): Rewrite in terms of 'add-data-to-store'.
* tests/store.scm ("add-data-to-store"): New test.
2017-01-30 10:52:45 +01:00
Ludovic Courtès
2535635f18
Use (ice-9 binary-ports) instead of (rnrs io ports).
This reduces the closure of (guix ui) from 123 to 106 modules.

* guix/derivations.scm: Use (ice-9 binary-ports) instead of (rnrs io
ports).
(map-derivation)[substitute-file]: Use 'read-string' instead of
'get-string-all'.
* guix/ftp-client.scm: Likewise.
* guix/hash.scm: Likewise.
* guix/http-client.scm: Likewise.
* guix/pki.scm (ensure-acl, current-acl): Likewise.
* guix/scripts/archive.scm (authorize-key)[read-key]: Likewise.
* guix/scripts/authenticate.scm (read-canonical-sexp)
(read-hash-data): Likewise.
* guix/scripts/download.scm: Likewise.
* guix/scripts/offload.scm (register-gc-root, remove-gc-roots)
(send-files): Likewise.
* guix/scripts/publish.scm (lazy-read-file-sexp): Likewise.
* guix/scripts/refresh.scm: Likewise.
* guix/scripts/substitute.scm (check-acl-initialized): Likewise.
* guix/serialization.scm (read-maybe-utf8-string): Likewise.
* guix/scripts/hash.scm (guix-hash): Use 'force-output' instead of
'flush-output-port'.
* guix/store.scm (process-stderr): Likewise.
* guix/tests.scm: Likewise.
* guix/utils.scm: Use (ice-9 binary-ports) and autoload (rnrs io ports)
for 'make-custom-binary-input-port'.
2016-10-19 15:54:10 +02:00
Ludovic Courtès
fe585be9aa
serialization: Add #:select? parameter to 'write-file'.
* guix/serialization.scm (write-file): Add #:select? parameter and honor it.
* tests/nar.scm ("write-file #:select? + restore-file"): New test.
2016-06-12 23:55:22 +02:00
Ludovic Courtès
958dd3ce68
utils: Move combinators to (guix combinators).
* guix/utils.scm (compile-time-value, memoize, fold2)
(fold-tree, fold-tree-leaves): Move to...
* guix/combinators: ... here.  New file.
* tests/utils.scm ("fold2, 1 list", "fold2, 2 lists")
(fold-tree tests): Move to...
* tests/combinators.scm: ... here.  New file.
* Makefile.am (MODULES, SCM_TESTS): Add them.
* gnu/packages.scm, gnu/packages/bootstrap.scm,
gnu/services/herd.scm, guix/build-system/gnu.scm,
guix/build-system/python.scm, guix/derivations.scm,
guix/gnu-maintenance.scm, guix/import/elpa.scm,
guix/scripts/archive.scm, guix/scripts/build.scm,
guix/scripts/graph.scm, guix/scripts/lint.scm,
guix/scripts/size.scm, guix/scripts/substitute.scm,
guix/serialization.scm, guix/store.scm, guix/ui.scm: Adjust imports
accordingly.
2016-05-04 23:35:55 +02:00
Ludovic Courtès
09d809db6a Remove assorted Guile 2.0.5 workarounds.
* guix/scripts/authenticate.scm (%default-port-conversion-strategy):
  Remove.
* guix/scripts/substitute.scm (fetch): Remove 2.0.5 special cases.
* guix/serialization.scm (write-file): Remove 'scandir' workaround.
* guix/ui.scm (command-files): Likewise.
2015-05-10 11:07:51 +02:00
Ludovic Courtès
bc7d089a9c serialization: Adjust the permissive UTF-8 decoder to Guile 2.0.12ish.
* guix/serialization.scm (read-maybe-utf8-string): Use
  'set-port-encoding!' and 'set-port-conversion-strategy!' instead of
  setting '%default-port-encoding' and
  '%default-port-conversion-strategy'.  This accounts for Guile commit
  d574d96, which changes bytevector input ports to use ISO-8859-1.
2015-05-01 13:10:18 +02:00
Ludovic Courtès
ce72c78074 store: Attempt to decode build logs as UTF-8.
* guix/serialization.scm (read-maybe-utf8-string): New procedure.
* guix/store.scm (process-stderr): Use it for the build log and errors.
* tests/store.scm ("current-build-output-port, UTF-8",
  "current-build-output-port, UTF-8 + garbage"): New tests.
2015-03-05 22:17:36 +01:00
Ludovic Courtès
472e4c4303 serialization: Factorize 'read-byte-string'.
* guix/serialization.scm (read-byte-string): New procedure.
  (read-string, read-latin1-string): Use it.
2015-03-05 22:17:36 +01:00
Ludovic Courtès
46b8aadbd6 serialization: Check for EOF and incomplete input conditions.
Fixes <http://bugs.gnu.org/19756>.
Reported by <sleep_walker@suse.cz>.

* guix/serialization.scm (currently-restored-file): New variable.
  (get-bytevector-n*): New procedure.
  (read-int, read-long-long, read-string, read-latin1-string,
  read-contents): Use it instead of 'get-bytevector-n'.
  (restore-file): Parameterize 'currently-restored-file' and set it.
* tests/nar.scm ("restore-file with incomplete input"): New test.
2015-02-07 23:16:55 +01:00
Ludovic Courtès
aa27b56083 serialization: Read Latin-1 strings with 'get-bytevector-n'.
* guix/serialization.scm (read-latin1-string): Use 'get-bytevector-n'
  instead of 'get-string-n'.  Use 'list->string' etc. to convert the
  bytevector to a string.
2015-01-18 22:17:09 +01:00
Ludovic Courtès
0363991a25 Break module cycle involving (guix store) and (guix ui).
Before, there was a cycle along the lines of:

  (guix store) -> (guix nar) -> (guix ui) -> (guix store)

This caused problems, as discussed at:

  http://lists.gnu.org/archive/html/guix-devel/2014-10/msg00109.html

This patch removes cycles in the (guix ...) modules.

* guix/nar.scm (&nar-error, &nar-read-error, dump, write-contents,
  read-contents, %archive-version-1, write-file, restore-file): Move to...
* guix/serialization.scm: ... here.
* guix/store.scm: Remove dependency on (guix nar).
* guix/scripts/hash.scm, guix/scripts/offload.scm,
  guix/scripts/substitute-binary.scm, tests/nar.scm, tests/store.scm,
  tests/substitute-binary.scm: Adjust accordingly.
2014-10-09 23:51:19 +02:00
Ludovic Courtès
6c20d1d0c3 store: Add #:timeout build option.
* guix/serialization.scm (write-string-pairs): New procedure.
* guix/store.scm (write-arg): Add 'string-pairs' case.
  (set-build-options): Add 'timeout' keyword parameter.  Honor it.
* tests/derivations.scm ("build-expression->derivation and timeout"):
  New test.
2014-03-09 23:01:18 +01:00
Ludovic Courtès
0f41c26f9b Add (guix nar) and (guix serialization).
* guix/store.scm (write-int, read-int, write-long-long, read-long-long,
  write-padding, write-string, read-string, read-latin1-string,
  write-string-list, read-string-list, write-store-path,
  read-store-path, write-store-path-list, read-store-path-list): Move to
  serialization.scm.
  (write-contents, write-file): Move to nar.scm.
* guix/nar.scm, guix/serialization.scm: New files.
* Makefile.am (MODULES): Add them.
2013-04-04 22:29:08 +02:00