ENHANCED: library(crypto): Retain a compact internal representation of the ciphertext throughout decryption.

This allows efficient decryption also of very large files and long
strings.

It is usually highly advisable to stick to common and portable library
predicates such as append/3. However, since append/3 does not yet
recognize this opportunity for improvement, I apply it manually in
this case, so that also very large files can be efficiently decrypted
using the compact internal string representation.

Without this change, decrypting a 1 GB file takes 48 GB of RAM,
whereas with this change, it only takes 2 GB (one for the string, one
for its copy with the appended tag).

src/lib/crypto.pl

@@ -1,5 +1,5 @@
 /* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
-   Written 2020, 2021 by Markus Triska (triska@metalevel.at)
+   Written 2020, 2021, 2022 by Markus Triska (triska@metalevel.at)
    Part of Scryer Prolog.
 
    Predicates for cryptographic applications.
@@ -46,6 +46,7 @@
 :- use_module(library(format)).
 :- use_module(library(charsio)).
 :- use_module(library(si)).
+:- use_module(library(iso_ext), [partial_string/3]).
 
 /* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    hex_bytes(?Hex, ?Bytes) is det.
@@ -594,7 +595,9 @@ crypto_data_decrypt(CipherText0, Algorithm, Key, IV, PlainText, Options) :-
         member(Encoding, [utf8,octet]),
         encoding_chars(octet, CipherText0, CipherText1),
         maplist(char_code, TagChars, Tag),
-        append(CipherText1, TagChars, CipherText),
+        % we append the tag very efficiently, retaining a compact
+        % internal string representation of the ciphertext
+        partial_string(CipherText1, CipherText, TagChars),
         (   Algorithm = 'chacha20-poly1305' -> true
         ;   domain_error('chacha20-poly1305', Algorithm, crypto_data_decrypt/6)
         ),