add tag
Say I've got a file on disk that doesn't fit in the computer's main memory. The file consists of two sections, a small section of less than 100 bytes at the beginning of the file, and a large section, consisting of the rest of the file.

I need to use AEAD (either ChaCha20-Poly1305 or AES-GCM) to encrypt the small section and authenticate the large section.

As far as I understand AEAD, in principle it should be possible to do this by loading the file into RAM piece-by-piece in small parts.

My problem is that the Python Cryptography AEAD API doesn't seem to be designed for this usecase, or at least it's missing similar examples.

This is the API documentation for ChaCha20-Poly1305:

The docs indicate that the `associated_data` (input that needs to be authenticated, but not encrypted) parameter must be a bytes-like object, defined here:

They further point to the Python Buffer Protocol:

Ideas for a solution:

1. Hash the large section and only authenticate the resulting digest. I'm wary of doing this as I'm not entirely sure about the security implications, although it seems like it would be OK.

2. Construct a bytes-like type that would represent the large section without trying to load it into memory all at once. I have little experience with Python so I'm not sure how to proceed in this direction.

EDIT: the first idea is fine:

Enter question or answer id or url (and optionally further answer ids/urls from the same question) from

Separate each id/url with a space. No need to list your own answers; they will be imported automatically.