If you use emacs, you can set up gnupg integration for easy encryption and decryption of files.
I would also read up on expiration of keys, and how you can export (backup) and import a key. The latter can also be useful if you want to use the same key on multiple devices.
If you are comfortable with the command line, mbsync / isync allow you to save your mail offline with the ability to search and read it later on. You can use a cron job to fetch the email however often you want it.