index.md 24KB
title: Apple Mail Stores Encrypted Emails in Plain Text Database, fix included!
url: https://medium.com/@boberito/apple-mail-stores-encrypted-emails-in-plain-text-database-fix-included-3c2369ce26d4
hash_url: 6aa439c2a9
Apple Mail Stores Encrypted Emails in Plain Text Database, fix included!
I was investigating how macOS and Siri suggest contacts and information to you, the user. This led me to the process called suggestd, run by the system level LaunchAgent com.apple.suggestd, and the Suggestions folder in the user-level Library folder, which contains multiple files and some potentially important database files (.db files). These are databases with information from Apple Mail and other Apple applications that enable macOS and Siri to become better at suggesting information.
The main thing I discovered was that the snippets.db database file in the Suggestions folder stored my emails. And on top of that, I found that it stored my S/MIME encrypted emails completely UNENCRYPTED. Even with Siri disabled on the Mac, it *still* stores unencrypted messages in this database!
**This is definitely not the expected behavior and can be considered an inadvertent information exposure.**
Let me say that again… The snippets.db database is storing encrypted Apple Mail messages…completely, totally, fully — UNENCRYPTED — readable, even with Siri disabled, without requiring the private key. Most would assume that disabling Siri would stop macOS from collecting information on the user. This is a big deal. This is a big deal for governments, corporations and regular people who use encrypted email and expect the contents to be protected. Secret or top-secret information, which was sent encrypted, would be exposed via this process and database, as would trade secrets and proprietary data.
Some may not know what S/SMIME encrypted email is. So, what is it? S/MIME uses a pair of keys to operate — a public key and a private key. Emails are encrypted with the recipient’s public key. The email can only be decrypted with the corresponding private key, which is supposed to be in sole possession of the recipient. If the private key is unavailable or removed, the message should not be readable, by anything. Unless the private key is compromised, you can be confident that only your intended recipient will be able to access the sensitive data in your email.
Unfortunately, snippets.db stores these encrypted messages completely UNENCRYPTED, not requiring the private key to read the message. This completely defeats the purpose of utilizing and sending an encrypted email.
Another database, entities.db, stores records of people’s names, email, and phone numbers you’ve corresponded with. Although the phone number may not be in your contact list, data from emails such as signature blocks and forward information are stored. It’s like an address book built for you. This could be touchy, as it may allow quick and easy access to some potentially sensitive information.
Although it has been suggested that you must disable Siri from learning from Apple Mail, even if Siri is disabled, I would argue that it still is the incorrect default behavior. Fortunately, I have discovered a way to mitigate the issue.
There are 3 ways to disable these processes from learning from Apple Mail.
- You can manually click the settings, but this does not scale well across an enterprise fleet of Macs.
System Preferences → Siri →Siri Suggestions & Privacy →Uncheck the boxes for Apple Mail.
2. As the user, not with elevated privileges, you can run this command in Terminal to turn off Siri from learning from Apple Mail.
defaults write com.apple.suggestions SiriCanLearnFromAppBlacklist -array com.apple.mail
3. You can deploy a System-Level (for all users) configuration profile to turn off Siri from learning from Apple Mail.
https://github.com/boberito/ConfigurationProfiles/blob/master/suggestd.mobileconfig
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>PayloadContent</key>
<array>
<dict>
<key>PayloadContent</key>
<dict>
<key>com.apple.suggestions</key>
<dict>
<key>Forced</key>
<array>
<dict>
<key>mcx_preference_settings</key>
<dict>
<key>SiriCanLearnFromAppBlacklist</key>
<array>
<string>com.apple.mail</string>
</array>
</dict>
</dict>
</array>
</dict>
</dict>
<key>PayloadEnabled</key>
<true/>
<key>PayloadIdentifier</key>
<string>MCXToProfile.85654d27-2ffe-4ee5-8ae3-ac63864d6bf3.alacarte.customsettings.73a7cb5a-550b-473d-a191-b8ec261302bd</string>
<key>PayloadType</key>
<string>com.apple.ManagedClient.preferences</string>
<key>PayloadUUID</key>
<string>73a7cb5a-550b-473d-a191-b8ec261302bd</string>
<key>PayloadVersion</key>
<integer>1</integer>
</dict>
</array>
<key>PayloadDescription</key>
<string>Disable Apple Mail Siri Suggestions</string>
<key>PayloadDisplayName</key>
<string>Disable Siri Suggestions</string>
<key>PayloadIdentifier</key>
<string>suggestd</string>
<key>PayloadOrganization</key>
<string></string>
<key>PayloadRemovalDisallowed</key>
<true/>
<key>PayloadScope</key>
<string>System</string>
<key>PayloadType</key>
<string>Configuration</string>
<key>PayloadUUID</key>
<string>85654d27-2ffe-4ee5-8ae3-ac63864d6bf3</string>
<key>PayloadVersion</key>
<integer>1</integer>
</dict>
</plist>The third option is the only way to permanently disable macOS and Siri from collecting this Apple Mail information for all users. Whichever way you disable it, you may want to delete the snippets.db file as well since disabling doesn’t remove previously saved emails. It just stops future ones from populating the database. This needs to be deployed to prevent suggestd from processing your emails even if Siri is disabled. The interface in the Siri System Preference panel may show it’s checked, but if you uncheck it you can’t re-check it. So, the user interface may not always report properly that it’s enabled or disabled.
You can run the following in Terminal to see what the setting’s value is.
defaults read com.apple.suggestions SiriCanLearnFromAppBlacklist
But even without this setting disabled, there are protections in place, most, not all, of your email messages go into this database. The user-level Library Suggestions folder is supposed to be protected by SIP (sandbox definition file found in /System/Library/Sandbox/Profiles/com.apple.suggestd.sb) and things like Bash, Zsh or Python are not intended to have access. It’s also marked with an extended attribute to prevent backups(com_apple_backup_excludeItem). Also, if you have FileVault enabled, then you have potentially another layer of protection if someone got their hands on your powered-off Mac.
However, be careful, because anything you grant Full Disk Access to with a Privacy Preferences Policy Control now has access to that directory. For example, when you turn on Remote Login(sshd) or File Sharing(smbd) you grant sshd and smbd Full Disk Access.
Additionally, Finder always has access, so with an AppleScript one could potentially get their hands on these files. Especially, since you’re presented with so many things asking for permission in macOS Catalina, the dialog fatigue is real. Though, if a vulnerability was discovered and taken advantage of in suggestd, that could be a way to access these emails through suggestd’s entitlement or honestly anything that’s been granted Full Disk Access. If System Integrity Protection (SIP) is disabled (please don’t disable SIP), then the directory is definitely not protected.
This sums up the positives, so there are protections in place and even a way to stop it, but it’s still an incorrect behavior because even with Siri enabled or disabled it should not be storing encrypted messages completely unencrypted.
The process that controls all of this is called suggestd. If you’re curious about suggestd, you can check it out in Activity Viewer. Then, with deeper inspection of the process under the Open Files and Ports tab, it shows what databases are being used by suggestd. If you want to watch suggestd do its thing live, you can run the following in Terminal.
log stream --predicate ‘(subsystem ==”com.apple.suggestions”)’ --level info
You can even start another instance of suggestd.
/System/Library/PrivateFrameworks/CoreSuggestions.framework/Versions/A/Support/suggestd
I discovered this database and what’s stored there on July 25th and began extensively testing on multiple computers with Apple Mail set up and fully confirming this on July 29th. Later that week, I confirmed this database exists on 10.12 machines up to 10.15 and behaves the same way, storing encrypted messages unencrypted. If you have iCloud enabled and Siri enabled, I know there is some data sent to Apple to help with improving Siri, but I don’t know if that includes information from this database. S/MIME is supported in iOS with Apple Mail, but I haven’t dug into it. I’d imagine it behaves the same but due to the app sandboxing and the security model of iOS, it’d take a lot more to get your hands on that database but maybe not impossible chaining vulnerabilities together. The focus on this has been macOS related.
Timeline
July 29 —
- Reported to product-security@apple.com that this exists in 10.14 and beta of 10.15.
- Submitted a ticket to Apple enterprise support.
- Submitted a bug report ticket through Feedback application using an Apple Business Manager account.
- Contacted professional points of contact within Apple.
- Received a response from Apple enterprise support.
- Received response from professional contacts within Apple.
July 30 —
- Reported and updated with all points of contact and tickets that this exists in 10.13.6.
- Received a response from enterprise support that this has been submitted to engineering.
July 31 —
- Reported and updated all points of contact and tickets that this exists in 10.12.6.
- Received response from enterprise support thanking me for testing but no update to report yet.
August 1 —
- Received a response from product security that they are investigating the issue.
September 4 —
- I sent an update to product security that the problem still exists in Catalina betas.
September 9 —
- Received a response from product security that they are still investigating the issue.
October 9 —
- Sent an update to product security that the public final release of Catalina still has the issue and that the Supplemental Updates to Mojave did not address it.
October 18 —
- Contacted personal contacts within Apple about the issue.
October 31 —
- Created a new product security ticket for the issue with Apple.
- Followed up with Enterprise Support on the severity of the issue again.
- Emailed Tim Cook.
November 5 —
- Received information from Enterprise Support on disabling learning from Apple Mail through System Preferences →Siri.
It’s been 100 days since I’ve alerted Apple, we’ve seen a security update to macOS Sierra 10.12, security updates to macOS High Sierra 10.13, Supplemental Updates to macOS Mojave 10.14, a security update to macOS Mojave 10.14, macOS Catalina 10.15.0 released, Supplemental Update to 10.15.0, and 10.15.1 release.
For a company that prides itself on security and privacy, the lack of attention to detail on an issue like this completely and totally surprises me. It brings up the question of what else is tracked and potentially improperly stored without you realizing it. For an operating system that you generally have to change controls to make it less secure, this is a setting that requires you to set to make it more secure and behave correctly. I also have to wonder why it took 99 days for someone to know the answer on how to prevent this. All parties at Apple were alerted multiple times before writing this blog and giving an ample amount of time before I published this. The two real main issues here are...
- Disabling Siri doesn’t stop macOS from collecting data for Siri.
- That with Siri enabled or disabled this process is storing encrypted emails in a database completely unencrypted.
If these issues are a concern for you and your organization, consider creating an enterprise support ticket, filing feedback, or contact your representative at Apple and let them know.
These views are my own and not the views of my employer. If you have any questions or want more information on this, feel free to contact me on the MacAdmins Slack(boberito), Linkedin(linkedin.com/in/bob-gendler-8702014) or email(bobgendler@gmail.com).
Below are some screenshots showing an encrypted message being displayed UNENCRYPTED with DB Browser for SQLite without requiring the private key. Apologies for all the blocked-out areas but I had to hide identifying information.
