Apple holds the keys to nearly all recent Mac software. This is a story of those keys, and how a Hewlett Packard (HP) error caused problems for a lot of people.
Code signing and certificates
First, it’s important to understand that when I say “keys,” what I really mean is “certificates.” These certificates are similar to the ones that are the basis for secure communication between a web server and your browser. With web traffic, these certificates are used to encrypt the data, but they support more than just encryption.
Certificates also allow for validation. For example, when you try to connect to your bank site, the site’s certificate will verify that the site really does belong to your bank. Not many people actually look at these certificates, of course, but doing so is a sure-fire way to avoid a phishing site.
How does this relate to Apple and HP, you ask? Good question. For quite a few years now, Apple has supported what is called “code signing” on macOS. Code signing involves using a certificate to cryptographically sign a piece of software. This allows the system, and the user, to verify which developer created the software, and check that it hasn’t been modified since it was created.
In recent years, Apple has done more than just support code signing… it’s come as close as is reasonably possible to requiring code signing. As a developer, if you don’t sign your Mac software, your users will have trouble running it, and you (or your support staff) will get countless help inquiries. Your software will also probably just get deleted by many people.
This obviously applies to apps you download from the Internet or the App Store, but it also applies to more prosaic software, such as print drivers. HP makes printers, and thus makes print drivers, and of course those drivers are signed, as they should be.
The certificates used to sign software on macOS (and iOS, for that matter) are provided and managed by Apple. The certificates used by HP are no exception.
So, what happened?
Last Thursday evening (October 22), we started seeing an influx of support requests from people complaining about some new malware that we weren’t detecting. At least, that’s what they were saying. As we dug into the issue, however, we saw that there was a pattern in the screenshots we were seeing.
The “malware” was being reported by the built-in anti-malware features in macOS, and there were a dozen or more different processes that macOS claimed “will damage your computer,” with a check box reading “Report malware to Apple to protect other users.” Sounds pretty scary, right?
However, we noticed that this “malware” was all (mostly*) related to HP printing drivers. The messages generally appeared when people were trying to print to their HP printers. Samples of the software that we obtained appeared to be legitimate, with no signs of malicious behavior.
Why did macOS think it was malicious?
Initially, there was a lot of finger pointing at a recent XProtect update. (XProtect is a basic form of anti-malware protection built into macOS, which aims to prevent malicious software from running.) The thought was that this was a false positive; in other words, XProtect was erroneously detecting legitimate files as malicious.
However, the timing of the last XProtect update didn’t line up with the very sudden and widespread emergence of the issue. With some digging, we found that the source of the issue was that the developer certificate used to sign these HP drivers had been revoked.
Revoking a certificate is usually done by Apple when a piece of malware is discovered to be signed using that certificate. It was initially assumed that Apple had erroneously revoked the certificate. However, it turned out, according to a statement from HP given to The Register, that HP itself had erroneously requested that the certificate be revoked.
We unintentionally revoked credentials on some older versions of Mac drivers. This caused a temporary disruption for those customers and we are working with Apple to restore the drivers. In the meantime, we recommend users experiencing this problem to uninstall the HP driver and use the native AirPrint driver to print to their printer.
Apple was able to reinstate the revoked certificate, which fixed the problem for some people, but not everyone. We’re still seeing new cases reported days later.
The impact of false positives
This isn’t the first time that certificates have been revoked in error. As an example, there was a case back in August where a developer named Charlie Monroe reported that his entire Apple developer account was deleted, and his code signing certificate was revoked. All his apps suffered the same issue as HP’s print drivers.
With any security software, false positives are always a potential problem. Mistakes happen, and Apple isn’t always to blame in cases like this. However, when there’s a certificate issue with a piece of Mac software, it affects everyone, everywhere, who is using that software.
The fallout of these events can hit the developers hard. I don’t know how Charlie Monroe is doing, but I suspect that a significant number of people who were using his software probably deleted it, and may never trust his software again.
At companies like Malwarebytes, these events have the potential to result in hundreds or thousands of support tickets from customers asking why we didn’t detect this “malware,” or even why we’re blocking something legitimate (on the mistaken belief that this message is being shown by Malwarebytes). Some folks may never have contacted our support teams, and simply uninstalled our software, thinking they’d gotten infected while under our protection.
Perfect conditions for scams
One of the most unfortunate aspects of events like these is that they provide incredibly fertile ground for scams. There has been an explosion in scam videos and web pages claiming to help you “remove” this “malware.” These scams work by taking advantage of common things people are searching for that they think are malware.
For example, if you search for “will damage your computer” on Google right now, you will get a number of results offering to help you “remove will damage your computer” (yes, in exactly that nonsensical language). Within hours on Friday, some of these sites – and fake YouTube videos referring to those sites – were already taking advantage of this chaos.
The goal of these sites is to trick you into thinking you’re infected, so that you will download the software they recommend to remove the “virus.” In reality, there often is no actual malware, and the site gets paid an affiliate fee for every referral to the software in question. Often, the software being recommended itself is a scam.
It’s very important to be skeptical in your use of Google (and other search engines). Don’t automatically believe that something is malware just because you Googled it and found sites calling it malware.
How to fix the Mac/HP printer issue
If you are among those who are still having the problem, here are some possible fixes that have worked for our customers:
1) Restart your computer, ensuring it’s on the network when it restarts
2) Check for HP software updates via the Software Update pane in System Preferences
3) Remove the HP printer from System Preferences -> Printers & Scanners, then try adding it again.
4) Check for newer HP software for your printer on the HP support site:
5) If all else fails, contact HP via its support site for assistance.
Earlier, we said that the issue was mostly related to HP printer drivers. There was another issue with a couple Amazon apps – Amazon Music and Amazon Workspaces – where users were seeing the same behavior. This led to a lot of speculation and finger pointing at Apple (in which yours truly regretfully participated), but this appears to have been an unrelated and coincidentally timed issue. Apple was not to blame, as was initially thought, and actually acted quite quickly to help HP rectify the error.