Intune Certificate Connector certificate expired

It was a bright sunny Monday morning of April 22, 2024, and all of a sudden we started receiving complaints from our service desk and regional IT units regarding failed Windows corporate enrolment.

Enrolment was failing at the certificates stage in ESP. From this, you already know where this is leading to. As we checked the NDES URL, we got the below error.

In the Intune admin center, we saw the certificate connector in Error status with last connected date time being April 20, 2024, at 19:03.

Upon connecting to the server, the first thing we found was that the internet connection on the server was broken.

The NDES server must be able to fetch CRLs from CDP locations to validate the required certificates.

In our environment, the Root CA CDP location is routed over the internet and thus, because of the internet connectivity issue, the Root CA CDP location became inaccessible.

As per our initial assumption, probably this resulted in the NDES service startup error.

As usual in any enterprise infrastructure setup, the internet network connectivity of the server is facilitated via a proxy, and then there is the network piece with firewalls. So all the concerned teams were engaged and the internet connectivity issue was rather quickly resolved.

With the internet connectivity back, we checked to confirm if the NDES can now reach the CDP location to retrieve the required CRLs for validation of the certificates required for NDES service startup.

All looks good above, so there shouldn’t be any problem with the NDES service anymore!

But against our expectations, even with the internet connectivity issue resolved and a restart of the services, the problem with the NDES service persisted and it was now up to my team to resolve.

With our initial assumption failing, we embarked on our troubleshooting journey.

We followed the same troubleshooting checklist as mentioned in one of my previous blogs on NDES troubleshooting, but for the sake of this blog, I will briefly get them listed here anyway.

As always, we started with the basics, checking the NDES service account permission/rights issue to ensure that the NDES service account

1. is not in a Locked state,
2. password is not Expired,
3. is a member of the local IIS_IURS group on the NDES box (ensure no GPO is modifying the membership of the particular group),
4. the IIS_IURS group is assigned the Impersonate a client after authentication user right (by default the right is present unless modified),
5. has Read permission to the private keys of both MSCEP RA certificates.

All assessments conducted returned positive results, indicating no issues. What, then, could be causing the problem?

Next, we checked the event viewer to see if there were any error events we could find for the NDES service. And for sure, we were greeted with the below events.

In a nutshell, the NDES service startup sequence is –

1. Locate RA cert from machine cert store – X509Objects
2. Acquire the Private Keys – CryptAcquireCertificatePrivateKey
3. Build cert chain – CertGetCertificateChainStart
4. Verify Revocation Status – CertVerifyRevocationStart

If all the steps above are completed successfully, the NDES service starts up and you get the success events as below.

Event Source: Windows Logs > Application > NetworkDeviceEnrollmentService

Event ID 47 - The Network Device Enrollment Service loaded the Registration Authority (RA) key exchange certificate with serial number ###### from the "MY" store.

Event ID 48 - The Network Device Enrollment Service loaded the Registration Authority (RA) signature certificate with serial number ###### from the "MY" store.

Event ID 1 - The Network Device Enrollment Service started successfully.

However, if the steps above fail, so does the NDES service start-up, as in our case. (Refer snap above.)

Event Source: Windows Logs > Application > NetworkDeviceEnrollmentService

Event ID 10 - The Network Device Enrollment Service cannot retrieve one of its required certificates (0x80070057).  The parameter is incorrect.

Event ID 2 - The Network Device Enrollment Service cannot be started (0x80004005). Unspecified error

Now this can happen if either or both of the MSCEP-RA cert(s) get expired or deleted, but the events, as above, do not necessarily mean the same. It can also be because of

• Expired CRL, or
• NDES server unable to retrieve the CRL from CDP location.

Because the failure events does not give much information by themselves, as such, the best way forward in this scenario, if you would like to get more detailed events, is to enable CAPI2 logging in events Applications and Services logs > Microsoft > Windows > CAPI2, then restart the Cryptographic Services and IIS [PS command iisreset] and check back in the CAPI2 Operational logs.

For our case though, we were not required to get the CAPI2 logs, as post restoring internet connectivity to the server, our check with the Enterprise PKI (pkiview.msc) MMC snap-in didn’t show any error related to CA connectivity or expiring CRL or CDP location not reachable by NDES server.

But still, to confirm that the NDES server was actually able to retrieve CRL from CDP, we used the certutil command as below to verify the same, and as can be seen, CRL is also retrievable.

Thus no problem with CA connectivity, or expired CRL or CDP location reachability. Then, did either or both of the MSCEP RA certificates got expired or deleted?

Checking the computer cert store, however, showed a different story!

Both the MSCEP-RA certs were present, valid, and not expired. Permission of the NDES service account to the private keys of both the MSCEP-RA cert was already checked previously and found to be good.

Then what? Did you notice the issue from the snap of the computer cert store already?

The certificate for the Microsoft Intune Certificate connector was set to expire on 20-04-2024, and it expired. Interestingly, if you remember at the beginning, it’s also the same date for the certificate connector’s last successful connection as can be seen from the Intune Admin Canter.

Now we do not know for sure, that for how long, the internet connection of the server was broken, but we can make an educated guess that it is because of this broken internet connectivity, the certificate renewal, which should have happened automatically, did not went through.

Remember that the Intune Certificate Connector communication with the Microsoft backend service is all outbound and connector initiated. Thus with no or blocked internet connectivity, the connector cannot initiate any communication to the service.

Thus in the event viewer, navigating to Applications and Services Logs > Microsoft > Intune > CertificateConnectors > Operational logs showed us all error events like event ID 5002, 2, and all others.

Now there is no technical way to renew the expired Certificate Connector certificate. The only way to get a new valid certificate is via re-enrolling the Certificate Connector, which essentially means reinstalling the Intune Certificate Connector.

However, you do not actually need to literally uninstall and re-install the connector and then reconfigure it back. Instead, on the server, you can just navigate to C:\Program Files\Microsoft Intune\PFXCertificateConnector\ConnectorUI and open PFXCertificateConnectorUI.exe with Run as admin elevation.

You get the familiar Intune Certificate Connector configuration wizard.

Follow through the wizard without making any changes to the existing configuration.

You will be required to provide the NDES service account credentials.

Make sure the proxy information is correct.

Now you will be required to do an Azure AD/Entra ID sign-in.

Previously this sign-in used to require a synced account having an Intune license with either Global Admin rights or at least Intune Admin rights assigned. However, I could not see the same requirement in the latest MS document and so tried with our usual cloud-only admin account which is also licensed.

But it did not work, so had to go back with a synced account doing the sign-in and it worked.

The connector instance is now re-enrolled successfully and you can close (Exit) the wizard.

Coming back to the Intune portal to check the connector status, you will see a new entry getting created for the same connector instance.

You can safely delete the old connector instance in the error state and rename the new one accordingly, as we did.

Now we need to head back to the NDES server and restart the services. Post restarting cryptographic services and IIS, we were finally greeted with success events for the NDES service startup.

The connector event log also showed successful SCEP request processing events.

And finally, the check for the NDES mscep URL resolved to the most desired HTTP 403.0 Forbidden Error.

With the NDES service back up and running, it was finally work done and relief. Job over for me and my team, and so was our Monday!