Deploy an OS over CMG using bootable media

[anyweb]

Introduction

Microsoft released update 2010 on December 1st and one of the many new features was the ability to deploy an OS over CMG using bootable media. I tested out this ability when it first arrived in aTechnical Preview release back in Technical Preview version 2009, you can read about that here to see how it worked then, there are some changes to the overall process since then.

This blog post assumes you've already setup a CMG as documented here. So let's try out the feature.

Update: 2020/12/19 Please review the following hotfix for anyone who installed ConfigMgr update 2010 using the fast ring. Below is related to OSD via CMG.

Quote

If you use a PKI-based certificate for operating system boot media, configure it for SHA256 with the Microsoft Enhanced RSA and AES provider. For later releases, including globally available version 2010, this certificate configuration is recommended but not required. The certificate can be a v3 (CNG) certificate.

Step 1. Create a task sequence

In the following step I'll create a simple task sequence, nothing special other than the following settings:

On the Install Windows step, select Enable the account and specify the local administrator password and enter a local administrator password.

Next, change the Configure Network settings step to Join a workgroup instead of joining a domain.

Select applications...

Close the wizard when done.

After the task sequence is created, edit the Enable BitLocker step and disable it or set it to Continue on Error.

This is because the step will fail as it won't have access to AD to escrow the recovery key.

Step 2. Distribute content to the CMG

Select the task sequence created in step 1, right click and choose Distribute Content.

On the Content Destination screen, make sure to select your Cloud Management Gateway (CMG).

Step 3. Verify the following settings

In the Administration pane of ConfigMgr, select Client Settings, then select Default Client Settings (or create a new one) and right click and bring up the properties of the client settings, select the Cloud Services section. Verify that the following settings are configured to Yes.

• Allow access to cloud distribution point
• Enable clients to use a cloud management gateway

as shown here.

Step 4. Create a device collection

In the next step, create a Device collection and limit it to something appropriate like CMG Clients. You can populate this collection with known computers that you intend to target with this task sequence, for example by importing the mac address and associating it with a computer name.

Note: This is my lab so I'm using All Systems, don't do that in production though !

Step 5. Deploy the task sequence

Right click the newly created task sequence and choose Deploy. Deploy it to the collection you just created. Make sure to configure the following settings when deploying the task sequence.

• Deployment settings page: Make available to an option that includes media, for example Only media and PXE.

• User experience page: Allow task sequence to run for client on the internet

• Distribution points page, deployment options: Download content locally when needed by the running task sequence.

I deployed the task sequence to both known (OSD via CMG boot media collection) and unknown computers.

Step 6. PKI certificate modifications

Note: As per the documentation, if you use a PKI-based certificate for the boot media, configure it for SHA256 like so...

And with the Microsoft Enhanced RSA and AES provider.

In case that's not clear, you'll need to modify your certificate template used for boot images to include the Microsoft Enhanced RSA and AES Cryptographic Provider instead of the default Microsoft RSA SChannel Cryptographic Provider.

I created a new template on my IssuingCA matching that requirement.

Note: Under the Request Handling tab make sure that Allow private key to be exported is checked.

After creating the new Certificate Template  on the Issuing CA, you'll need to request it on the ConfigMgr site hosting your distribution point.

Right click the newly imported certificate and choose All Tasks, then select Export. In the welcome to certificate export wizard click Next and choose to export the private key.

Save it on your desktop with a memorable name like CMG Cert.pfx.

Step 7. Create bootable media

Now that you've done everything above, it's time to create your bootable media, that media can be a USB boot key or a standalone ISO.

In the ConfigMgr console, in Software Library, Operating Systems, Task Sequences, choose Create Task Sequence Media from the options available in the Create section of the ribbon.

Choose Bootable media from the options.

On the Media Management page of the wizard, select the option for Site-based media.

Choose USB or ISO

As we limited the device collection to All Unknown Computers, make sure to select that option from those available, and specify your OSD certificate+password. Also, as is this bootable media, set a strong password.

Make sure to point to the newly created CMG Cert.pfx file created earlier if your site mode is HTTPS Only (PKI).

On the Boot Image page, select the Cloud management gateway for the management point settings.

continue through the wizard until complete.

Step 7. Boot a computer using boot media

After completing all the steps above and verifying that your task sequence content is on the CMG, it's time to boot a target computer (with a LAN internet connection) from the media (ISO or USB). This computer does not need a connection back to the Active Directory domain controller or the on premise ConfigMgr infrastructure. But it does need full access to your CMG.

As I'm using a virtual machine, here are the configured settings I'm using to boot from the ISO file.

The computer should boot very quickly from the media and wait at the password screen, enter your strong PXE password configured earlier.

Select the task sequence and off it goes. In this step, you can see it's downloading the content from the CMG

when it's done, login as Administrator,

and your computer is managed via Endpoint Manager, there's the client, and it's getting policy so you can deploy apps, and do all sorts of things to this newly deployed computer. You will of course note that this is not domain joined and that is because it does not have any contact with the on premise infrastructure throughout the entire process. You can use the ConfigMgr client to provide apps to domain join or install whatever you want after the user has logged on.

After the task sequence runs, the client uses token-based authentication.

What an incredible feature !

Troubleshooting

If and when problems occur look at the smsts.log file on the client (in x:\windows\temp\smsts.log) and the CMGService.log on the CMG, that log file and related logs can be found be clicking here. I've observed problems such as "failed to retrieve reg token from media token 0x87d00215 ", that should be resolved with this hotfix. If you see "WINHTTP_CALLBACK_STATUS_FLAG_CERT_REV_FAILED" check the Verify Client Certificate Revocation check box on the settings tab of your CMG properties.

Recommended reading

• How to estimate the cost of OSD via the CMG
• The right way to find logs from your CMG
• Deploy an OS over CMG using bootable media
• Update 2010 for Microsoft Endpoint Configuration Manager Current Branch
• Plan cloud management gateway
• Creating boot media
• How can I configure MEM in HTTPS mode - part 1
• How can I configure MEM in HTTPS mode - part 2

 

[Imran]

On 12/6/2020 at 4:34 PM, anyweb said:

Great blog. As you have created BitLocker videos. Can you please create a video on this as well ... looking at your videos has made my life so easier and for me its easy to follow videos 

 

[anyweb]

here's the video !

https://www.niallbrady.com/2020/12/10/video-deploy-an-os-via-cmg-using-bootable-media/

[Teffan Hawk]

Hi, I can login and see the all TS published. During the download, however, I receive a error list in my smsts.log "Content location request for XXX00002: 12 failed error. (Code 0x80004005)". All the content of the selected TS is obviously distributed on the CMG. Thanks 1000 for what you do

[anyweb]

Hi @Teffan Hawk

thanks for the thanks, are you absolutely sure your content is on the CMG, in the above snippet it's the 12'th source version of the content that's missing

cheers

niall

[Teffan Hawk]

Unfortunately I'm sure and the evidence also gives me the distmgr.log.

Successfully updated the package server status for ["Display=\\XXXcmg.cloudapp.net\"]MSWNET:["SMS_SITE=XXX"]\\XXXcmg.cloudapp.net\ for package XXX00002, Status 0

I tried to remove XXX00002 package from the TS but it also behaves exactly with others in the same TS. The strange thing is that I can authenticate, I see the list of available TS but, at the time of download, I get the error.
I'm really sad as I think I'm one step away from the goal

Cheers, Hawk

[HeroicBandit]

Thanks for the write up and video! After following everything exactly, I've run into an immediate failure when attempting to download policy from WinPE once booted. By all accounts it looks cert related but I can't for the life of me figure out what's happening. CRL enforcement isn't being enabled on my CMG nor is TLS 1.2. I've also tried using the same cert we use for imaging on prem but that resulted in the exact same error. Sample log below.

 

Client is not allowed to use or doesn't have PKI cert while talking to HTTPS server. Request may fail.    TSMBootstrap    12/22/2020 3:22:59 PM    1684 (0x0694)
Using port 443 for CMG request even customer configured customized port.    TSMBootstrap    12/22/2020 3:22:59 PM    1684 (0x0694)
SMS CCM 5.0: Host=redacted.CLOUDAPP.NET, Path=/CCM_Proxy_ServerAuth/10119/CCM_STS?RequestTokenType=Bulk, Port=443, Protocol=https, CcmTokenAuth=0, Flags=0x1204, Options=0x40000000    TSMBootstrap    12/22/2020 3:22:59 PM    1684 (0x0694)
Created connection on port 443    TSMBootstrap    12/22/2020 3:22:59 PM    1684 (0x0694)
Target URL scheme is HTTPS: https://redacted.CLOUDAPP.NET/CCM_Proxy_ServerAuth/10119/CCM_STS?RequestTokenType=Bulk    TSMBootstrap    12/22/2020 3:22:59 PM    1684 (0x0694)
Trying without proxy.    TSMBootstrap    12/22/2020 3:22:59 PM    1684 (0x0694)
[CCMHTTP] AsyncCallback(): -----------------------------------------------------------------    TSMBootstrap    12/22/2020 3:22:59 PM    1684 (0x0694)
[CCMHTTP] AsyncCallback(): WINHTTP_CALLBACK_STATUS_SECURE_FAILURE Encountered    TSMBootstrap    12/22/2020 3:22:59 PM    1684 (0x0694)
[CCMHTTP]                : dwStatusInformationLength is 4
    TSMBootstrap    12/22/2020 3:22:59 PM    1684 (0x0694)
[CCMHTTP]                : *lpvStatusInformation is 0x8
    TSMBootstrap    12/22/2020 3:22:59 PM    1684 (0x0694)
[CCMHTTP]            : WINHTTP_CALLBACK_STATUS_FLAG_INVALID_CA is set
    TSMBootstrap    12/22/2020 3:22:59 PM    1684 (0x0694)
[CCMHTTP] AsyncCallback(): -----------------------------------------------------------------    TSMBootstrap    12/22/2020 3:22:59 PM    1684 (0x0694)
spNamespace.Open( c_szEventingNamespace, true, 0, (uFlags & CcmEvent_UseAdminLocator) != 0 ), HRESULT=8004100e (..\Event.cpp,280)    TSMBootstrap    12/22/2020 3:22:59 PM    1684 (0x0694)
Failed to create event "CCM_CcmHttp_Status" (8004100E)    TSMBootstrap    12/22/2020 3:22:59 PM    1684 (0x0694)
CreateCcmEventV(pszEventName, 0, &spEvent, va), HRESULT=8004100e (..\Event.cpp,353)    TSMBootstrap    12/22/2020 3:22:59 PM    1684 (0x0694)
HRESULT_FROM_WIN32( dwErrorCode ), HRESULT=80072f8f (..\requestresponse.cpp,799)    TSMBootstrap    12/22/2020 3:22:59 PM    1684 (0x0694)
Failed in WinHttpSendRequest API, ErrorCode = 0x2f8f    TSMBootstrap    12/22/2020 3:22:59 PM    1684 (0x0694)
[CCMHTTP] ERROR: URL=https://redacted.CLOUDAPP.NET/CCM_Proxy_ServerAuth/10119/CCM_STS?RequestTokenType=Bulk, Port=443, Options=1073741824, Code=12175, Text=ERROR_WINHTTP_SECURE_FAILURE    TSMBootstrap    12/22/2020 3:22:59 PM    1684 (0x0694)
[CCMHTTP] ERROR INFO: StatusCode=<unknown> StatusText=    TSMBootstrap    12/22/2020 3:22:59 PM    1684 (0x0694)
HttpRequestResponse( m_sUserAgent.c_str(), szUrl, szMethod, szHeaders, pPayload, dwPayloadLen, 0, uFlags, &httpOptions, ResponseHandler, (LPVOID)&responseData, false, m_eCertAuthResult, m_dwStatusCode, m_sStatusText ), HRESULT=80072f8f (..\ccmhttpget.cpp,815)    TSMBootstrap    12/22/2020 3:22:59 PM    1684 (0x0694)
RequestResponseImpl( szUrl, L"GET", szHeaders, 0, 0, 0, 0, uFlags, &pbResponse, &ulResponseLen), HRESULT=80072f8f (..\ccmhttpget.cpp,297)    TSMBootstrap    12/22/2020 3:22:59 PM    1684 (0x0694)
GetURLSyncInStreamEx2(szUrl, szHeaders, uFlags, &spStream), HRESULT=80072f8f (..\ccmhttpget.cpp,372)    TSMBootstrap    12/22/2020 3:22:59 PM    1684 (0x0694)
spHttpGet->GetURLSyncInStringEx2( sUrl, sAuthHeader, dwFlags, &csResponse), HRESULT=80072f8f (..\ccmtoken.cpp,478)    TSMBootstrap    12/22/2020 3:22:59 PM    1684 (0x0694)
RetrieveTokenFromStsServerImpl failed with error 0x80072f8f    TSMBootstrap    12/22/2020 3:22:59 PM    1684 (0x0694)
Failed to create SMS client object. Error 0x80040154    TSMBootstrap    12/22/2020 3:22:59 PM    1684 (0x0694)
spNS.Open(L"root\\ccm"), HRESULT=8004100e (..\CcmUtilLib.cpp,4350)    TSMBootstrap    12/22/2020 3:22:59 PM    1684 (0x0694)
!sCcmToken.empty() && (ulExpiresIn > 0), HRESULT=87d00215 (..\ccmtoken.cpp,404)    TSMBootstrap    12/22/2020 3:22:59 PM    1684 (0x0694)
CCcmTokenMgr::RetrieveTokenFromStsServer(szPotentialServerUrl, szQueryString, sAuthToken, sToken, ulExpiresIn), HRESULT=87d00215 (..\clientauthutil.cpp,2734)    TSMBootstrap    12/22/2020 3:22:59 PM    1684 (0x0694)
CCM::Authentication::CCMGetTokenForMedia(CCM_PREAUTH_TOKEN_REGISTRATION, sSMSTSMP.c_str(), sMediaToken.c_str(), sMediaGuid.c_str(), pClientCertContext, sMediaBulkToken), HRESULT=87d00215 (tsmediawizardcontrol.cpp,948)    TSMBootstrap    12/22/2020 3:22:59 PM    1684 (0x0694)
Failed to retrieve registration token from the media token, Error code: 0x87d00215    TSMBootstrap    12/22/2020 3:22:59 PM    1684 (0x0694)

Any help is greatly appreciated!

[anyweb]

thanks

can you please confirm that you've done step 5 of this guide here ?

[HeroicBandit]

Yup! Confirmed using the trusted root cert. I've also got that applied to the CMG along with the intermediate. Not sure if its worth noting but part of the reason the CRL checks are disabled in my environment is I had a heck of time troubleshooting all the rejected attempts to have a client communicate with it until I realized our root cert isn't published externally and will not be due to company security and politics.

[anyweb]

If CRL checks are disabled in your environment, then how is the CRL check setting in the properties of your CMG set to ?

[HeroicBandit]

This setting is also unchecked in the CMG properties.

*Edit

Looking at the failure logs further, it seems like its trying to use a token. Having some understanding of the bulk token option for CMG internet only clients, is there any sort of prerequisite for this and such tokens?

[zawtowers]

I'm getting the same error as HeroicBandit in SMSTS.log with the generated boot media.

There's no PKI, and the site is using the Configmgr cert for HTTP site systems.  The only certificate issued by the internal CA is the one with the subject name of our CMG, which is present on the CMG.  Clients are authenticating with token fine, so I know the CMG is working for existing clients.

Is this a limitation because there's no PKI on the site system itself and so has no path to authenticate via the internal CA when using boot media?

[anyweb]

I haven't tested it in non-pki environments and a quick look at the documentation only states this " For version 2010 early update ring, if you use a PKI-based certificate for the boot media, configure it for SHA256 with the Microsoft Enhanced RSA and AES provider. For later releases, including globally available version 2010, this certificate configuration is recommended but not required. The certificate can be a v3 (CNG) certificate. "

In other words, it doesn't call out non-PKI environments or token based auth in this scenario, i'll ping the product group and ask if it's actually supported

 

cheers

niall

[anyweb]

so I got this back from the very knowledgeable Jason in the PG "The problem here appears to be that the cert on the CMG is issued by an internal CA and thus not trusted by the WinPE environment. Using a cert from a public PKI is the only way I know of to get past this (or using a pre-start script to add the issuing PKI as trusted before the TS engine launches)."

 

[zawtowers]

Cheers to you both - that's pretty much what it looks like to me.  The cert on the CMG is the server auth cert (so server auth between the ConfigMgr server and the CMG works, as does token based auth via the CMG to the MP) but of course there's no client auth in there.  In theory replacing the internal CA cert with a public CA cert (and of course ensuring the DNS CNAME is set as well so the CNAME references the CMG instance) would mean WinPE uses its approved CA list to connect to the cert on the CMG. 

It'd definitely be worth testing further from the MS product team to see if that's a viable solution for those using token-based auth, or if there's a way that the bulk registration token could be injected into the client in the boot media (so it has a valid token in the media itself) which could alleviate.

[anyweb]

also, have you seen this post from Eswar, he's not using PKI either but it's working for him

http://eskonr.com/2021/01/certificate-error-while-deploying-an-os-over-cmg-using-bootable-media/

[zawtowers]

23 hours ago, anyweb said:

also, have you seen this post from Eswar, he's not using PKI either but it's working for him

http://eskonr.com/2021/01/certificate-error-while-deploying-an-os-over-cmg-using-bootable-media/

Cheers for that.  I looked at that, which I was already along the same lines of, but realised a couple of things - as below:

• As the PFX cert on the CMG has a path to intermediate and root CA, I specified both .cer files in the site properties
• The CMG config (in the Admin page of the Configmgr console) showed a new version increased by 1, so then synced the config
• I then created the boot media, using a self-signed cert.  It also added the intermediate and root CA certs into the boot image
• I booted, and the SMSTS.log showed it could auth to the CMG (because the certs were both there and full path to the cert on CMG was able to auth)
• I could see the task sequences.

Specifying the root CA alone did not work in the site properties, I had to do the root and intermediate, then update the CMG config and boot media. 

[Marcel_T]

I am getting a Error stating   This task sequence cannot be run because the program files for NLD00005 cannot be located on a distribution point. For more information, contact your system administrator or helpdesk operator.

I have checked the files one the CMG it all line fine. Also  redistribute to it without issues. I am kind of lost now. 

 

SSL, using authenticator in request.    TSMBootstrap    5/28/2021 1:05:29 PM    1480 (0x05C8)
In SSL, but with no client cert.    TSMBootstrap    5/28/2021 1:05:29 PM    1480 (0x05C8)
Request was successful.    TSMBootstrap    5/28/2021 1:05:30 PM    1480 (0x05C8)
::DecompressBuffer(65536)    TSMBootstrap    5/28/2021 1:05:30 PM    1480 (0x05C8)
Decompression (zlib) succeeded: original size 283, uncompressed size 694.    TSMBootstrap    5/28/2021 1:05:30 PM    1480 (0x05C8)
Location Reply:
<![CDATA[<ContentLocationReply SchemaVersion="1.00" BGRVersion="1"><BoundaryGroups BoundaryGroupListRetrieveTime="2021-05-28T11:05:30.120" IsOnVPN="0"><DOINCServers/></BoundaryGroups><ContentInfo/><Sites><Site><MPSite SiteCode="NLD" MasterSiteCode="NLD" SiteLocality="FALLBACK"/><LocationRecords/></Site></Sites><RelatedContentIDs/></ContentLocationReply>]]>    TSMBootstrap    5/28/2021 1:05:30 PM    1480 (0x05C8)
Processing 0 locations.    TSMBootstrap    5/28/2021 1:05:30 PM    1480 (0x05C8)
No static content server.    TSMBootstrap    5/28/2021 1:05:30 PM    1480 (0x05C8)
(LocationsList.size() + slistHttpPaths.size() + slistSMBPaths.size()) > 0, HRESULT=80040102 (..\resolvesource.cpp,2665)    TSMBootstrap    5/28/2021 1:05:30 PM    1480 (0x05C8)
FALSE, HRESULT=80040102 (..\tspolicy.cpp,2441)    TSMBootstrap    5/28/2021 1:05:30 PM    1480 (0x05C8)
Content location request for NLD00005:13 failed. (Code 0x80040102)    TSMBootstrap    5/28/2021 1:05:30 PM    1480 (0x05C8)
hr, HRESULT=80040102 (..\tspolicy.cpp,3346)    TSMBootstrap    5/28/2021 1:05:30 PM    1480 (0x05C8)
Failed to resolve PackageID=NLD00005    TSMBootstrap    5/28/2021 1:05:30 PM    1480 (0x05C8)
(*iTSReference)->Resolve( pTSPolicyManager, dwResolveFlags ), HRESULT=80040102 (..\tspolicy.cpp,4421)    TSMBootstrap    5/28/2021 1:05:30 PM    1480 (0x05C8)
m_pSelectedTaskSequence->Resolve( m_pPolicyManager, TS::Policy::TaskSequence::ResolvePolicy | TS::Policy::TaskSequence::ResolveSource, fpCallbackProc, pv, hCancelEvent), HRESULT=80040102 (tsmediawizardcontrol.cpp,1642)    TSMBootstrap    5/28/2021 1:05:30 PM    1480 (0x05C8)
Failed to resolve selected task sequence dependencies. Code(0x80040102)    TSMBootstrap    5/28/2021 1:05:30 PM    1480 (0x05C8)
hrReturn, HRESULT=80040102 (tsmediaresolveprogresspage.cpp,445)    TSMBootstrap    5/28/2021 1:05:30 PM    1480 (0x05C8)
ThreadToResolveAndExecuteTaskSequence failed. Code(0x80040102)    TSMBootstrap    5/28/2021 1:05:30 PM    1480 (0x05C8)
ThreadToResolveAndExecuteTaskSequence returned code 0x80040102    TSMBootstrap    5/28/2021 1:05:30 PM    1188 (0x04A4)
Setting wizard error: This task sequence cannot be run because the program files for NLD00005 cannot be located on a distribution point. For more information, contact your system administrator or helpdesk operator.    TSMBootstrap    5/28/2021 1:05:30 PM    1188 (0x04A4)
ResolveProgressPage::OnWizardNext()    TSMBootstrap    5/28/2021 1:05:30 PM    1188 (0x04A4)
Activating Finish Page.    TSMBootstrap    5/28/2021 1:05:30 PM    1188 (0x04A4)