Quick and Dirty Image Factory with MDT and PowerShell

I haven’t written a blog in a while, been busy with the new job at Tanium, but I did write this script recently, and thought I would share, in case anyone else found it interesting. Share it forwards.

Problem

Been working on solutions to upgrade Windows 7 to Windows 10 using Tanium as the delivery platform (it’s pretty awesome if I do say so my self). But as with all solutions, I need to test the system with some end to end tests.

As with most of my OS Deployment work, the Code was easy, the testing is HARD!

So I needed to create some Windows 7 Images with the latest Updates. MDT to the rescue! I created A MDT Deployment Share (thanks Ashish ;^), then created a Media Share to contain each Task Sequence. With some fancy CustomSettings.ini work and some PowerShell glue logic, I can now re-create the latest Windows 7 SP1 patched VHD and/or WIM file at moment’s notice.

Solution

First of all, you need a MDT Deployment Share, with a standard Build and Capture Task Sequence. A Build and Capture Task Sequence is just the standard Client.xml task sequence but we’ll override it to capture the image at the end.

In my case, I decided NOT to use MDT to capture the image into a WIM file at the end of the Task Sequence. Instead, I just have MDT perform the Sysprep and shut down. Then I can use PowerShell on the Host to perform the conversion from VHDX to WIM.

And when I say Host, I mean that all of my reference Images are built using Hyper-V, that way I don’t have any excess OEM driver junk, and I can spin up the process at any time.

In order to fully automate the process, for each MDT “Media” entry. I add the following line into the BootStrap.ini file:

    SkipBDDWelcome=YES

and the following lines into my CustomSettings.ini file:

    SKIPWIZARD=YES            ; Skip Starting Wizards
    SKIPFINALSUMMARY=YES      ; Skip Closing Wizards 
    ComputerName=*            ; Auto-Generate a random Computer Name
    DoCapture=SYSPREP         ; Run SysPrep, but don't capture the WIM.
    FINISHACTION=SHUTDOWN     ; Just Shutdown
    AdminPassword=P@ssw0rd    ; Any Password
    TASKSEQUENCEID=ICS001     ; The ID for your TaskSequence (Upper Case)

Now it’s just a matter of building the LitetouchMedia.iso image, mounting to a Hyper-V Virtual Machine, and capturing the results.

Orchestrator

What I present here is the Powershell script used to orchestrate the creation of a VHDX file from a MDT Litetouch Media Build.

• The script will prompt for the location of your MDT Deployment Share. Or you can pass in as a command line argument.
• The script will open up the Deployment Share and enumerate through all Media Entries, Prompting you to select which one to use.
• For each Media Entry selected, the script will
  • Force MDT to update the Media build (just to be sure)
  • Create a New Virtual Machine (and blow away the old one)
  • Create a New VHD file, and Mount into the Virtual Machine
  • Mount the LitetouchMedia.iso file into the Virtual Machine
  • Start the VM
• The script will wait for MDT to auto generate the build.
• Once Done, for each Media Entry Selected, the script will
  • Dismount the VHDx
  • Create a WIM file (Compression Type none)
  • Auto Generate a cleaned VHDx file

Code

The code shows how to use Powershell to:

• Connect to an existing MDT Deployment Share
• Extract out Media information, and rebuild Media
• How to create a Virtual Machine and assign resources
• How to monitor a Virtual Machine
• How to capture and apply WIM images to VHDx virtual Disks

Notes

I’ve been struggling with how to create a MDT VHDx file with the smallest possible size. I tried tools like Optimize-Drive and sDelete.exe to clear out as much space as possible, but I’ve been disappointed with the results. So here I’m using a technique to Capture the VHDx file as a Volume to a WIM file (uncompressed for speed), and the apply the Capture back to a new VHDx file. That should ensure that no deleted files are transferred. Overall results are good:

Before:   19.5 GB VHDx file --> 7.4 GB compressed zip
After:    13.5 GB VHDx file --> 5.6 GB compressed zip

Links

Gist: https://gist.github.com/keithga/21007d2aeb310a57f58392dfa0bdfcc2

https://wordpress.com/read/feeds/26139167/posts/2120718261

https://community.tanium.com/s/article/How-to-execute-a-Windows-10-upgrade-with-Tanium-Deploy-Setup

https://community.tanium.com/s/article/How-to-execute-a-Windows-10-upgrade-with-Tanium-Deploy-The-Sensors

https://community.tanium.com/s/article/How-to-execute-a-Windows-10-upgrade-with-Tanium-Deploy-Setup

 

Advertisements

Dell XPS 13 9360 Hardware Reset

TL;DR – If you are having some spontaneous errors start your laptop try disconnecting your batteries for an hour, and try again.

XPS 13 9360

Got a new laptop last month, it was time to replace the old one. Did some searching online and found something light, powerful, and at a good price. Dell XPS 13 9360:

•  8th Gen Intel® Core i7-8550U  (*Quad Core*)
• 512GB PCIe Solid Drive Drive (*NVMe Drive*)
• 16GB LPDDR3 1866MHz RAM
• 1x Thunderbolt port
• 13.3″ Touchscreen InfinityEdge QHD+ (3200 x 1800) Display

On sale at Costco for $1400. Overall a good value for a quad core laptop with NVMe.

The Break

Came back from a meeting (Starbucks? :)) Friday and the machine failed to boot. Got some display errors, rebooted, but got the recovery screen. So I shutdown for a while, when I rebooted, nothing. No Screen nothing.

However I did notice that the LED on the front was blinking, and I was able to catch the pattern, 2 and 7. Looking up in the service manual:

LCD Error!?!?! Crap.

A call to Dell Support confirmed the error, and a RMA ticket was generated, it could be two weeks before I get it back.

Battery

I wanted to archive the contents of the Disk before I sent it off to dell, so I got out my Torx screw driver.

But while I had the case open I disconnected the main Battery and the CMOS battery.

With most modern PC’s each of the components have small computers built in them. If they develop errors, do they reboot like the main OS when the power is off? If the battery is always connected that might not be true.  I had a similar problems recently with my SuperMicro test box, where flashing the BIOS wasn’t helping to resolve a complex problem I had with the box. Draining the CMOS battery and re-flashing the BIOS did work!

After an hour, I plugged in the batteries, and tried booting again. Yea, the machine works! It’s alive! I don’t have to send my machine in for repair.

Hopefully the machine will work a little bit longer than 45 days. We’ll know soon.

 

Silence is Golden during Setup

Thanks to @gwblok for pointing me to this twitter thread about Windows OOBE Setup.

Or rather, disable it first, and then enable it after OOBE is completed (in the task sequence). pic.twitter.com/ZCFTjr0lOs

— Johan Arwidmark (@jarwidmark) December 12, 2017

When Unattended is not Silent

During Windows 10 OOBE, the Windows Welcome process uses the Cortana voice engine to speak during Windows Setup.

Now we can go look for any updates

Shut up!

Yes, I’m one of those guys who sets my Sound Profile to “silent”, Silence is Golden!

And if I’m going to be running several Windows Deployments in my lab (read my home office), then I would prefer the machine to be silent. Reminds me of the XP/Vista days when we had boot up sounds. How rude.

So how to disable… Well the answer doesn’t appear to be that straight forwards.

SkipMachineOOBE

At first I suggested SkipMachineOOBE, and works on my test machine! Yea!

Then I got a reminder that SkipMachineOOBE is deprecated according to documentation.

DisableVoice

Thanks to @Jarwidmark for pointing me in the thread above to:

reg.exe add HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\OOBE /v DisableVoice /t REG_DWORD /d 1

However, Microsoft Documentation also states that you should only use this for testing, and that Cortana Voice should be-enabled for users. OK… Fine, we’ll delete the key after setup is complete.

So where to place all this stuff?

Specialize

Several people suggested modifying the local registry within the imaging process, but I would prefer to avoid that, instead trying to see if we can perform the action during Setup using our unattend.xml file.

The command to disable would need to be *before* “OOBE”, sounds like the perfect job for the “Specialize” process.

Some quick testing, verified, and we are ready to go.

Automating OOBE

So, given the guidance from Microsoft on how to automate Windows 10:

https://docs.microsoft.com/en-us/windows-hardware/manufacture/desktop/settings-for-automating-oobe

Here are my changes:

• We disable Cortana during the Specialize Pass before OOBE.
• Then during OOBE, we clear the Cortana setting, and continue.

 

Windows 10 In-Place Security Considerations

TL;DR – When performing a Windows 10 In-Place upgrade, you must temporarily suspend any Disk Encryption protections, for BitLocker *AND* 3^rd party disk encryption too!

In Place Upgrade

So, how do we upgrade an Operating System? You know, the one we are currently using? Can we still upgrade it while still in use? Unfortunately, no. The Windows 10 In-Place process is very complex, and it requires full access to all the files on the machine. So how do we do that? Well, the upgrade process needs to shift to another OS, just temporarily, to modify the OS on our C:\ drive, we can use WinPE (Windows Pre-Installation Environment), or in this case WinRE (Windows Recovery Environment).

WinPE and WinRE are lightweight OS’es that are contained in a compressed boot.wim file, about 300MB to 500MB in size, and placed somewhere on the disk. We can boot into WinPE/RE and have it completely reside in memory. Now we have full access to the C:\ drive on the machine, and we can move files around and including a new OS.

3^rd Party Drivers

One of the challenges of shifting to a separate OS like WinPE/WinRE is that we’ll need to re-load any drivers required to access the system, including Disk and File System Drivers. For the most part, the latest versions of WinPE/WinRE will have very excellent support for the latest disk controller drivers. And it’s very rare that I’ve had needed to supply 3^rd party drivers for mainstream hardware. Starting with Windows 10 1607, Microsoft gives us the ability to add 3^rd party Drivers to the WinPE/WinRE using the /ReflectDrivers switch. This includes the ability to supply drivers for a Storage Controller or even a 3^rd party Disk Encryption tool. Anything that is required to access the machine.

Suspending Encryption

First some background…

At my house I have a Lock Box like this. I can place my house key in the box, and if someone needs to get into the house, I can just give them the code to the lock box. This is much better than giving everyone their own key, or just leaving the main door unlocked while I’m out. If I want to revoke access, I just change the code on the lock box, rather than re-keying my whole house.

If you have an OS disk that is encrypted, and you want to upgrade the OS, you probably don’t want to decrypt the ENTIRE disk before the OS upgrade, and re-encrypt the disk when the new OS is ready, that would take time to read and write data to the entire disk. Instead it would be better if we could leave the disk encrypted, and just temporarily give the setup system full access. It’s similar to the Lock box analogy above, we don’t want to give everyone access to the main encryption key, but the system will allow access at the right time to the right users.

For Microsoft BitLocker, the process is called “suspending”. We leave the disk encrypted, but the encryption keys for the disk are no longer protected. When the new OS is installed, we can re-establish protection via our usual protectors like TPM, SmartCard, Password, etc…

3^rd party encryption products need to function in the same way.  We would like to leave the disk encrypted, but any protections like “Pre-Boot authentication”, should be disabled, so the WinPE/WinRE Operating System, with the correct Encryption filter drivers have full access to the main OS. When finished, we can re-establish any Pre-Boot authentication protections supported by the encryption software like Passwords, TPM chips, Smart Cards, etc…  If the 3^rd party disk encryption product does not offer this then the WinPE/WinRE OS won’t be able to access the local C:\ Drive.

Misconceptions

I’ve been working with a client lately whose security team has correctly identified the In-Place Upgrade-Suspending Encryption behavior I described above. However, they incorrectly prescribe this as a vulnerability of BitLocker, and have not acknowledged that it is also a vulnerability of other 3^rd party disk encryption tools.

First off, yes, this is a known security Vulnerability in the way Windows 10 handles In-Place Upgrades, we simply must temporarily suspend protections as we move off to offline OS, this is by design. More below…

Secondly, It’s disingenuous to claim that this is only a BitLocker problem, by the design of the current Windows 10 In-Place upgrade system with the /ReflectDrivers hook, 3^rd party disk encryption tools must also suspend protections so the WinPE/WinRE offline OS’es.

This is really important for fully automated In-Place upgrade scenarios like MDT Litetouch or System Center Configuration Manger (SCCM) OSD (Operating System Deployment) tools.

Mitigations

Well, it’s not all gloom and doom, It’s not perfect, but like most things related to security, there are compromises, and tradeoffs.

Note that your data at-rest, protected by encryption, is only one potential threat vector where bad guys can get your data. There is also Malware, OS bugs, and other vectors that are made more secure with the latest Windows Releases. It *IS* important to keep your machine up to date and healthy with the latest OS and security tools, and simply avoiding upgrades because you don’t want to expose your machine, isn’t the best solution.

But there are also techniques/mitigations we can do to limit the exposure of your data during In-Place Upgrades. You will, of course, need to perform your own threat analysis. Some ideas might be:

• Don’t allow Upgrades to be performed in an automated fashion, always run attended. (not possible in some large environments).

• Only allow Upgrades to be performed on site, in semi-secured environments. Never over VPN or Wi-FI
• If you are running in a SCCM environment, we could develop some scripts/tools to monitor Upgrades. If a machine hasn’t returned from In-Place upgrade after XX minutes, then auto-open a Support Ticket, and immediately dispatch a tech.

-k

Install Windows 10 on Surface 1TB with MDT

TL;DR – Here is a script to get past the Disk0 to Disk2 mapping issue with the new Surface Pro with a 1TB drive.

Surface Hardware

OK, first a bit of history, I used to work for the Surface Imaging team back in 2015 to 2016. Overall a cool job, I learned a lot, and got to sharpen my PowerShell coding skills.

During that time I got to see my first Surface Studio device, even before it was released. Once of the unique features of the device was it’s unique disk architecture, it contains TWO disk drives, one a SSD in a M.2 format, and a Spinning Hard disk in a 2.5″ format. The OS contains a driver that uses the SSD as a cache. The idea is that you get the size of the 2TB hard disk, with (generally) the speed of the SSD.

Of course this creates a problem for OS deployment because we need to load the Special Caching driver into WinPE before deployment, so both drives are properly identified.

The Surface Pro with the 1TB drive is also unique in this manner, on the inside it isn’t a single 1TB drive, instead it’s two 512GB drives running in a Raid 0 configuration.

So you’re probably wondering how this works within WinPE on MDT, well the good news is that the built in 1709 drivers can correctly identify the two SSD disk as a single 1TB drive…

… The only problem is that it’s identified as Disk(2), and that breaks stuff.

ZTIDiskPart

Yes, yes, I know… mea culpa…

MDT (and SCCM/OSD) make an assumption on the “Format and Partition Disk” step: The target disk number is fixed for each Task Sequence. Now, we can change the target disk dynamically within the Task Sequence by chaning the OSDDiskIndex variable. But it will require some coding.

Fix 1

One fix, if you are OK with some WMI queries, is to test for a “Surface Pro” model and a 1TB disk at index 2. I would prefer to test for the ABSENCE of a disk at index 0, but not sure how to do that.

Fix 2

The following is a modification of my ZTISelectBootDisk.wsf script. Designed specifically for this configuration. Just drop it into the scripts folder and add a step in the Task Sequence before the “Format and Partition disk step.

Testing

Now this script has NOT been tested on a 1TB Surface device. However I *AM* available for testing the 1TB surface device. I can forward my home mailing address, if you want to send me one :^).

 

Create a Remote Desktop Conn. Mgr. file for Azure

Every now and then, I will spin up a test environment in azure with a couple of Virtual Machines. And as much as enjoy the User interface of the Portal, there have been times that I longed for the ease of use of the Remote Desktop Connection Manager.

One of the frustrations is that the *.rdp files downloaded from Azure run at full screen, which I find awkward, so the first thing I do is bring down the resolution, which is a pain, when I have several machines.

So I decided to create a tool to auto-generate a Remote Desktop Configuration Manager configuration file.

This also gave me the opportunity to play around with the way *.rdg stores passwords.

RDG security

First a word about security.

The Remote Desktop Connection Manager uses the “Data Protection API” to “Encrypt” passwords stored within the *.rdg file.

The great thing about this API, is that if another user were to open this file on another machine, it can’t be read. Only the user running on the same machine can extract this password.

Note that any program or script running under your user’s context can read this password as plaintext, works great for this script, but my one design change to the “Remote Desktop Connection Manager” would be to add a “PIN” or other layer of security abstraction to prevent other “rouge” processes or users from gaining access to the passwords stored locally on my machine.

Azure

This script will log into your Azure Account, enumerate all the service groups, and get a list of all Virtual Machines within these groups, Creating entries within the “Remote Desktop Connection Manager” for each Azure Virtual Machine.

Script

-k

Notes on Microsoft ADV170012 – TPM Madness.

Hidden within the latest Microsoft Security Advisory is a Whooper: ADV170012

https://portal.msrc.microsoft.com/en-US/security-guidance/advisory/ADV170012

The summary is that some of the Infineon TPM chip implementations have a bug. And appears that someone has produced a Proof of Concept exploit. Wow.

Microsoft and Infineon have arguably done the right thing here and have announced the issue, produced a Hotfix to help customers better identify the issue, and have developed tools to update the issue through firmware.

What’s not clear to me is just what the issue is, and what the hotfix does. Unfortunately, it may be a while before Microsoft releases more information, while they give companies a head start working on application of the hotfixes.

A link in the article above suggest that exploit is not easy:

A successful attack depends on conditions beyond the attacker’s control. That is, a successful attack cannot be accomplished at will, but requires the attacker to invest in some measurable amount of effort in preparation or execution against the vulnerable component before a successful attack can be expected.

Which leads me to believe that any exploit is hard, requiring a highly skilled attacker, not someone who is going to steal my laptop from the local Starbucks in the hopes of getting my Credit Card number, saved somewhere on the machine.

Stay tuned…

Script

In the mean time, I decided to re-write the PowerShell script in the article above. The latest version works great when issuing commands remotely and collecting the data in a centralized location.

For example I could run the command:

icm { iwr 'https://gist.githubusercontent.com/keithga/22aa4500de40bc174f2f4921052e3b87/raw/Test-TPMReimann.ps1' | iex } -Cred $cred -Computer Pickett1,Pickett2,Pickett3,Pickett4 | ft *

And see the output:

Lucky for me I have Four machines that are affected with the Bad TPM Module.

• One machine is my work machine (Version 6.41)
• Two machines don’t have bad Infineon version numbers (Verison 3.19), but may need to be cleared anyways. Easy to do.
• One machine has the bad Infineon version (Version 4.32), but the TPM Module is on a replacement Riser card, and I can purchase a new one for $50.

Now to figure out how to address this at work.

Code

Notes

Don’t recall why I  named it Reimann, I think I saw that as a code word in an article somewhere, and it stuck. Is the name of the researcher who found the issue, or just an arbitrary code name?

Not sure why you need to know when the last Provision Time was on machines *without* the issue. Either the TPM chips works or not!?!?