Formatting a removable USB drive with 2 partitions

TL;DR – Starting with Windows 10 Insider Preview Build 14965, you can format any “Removable” USB Flash Drive with more than one partition. Perfect for installation of large (over 4GB) WIM files on UEFI machines!

 

Hey all, back from a week at the Microsoft MVP summit, a Week in the UK, and a week in Arizona.

A few weeks ago at the Microsoft MVP summit, an engineering manager with the Windows Product group made an offhand comment about formatting a removable USB drive with two partitions. This took several of us by surprise, because historically, this hasn’t been supported widely without converting to a Fixed disk or something.

Mike Terrill (and Mike Niehaus) already beat me to the punch with some posts, but I wanted to share my results. :^)

The Background

Why is this important? Well as I mentioned in another blog post, as more and more people are booting to UEFI, on USB flash drives formatted with Fat32, with WIM images over 4GB in size, that causes a problem because Fat32 can’t hold files over 4GB in size.

Another solution would be to use the Rufus tool to split a USB drive into multiple partitions with a hidden fat32 partition. However, the problem here is that the hidden partition uses a special UEFI app that is not signed, so it won’t work on UEFI machines with Secure Boot enabled.

This has become even more interesting since Windows Server 2016 came out, with a base WIM image for standard Server SKU that is over 4GB in size. Hum…

The Hardware

20161126_200856.jpg

I tested on several different USB makes using my Windows 10 (version 1607) laptop. Some would allow me to create a 2nd partition on a removable Flash Drive, others would not giving me an error:

DISKPART> create part pri

No usable free extent could be found. It may be that there is insufficient 
free space to create a partition at the specified size and offset. Specify
different size and offset values or don't specify either to create the maximum 
sized partition. It may be that the disk is partitioned using the MBR disk
partitioning format and the disk contains either 4 primary partitions, (no
more partitions may be created), or 3 primary partitions and one extended
partition, (only logical drives may be created).

Mostly the older and/or cheaper drives didn’t work, but most of the newer and/or name brand drives did work.

Finally I narrowed it down to two different models, both my favorites:

Then I tested against three Operating Systems: Windows 10 Version 1607, Windows 10 Preview, and Windwos 7.0 SP1. All using Diskpart to create multiple partitions.

The script

Diskpart.exe –>

sel disk 1
clean
create part pri size=450
format quick fs=fat32
assign
create part pri
format quick fs=ntfs
assign
exit

The Results:

                                 SanDisk           Transcend
Windows 7 SP1 Build 7601           Pass               Fail
Windows 10  Version 1607           Pass               Fail
Windows 10 Preview 14965           Pass               Pass   

I was able to format my SanDisk into multiple partitions using Windows 7 and beyond.

But I was not able to format the Transcend drive into multiple partitions using Windows 7 or Windows 10 Version 1607, but I was able to partition into multiple partitions on the new Windows 10 Insider Preview 14965.

That’s new!

I haven’t done enough testing using the removable flash drives on older machines, to see if the partitions are still visible, but the results look promising for a start.

Update #1 – 11/28/16:

Found out today that the reason that my SanDisk Extreme disk worked on Windows 7 and Windows 10 1607 may be because the removable Flash disk is reported as “Fixed” rather than “Removable” to the OS. Link.

Update #2 – 11/28/16:

I noticed that when taking the “removable” disk formatted with 2 partitions from Windows 10 Preview 14965 over to Windows 10 Version 1607, only the first partition was visible. As a work around I tried moving the main NTFS partition first and the Fat32 partition second.

sel disk 1
clean
create part pri
shrink desired=450
format quick fs=ntfs
assign
create part pri
format quick fs=fat32
assign
exit

Display WPF XAML code in PowerShell

Last week I went to the Minnesota Management Summit at the Mall of America #MMSMOA, and I got inspired to work on a few projects in my backlog.

One of the presentations I went to was with Ryan Ephgrave (@EphingPosh on Twitter.com), and his talk on “Better Know a PowerShell UI“.

Overall it was a great presentation, and I learned a lot. And got me thinking about some of the things I could do with a framework I started earlier in the year but never got around to finishing.

WPF4PS

Without further adieu, I present Windows Presentation Framework for PowerShell (WPF4PS). It is also the first project I’ve released as source code on GitHub:

https://github.com/keithga/WPF4PS

Background

Most WPF + PowerShell examples are created with a lot of custom code to add in event handlers for the User Interface elements. The goal is to find all control elements on the page and if there is a pre-defined function created, then use it. Which means minimal code for overhead.

Example

Here is a fully functional example:

  • Load the WPF4PS module
  • Import a XAML defined in Visual Studio
  • Create a scriptBlock to handle the button Click
  • Create a HashTable to pass data between our script that the XAML Window
  • Call the Show-XAMLWindow function
  • Get the value of the TextBox from the Hash

wpf4ps

<#
.SYNOPSIS
WPF4PS framework Examples

.DESCRIPTION
Simple Example

.NOTES
Copyright Keith Garner, All rights reserved.

#>

[cmdletbinding()]
param()

import-module $PSScriptRoot\wpf4ps -force

$MyXAML = @"
<Window x:Class="WpfApplication1.MainWindow"
 xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
 xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
 xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
 xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
 xmlns:local="clr-namespace:WpfApplication1"
 mc:Ignorable="d"
 Title="MainWindow" FontFamily="Segoe WP Semibold" Width="400" Height="300" Name="WindowMain" >
 <Grid>
 <Label>Hello World</Label>
 <Button x:Name="Button1" Content="Big Red Button" Width="125" Height="25" Background="#FFDD0000" Margin="0,60,0,0"/>
 <TextBox x:Name="textBox1" Height="23" Width="200" />
 </Grid>
</Window>
"@

$MyControl = [scriptBlock]{

    function global:button1_click()
    {
        "Click the Big Red Button`n" + $TextBox1.TExt  | show-MessageBox 
        $WindowMain.Close()
    }

}

$MyHash = [Hashtable]::Synchronized(@{ textBox1 = "Hello World" })

Show-XAMLWindow -XAML $MyXAML -ControlScripts $MyControl -SyncHash $MyHash

$MyHash.TextBox1 | Write-Host

Next

My goal is to work out the kinks and eventually upload/share this on PowerShellGallery.com.

For example:

  • I created two Show-XAMLWindow() functions in the library, one inline and another Async. I still don’t know what the usage case of Async is.
  • Ryan Ephgrave did some XAML + Powershell examples in his “Better Know a PowerShell UI” blog series with XAML “Binding” elements, something I have not used in the past, so I excluded them from this package.
  • I had to do some weirdness with the declaring the functions above as “global” to make them visible to the Module

If you have feedback on the layout or usage, please let me know.

-k

Fix for Windows 1511 ADK bug

First off, yes, I have a new job working for 1e! I’m super excited, and I should have posted something about it, but I’ve been super busy. My first day on the job was at a customer site in Dallas, and I’ve been on the go ever since, working on this and that (stay tuned :^).

As many of you may have known, there has been a pretty big bug in the Windows 10 Version 1511 ADK, it’s caused all kinds of interop problems with Configuration Manager. Well Microsoft released a fix today! KB3143760. Yea!

Well I opened up KB3143760, and yikes! The instructions are a bit dry. Mount this, patch that, watch out for the data streams!

I needed to patch my local Windows 1511 ADK installation because I’m working on a SCCM+MDT Refresh scenario, and I don’t want to uninstall the 1511 ADK. Perfect timing, if only there was a way to automate this..

Repair-1511ADK.ps1

Here is a link to a PowerShell script I wrote to auto-magically patch your WinPE files!

https://onedrive.live.com/redir?resid=5407B03614346A99!158500&authkey=!AHWArN5C7FyRPIY&ithint=file%2cps1

This script will:

  • Download the patch (no need to go through the E-Mail process)
  • Take care of all the stream issues (really I don’t use IE/Edge, so no security streams)
  • Auto extract the patch contents
  • Mount the wim file
  • Patch the appropriate dat files
  • Fix the permissions
  • Dismounts the WIM
  • Cleans up all left over files

So, for example, if you wanted to patch all of the WinPE Wim files in the ADK directory (before importing them into SCCM), you can run the following command:

get-childitem 'C:\Program Files (x86)\Windows Kits\10\Assessment and Deployment Kit\*.wim' -recurse | .\Repair-1511ADK.ps1 -verbose

Lately, when programming in PowerShell, I have taken the “write-host considered harmful” rule to heart, so by default, there is *NO* std console output. Instead, I redirect most information output to “verbose”, so if you want to see what is happening in the background, use the -verbose switch.

-k

Hopefully, moving forwards, this will be the *last* time I place a new script up on OneDrive, really I should be moving towards something more… modern… like GitHub.

MDT package now on Chocolatey.org ready for Windows 10!

Been a while since I posted, I’ve been busy with Surface, Windows 10, and other Kits. But my chocolatey package just got approved, so I thought I would share.

I’ve been following the progress of PowerShell’s OneGet, and http://Chocolatey.org for a while now, and thought it was time to stick my toes in and create a package for public use. MDT seemed like a great start.

As you may already know OneGet is a new feature of PowerShell, included in Windows 10 and available through WMF 5.0 that allows for the installation of packages over the internet. Chocolatey is one of the back-end providers, with a great collection of apps ready for installation.

With the recent release of MDT 2013 Update 2, it seemed like a great opportunity to practice my packaging skills. Eventually I created a PowerShell script to auto generate the chocolatey package (not shown here), it would download the MSI files, and extract out the MSI Product Code and Checksum values. You can see the code generated on the Chocolatey MDT page.

Now to install MDT on Windows 10 (or Windows Server 2016), we can run the commands:

set-executionpolicy RemoteSigned; 
Install-Package -Name MDT -ProviderName Chocolatey `
-ForceBootstrap -Force -Verbose

How it works

First step we need to do on clean machine is to set the execution policy:

set-executionpolicy RemoteSigned

Chocolatey has some PowerShell scripts that run in the background, so we need to allow PowerShell to run these commands with the Set-ExecutionPolicy command. Most Powershell users run this command anyways, so it’s not that uncommon.

Then we install the package using the PowerShell 5.0 “Install-Package” cmdlet built into Windows 10:

Install-Package -Name MDT -ProviderName Chocolatey

We must specify the “-ProviderName Chocolatey” parameter the fist time we call Install-Package so the chocolatey Provider is installed, MDT is only known to Chocolatey at this time.

Install-Package will prompt us to confirm installation of the chocolatey provider, we can skip this with the -ForceBootStrap parameter. Additionally, Install-Package will also ask for confirmation before installing MDT, and we can sip the confirmation with the -Force Paramater.

I like to see what is going on the background, so I add the -verbose parameter, and my screen fills with yellow:

Capture

We can see Install-Package downloading MicrosoftDeploymentToolkit2013_x64.msi from the Microsoft web servers.

ADK

The Windows 10 ADK package has also been uploaded to Chocolatey, but hasn’t been officially approved yet, so when you try to run the “windows-ADK” package it will install the older Windows 8.1 version. We can force the Windows 10 ADK to install with a version parameter. Additionally, the default version of the “Windows-ADK” package does not install USMT, so to install everything we will need the “windows-adk-all” package (which is a lot of stuff, sorry).

install-package -ProviderName Chocolatey -Name Windows-ADK-All `
-force -Verbose -MinimumVersion 10.1.10586.0

More information:

https://chocolatey.org/packages/MDT

-k

MDT UberBug11 – Security vs Usability

(Haven’t posted in a while, been busy with my day job(s), travel, scripting, sleep :^).

Some of you have recently noticed that MDT 2013 Update 1 has changed the way it sets the permissions on the network shares when it creates them for the first time.

When you create a new deployment share in MDT, it will ask you what the network share should be, and the wizard will automatically create the necessary bindings for the share and your local path, including setting up the permissions. Super!

Capture

The old permissions used to be “Everyone” has full access, it’s now set to “CREATOR OWNER”. I was somewhat confused by this change, and seemed a bit arbitrary to me. I suspect that someone filed a bug against MDT thinking that locking down the deployment share in the most restrictive way possible would somehow be better, because you know… Security! Think of the security breaches at Home Depot, and Target. PKI. Oh the humanity.

Well, MDT deployment shares don’t really store sensitive information, if you *DO* store any sensitive information on a MDT deployment share, then you are doing it wrong.

But anyways, someone made the decision to lock down the share, although “CREATOR OWNER”, this is kind of confusing to me, only one user? Why not use local “Administrators” group, local administrators already have full access to the files. “CREATOR OWNER” might only give access to one of several local “Administrators”.

Additionally, “Everyone” isn’t really that bad, access files over the network, you are still limited to the “File” level permissions on each file, which are better IMHO, I can create a Logging directory with Create/Write permissions, and set everything else to “Everyone” Read, with “Administrators” “Full R/W”

See: https://keithga.wordpress.com/2015/01/06/security-week-locking-down-your-deployment/

Anyways, this new permissions change for MDT 2013 Update 1 hasn’t caused much of a problem, as most users can easily work around the issue by adding extra approved users to the share afterwards.

Bootstrap.ini

Got a question today about a missing DeployRoot varaiable in BootStrap.ini.

MDT uses BootStrap.ini in WinPE to remember where to find the DeploymentShare to do the heavy lifting.

Normally, when creating a new DeploymentShare, MDT will automatically update the DeployRoot variable in BootStrap.ini, however several users were observing that this was no longer getting updated.

I used my trusty ILSpy to disassemble “C:\Program Files\Microsoft Deployment Toolkit\Bin\Microsoft.BDD.PSSnapIn.dll” and look at the “Provider class”, what I observed is this code segment:

IniManager iniManager = new IniManager(deploymentPointSettings["UNCPath"] + "\\Control\\Bootstrap.ini");
iniManager.Write("Default", "DeployRoot", deploymentPointSettings["UNCPath"]);
asdfsdf

You can see here that MDT is attempting to open the Bootstrap.ini file and write the Path to the DeployRoot Value.

Note that MDT is trying to load the Bootstrap.ini file using the same “UNCPath”? I suspected that MDT was failing to open the file due to the restrictive “Creator Owner” permissions, Sure enough, I tried opening the file over the network and it failed. Found!

Work Around

After creating a new deployment share in MDT, be sure to go back and fix some of the defaults:

  • Change the permissions, Something more permissive like local “Adminsitrators”
  • Change the \\server\deploymentshare$\control\bootstrap.ini to include
    deployroot=\\server\deploymentshare$
  • <More to follow I as do more testing>

MDT Bug: 451130 (known issue)

-k

Deployment Gotcha – Get Windows 10 App doesn’t work with MDT

Did you know that you can download *.iso images of Windows 10 using the Get Windows 10 app?

http://www.microsoft.com/en-us/software-download/windows10

It’s a tool that will allow you to download *.iso images for deployment.

The only problem is that it doesn’t actually download *.iso images, instead, it downloads *.esd images, which are highly compressed *.wim files, that are encrypted/encoded. The tool then decrypts/decodes the *.esd file into *.wim files, and constructs a *.iso image for use.

Additionally, the *.wim file is still a highly compressed *.esd file, so legacy tools that leverage the WIMGAPI library like imagex.exe don’t understand it.  Dism works fine.

C:\>"c:\Program Files (x86)\Windows Kits\10\Assessment and Deployment Kit\Deployment Tools\amd64\DISM\imagex.exe" /info g:\getwin10\Windows\sources\install.wim
ImageX Tool for Windows
 Copyright (C) Microsoft Corp. All rights reserved.
 Version: 10.0.10011.16384

Error opening file [g:\getwin10\Windows\sources\install.wim].

An attempt was made to load a program with an incorrect format.
C:\>dism /get-wiminfo /wimfile:g:\getwin10\Windows\sources\install.wim

Deployment Image Servicing and Management tool
 Version: 6.3.9600.17031

Details for image : g:\getwin10\Windows\sources\install.wim
 Index : 1
 Name : Windows 10 Pro
 Description : Windows 10 Pro
 Size : 9,343,966,034 bytes

The operation completed successfully.

That means that you can’t import the images into MDT, because MDT uses the WIMGAPI libarary, and powershell commands might not work either.

I tried to export the *.wim file to another *.wim file, but DISM.exe doesn’t like doing that either.

As a work around you might be able to export the *.wim file to a *.vhd container, and re-capture back to *.wim file with normal compression.

Johan has been documenting esd to wim techniques on his blog.

http://deploymentresearch.com/Research/Post/399/How-to-REALLY-create-a-Windows-10-ISO-no-3rd-party-tools-needed

Otherwise, I hope the DISM team fixes this for TH2 :^) <hint> <hint>

 

 

MDT UberBug09 – BitLocker broken on uEFI machines for Win10

Got a IM today from a Consultant who was having trouble creating an end to end deployment of Windows 10 for some Surface Pro 3 devices.   Hey, I love Surface Pro 3 devices ( I have 3 in my office :^), so I thought I would help out.

The error message was when he tried to enable BitLocker, but that turned out to be a red-herring.

The Error

He had enabled BitLocker Pre-Provisioning and was trying to enable the protectors at the end, but kept getting errors like

BLError

This PC Doesn’t support entering a BitLocker recovery password during startup. Ask your administrator to configure Windows Recovery Environment so that you can use BitLocker.

What does WinRE have to do with bitlocker? OK, we ran REAgentC.exe /Info to see what the status of Recovery was, and turns out that it was *NOT* installed.

OK Let’s run the REAgentC.exe /enable command to turn it on:

reagentc_enable

REAGENTC.EXE: Windows RE cannot be enabled on a volume with BitLocker Drive Encryption enabled.

BitLocker wont’ start because it’s missing WinRE, and WinRE won’t install because of BitLocker. Super!!

Well after digging some more, we found out from the panther logs that the REAgentC.exe command was attempting to copy the WinRE.wim file from the OS partition to the WinRE partition created by MDT’s ZTIDiskPart.wsf but failed because the size was too small.  Too small?

The Root Cause

I Went back and looked at some WinRE sizes from current and past OS’es:

  • Windows 7 SP1 x86 – 145,287,084
  • Windows 7 SP1 x64 – 169,213,970
  • Windows 8.1 Update x86 – 193,132,205
  • Windows 8.1 Update x64 – 236,122,267
  • Windows 10 x86 – 238,226,804
  • Windows 10 x64 – 302,808,595

Wow, the Windows 10 x64 WinRE image is *far* bigger than the other versions.

Well, sometimes MDT can use the LitetouchPE WIM file as a WinRE file, how does the Win 10 ADK look:

  • LiteTouchPE_x64.wim  – 305,928,022
  • LiteTouchPE_x86.wim – 246,698,134

Well, since the Surface Pro 3 will *only* use x64, we better account for at least 310MB on our Recovery Partition.

What does ZTIDiskPart.wsf allocate? Easy it’s hard coded (in Mega Bytes):

Const WINRE_DRIVE_SIZE = 300

Our 302MB image is just *not* going to fit on a 300MB partition. Nope.

Bug

I opened this bug against the small partition size back in October of 2014:

https://connect.microsoft.com/ConfigurationManagervnext/feedback/details/1012326/ztiwinre-fails-to-install-winre-images-on-correct-recovery-partitions-in-uefi-scenarios

Additionally, can you up the size of WINRE_DRIVE_SIZE
in ZTIDiskPArt.wsf to something like 400?

If you would like MDT to install WinRE.wim for Windows 10 scenarios, please log into connect and up-vote this connect bug.  :^)

 

-k

ZTIWinRE.wsf

P.S. Additionally, please do *NOT* ask me about ZTIWinRE.wsf, it’s a piece of junk and should be completely removed from MDT. I will not speak of it (unless you are on the MDT team, and I will be happy to explain why it is evil and wrong, and should be deleted ). :^p