MDT 2013 UberBug01 – MDAC and the Fast Machine

Well MDT 2013 Update 1 is out. Yea!

Time to test and evaluate to see if there are any regressions in my test and environment.

Wait… Something went wrong…

dism

Fast System

As I mentioned in an earlier blog post, I recently purchased a super fast Intel 750 SSD to make my MDT Build machine run faster. Blazing Fast

WP_20150507_19_51_57_Pro

You might think: “Wow, that’s super cool, everything would be so much better with a faster machine”

Essentially, yes, except when it’s faster than the Operating System can handle.  :^(

The Bug

When updating a deployment share you may get the following error message:

Deployment Image Servicing and Management tool
Version: 10.0.10240.16384
 
Image Version: 10.0.10240.16384
 
Processing 1 of 1 - Adding package WinPE-MDAC-Package~31bf3856ad364e35~amd64~~10.0.10240.16384
 
Error: 1726
 
The remote procedure call failed.
An error occurred closing a servicing component in the image.
Wait a few minutes and try running the command again.
 

Dism.log shows nothing interesting:

 
2015-07-15 13:55:00, Error                 DISM   DISM.EXE: DISM Package Manager processed the command line but failed. HRESULT=800706BE
2015-07-15 13:55:00, Error                 DISM   DISM Manager: PID=2364 TID=2424 Failed to get the IDismImage instance from the image session - CDISMManager::CloseImageSession(hr:0x800706ba)
2015-07-15 13:55:00, Error                 DISM   DISM.EXE:  - CDismWrapper::CloseSession(hr:0x800706ba)

I asked someone knowledgeable at Microsoft (MNiehaus), and he mentioned that he saw it a couple of times, but couldn’t repro it consistently. However, I could easily reproduce the problem on demand with my hydration/buildout scripts.

Turns out that there is a narrow case where this bug manifests:

  • If you add any optional components to WinPE within MDT
  • If you have a fast hard disk (like my Intel 750 SSD)
  • If you have <UseBootWim> defined in you settings.xml, it may get worse.

The fast disk is most likely why the Windows Product Group teams never saw this bug in testing.

Well, I provided a repro environment to the Windows Product Groups involved in this component, even letting them log into my machine to reproduce the issue.

Good news is that they were able to determine what the root cause is ( timing related during unload of various components ), and even provided me with a private fix for testing! The private passed!

Now the real fun begins, there is a legitimate challenge here, because the error exists in the Windows 10 Servicing Stack, and that stack is embedded *into* WinPE.wim on the ADK and Boot.wim on the OS Install disk.

How do you update these files with the updated servicing stack, it’s not a trivial matter. They could send out a KB QFE Fix, and let customers manually update the files manually with dism.exe, they could also repackage/rerelease the ADK Itself, or worst case wait till the next release of the Windows OS ISO images.

I have been monitoring status, and there are several team members working on the release issues, and even someone from Customer Support acting as a customer advocate. My work is done.

Work Around

In the mean time, you can work around the issue:

  • Removing optional components from MDT. :^(
  • Of course, you move to a machine with a slower disk.
  • I had some luck getting optional components added when I set <UseBootWim> to false in the settings.xml file.
  • Additionally, Johan has mentioned that he can get it to pass if the OS running MDT is Windows 10 ( I was running Windows Server 2012 R2 ).

For me, it was easy, I don’t use the MDAC components in my environment, so I just removed them from the settings.xml file. Lame, I know.

-k

More Deployment bugs to follow!

Updated Dell Driver Catalog Script for Windows 10

Woot!

Schema

Dell has updated the Schema on their DriverPackageCatalog.xml to include full SMBIOS Make and Model information for their systems!

I updated the Install-DellDriverCatalog.ps1 script to take advantage of the new metadata and tried importing drivers for my trusty Dell Venue 8 Pro.  As you may recall, in previous versions of the DriverPackageCatalog.xml file, the Dell Venue 8 Pro had a weird 5830 number for the Model, which didn’t match what was in the Firmware.

Anyways, I tried importing into MDT, and got the following…

DellModelVenue

Looks Great!!

WinPE

Additionally, with the addition of the Windows 10 ADK, I have updated the script to support this platform, and other changes

mdt selection profiles

Scripts:

Install-DellDriverCatalog.ps1

Install-DellWinPEDriverCatalog.ps1

 

Thanks to @WarrenByle for putting up with my pestering :^) Great stuff from dell.

Updated Dell Driver Catalog script

So I provided feedback to Dell last year about the schema on their DriverPackCatalog.xml file. I wanted to tie the Make & Model information from XML file directly with the SMBIOS data returned in WMI queries like Win32_ComputerSystemProduct. Right now Dell provides some model information in the XML:

<Brand key=90 prefix=TABLET>
<Display lang=en><![CDATA[ Tablet ]]></Display>
   <Model systemID=0630>
      <Display lang=en><![CDATA[ 5830 ]]></Display>
   </Model>
</Brand>

But how to cross reference that with my Dell Venue 8 Pro:

PS C:\> Get-WMIObject Win32_ComputerSystemProduct

IdentifyingNumber : 123QWER
Name              : Venue 8 Pro 5830
Vendor            : DellInc.
Version           : Not Specified
Caption           : Computer System Product

I could try to do some sub-string matching, but I can’t seem to come up with the ideal pattern matching system, in addition, I want to support Johan’s Total-Control system, and if Dell doesn’t support the full Name “Venue 8 Pro” out of the “Venue 8 Pro 5830” I can’t construct the correct folder name.

Request: I would like Dell to supply the *FULL* SMBios name for each platform in the DriverPackCatalog.xml file.

Well, Dell went and updated the schema, but instead of adding full Model strings for the SupportedSystems, they modified the schema in ways that broke some other things in my scripts. My scripts have been hopelessly out of date and broken for some time. :^(

Updated Scripts

I updated the Install-DellWinPEDriverCatalog.ps1 file to match the new Dell Schema.

Fair warning, I don’t have enough Dell machines to do proper testing, but from what I can tell it was able to import WinPE and Some Latitude drivers into my test MDT environment.

Dell

(perhaps Dell can send me a new Precision Workstation so I can do proper testing ;^)

Updated Scripts:

Install-DellDriverCatalog.ps1

Install-DellWinPEDriverCatalog.ps1

 

Adding Help to your PowerShell Script

So, after working on your PowerShell script for a while, you figure it’s time to cleanup the comments and add some help so other people can use it.

PowerShell has an existing framework setup for getting help that most people already understand, it’s the “Get-Help” cmdlet, but most PowerShell scripters don’t realize that you can use “Get-Help” for your script, if formatted correctly.

The wrong way

First off, let’s discuss how not to do help, occasionally, I will see scripts that try to parse the arguments passed into the script:

if(($args.Count -gt 0) -and (
        ($args[0].ToLower() -eq "/?") -or 
        ($args[0].ToLower() -eq "/help") -or 
        ($args[0].ToLower() -eq "-h") -or 
        ($args[0].ToLower() -eq "--help") -or 
        ($args[0].ToLower() -eq "-help")))  {

    Write-Host("USAGE:")
    Write-Host(" - Get Some Info: GetInfo")
    Write-Host(" - Get Remote Info: GetInfo <COMPUTER NAME | IP>")
    Write-Host(" - Get Many Remote Info: GetInfo <COMPUTER NAME | IP> <COMPUTER NAME | IP> (Repeat as many times as needed")
    Write-Host(" - Show this help: GetInfo /? | /help | -h | --help")
} Else {
    if($args.Count -eq 0) {$args = ("localhost")}
    Foreach($computer in $args) {
        Write-Host "Do Something $Computer"
    }
}

This is very much a C++/C#/Java way of parsing arguments, and doesn’t leverage built-in PowerShell functionality.

about_comment_based_help

Powershell has built in functionality for self documenting help based on the comment block of your function or script:

#Requires -Version 3
#Requires -RunAsAdministrator 

<#
.SYNOPSIS
Make a VHD File bootable
.DESCRIPTION
Converts a raw *.vhd file to a bootable *.vhd file
.PARAMETER Path
File path to a VHD File to be made active
.EXAMPLE
.\Make-VHDBootable.ps1 -verbose -path C:\15\install.vhdx
.NOTES
Copyright Microsoft Corp, All Rights reserved.
#>

[CmdletBinding()]
param(
       [Parameter(Mandatory=$true)]
       [string] $Path
)

$MountedVHD = Mount-VHD -Path $Path –Passthru
if ( $MountedVHD ) {
    $MountedVHD | Out-String | Write-Verbose
    $Drive = $MountedVHD | Get-Disk | Get-Partition | Get-Volume | Select-Object -ExpandProperty DriveLetter
    if ( Test-Path "$Drive`:\Windows\System32\BCDBoot.exe" ) {
        & "$Drive`:\Windows\System32\BCDBoot.exe" "$Drive`:\Windows" /s "$Drive`:" /F ALL | out-string |write-verbose
    }
    Dismount-VHD -Path $Path
}

Note how I have two #requires strings at the top of the script, this is an easy way to enforce basic script requirements. Most of the scripts I write require Elevated Administrative privileges, and this is the quickest way to make sure this works.

There is a large comment block at the top of the script describing what the script does. See: about_comment_based_help

Also note that I try to match the Parameters for my function with any cmdlets I will pass the parameters to within the function, that way I can convert to Splatting down the road if beneficial.

Get-Help

We know how get-help works for most of our cmdlets, and it’s a great way to learn the details of what functions do. But did you know that get-help also works for scripts?

Just run get-help with your full script path as the parameter:

PS C:\Users\Keith\Desktop> get-help .\Make-VHDBootable.ps1

NAME
    C:\Users\Keith\Desktop\Make-VHDBootable.ps1
    
SYNOPSIS
    Make a VHD File bootable
    
SYNTAX
    C:\Users\Keith\Desktop\Make-VHDBootable.ps1 [-Path] <String> [<CommonParameters>]
    
DESCRIPTION
    Converts a raw *.vhd file to a bootable *.vhd file
    
RELATED LINKS

REMARKS
    To see the examples, type: "get-help C:\Users\Keith\Desktop\Make-VHDBootable.ps1 -examples".
    For more information, type: "get-help C:\Users\Keith\Desktop\Make-VHDBootable.ps1 -detailed".
    For technical information, type: "get-help C:\Users\Keith\Desktop\Make-VHDBootable.ps1 -full".

Some Notes about WinRE on Windows 8.1 and Windows 10

Had some questions this week on what the behavior is for the WinRE.wim file.

Background

WinRE is a lightweight Utility OS similar to WinPE that sits on a Fat32 partition (not encrypted) on the local disk. If the local Operating System detects that there is a problem, typically during the boot process, it can boot to WinRE and allow for some recovery.

The problem we had this week is where the file was getting placed, sometimes it was in C:\Windows\System32\Recovery\WinRE.wim, sometimes it was in c:\Recovery, and other times it was in a the WinRE recovery partition.

Windows Setup

In a standard Windows Install.wim file the file should be located in the WIM at \Windows\System32\Recovery\WinRE.wim.

During windows Setup the OS will move the WIM file to another location depending on the state of the machine:

  1. It will install to the “Recovery” partition. Marked with the ID:
    de94bba4-06d1-4d40-a16a-bfd50179d6ac (For uEFI/GPT machines)
  2. Copied to the Fat32 “System” partition. ( For MBR machines )
  3. On the “Windows OS” partition, this is not recommended for machines encrypted with BitLocker.

Capture

After OS installation, we may want to capture the machine back to an Install.wim file, and it would be nice to ensure that the WinRE.wim file get’s put back under the c:\Windows\System32\Recovery folder.

To do this, perform *either* of the following

  • reagentc.exe /disable
  • sysprep.exe /generalize

Most people will correctly run sysprep before capturing the image, so that is good.

Please do not copy the WIM files back to c:\Windows\System32\Recovery directly, this may break Recovery.

MDT and ZTIWinRE.wsf

Right now ZTIWinRE.wsf in the MDT kit is looking a little old. It was written many years ago, and rather than use the c:\windows\System32\Recovery\winre.wim file, it tries to use the existing WinPE wim file which can be retrofitted with the necessary WinRE components. Really this method is broken, especially if you are deploying more than one version of windows, say installing Windows 10 with a Windows 8.1 version of WinPE.

Really we should be injecting any boot critical drivers into WinRE, either when generating the captured OS, or during deployment.

Something for the future.

Sysprep Windows 10 Build 10074 Fix

There have been some reported SysPrep errors in Windows 10 Build 10074. Something to do with AppX controls (of course) in the panther logs.

There is a work around that has been floating here and there, hopefully it’s only a temporary fix for Windows 10:

Fix:

  • Stop the ’tiledatamodelsvc’ Service (ensure it has *actually* stopped)
  • Stop the ‘staterepository’ Service (ensure it has *actually* stopped)
  • Prevent both services from starting again by modifying the ImagePath
  • Run Sysprep normally
  • Then restore the ImagePath for both services

Code:

I have updated my private LTISysprep.wsf script for MDT 2013 Update 1 (Preview) here:

http://mdtex.codeplex.com/SourceControl/latest#Templates.2013.u1/Distribution/Scripts/LTISysprep.wsf

One of the cool things about Codeplex, is that you can compare it with the previous version to see what I changed. :^)

Note that this fix should only be temporary, and it is my intention to delete this post in the near future when fixed, hopefully before RTM. :^) Welcome to the fun of Beta Software.

Find errors quickly in a SCCM or MDT log file.

I have an automated system to build out my Windows Images locally, 9 images in total using MDT 2013 U1 Beta Litetouch

  • Windows Server 2008 R2
  • Windows Server 2012 R2
  • Windows Server Technical Preview 2 (build 10074)
  • Windows 7 SP 1 – Both x86 and x64
  • Windows 8.1 Update 3 – Both x86 and x64
  • Windows 10 Build 10074 – Both x86 and x64

But I wondered today if everything was *really* going as well as I thought… The recommended way to check is to open the bdd.log file in cmtrace.exe and look for any Yellow or Red lines, warning and Errors!

SCCM and MDT use a special logging format so it’s pretty easy to search for errors using a script.

In powershell I can type in:

gci -recurse bdd.log | gc | Select-String "type=""(2|3)"""

I didn’t like the output, as it only returns the error line, without the file and line number, so I started to play around the script, and noticed that by removing the “get-content” command, Select-String did the right thing and opened the file in question

gci -recurse bdd.log | Select-String "type=""(2|3)"""

Yea! it worked, and found errors… :^)

Now I have to fix the errors. :^(

Driver Packages for Lenovo and HP

I’ve had several requests to extend my Dell Driver Pack Catalog Tool to Lenovo and HP.

I got as far as developing a powershell script to get a Lenovo driver manifest by web-scraping the support.lenovo.com web site:


#requires -version 3
#Requires -RunAsAdministrator

[CmdletBinding()]
param()

Write-verbose “Download Model List”
$AllModels = Invoke-WebRequest http://support.lenovo.com/en/documents/ht074984
$ModelPages = $AllModels.links.href |
   select-string -allmatches “(?:/docs/|/us/en/downloads/)(ds[0-9]+)” |
   ForEach-Object { $_.matches.groups[1].value } |
   Select-Object -Unique

Write-Debug “Done: Count[$($ModelPages.count)]”

$i = 0
$manifWrite-Verbose “Download each model page”
ForEach ( $Model in $ModelPages ) {
Write-Progress -Activity “Download $Model -PercentComplete ($i * 100 / $ModelPages.Count)
    $i++
    Invoke-WebRequest http://support.lenovo.com/us/en/downloads/$Model |
        ForEach-Object {
            [PSCustomObject] @{
                 PackageID = $Model;
                 Title = $_.ParsedHTML.Title;
                 Download = ($_.links.href -match “.*exe”)
            }
        }
}

Write-Progress -Activity “Done” -Completed


The next step is to download and extract the *.exe package (with the /silent or /VerySilent switch), and then import into MDT Litetouch or SCCM

Unfortunately, I’m not very happy about the way the Model numbers are displayed/parsed by the web site and my script, and I would prefer something that is machine readable (XML file, for example). This would allow us to follow the Johan “Total Control” method of driver management.

If you know how to get the Model Numbers for Lenovo machines given the packages from the web site, let me know.

http://support.lenovo.com/en/documents/ht074984

ftp.hp.com/pub/caps-softpaq/cmit/HP_Driverpack_Matrix_x64.html
ftp://ftp.hp.com/pub/caps-softpaq/cmit/HP_Driverpack_Matrix_x86.html

 

 

How to change uEFI boot order

I’ve been doing a lot of USB and PXE installations on uEFI physical hardware lately. Most of the machines I’ve used in the past have been Lenovo/Dell/HP machines that support F9 or F12 boot overrides, however my Surface test machines ( Surface Pro (1), Surface Pro 3, and Surface 3) don’t support the typical F9 or F12 overrides, instead you have to hold down the Vol-Up key while pressing power on.

Well sometimes I get distracted, and forget to press the button, or sometimes my machine is just slow.

BIOS

Back in the BIOS days, it was easy for me to change the default boot order, for starters, I always set the Hard Disk to 1st priority, I never set PXE or USB to higher priority, that’s a security vulnerability. So on a BIOS machine, all you have to do is disable the active bit on the local hard disk system partition, and the BIOS will boot to the next item in the list.

Here is a script I wrote to do this:
SABOTAGE.CMD

 
@if not defined debug echo off

@echo.
@echo  This script will SABOTAGE the main hard disk and reboot the machine.
@echo.

pause

if exist c:\minint rd /s /q c:\Minint
if exist d:\minint rd /s /q d:\Minint
if exist e:\minint rd /s /q e:\Minint
if exist c:\_SMSTaskSequence rd /s /q c:\_SMSTaskSequence

(
@echo List Disk
@echo Select Disk 0
@echo List Partition 
@echo Select Partition 1
@echo Inactive
@echo Select Partition 2
@echo Inactive
@echo Exit
) | diskpart.exe

WPEUtil.exe reboot
shutdown.exe -s -f -t 0

The script will try to inactivate two different partitions just to be sure, and I run two different commands to reboot, one for Full Windows, and one for WinPE.

Additionally, if you decide later on that you don’t want to wipe the machine and install a new OS, you can boot into WinPE, and re-activate the System partition, and you got your full OS back.

uEFI

Well uEFI is a bit harder, and I finally think I’ve come across a way to disable booting from the system partition. The challenge is that the files on the System Partition are on a volume that does not have a drive letter, so it’s harder to gain access, but not totally impossible. Turns out that we can use the volume mount points to gain access.

On most of my test machines the System partition is on Disk 0 Partition 2, so the mount point is \\?\HardDiskVolume2

Example:
mount

We just enumerate through all partitions till we find the correct one.

(Please don’t ask about accessing these \\?\HardDiskVolume2 mount points in Powershell, it’s hard).

uEFISabotage.cmd

@if not defined debug echo off 

@echo.
@echo  This script will SABOTAGE the main hard disk and reboot the machine.
@echo.

pause

for /L %%i in ( 1,1,10 ) do (
  if exist \\?\HarddiskVolume%%i\efi\boot\bootx64.* (
    echo Found uEFI drive \\?\HarddiskVolume%%i
    rename \\?\HarddiskVolume%%i\efi\boot\*.efi *.bak
    rename \\?\HarddiskVolume%%i\efi\Microsoft\boot\*.efi *.bak
    WPEUtil.exe reboot
    shutdown.exe -s -f -t 0
  )
)
echo No uEFI drive found
pause

Be careful about USB drives that are mounted on the local system, the script should search and find the local volumes first, but just to be sure remove your USB stick.

Additionally, if you find that you want to keep your existing OS, just boot into WinPE, and rename all the *.bak files to *.efi.

MDT 2013 Update 1 private – What’s new!

On Friday Microsoft released a quick update to the MDT 2013 Update 1 Beta.

http://blogs.technet.com/b/msdeployment/archive/2015/03/28/mdt-updates-for-windows-10-technical-preview-build-10041.aspx

Within the script it’s being called 6.3.8231.Private

Overall there are just a few small script changes, but one big

What’s new

  • Client.xml and StateRestore.xml – Removed hard coded references to build 6000 ( Server.xml was fixed for the first Beta, but somehow these files got missed).
  • Sysprep.wsf – Code added to properly distinguish between Windows above and below Build 6000. This is strange to me, since MDT 2013 only supports Windows 7 and above, why add *EXTRA* code to support XP?
  • ztigather.wsf – Code added to distinguish between Windows 5.0 vs 6.0 by checking for build number 6000.
  • ZTIUtility.vbs – Whoops, Bug! While modifying the version in the header (line 10), the MDT team forgot to modify the version on Line 40, meaning that all the bdd.log files will reflect version “<version>” instead of a real number, go ahead and copy the version from Line 10 and add to line 40. The MDT team added some new code to support parsing of build and revision numbers.
  • ZTIPreReq.vbs – This change is a concern. Looks like the MDT team added code to handle processing the Windows Script Host Version number. In XP it was 5.6, in Windows 7 through 8.1 it’s 5.8, and now in Windows 10 Build 10041 it’s 5.12, for those of you have been following along with the 10.0 > 6.3 and the 10041 > 7600 version number problems you can see that there will be problems here as well since 5.6 > 5.12. However the fix may have some unintended consequences, as the author also changed the version check from 5.6 to 5.8, meaning that it should fail on any OS *Before* Windows 7 and Server 2008 R2. This script, as you may recall is used to verify the OS Version within Litetouch.vbs. The unintended consequence is that it should block running Litetouch.vbs from Windows XP, Vista, 2003 and 2008 (R1), meaning you can’t refresh *from* these OS’es.

Scorecard

OK, let’s check the MDT fix scorecard to see what they have addressed, and what they did not.

https://keithga.wordpress.com/2015/03/04/more-about-mdt-2013-update-1-preview/

  1. Perhaps the biggest missing component is a new task sequence for the new Windows 10 “OS Upgrade”. There have been several blog posts on the subject, so I would have expected to see it by now.
  2. .\Templates\Server.xml has a fix for ImageBuild > 6 in Sysprep, but .\Templates\Client.xml is missing that fix.
  3. Still a lot of XP and Vista legacy code in LTIApply.wsf and LTISysprep.wsf.
  4. Readers of this blog may recall my fix for LTIApply.wsf and split WIM’s. Johan has mentioned some concerns about the performance here.
  5. Only minor fixes to ZTIWindowsUpdate.wsf. I have several other updates on my blog.
  6. I have a minor quibble about the new Function GetMajorMinorVersion. It’s not a function, it’s a Sub since it does not return any values. I would have preferred that the function return the Major and Minor OS version as a Float.
  7. I have 20 or so bugs on Connect, and so far I can see only the split wim fix in the Preview.
  8. Although MSFT has fixed several bugs related to the version number jump “6.4” > “10.0” they didn’t fix any build number problems “6000” > “10007”.
  9. I can see that Microsoft added some entries for ServerManager.xml, but they copied the Windows 10 entries from above (see how Roles ID 12 is repeated twice), I wrote a script a while back to programmatically generate the ServerManager.xml file from the OS itself. You can see my changes on MDTEx.
  10. Would also like to see some LTI support for Windows 10 new Component Package model.