MDT Performance (Microsoft Time)

Time is relative. Whatever. I mean, who is the “Einstein” who came up with that one? :^)

Have you ever heard of a New York Minute? I guess time is different in the big apple.

What about “Microsoft Time”, you know where the progress bar says it’s 90% complete with 2 minutes remaining, but the next time you look. Pow! The progress bar now says 91% complete with 5 minutes remaining! How can it take negative 3 minutes to go 1%?

Of course the problem is that in a Modern OS there are thousands of different components that can affect performance. Memory Exhaustion, other background processes and other systems each fighting for limited resources.

Anyways, in the realm of OS deployment performance, things are relative. An OS deployment that took 1 hour might seem fast to you on a Virtual Machine, but it’s slow to me on a new Desktop.

How do we make MDT go faster? Easy, find the bottlenecks! One of the best places to start when using MDT LiteTouch deployments is by looking at the BDD.log file. The BDD.log file will show all the steps taken during a full OS Deployment.

  • Booting to WinPE
  • Interacting with the user with the Wizard
  • Formatting and Reparitioning the disk
  • downloading the OS *.wim file
  • Full OS Installation
  • State Capture and State REstoration
  • Installation of Applications
  • Etc..

Of course, when reading bdd.log files, I highly recommend using the CMTrace.exe tool found in the
System Center 2012 R2 Configuration Manager Toolkit. The CMTrace.exe reader can parse otherwise complex bdd.log files and display the relevant information in a table (these tables can even be imported/exported into Excel).

We can follow the date/time that each steps was performed, and determine how long operations took. One cool feature is the ability to highlight a set of commands and see what the elapsed time was to perform the command. Note the “Elapsed Time” at the bottom of the CMTrace.exe window?

performance

We can use this information to make educated guesses on where to improve our processes. For example if we can see that it takes 2 hours to apply a 4GB installation image from our MDT server when the server is under load, I would suggest looking into configuring the server for multicast.

Or perhaps we can see that a single application installation is taking too long. Can we can then investigate why that application is taking too long.

Happy hunting!

ListItem processing in CustomSettings.ini file.

Got thrown for a loop today when someone on the TechNet forms asked why this CS.ini file didn’t process both the Default and Secondary sections:

[Settings]
Priority=Default, Secondary

[Default]
MandatoryApplications001={AAAAAAAA-AAAA-AAAA-AAAA-AAAAAAAAAAAA}

[Secondary]
MandatoryApplications002={DDDDDDDD-DDDD-DDDD-DDDD-DDDDDDDDDDDD}

When done, only MandatoryApplications001 was processed. Now I would think that the MandatoryApplications item both 001 and 002 would be unique across the INI file, without regard to what section it is in. However that is not the case.

Turns out that within each section, any “List Item” must be a contiguous collection of numbers starting at 001( 001-00n). If you have 001,002,003 and 005 (where 004 is missing) only 001-003 would get imported. In the example above, we are missing 001 in the [secondary] section.

Also, even though there are two sections that contain the unique MandatoryApplications001, when imported ZTIGather.wsf won’t overwrite the 2nd instance of MandatoryApplications001, instead it will just add it to the end of the list.

So for example, if we have two sections with MandatoryApplications lists starting at 001, it should work.

[Settings]
Priority=Default, Secondary

[Default]
MandatoryApplications001={AAAAAAAA-AAAA-AAAA-AAAA-AAAAAAAAAAAA}
MandatoryApplications002={BBBBBBBB-BBBB-BBBB-BBBB-BBBBBBBBBBBB}

[Secondary]
MandatoryApplications001={CCCCCCCC-CCCC-CCCC-CCCC-CCCCCCCCCCCC}
MandatoryApplications002={DDDDDDDD-DDDD-DDDD-DDDD-DDDDDDDDDDDD}

And ran a test with the command:

cscript.exe c:\DeploymentShare\Scripts\ZTIGather.wsf /nolocalonly /inifile:c:\temp\test.ini

Will create the following in the bdd.log:

Property MANDATORYAPPLICATIONS001 is now = {AAAAAAAA-AAAA-AAAA-AAAA-AAAAAAAAAAAA}
Property MANDATORYAPPLICATIONS002 is now = {BBBBBBBB-BBBB-BBBB-BBBB-BBBBBBBBBBBB}
Property MANDATORYAPPLICATIONS003 is now = {CCCCCCCC-CCCC-CCCC-CCCC-CCCCCCCCCCCC}
Property MANDATORYAPPLICATIONS004 is now = {DDDDDDDD-DDDD-DDDD-DDDD-DDDDDDDDDDDD}

Note that [secondary]MandatoryApplications001 is now MandatoryApplications003?

Cool!

Re-ordering applications in MDT

Spent some time today updating tools that were out of date and needed a refresh.

One of the applications is my MDT Application Re-ordering tool.

Application Reordering

There are two different scenarios where you might want to rearrange applications in MDT.

1. Where application X needs to be installed *before* application Y. If so, use the dependencies option in the applications entry page in the Deployment Console. This is the only way to ensure applications are installed in any specific order.

2. However if you want the applications to *appear* in a specific order in the Deployment Wizard, then you can use the MDT2010Ordering.zip tool to perform this task. This also has a side effect any application selected in the MDT wizard will be installed in the order displayed in the wizard.

Behind the Scenes

After the user arranges the applications in the MDT2010Ordering tool, the tool will construct a powershell script to do the heavy lifting. The script essentially just moves the application out of the folder and back again in the correct order.

If you wanted to do all of this by hand, you could also just modify the control\ApplicationGroups.xml file to perform the actions. Be careful! :^)

Link

Scripts to dump installed drivers and applications

I was looking at my old ZTIYellowBang.wsf script on my old Blog site and noticed that the script version posted is an older version. So it’s time to update the script! :^)

Background

Back when I did a Deployment contract for Microsoft’s own IT department we developed a MDT Litetouch Deployment solution (Another team was working on the SCCM-OSD deployment). And I wanted to ensure that we were getting good driver/application coverage.

One of the first things I did in our test lab was to setup a SLShare so all of the bdd.logs generated by the test machines were sent to a centralized location for processing later. BUt how to make sure that the drivers installed were correct.

Updated Tool

I created ZTIYellowBang.wsf to track which devices on a machine had no drivers installed (or when drivers failed to load). The term “Yellow Bang” is in reference to the Yellow Exclamation point visible in the Windows Device Manager for any devices that are not working properly.

The updated tool I developed for MSIT doesn’t just display devices that are missing drivers, it also displays if the drivers installed are inbox Microsoft Drivers, 3rd party signed (WHQL), or not signed at all (Bad).

I added this to the standard Task Sequence near the end of the “State Restore” phase. It was also easy to write a script to find (Grep or findstr.exe) devices that did not find any drivers.

Applications

ZTIAppVerify.wsf
The other tool included here is a tool that displays all “installed” applications on the local computer. It searches the registry for all installed applications that appear in the “Add/Remove Programs” section of the Control Panel.

This tool can be helpful if you have drivers that also installed some associated applications that appear in the Control Panel.

Additionally, it will also go through the list of Applications and ManadatoryApplications selected for installation by MDT ZTIApplications.wsf. If the Application entry has a “UninstallKey” defined, the script will verify that the application was installed.

Steps

Simply copy the ZTIYellowBang.wsf and/or ZTIAppVerify.wsf script to the Deployment Share under the …scripts directory. Then add the script as a Step to your Task Sequence (just before the Capture section):

cscript.exe %ScriptRoot%\ZTIYellowBang.wsf

Links

Keith Garner is a new Microsoft MVP!

Cool, looks like I’ve been nominated by Microsoft as a MVP:

MVP

Dear Keith Garner,

Congratulations! 
We are pleased to present you with the 2014 Microsoft® MVP Award! 
This award is given to exceptional technical community leaders 
who actively share their high quality, real world expertise with 
others. We appreciate your outstanding contributions in Windows 
Expert-IT Pro technical communities during the past year.

Still exploring the MVP program, but it looks like I get a 1 year subscription to MSDN download center (which strangely I have never had).

MDT 2012 uEFI on large Media

Starting go get some more questions on how to image uEFI machines. It’s one of the biggest architectural changes to the PC in recent memory, and requires some new skills for us IT Professionals.

Problem

One of the most perplexing problems I’ve come across is of Large files on USB Flash Drives. You see, Microsoft licensed the FAT file system for use in the uEFI firmware so the firmware itself can read files off Hard Disks and USB Flash Media, uEFI does not support booting directly from NTFS partitions which assumes some kind of Boot Sector. Most machines with large internal Hard Disks break down the local disk into several partitions, One partition used to contain the uEFI boot files, and another NTFS partition to contain the Windows Operating System.

However, the problem with Fat32 Partitions is that they do not support files that are larger than 4 GB in size. If you want to install a Windows OS using MDT offline media, and your core image is larger than 4GB you can’t place the file on a Fat32 USB Stick. Additionally, USB Flash drives can not be given multiple partitions, you can only have one Fat32 or one NTFS partition on a removable flash drive, this means we can’t do the same trick as a internal SATA Hard Disk where we give the drive several partitions.

Work Arounds

There are several tricks we can use in MDT to work around the issue:

  • Don’t use Offline USB Media, instead download the large WIM file from your Deployment Share over the network. Use either PXE boot, or boot from a USB Drive and connect over the network. In some instances, the Gigabit Ethernet connection to the server will be faster than the local USB 2.0 disk. Remember, you can boot from a small 256MB USB flash drive with the contents from c:\deploymentshare\boot\LitetouchPE_x86.iso extracted to a USB Flash Drive.
  • Use *two* flash drives! Create Two flash drives, One formatted with NTFS that contains the offline USB media with the large *.wim file, and a second drive formatted with FAT32, that contains the contents from contents of the c:\deploymentshare\boot\LitetouchPE_x86.iso. uEFI will be unable to read the NTFS drive, and boot off the Fat32 drive instead, however once in WinPE, MDT will search all the drives and continue as if it was a Offline Media Installation. (Normally I don’t recommend leaving two MDT USB Drives in a machine at once, can cause confusion).
  • You may not re-partition USB Flash drives marked as “removable”, but some USB drives marked as “Fixed” can be repartitioned to include both a Fat32 and NTFS partition(s). Windows To Go drives are designed to do this. Additionally, you might be able to repartition USB Hard Disk Drives.
  • Use DVD-DL (DVD Dual Layer) which has a capacity of 8.5GB, and can support files larger than 4GB. Native support for BIOS and uEFI machines.
  • Finally, you might want to split the super large install.wim file into multiple parts to make more manageable.

Splitting Wims

First of all, let me mention that splitting WIM files like what is mentioned below will cause some compatibility problems within MDT itself. Although *.SWM files are a supported format within the *.wim file standard, MDT was not designed to handle *.SWM files. So you can’t import and/or service these files within the MDT console, and the client scripts will also get confused if used.

I developed a prototype for WIM splitting a couple of years ago, and wanted to share the solution:

1. Create an Offline Media Share with the OS you want to support.
2. Split your large WIM file using the command:

imagex.exe /split install.wim install.swm 4095

3. Modify the …\Deploy\Control\OperatingSystems.xml file and change the “Install.WIM” entry to “INstall.SWM”
4. Modify the …\Deploy\Scripts\LTIApply.wsf script to:

If GetWDSServer "" then
   '...
ElseIf ucase(right(sImagePath,4)) = ".SWM" then
   sRWMPath = mid(sImagePath,1,len(sImagePath)-4) & "*.swm"
   oLogging.CreateEntry "Found SWM file, adding other SWM files for concatenation: " & sRWMPath, LogTypeInfo
End if

5. Copy to your USB Flash Drive and use as normally.

Update

Also found this solution for Splitting WIM files that is similar, but slightly different.

http://www.osd-couture.com/2013/11/how-to-allow-mdt-2013-to-use-wim-file.html

-k

Failed to save the current environment block

Problem

I have received some questions lately about a weird blocking MDT bug on some uEFI machines.

A quick scan of the bdd.log does not reveal any clue as to what the failure is, however when we check the SMSTS.log file we can see the following:

Set a global environment variable _SMSTSNextInstructionPointer=53     TSManager         1/29/2013 6:41:40 AM    1532 (0x05FC)
Set a TS execution environment variable _SMSTSNextInstructionPointer=53       TSManager         1/29/2013 6:41:40 AM                1532 (0x05FC)
Set a global environment variable _SMSTSInstructionStackString=46       TSManager         1/29/2013 6:41:40 AM    1532 (0x05FC)
Set a TS execution environment variable _SMSTSInstructionStackString=46         TSManager         1/29/2013 6:41:40 AM                1532 (0x05FC)
Save the current environment block       TSManager         1/29/2013 6:41:40 AM    1532 (0x05FC)
pszPath[0] != L'', HRESULT=80070057 (c:\qfe\nts_sms_fre\sms\framework\core\ccmcore\path.cpp,58)                TSManager         1/29/2013 6:41:40 AM    1532 (0x05FC)
Filesystem::Path::Add(sEnvPath, EnvDataFileName, sEnvPath), HRESULT=80070057 (e:\nts_sms_fre\sms\framework\tscore\environmentlib.cpp,639)         TSManager         1/29/2013 6:41:40 AM    1532 (0x05FC)
Failed to save environment to  (80070057)           TSManager         1/29/2013 6:41:40 AM    1532 (0x05FC)
TS::Environment::SharedEnvironment.saveEnvironment(szPath), HRESULT=80070057 (e:\nts_sms_fre\sms\client\tasksequence\executionengine\executionenv.cxx,842)     TSManager         1/29/2013 6:41:40 AM         1532 (0x05FC)
Failed to save the current environment block. This is usually caused by a problem with the program. Please check the Microsoft Knowledge Base to determine if this is a known issue or contact Microsoft Support Services for further assistance.
The parameter is incorrect. (Error: 80070057; Source: Windows) TSManager         1/29/2013 6:41:40 AM    1532 (0x05FC)
SaveEnvironment(), HRESULT=80070057 (e:\nts_sms_fre\sms\client\tasksequence\executionengine\executionenv.cxx,420)     TSManager         1/29/2013 6:41:40 AM         1532 (0x05FC)
Failed to persist execution state. Error 0x(80070057)       TSManager         1/29/2013 6:41:40 AM    1532 (0x05FC)
Failed to save execution state and environment to local hard disk             TSManager         1/29/2013 6:41:40 AM    1532 (0x05FC)
Reboot to local harddisk               TSManager         1/29/2013 6:41:40 AM    1532 (0x05FC)
FALSE, HRESULT=80004005 (e:\nts_sms_fre\sms\client\tasksequence\executionengine\engine.cxx,584)                TSManager         1/29/2013 6:41:40 AM    1532 (0x05FC)
The task sequence execution engine can not reboot the machine because we failed to persist execution environment     TSManager         1/29/2013 6:41:40 AM    1532 (0x05FC)
CheckForRebootRequest(&bRebootInitiated), HRESULT=80004005 (e:\nts_sms_fre\sms\client\tasksequence\executionengine\engine.cxx,274)   TSManager         1/29/2013 6:41:40 AM                1532 (0x05FC)
Fatal error is returned in check for reboot request of the action (Restart computer). 
Unspecified error (Error: 80004005; Source: Windows)   TSManager         1/29/2013 6:41:40 AM    1532 (0x05FC)
An error (0x80004005) is encountered in execution of the task sequence              TSManager         1/29/2013 6:41:40 AM                1532 (0x05FC)
Sending status message . . .        TSManager         1/29/2013 6:41:40 AM    1532 (0x05FC)
Executing in non SMS standalone mode. Ignoring send a task execution status message request               TSManager                1/29/2013 6:41:40 AM    1532 (0x05FC)
m_TSEngine.Execute(& m_eExecutionResult), HRESULT=80004005 (e:\nts_sms_fre\sms\client\tasksequence\tsmanager\tsmanager.cpp,763)       TSManager         1/29/2013 6:41:40 AM                1532 (0x05FC)
Task Sequence Engine failed! Code: 80004005    TSManager         1/29/2013 6:41:40 AM    1532 (0x05FC)

The important line here is when we see: “Failed to save the current environment block.” MDT uses the SMS Stand Alone Task Sequencer, and it must save it’s state, including environment variables and other instruction pointer steps to the hard disk before it can reboot so it can continue where it left off.

For whatever reason, the SMS Stand Alone Task Sequencer (SMSTS) can’t save to the local disk, perhaps it’s having trouble trying to enumerate through the uEFI GPT Partitioning structure.

Resolution

Upon further review we can see that for most of these machines the problem is occurring during the WinPE Phase, and the WinPE version is 7600, meaning that it was created from the WAIK. Normally this would be OK, as MDT 2012 and the WAIK have been tested with uEFI, however there is this one note in the MDT 2012 Update 1 “Release Notes” that should be taken into account:
• 64-bit versions of Windows are unable to deploy to computers with Unified Extensible Firmware Interface (UEFI) 2.3.1, because the Windows AIK is being used to perform the deployment. The Windows ADK is required when deploying Windows to a computer with UEFI 2.3.1.

It appears that the ADK is required for deployment to uEFI machines, at least with uEFI 2.3.1, which is most modern Windows 8 class machines. So the resolution is to upgrade to ADK on your build machine to see if the problem goes away.