MDT UberBug08 – Task Sequencer Crash

My 3rd bug related to the new Stand Alone Task Sequencer has to do with a Crash in the product. I’ve seen this pretty consistently when hydrating/building out my own SCCM 2012 Server.

At the end of the task sequence I will see the following:

error

SMSTS.Log shows:

<![LOG[Process completed with exit code 2147942402]LOG]!><time="23:36:24.402+420" date="07-12-2015" component="TSMBootstrap" context="" type="1" thread="3864" file="commandline.cpp:1123">
<![LOG[Exiting with return code 0x80070002]LOG]!><time="23:36:24.417+420" date="07-12-2015" component="TSMBootstrap" context="" type="1" thread="3864" file="tsmbootstrap.cpp:1238">
<![LOG[shSmsTsKeyForScope.Open (HKEY_LOCAL_MACHINE, sSmsTsKeyForScope.c_str(), KEY_READ), HRESULT=80070002 (e:\nts_sccm_release\sms\framework\tscore\environmentscope.cpp,311)]LOG]!><time="23:36:24.417+420" date="07-12-2015" component="TSMBootstrap" context="" type="0" thread="3864" file="environmentscope.cpp:311">
<![LOG[Failed to open key Software\Microsoft\SMS\47006C006F00620061006C005C007B00350031004100300031003600420036002D0046003000440045002D0034003700350032002D0042003900370043002D003500340045003600460033003800360041003900310032007D00\SMSTS]LOG]!><time="23:36:24.417+420" date="07-12-2015" component="TSMBootstrap" context="" type="3" thread="3864" file="environmentscope.cpp:311">
<![LOG[GetKeyFromRegistry (m_sScopeRegKey.c_str(), Key), HRESULT=80070002 (e:\nts_sccm_release\sms\framework\tscore\environmentscope.cpp,764)]LOG]!><time="23:36:24.417+420" date="07-12-2015" component="TSMBootstrap" context="" type="0" thread="3864" file="environmentscope.cpp:764">

It’s a bit troubling, as I mentioned earlier with my other bugs related to the Stand Alone Task Sequencer, the Task Sequencer in MDT should be “rock solid”, and is a mission critical component, the fact that it crashes due to missing or corrupt registry is a problem.

I have filed a bug against MDT 2013.

https://connect.microsoft.com/ConfigurationManagervnext/Feedback/Details/1712354

I have no work around, but thought I would share in case anyone else encounters the same error.

MDT 2013 UberBug07 – Not enough space on disk.

Here are some quick notes on a different kind of bug with MDT that I’m posting if anyone encounters it.

MDT 2013 Update 1 has a small bug in it that can cause refreshes from previous MDT deployments to fail.

It’s a complex interaction bug with MDT, the System and Recovery partitions, and WinRE.wim.

Symptoms

What happens is that when you try to “refresh” a machine from Windows 8.1 to another OS, MDT will attempt to place a copy of WinPE.wim on the “System” partition. However Windows 8.1 Setup may have already placed the WinRE.wim file on the System partition (rather than the WinRE partition where it should have been placed). This can happen even though you may not have enabled the ZTIWinRE.wsf script in your task sequence, because Windows 8.1 will always copy the WinRE.wim file to a non “OS/Boot” partition during setup.

When MDT tries to copy the WinPE.wim file to the local disk, it will first try to copy to the System partition, but will fail, because there is not enough space (thanks to the WinRE.wim file hogging up all the space). And you will get the error message:

Not enough space for boot image on boot partition using system partition

 

Fix

I filed a bug against MDT a while back:

https://connect.microsoft.com/ConfigurationManagervnext/Feedback/Details/1136099

IN the mean time, You can view my private fix at:

http://mdtex.codeplex.com/SourceControl/diff/file/view/39534?fileId=Templates.2013.u1%2FDistribution%2FScripts%2FLTIApply.wsf

Lines 232 – 245

I basically delete the WinRE.wim file from the system partition. the older WinRE.wim file won’t be used in the new OS anyways, so it’s OK to delete. :^)

MDT UberBug05 – Batch scripting is dead.

Found this bug on the TechNet social group for MDT:

https://social.technet.microsoft.com/Forums/en-US/8b471021-4a84-4366-9467-acf6d03cba38/mdt-2013-update-1-batch-files-are-searched-in-cwindowssystem32-not-in-application-directory

Basically if you call a *.bat or *.cmd file in MDT as an Application, MDT will do some special processing to ensure that the default directory is in a good location for the bat or cmd file.

If not oNode.selectSingleNode("WorkingDirectory") is nothing then
  sDir = oUtility.SelectSingleNodeString(oNode,"WorkingDirectory")
  If Trim(sDir) <> "" and Trim(sDir) <> "." then
    If Left(sDir, 2) = ".\" then

      If (Instr(1, sCmd, ".CMD", 1) > 0 or Instr(1, sCmd, ".BAT", 1) > 0) and oEnvironment.Item("ResourceDrive") <> "" then
        If oEnvironment.Item("DeploymentMethod") = "MEDIA" then
          sDir = oEnvironment.Item("ResourceRoot") & Mid(sDir, 2)
        Else
          sDir = oEnvironment.Item("ResourceDrive") & Mid(sDir, 2)
        End If
     Else
       sDir = oEnvironment.Item("ResourceRoot") & Mid(sDir, 2)
    End if
  End if

  sDir = oEnvironment.Substitute(sDir)
  oUtility.ValidateConnection sDir
  oLogging.CreateEntry vbTab & vbTab & "Change directory: " & sDir, LogTypeInfo
  On Error Resume Next
  oShell.CurrentDirectory = sDir

ResourceDrive is a drive letter (Z:\) mapped to the Deployment share rather than the network path like ( \\MDT1\DeploymentShare$ ) Batch scripts like the drive letter better than the full path.

However in the RTM version of MDT that special test doesn’t work since sCMD is always empty for the block above.

Haven’t had a chance to code a fix, sadly it doesn’t directly affect me right now, I don’t use any cmd scripts.

If you want to upvote for a fix:

https://connect.microsoft.com/ConfigurationManagervnext/Feedback/Details/1682714

MDT Gotcha – No Progress is good progress.

Quick note:

from the MDT release notice:

http://blogs.technet.com/b/msdeployment/archive/2015/08/17/mdt-2013-update-1-now-available.aspx

One of the changed items is:

Switched to using DISM for imaging processes (instead of deprecated ImageX)

Some people might think this is a good thing, but I actually think this fix is bad.

ImageX.exe is the old tool used by Windows Vista to capture and apply *.wim images. The tool was superseded by dism.exe /apply-image and /capture-image. Really it didn’t matter since they *both* used the same wimgapi.dll API calls.

Why is Dism.exe better? Well for capture scenarios, it makes the process run much faster, turns out there was a bug in imagex.exe that would cause it to stop processing for 15 minutes at the end of the capture. Well it would be better to use dism.exe for capturing.

*However…*

The problem with dism.exe is that MDT doesn’t have the ability to capture the progress output from the program so it can’t display the pretty progress on the Task Sequencing Progress Bar. For Capture images, that’s not a big deal, but for image apply it would be better to keep imagex.exe, since MDT can capture the output from the program and display progress for Apply. imagex.exe apply didn’t have the imagex.exe 15 minute wait at the end so there was no need for it there.

 

During MDT beta I sent feedback to the MDT team that we should revert back to ImageX.exe for captures, but they declined to fix.

Please feel free to “upvote” my bug on the Microsoft Connect Site if you agree:

https://connect.microsoft.com/ConfigurationManagervnext/feedback/details/1685836

-k

MDT UberBug04 – Litetouch Success may mean Failure

I once knew a guy named Ted who didn’t like it when the “Check Engine” light came on in his car. He fixed the problem by covering up the Check Engine light with some black electrical tape. “There, now I won’t be bothered by the light anymore.” “But that’s not how to fix that.” I said… <sigh>

One of the really cool things about MDT LiteTouch is the fact that +90% of the product is included with source, VBScript and PowerShell. Even the Console and the PowerShell Providers written in C# can be decompiled and checked if there is a serious error. That means that if there are any problems, most of them can be fixed with some code changes on site.

ZTI (most of SCCM) and UDI are also closed boxes, written in C/C++, if you have a problem with them, then you need to file a bug with Microsoft Product Support <ouch>.

About the only thing that is a black box in MDT LiteTouch is the Stand Alone Task Sequencing Engine. It’s written in C/C++, but as long as I have worked on MDT, it’s been rock solid, only failing in a few obscure scenarios.

The bug

I had some weird errors in my task sequences for a couple of Windows 10 deployments, the “Deployment Summary” screen showed success, but the install failed somehow, what happened?

I tried creating a task sequence with an extra step to reproduce the problem:

tsstep1

The step is simple, it runs cmd.exe, and promptly fails. If we define “Continue on Error” as “False”, any failure returned in this step means that the *entire* Task Sequence should fail.

tsstep2

The result is that you get the Red (Or salmon) screen at the end of deployment.  That means that if I were to run this Task Sequence, I should see an error in MDT Litetouch.

However, what I see in my deployment is “Success”, not “Failure”

result1

What’s frustrating, is that the smsts.log is no help, it doesn’t show that the cmd.exe /c exit 42 command ran, it also doesn’t show what the error returned is in smsts.log.

...
Reading  logging settings from TS environment to set TS logging
Process completed with exit code 0
Exiting with return code 0x00000000
Used smsts.ini to set logging settings.
...

I can reproduce this problem quite easily in my Hydration/Buildout System, and I even provided a repro VHDX file to the MDT Product group team, but so far this bug has been marked as “Low Priority” and postponed till the next release.

Generally, I can work around most of the bugs in MDT, I just fix the script and move on, but if the error in a black box, with no work around there is not much I can do. <sigh>

That combined with MDT UberBug03 creates a dangerous scenario. Now we have task sequences that are failing, randomly, but returning success, and no logging to tell when and where it happened. This is a clear bug, and a regression from MDT 2013.

These two bugs alone (not to mention the other UberBugs I will be filing in the next few days), lead me to believe that MDT has some serious problems that could lead to problems in production.

Please let me know if you see this in your deployments. And upvote my Connect bug.

Bug: 1685295

MDT 2013 UberBug03 – Less Logging is not better

During the announcement of MDT 2013 Update 1 yesterday it was mentioned that one of the new features is:

Updated task sequence binaries from System Center 2012 R2 Configuration Manager SP1

Deployment pros know what a Task Sequence is, it’s the steps performed in order that make up a full deployment.

ts

Even when running LiteTouch, MDT will use System Center binaries to do the heavy lifting of calling each program, managing variables, and logging the results.

Logging

During task sequence execution, the SCCM Stand Alone Task Sequencing Engine will log each step, and log the results, most of the time in %temp%\smsts.log. For example, say we add a bad step to our task sequencer. In our SMSTS.log file would we see:

!------------------------------------------!
Expand a string: WinPEandFullOS
Executing command line: cmd.exe /c Echo Hello TS & exit 28
Process completed with exit code 28
!------------------------------------------!
Failed to run the action: Run Command Line.
The printer is out of paper. (Error: 0000001C; Source: Windows)

That’s great, I created a step in the Task Sequencer that retuned back a non-zero error code, and the step was not marked as “continue on error”, so the Task Sequence stopped because of it. At least we know what step failed.

Updated for MDT 2013 Update 1

Now with MDT 2013 Update 1, they have updated the core Task Sequencing binaries, and there is new functionality. The new functionality is to *not* log the steps, even when there is an error.

No Logging, this sucks.

This means that if one of your steps fails, and returns a non-zero result, the MDT Task Sequence may fail, and you the MDT Administrator may not be able to figure out why or where it failed.

I consider this to be a major flaw of the New Task Sequencing engine. I have filed a bug against the MDT team a while back, but they have indicated that they won’t look at a fix until the fall 2015.

If you think you can create task sequences that never have problems, then you should be OK, but logging after the fact is an important part of any *Production* MDT environment, as it blocks debugging.

There is no known work around.

MDT 2013 UberBug02 – English keyboards only please!

Got a question from a private DL asking about a possible bug in MDT 2013 Update 1:

locale

When you change the Keyboard Locale in the MDT LiteTouch wizard, you get this error:

“An error has occurred in the script on this page”

“Type mismatch: ‘SetNewKeyboardLayout'”

The bug

Looking at the differences in MDT 2013 RTW (Release to Web), MDT 2013 Update 1 Tech Preview, and the final MDT 2013 Update 1 RTW ) version, I can see that someone added some debugging code into DeployWiz_LanguageUI.xml for the beta:THe

<SELECT NAME="KeyboardLocale_Edit" class=WideEdit onchange="SetNewKeyboardLayout">

and into DeployWiz_LanguageUI.vbs

Function SetNewKeyboardLayout
   MsgBox(KeyboardLocale_Edit.Value)
   MsgBox(GetLocale)
   MsgBox(hex(GetLocale))
End Function

Clearly this is some kind of debugging used in development, adding a MsgBox during onchange in production is bad.

The Fix

The work around is easy, click “yes” on the dialog above, and continue.

The fix is fairly easy. Simply change the DeployWiz_LanguageUI.xml file line 62 from the text above to:

<SELECT NAME="KeyboardLocale_Edit" class=WideEdit>

Connect bug: 1684128