Using MSI 5 Fully Cached Packages On the Sourcelist Print E-mail

Windows Installer on Windows 7 has a nasty little secret!  It caches the ENTIRE MSI file.  Surely it must then use it as the source location right?  Nope!  But where there is a will, there is a way (sometimes the wrong way, but that's for another time ;)

Legacy Packaging Caching Overview

Windows Installer has always cached a copy of the .MSI file in the "Installer" folder under the Windows folder.  It is used for uninstall.  Before Windows 7, this cached copy of the .MSI was always stripped of the application source files if they were compressed within the original MSI to save space.

The names in the cache folder are randomized because in situations where per-user installs are occuring, each user may have a slightly unique, minor version of the package.  For install (and source list processing), minor versions must have the same file name.  So if Joe has Abc.msi 1.1 installed per-user and Sally has Abc.msi 1.2 installed per-user we'd have a name clash in the cache unless we randomized.  Since the .MSI packages could be different internally, each individuals self-healing must point to an accurate copy of what they have installed.  So the file names are randomized.  You probably don't do may per-user installs, but that's why it works this way.

What Changes in Windows Installer 5 (Windows 7 and later)

Windows Installer version 5 (Windows 7 release) caches the entire .MSI package to its cache location - including any internal application source files.  So if you have  1.5 GB .MSI, you will find the entire 1.5 GB file in the cache with a random name.  So you may start finding some very large .MSI files in c:\windows\installer on Windows 7.

Why does it do this?  It turns out that a 1.5 GB package that is signed will give a nice blue UAC prompt indicating that everything is cool - the package is signed!  If, however, the files are stripped out of it during caching, the signature is broken and an uninstall from Programs and Features gives the nasty alarming Yellow UAC prompt indicating that software from an unknown source wants to elevate on your system.  

This is, in fact, exactly how it works on Vista.  Microsoft struggled with how to fix this situation all during Vista's lifetime.  The solution chosen was caching of the entire package.  

There is no way to disable this behavior and simply take the yellow UAC prompt hit.  Even if you are a corporation who never let's users uninstall software from Programs and Features!

This behavior also happens even if the package is not signed.

How Do I Prevent Full Caching?

If your company made the packages, then change your package format to have external .CAB files.  The external cabs are not copied into the cache.

Enjoying your read? Subscribe to our newsletter (without loosing your place in this article).
(Please ensure that the confirmation email clears your spam filter so that you will see future mailings.)

If the .MSI is from another company, they you *may* be able to do an administrative install which pulls the files out of the .MSIs and puts them next to a copy of the .MSI without the internal sources.  The challenges to this approach are: (a) Takes double the disk space for the .MSI + uncompressed files on your distribution server, (b) Breaks the .MSI file signature (if there was one - even if there is one, in 99% of corporate distribution scenarios it does not matter or you can resign the .MSI with your own certificate), (c) the individual source files are in the file system and can be played with - affecting any installs that occur after that. (by someone with the best of intentions of course).

It's better just to explain the situation to the company providing the .MSI and ask them to change to the external .CAB source format.

Is the Cached Package Used as a Source?

Well at least it must use the fully cached package as a source location so you get the benefit of taking the hit on disk space, right?  Nope.  The reason is the little problem with .MSI file name randomization we talked about above.  When any search for source occurs, it always uses the original MSI file name - so adding C:\windows\installer to the source path does not allow the package to be found.

I spent some time trying to rename the cached copy and see if I could trick the whole system, but windows installer would rewrite a new copy with the random name (it is randomized at install, but the same for an installed product after that point).  I suspect I'd have to hack on the repository as well and even then I am working against MSI behavior in a fairly fundamental way - not something you can use in managing packages in the enterprise.

While teaching a class a couple weeks ago I had a blinding flash of intuition.  (It is amazing how many times this happens during class when I am trying to explain concepts to a group - proves to me beyond a shadow of a doubt that  articulating to others is a catalyst to our own innovation.)

Symbolic Links to the Rescue

In theory, if a symbolic link is created from the cached version to a known path with the original package name, we could put that symbolic link location on the source list and use the cached copy as a source.

Well it works!

If you've never heard of symbolic links before, here is the BEST introduction and BEST free tool for working with them on Windows.  BTW, admins are increasingly using file system linking to create solutions and work arounds for many different challenges when running windows - so it is a good idea to become familiar with them.

Proof Of Concept Script and Test Package

Attached to this post is a proof of concept script.  Without any modifications it can run under Windows Script Host or as a Windows Installer custom action.

The attached also includes a copy of our test harness package with the script embedded as a custom action.

One thing to beware of with file system links (hard links, symbolic links) is that backup software and scripts and software that independently calculate available disk space will count the bytes for any linked file each time a link is encountered - it looks like a normal file.  This isn't true of every software package, but the vast majority of Windows file system utilities do not check whether files are real or links.  This has to be done explicitly on each file because the linking system is designed to fool anything accessing it into thinking the file is the real file.

Implementation Details

Symbolic links are used so that when the package is uninstalled the file will be removed and no longer take up space.  If hardlinks were used, when MSI removed the cached copy, the actual file contents would stay in place and not release the storage.  The symbolic link will be broken in this case, but since the package referencing it is not on the system anymore, it shouldn't create any problems.

In the provided script the folder used for links is: "%windir%\Installer\DerandomizedSymboliclinksForSourceLists"  I suggest using this location and and name because: (a) It has a self-documenting name, (b) it is under a known "throw away" cached location and under Windows - so it is *less* likely to cause data backups to grab the files since they won't be mistaken for user data files.  If we used C:\packages, backup software might grab it and curious admins would play with it.  By placing it under the Installer cache folder it communicates it is related to the files in the Windows Installer cache and this folder is rarely viewed by individuals who do not already understand some of the internal workings of Windows Installer.

The proof of concept script automatically clears the existing sourcelist, including the location it ran from.  If you are wanting to use the local location as the FIRST location on the list, you will need to add the current location back to the list (remove the commented code).  If you want to have the local cached location as SECOND, you can comment out the line when clears the source list.

Requirements: MSI Version, .MSI Package, MSIEXEC Run Context

There are, however, some restrictions (these are also detailed in the proof of concept script attached to this post):

  1. You must be running on Windows 7 or later, Vista and earlier do not have the MSI 5 behavior of placing the entire .MSI in the cache (Script does nothing if MSI version not >= 5)
  2. The MSI used MUST have source files compressed internally.
  3. The MSI must have NO external dependencies on other files. (unless you already copy your package source locally - see section"Already Maintain a Local Cache?" in this article.)
  4. The MSI package is ALWAYS installed Per-Machine (ALLUSERS=1) (Since the hardlink will be removed and rewritten to the latest install).
  5. The process that calls the code as a .VBS or the process that calls MSIEXEC.exe when the code is a custom action, must already be FULLY elevated. Any software installation system that runs a background service logged in as an administrator meets this requirement (UAC does NOT affect services).
Additional requirements and considerations to run as a custom action:
  1. NOTE: The proof of concept VBS code will run as a custom action with NO ALTERATIONS.
  2. The custom action must be set to run AFTER the "RegisterProduct" standard action (the product must be registered before we can mess with the source list)
  3. The custom action is NOT ready to be used as a deferred system context custom action because it directly sources several properties for simplicity.  These properties (OriginalDatabase and LocalPackage) would need to be passed in on CustomActionData if reworking for deferred system context.

Already Maintain a Local Cache?

If you already copy package source files locally as a permanent managed cache, you could use this method to link into your existing local cache.  In fact, if the MSI has other associated files that it depends on in order to run, you could replace ONLY the .MSI with a link and then you would still save the disk space of the .MSI.  In this case you may wish to use a hardlink so that the removal of the cached version does not result in removing your cached version (would be a problem if you were running a "reinstall" script against the original package as your managed cached copy would disappear after the uninstall of the package).

In this scenario you would also not need to play with the source list at all since the default first source location is the same file you are overwriting with a hardlink.

A Simpler Way

If you are familiar with EXE custom actions and only wanted to use this concept as a custom action, it could be reduced to one file search (AppSearch) for a previous version of the link and two cmd.exe custom actions - one conditioned to delete a previous version of the link if it exists and one to create the new link.

Digging Deep

As CSI-Windows we like to dig deep and find out the real details because we know these details allow you to add value at work and feel good about making significant contributions!  We build our training with the same approach.  Check out our training.

Download this file ([Does not contain sample .MSI file]2 Kb
Download this file ([ ]667 Kb

Add comment

Security code