Ryan Gallier

Jan 292014
 

 

I’m sure we are all familiar with the Shutdown Event Tracker. Hypervisor crash for “no reason”? Have a bunch of servers power down hard “by accident”? It happens to all of us. What’s annoying about this, specifically in a XenApp/RDS environment is the fact that when a regular user logs in they will see this message unless an administrator has already gone in and removed it.

Now, you could just remove it via GPO all together, but I’m not really a fan of that. I would think that this would be available for administrators only, and not regular users. The GPO supplied is a computer based GPO and does not allow that type of granularity. This is in Computer Configuration / Policies / System. As you can see it basically has no options for users.

Simple fix though. After about two seconds of troubleshooting I found that this tracker is controlled by c:\windows\system32\shutdown.exe. So, you could simply just take ownership of this file and remove users read access to this and that works fine. However, if you want to do this in some scale, you can setup a Software Restriction policy and apply it to your RDS/XenApp users. This is also pretty simple.

Drill down to User Configuration / Policies / Windows Settings / Software Restriction Policies. Go to Action and select “New Software Restriction Policy”.

This will create some new folders under Software Restriction Policies. Drill down to Additional Rules and right-click “New Path Rule”.

Simply type in the path and hit “ok”

Make sure this policy is applied only to non-admin users and not administrators. I have a large GPO that I apply to all regular users that access XenApp, so I simply applied it there. That’s about it. Now when your non-admin users’ login they will not be allowed to launch shutdown.exe, which in turn will stop the Shutdown Event Tracker from appearing.

You can validate this by running a command prompt as a regular user. They should be getting this message.

Have fun!

Jan 072014
 

I’m not going to go into the details about what Multi-Stream ICA (MSI) is in this article. I assume you already have a basic understanding of what this is and you are really just here to figure out how to configure it. If you do not, a great Citrix blog about Multi-Stream ICA has been written up here. In a nutshell Multi-Stream ICA allows you to break out different portions of Citrix traffic into dedicated TCP ports. The basic breakdown is below:

Enabling this on the Citrix side is pretty easy. We need to first enable Multi-Stream in a computer policy.

 

Then we need to setup the Multi-Port Policy. In this example I’m using 2599,2600,2601 for the other ports. The Default Port is the standard 2598 port for Session Reliability.

 

MSI requires Session Reliability to be enabled, so make sure you don’t have it disabled in a Computer Policy. I like to enable it so my other engineers know for a fact that it is enabled.

This takes care of the Citrix side. Not too much configuration there. The Cisco side gets a bit more involved. You do need to have a bit of knowledge on basic Cisco configuration and more specifically, an understanding of how QOS works, in general, and specifically how Cisco QOS works. Cisco has released an article here that talks in detail about how to implement MSI in a Cisco Enterprise environment. The Cisco document calls for mapping specific DiffServ classes to each MSI priority level. This is certainly the best practice when you want this to traverse a larger network with multiple hops. I’m going to post a simple example where you have Users—–RTR1—–RTR2—–Citrix. We will be using a 5Mbps Point to Point link between the 2 routers. In this setup we are going to define the ports, and assign bandwidth values to them.

First, you want to define the port groups. I will label these with Very High, High, Medium, Low (vh, h, m, l). All configurations need to be performed on both routers.

ip access-list extended citrix-vh
permit tcp any eq 2599 any
 
ip access-list extended citrix-h
permit tcp any eq 2598 any
 
ip access-list extended citrix-m
permit tcp any eq 2600 any
 
ip access-list extended citrix-l
permit tcp any eq 2601 any
 
Next we will define the Class-Maps.
 
class-map match-any citrix-vh
match access-group name citrix-vh
 
class-map match-any citrix-h
match access-group name citrix-h
 
class-map match-any citrix-m
match access-group name citrix-m
 
class-map match-any citrix-l
match access-group name citrix-l
 

Now that the variables are defined we move on to creating a queuing policy. Let’s assume we have 20 users. My main concern here is the “high” queue. At periods of congestion I want to make sure that the Screen/Keyboard/Mouse have enough bandwidth so users can continue to work and not have service degradation. Let’s say 20 users @ 50Kbps each = 1000Kbps. That’s about 25% of a 5 meg link, however we can give it a little wiggle room, so let’s bump that up to 35% (1.7Mbps). Keep in mind that this doesn’t LIMIT the bandwidth to 35%, it can use as much bandwidth as it wants until periods of congestion (link saturation) where it’s guaranteed the 35% of the bandwidth we configure. The medium and low queues which contain printing and file redirection aren’t as critical and I will set these to use 1Mbps during congestion (25%). Lastly, is the Very High queue. In my specific environments we don’t use too much audio, therefore I have set this to 15% of the link speed.

 

Keep in mind that your numbers and percentages will vary based on your use case and your amount of bandwidth available. Do you have a lot of users upload photos? Do you print a lot of large documents? Do you use a lot of real-time audio? You will need to evaluate your specific scenario and alter this example to fit your needs. You may also have existing QOS on your link and need to integrate this configuration. We have customers using SIP/RTP for voice on most of our links and I have integrated that into my configuration. I have removed that configuration for simplicities sake.

 

Here is the configuration for my policy map.

 
policy-map CITRIX-QUEUE
class citrix-vh
bandwidth percent 15
class citrix-h
bandwidth percent 35
class citrix-m
bandwidth percent 25
class citrix-l
bandwidth percent 25
class class-default
fair-queue
 

After this policy-map is configured we will need to embed this into a shaping policy to make sure the percentages line up with the available bandwidth. If you do not do this, QOS will assume the link speed of the interface which is generally 100/1000 Mbps. Obviously 25% of 100Mbps is more than the 5Mbps the link speed actually is. The below configuration will shape the bandwidth at the site to 4.9Mbps then apply the queue.

 
policy-map CITRIX-SHAPE
class class-default
shape average 4900000
service-policy CITRIX-QUEUE
 

After all is said and done you will need to apply this to the each end of the interface in an OUTBOUND direction.

 
interface FastEthernet4
service-policy output CITRIX-SHAPE
 

Make sure your configuration is working by doing a “show policy-map interface FastEthernet4”. The first example shows citrix-vh. You will want to make sure its incrementing packets under the “Match” area. You may see some drops (164), but this is normal. The second snip is class-default. If you have congestion on your link you will start to see drops on this class-map. In this example we have 143878 drops. This is a 5Mbps link and QOS is working great keeping non-critical traffic from starving my Citrix traffic.

 

Class-map: citrix-vh (match-any)
752287922 packets, 53798289807 bytes
5 minute offered rate 0 bps, drop rate 0 bps
Match: access-group name citrix-vh
752287923 packets, 53798290067 bytes
5 minute rate 0 bps
Queueing
queue limit 64 packets
(queue depth/total drops/no-buffer drops) 0/164/0
(pkts output/bytes output) 752287754/53798175224
bandwidth 25% (1024 kbps)
 

Class-map: class-default (match-any)
315951400 packets, 75108131350 bytes
5 minute offered rate 94000 bps, drop rate 0 bps
Match: any
Queueing
queue limit 64 packets
(queue depth/total drops/no-buffer drops/flowdrops) 0/143878/0/143878
(pkts output/bytes output) 315807520/74835090017
Fair-queue: per-flow queue limit 16
 

Hope this helps you in your Citrix Multi-Stream ICA configurations. This is a pretty basic setup, but I hope this points you in the right direction to get started using this cool newish feature. I hope to write up a blog later on how this integrates with CloudBridge and how to use MSI in conjunction with that.

Jun 272013
 

 

I’ve been using PNAgent to create Desktop/Start Menu shortcuts for years. I’ve always thought it was a good method to give users only what icons they have access to launch. With the new Group Policy Preferences that have come out with Server 2008, we now have another option. I find that GPP based shortcuts are more stable than PNAgent based shortcuts as they do not rely on XML to be functioning for users to launch applications. Keep in mind however, for this method to work you will need to have the applications actually installed on the XenApp servers themselves.

|Atum| on CitrixIRC started with a simple little script that was able to take some information and convert it to XML to import into a GPP. I gave this information to one of our developers, losethisurl, in CitrixIRC to enhance it. Below will be the scripts and steps to convert PNA based shortcuts to GPP based shortcuts.

The first thing we want to do is check your execution policy to make sure its set to something that will allow this script to run. In my test environment it’s currently set to RemoteSigned.

The first script is called Get-CitrixAppList.ps1. This basically runs a Get-XAApplicationReport and parses the information out in a format that we can use to import into another script. We run this from one of your XenApp Servers. The script is smart enough to load the Citrix snap ins if they are not already loaded. You need to use OutputFileName to specify a filename and optionally you can use OutputFilePath to copy it somewhere. Otherwise it will just save in the current directory.

Notice how XAApplications.csv is now in the folder. Let’s take a look at what’s in there.

So, let’s take this file and convert it to some XML for our GPP. Copy this to your AD controller, or any box that has the Quest AD PowerShell command templates. You can download those here. Now we can use the script Convert-CitrixAppCSVtoGPPShortcutsXML.ps1 file. All we need to do here is run the script and point it to the CSV file.

Notice this creates an Out.xml file. Let’s take a look. If you are familiar with GPP XML files, this should look pretty familiar to you.

So, what do we do with these? Well, that’s pretty simple. All we need to do is copy/paste these into the existing GPP shortcuts policy. I assume you should already have some kind of XenApp Users Policy. If so edit that one. If you do not have one, simply create one with a dummy shortcut. I’ll show you how. First, create a GPO object (or edit an existing). Go to User Configurations/Preferences/Windows Settings/Shortcuts. Right-Click and create a new shortcut.

Click Ok and close the window. Next, go to the “Details” tab on the GPO object and note the Unique ID.

Browse to \\fqdn\SYSVOL\fqdn\policies\<Unique ID>\User\Preferences\Shortcuts\ and edit Shortcuts.xml

You should already see some crap in there for where you created a shortcut earlier. Remove everything right of the blue line.

It should then look like this.

Now copy/paste your Out.xml and paste it on the blank line above </Shortcuts>.

Save the file then close it. Now go back into your GPO and look at the Shortcuts GPP section.

Excellent! Now we have GPP based shortcuts. You may need to go in there and delete things you don’t want, such as “Full Desktop 6” in this example. You can also adjust their locations and item-level-targeting as needed. By default the script will already use item level targeting based on the user group that was assigned to the application in AppCenter (or DSC).

So now that this is populated you don’t want to have duplicate icons published from PNA and GPP.  Make SURE that you have the shortcuts delete on logoff inside of the services site before you start this whole thing.

shortcutremoval

I also have a script that will disable all of the applications in the farm that you are publishing. Keep in mind to go back and re-enable any applications that aren’t locally installed on the XenApp server. So, let’s go back to the XenApp server and take a look at the Console.

All you need to do is run Disable-AllPublishedApps.ps1 from the XenApp server.  This will disable all apps, and remove “Show on Desktop” and “Show on Start Menu”.

Take a look at the console now. Notice that it excludes any published desktop. That’s a feature. J

That’s about it. This is a real time saver, especially if you have 50 or so applications created through PNA.

Feel free to look at the scripts below and jump in CitrixIRC to talk about it! You can join the conversation at http://join.citrixirc.com. If you already have an IRC client you can simply join #Citrix on the Freenode IRC network.

Get-CitrixAppList.ps1

Convert-CitrixAppCSVtoGPPShortcutsXML.ps1

Disable-AllPublishedApps.ps1

Apr 162013
 

 

Why Windows didn’t enable this feature in the built in GPOs is beyond me. Regardless, I needed a way to disable Windows Defender automatic scans to keep my hundreds of XenApp servers from running a scan at 2am and most likely crushing my storage infrastructure. So, what am I talking about here? How to disable this:

As you can see, the default GPOs do nothing for us.

So, how does this actually work? Well, when you configure this automatic scan, it creates a scheduled task, and writes a file in C:\Windows\System32\Tasks\Microsoft\Windows Defender\

Now, you can just delete the MP Scheduled Scan file, but this doesn’t remove the configuration from Windows Defender, so that won’t work. After a small bit of digging I found these registry keys in HKLM\Software\Microsoft\Windows Defender\Scan

The key in question here is “ScheduleDay” 0 = daily, and 1=Sunday, 2=Monday, etc. 8=off. So. Simple GPP configuration here to set the key to 8.

Do a GPUpdate /force and Viola! It has been removed from Scheduled Tasks, the file is gone, and its configuration removed from the Windows Defender GUI.

 

Feb 182013
 

Profile Optimization and “How do I speed up login times?” generally go hand-in-hand. These have to be two of the most important and most talked about items when it comes to delivering XenApp desktops. There are lot of different philosophies and strategies with regards to this, and in this article I’ll simply talk about what I have implemented in my environment. I have gone through extensive testing, tracing, logging, and analyzing of my settings and will show you what has worked for me. You can use some of these techniques to troubleshoot your own environment and see if you can get some gain in yours.

To start, I have leveraged many whitepapers, blogs, and Citrix KBs to generate my settings. I’d like to give credit where credit is due. First the Citrix XenApp and XenDesktop Policy Planning Guide was a good resource and baseline for everything. Second, this Citrix blog about Citrix Profile Management had a lot of great information. Also, CitrixIRC, of course, has been a great reference to talk things through with a bunch of great Citrix Admins. Join our chat at http://join.citrixirc.com. I have also read many other things on these topics, but I don’t recall them well enough to cite them.

Let’s get the framework in perspective here. I work for a Citrix CSP (Citrix Service Provider) and we currently have a couple dozen farms mostly in the SMB space (<250 users) I don’t do any enterprise work, so my tools and tricks are built around an SMB mindset. I use Citrix Profile Manager and GPOs, exclusively. I do not use any other third party tools to manage my profiles. I try to keep my environments simple enough for our other admins to be able to manage them. I think that if you can configure and test these tools properly they can do the job well enough to not need additional cost factors in our environments.

That being said lets start with Folder Redirection! Simply put, I redirect everything, except for AppData, utilizing GPOs. I manipulate AppData with UPM and we will talk about that later. Redirecting everything keeps it out of the profile and keeps the profile small. Simple enough.

Folder redirection isn’t the only culprit for large profiles. There are other commonly used programs that keep crap in the profile. I use GPOs to redirect these items as well. Outlook PST and OST files. Download the Office admx templates and USE THEM. “Microsoft Outlook 2010/Miscellaneous/PST Settings”. I set “Default location for PST/OST files” to a network drive. Well, I’m not using cached mode you say? Other things are stored in PST files as well, such as SharePoint Lists, so keep this in mind. AutoArchive? This will create a PST also, so if you are using this, you will want to make sure PST/OST files are moved. AutoRecover files are also stored in the profile. You can redirect Excel and Word Autorecover using the same admx templates.

How about Evernote? A lot of my users use Evernote, and by default the database is stored in AppData\Roaming. I redirect this to a network drive with a GPP Registry key. “HCU\Software\Evernote\Evernote” REG_SZ “DatabasePath”. I have seen very large databases and this is a good tweak to keep the profiles small.

Let’s talk AppData. First, I use UPM to exclude AppData\Local and AppData\LocalLow at the root. I keep AppData\Roaming in the users profile mainly for the performance implications of this being redirected on a large scale. However, I use the UPM to exclude a bunch of directories to keep it as small as possible. I will attach my UPM GPO for you to look at these settings in more depth. I exclude about 12 directories from AppData\Roaming that were gathered from the various best practices documents. Using Chrome? Chrome keeps all of its settings in AppData\Local. Shame on you, Google! With UPM, this is no problem. I do 2 things with Chrome. First, I include AppData\Local\Google in Synchronization. Second, I exclude AppData\Local\Google\Chrome\User Data\Default\Cache, Cached Theme Images, and JumpListIcons. This allows my users’ Chrome settings to save, but excludes the not-needed bloat directories.

Don’t forget the cookies! I have written another blog here on that. Read it!

How about the rest of the UPM settings? Again, I’m going to attach my UPM policy in here somewhere, but we can run through the basic settings. I delete cached copies of local profiles. We always want to load a fresh profile each time. This will lower profile corruptions. How about profile streaming and active writeback? Well, I turn these off. Most people will say that’s dumb, and those are great features, and you should keep those on. Well, I can see how these are great features, but again, I’m tuning these settings for my environments. With the tweaks I am implementing I have an average profile size of 30meg. The profile itself can load in less than 1 second on a gigabit network, so I’m not too concerned about this. These settings are nice for larger setups, but in my environment I’ll keep it as simple as possible.

Did you use the 2008 R2 Optimization Guide for XenApp 6/6.5? Well, don’t forget this blog post about one of the settings you need to change if you are using the UPM. Without changing it, UPM times out a lot and slows logon/logoff processing.

Don’t forget to exclude all of the un-needed folders inside of the profile as well. This is done with a GPO.

When all is said and done, here is what my profile looks like. Of course, this is a test user, but this is a great foundation to build user profiles on. Notice there are not any folders in there except for Windows and AppData.

Do you already have an environment built and would like to tweak these settings? I certainly did. I actually wrote a script that would go through the Profile Store and delete out all of the bloat from the users’ existing profiles. You can check that out script here. Its powershell, so have fun. I had users with 1gig profiles and was able to lower then to 30-60meg in our internal environment. Note, this must be run from the Profile Store directory.

One setting that works for me, but will require testing, is the GPO to wait for network at computer startup and logon. I was able to gain about 13 seconds on my logon times when I disabled this setting. Your mileage may vary.

Are you using GPPs for shortcuts and printers and such? I did a lot of GPP tracing to analyze these mappings and found this to be true inside my environments. If you create GPP Shortcuts using “update” it takes about 200ms for each item at each logon to parse. If you have 60 shortcuts between the start menu and desktop, that’s 12 seconds right there. That’s not a short amount of time. Setting these to “create” will speed this up to about 5ms per item at each login. You can change it to “update” if you actually want to change something in the future. I gained another 15 seconds on my logins when I changed all of my GPP shortcuts to “create”. The same basic numbers apply for printers too, however, I have not traced them to get exact numbers yet.

Login times have a lot to do with how many GPOs that you have in your environment. Remember these tips. Always prefer fewer larger GPOs opposed to many smaller ones. Each GPO has a set base processing time that can be avoided by consolidating GPOs into one larger one. Make sure you disable Computer/User settings in a GPO if you aren’t using them. This lowers login time a second or so per GPO.

So, what did I use to troubleshoot all of these things? I do curse Microsoft for getting rid of the userenv.log detailed logging. Nothing works quite as good. UPM logging is a really good place to start, however. You can turn it on in the UPM Policy GPO, and parse the logs with the UPM Log Parser. You should also be using the GPSvc.log. You can set that up using this blog. Don’t forget to create the “usermode” directory if it doesn’t exist, or the logs won’t work. You can also turn on GPP Tracing in a GPO under “Computer Configuration\Policies\Administrative Templates\System\Group Policy\Logging and Tracing”. You can turn all of these on, and enable tracing, to get detailed information of your GPPs. Some people like to use Policy Reporter to go through the logs. This is a nice tool, but I just read the logs manually.

Using all of these tips and tricks above, I was able to get my test user in my test environment to log in after about 9 seconds on the 3rd login. Obviously the first 2 logins are a tad slower as it builds the profile from scratch and runs some other scripts that I have in my environment. Now, keep in mind that’s a bare environment and your mileage will vary here as well. In my internal environment I was able to speed my logins up from around 75 seconds to 23 seconds. This is keeping in mind that our internal environment has about 847 GPOs and isn’t optimized at all. In my customer facing CSP environments I have gotten about a 75% improvement time in the environments that I have implemented these changes.

Take a look at my detailed UPM policy is here

Jan 312013
 

 

I’ve been running in my 6/6.5 environment since the beginning with UPM and didn’t even realize that the cookies were not saving properly. Apparently this is a known issue and the fix is simple! You need to add 2 things to your UPM Group Policy object. First, Add “AppData\Roaming\Microsoft\Windows\Cookies” to the “Folders to Mirror” policy. Second, enable “Process Internet cookie files on logoff” under the “Advanced settings” folder. When the user logs out and back in again, UPM will start properly processing cookies. Note: This does add a couple of seconds to the logoff time for users. Keep in mind if adding this to an existing environment that it could add minutes to the logoff time the very FIRST time a user logs off if they have a lot of cookies.

Dec 102012
 

Orazz from CitrixIRC found a great forum post that really made an impact on our environment. In a nutshell, when using the 2008 R2 Optimization Guide for XenApp and the Citrix Profile Manger, there is a registry setting that causes the UPM to timeout. A lot! After making the changes below, I noticed an immediate improvement in performance of the UPM. Logon/Logoff times have been reduced significantly. I’m also hoping this is going to lower the profile corruption issues that also pop up from now and again. We don’t have many since upgrading to UPM 4.x, but we still do have some.

  1. Upgrade to UPM 4.1.2
  2. Change HKLM\System\CurrentControlSet\Control\FileSystem\”NtfsDisable8dot3NameCreation” from “1” to “0” ( I did this using a GPP object as part of the optimization guide. I simply edited the GPP and changed to “0”)
  3. Backup and delete the keys from “HKLM\Software\Policies\Citrix\UserProfileManager”
  4. Reboot

Upon rebooting I verified the keys were recreated, and the NtfsDisable8dot3NameCreation was set to 0.

Note, some interesting changes in UPM 4.1.2 also. We use “Delete locally cached profiles on logoff”. This process now takes about 3 minutes for the folder to delete from the XenApp server. This is part of the new design. See UPM 4.1.2 in http://support.citrix.com/article/CTX134616

Citrix Forum Post

Nov 092012
 

Do you have any HP Printers that are using the Windows LaserJet Family Driver and they are only printing 1 page no matter how many copies you select? Disabling Mopier mode (which is enabled by default) fixes the problem.  We have seen this problem in XenApp 6 and 6.5.

Steps to fix:
1. Start > Devices and Printers
2. Right Click Printer >Printer Properties
3. Click “Device Settings” tab
4. Scroll to bottom of page and change “Mopier Mode” to Disabled under the “Installable Options” group