NYC Powershell User Group Update

I got busy today!  The user group officially has a website.  Granted it’s probably just temporary until I solidify the location/date.  I placed a call into the Microsoft 6th ave. office near the end of the day.  Hopefully we’ll have some venue choices by the end of the week.  Amusingly a friend said we should do it in Brooklyn at Monkeytown, but it closed down recently.  That would have been a cool place for a meeting.  I’m still open to other interesting artsy ideas if anyone has any connections or thoughts….  It has to be in Manhattan though.

Scriptomania! 2500 dollar prize!

VMWare is running a scripting contest.  They want the most creative ESXi management scripts possible!  So put on those thinking caps and get scripting!

http://communities.vmware.com/community/vmtn/vsphere/automationtools/scriptomania

NYC Powershell User Group

I’ve decided to put together a user group for New York City. I have received a lot of interest from friends and colleagues. I put together a few presentations this weekend. I’m looking to have the first meeting in March. We’ll start with a simple format getting two speakers or a speaker and a vendor to do 30-45 mins. I’m going to try and keep it balanced with one advanced and one beginner discussion. This way people with all skill levels can benefit from the meetings. Once I get a venue I’ll put up an official page.

If you’d like to help by being a speaker or help me find a place to host it please contact me. Also drop me a line if you’re interested in attending so I can email you when the official page is up.

Excuse Me While I Stroke My Ego

I have just completed the CompTIA “Train the Trainer” course to get my CTT+ and MCT certifications.  Part of the process is to design and teach a 20 minute class in front of a minimum of 5 students.  This all needs to be video taped, and you are judged on certain criteria.  I decided to do a little Intro to Powershell class.  Unfortunately, that was easier said than done.  It took me 3 days of working with the students to realize that a hands-on demo just wouldn’t be possible.  The majority of the people in the class are becoming MCTs in order to teach Office products, and things like getting them to set-executionpolicy remotesigned and run a few scripts was a taller order than I anticipated.

After some late nights of planning, and suffering through a stuffed nose and sore throat due to an attack by the common cold I succeeded in my task.  What I wound up with is a decent introduction to Powershell from a big-picture view with an emphasis on why we need scripting, and why we should use Powershell.  Ideally the class would have been an hour, but one of the criteria is that you can ensure that everything is done in 20 minutes so some things definitely were breezed through, and some things I hoped to discuss were completely skipped.  I probably should have dropped the last two scripts, and at the very least had the class type get-process possibly with a pipe to out-gridview.  Regardless I think it went well enough to post here.  Granted if you’re reading this site then you are not the target of the presentation, but if anyone has any teaching gigs for me feel free to drop me a line.

View the Introduction To Powershell videos

Powershell and Scheduled Task Logs

There was a thread in the Powershell forum that posed a question of how to report on the status of scheduled tasks by using powershell on remote machines. My first thought was to use WMI:

$servername = "tome-mac"
$items = get-wmiobject -class "Win32_ScheduledJob" -namespace "root\CIMV2" -computername $servername
$items |select Command, Status

The above, however, only showed items scheduled via WMI or the AT command. I found a nice COM interface from this link that would let you look at the properties of scheduled tasks. I put together a quick script to look at the status of the tasks:

$servername = "tome-mac"
$schedule = new-object -com("Schedule.Service")
$schedule.connect($servername)
$tasks = $schedule.getfolder("\").gettasks(0)
$tasks |select name, lasttaskresult, lastruntime

This seemed to accomplish the goal, but Marco Shaw raised a very interesting point that had me go down the rabbit hole a little bit farther. His point was that it was possible to miss a failed task based on frequency of the task, and the possibility of failed runs from your script that checks the status.

I was reflecting on this point. In my environment I use a SQL agent server to manage all of our scheduled tasks for the company from a central location. We need it anyway for SSIS so it just makes sense to use it for all of our scripts too. It has very robust logging built in. I started thinking of little hacky ways to have your jobs within scheduled taks log their results to their own log files, but imagined how unmanageable this could become. Imagine creating a .bat for every task that would redirect the output to txt files that your script would monitor. A very cludgy way to defeat this problem.

I was wondering why Scheduled Tasks has no logging built in so I started looking to see if there are ways you can either do this or at least have a completion message sent to your app or system logs. Then I learned that there is a built-in log that I never knew existed:

You can view it in scheduled tasks under the advanced menu, but I found that the log location is located within HKLM\Software\Microsoft\SchedulingAgent\LogPath. The default is: %SystemRoot%\Tasks\SchedLgU.Txt

You could write a powershell script that parses this log file very easily. You could then have it output the last “Result:” listed for each command:

get-content schedlgu.txt |foreach {
    if ($_ -match '\"')
    {
        $name = $_
    }
    if ($_ -match "Result:\s([\s\S]*)")
    {
        echo "" $name $matches[1]
    }
}

The question is how to run it remotely. At this point you could use \\servername\c$ administrative share if you always know that the path is going to be the same or you could enable powershell remoting on each server and run it that way.

The only caveat with the above is that will not have your AT command history. Since win7/vista/2k8 now list your at commands within task scheduler I looked at the log there and found that it has even more information than in the above log, but the contents in the gui were not what was in schedlgu.txt. The contents of this log can be seen in event viewer under Microsfot\Windows\TaskScheduler\Operational. So I loaded that into powershell using get-winevent with level 2 to only get the errors. I also added a filter so that it would only return results from the past 24 hours.

$yesterday = (get-date) - (new-timespan -day 1)
$events = get-winevent -FilterHashtable @{logname = "Microsoft-Windows-TaskScheduler/Operational"; level = "2"; StartTime = $yesterday}
$events |foreach {
    $_
}

In the end the get-winevent method would be the best, but it will only work if you are using 2008/Vista/Win7.  If you are using Windows 2k3/XP your best bet is to read the log file directly and ensure that users do not use AT jobs for monitored tasks.  If you must monitor AT jobs as well you should probably use the Schedule.Service Com object, and just run the script frequently enough to pick up all failures if they occur.

Migrating a VM SCSI controller from Bus Logic to LSI Logic

This is an old topic, but it’s something that I get asked all the time from my colleagues who are trying to make heads or tails of the options available for SCSI controllers in virtual machines running on a VMWare ESX server.  For the most part when you create a new vm, and specify the proper guest OS it will provide you with the most suitable controller, but that is not always the case.  When creating a VM from scratch the client will choose the most recent controller that your OS has a driver for on the installation media i.e. Windows 2000 will be installed with a bus logic driver because that driver is on the CD.  The bus logic driver, however, is not the optimal driver for any Windows OS.

The following is the recommended best practice for SCSI controllers:

ESX 3.X – All versions of Windows should use the LSI Logic controller

ESX 4

Windows 2000, XP, 2003 – LSI Logic Parallel

Windows 2008 – LSI Logic SAS

With the latest patch, ESX 4 update 1, you can now use a VMWare Paravirtual controller (PVSCSI) on your boot disk.  If you are using a SAN or iSCSI you will definitely want to explore this option.  The benchmarks being released are showing huge gains.  According to page 6 of the VSphere Performance Enhancements White Paper, “Efficiency gains from PVSCSI can result in additional 2X CPU savings for Fibre Channel (FC), up to 30 percent CPU savings for iSCSI”.  If you are using DAS you will see no benefit.  I currently have a post in the VMWare forum to find out if there are any gains with NFS storage.  I’ll make sure to update this post/site once I have an answer.

In the meantime this post will focus on converting your VMs to the LSILogic controller.  This is not as easy as it should be, but it’s not that difficult once you understand the steps.  The problem is that if you change your primary adapter from one type to another you will blue screen on boot up since the driver is not yet installed.  The below procedure will add a second SCSI adapter of the type you are changing to so that the driver will be installed when you change the primary controller.  Unfortunately, you cannot add the second controller through VCenter so you will need to modify the .vmx file directly.  This can easily be done through vcenter by browsing the datastore to the file and using download and upload to modify the contents.  You can just as easily do this through a shell, but for our purposes I will document it the VCenter way in case you are using ESXi or don’t have credentials to log in to a shell.

1. Log onto the VM with your credentials or the local administrator.  This is done to ensure that you have a profile available on the system.  Later you will lose network connectivity so you need to make sure you can logon first.
2. If this is a Windows 2000 server you will also need to upload the lsi logic driver to the system at this time.  You can get this driver on the lsi website.  The driver you are looking for is LSI20320.
3. Shut down the VM
4. Browse the data store and folder where the vmx file is located for the VM.
5. Download the vmx file and make an extra copy of the file somewhere on your system for backup purposes.
6. Edit the vmx file in word pad or some other text editor besides notepad:
   1. Find the line:
      1. scsi0.present = “true”
   2. Add two lines underneath that one (location doesn’t matter, but it’s easier to find it later if you always use the same place):
      1. scsi1.present = “true”
      2. scsi1.virtualdev = “lsilogic”
   3. Save the .vmx file.
7. Upload the saved file over the existing one for the VM.
8. Power on the VM and log on.
9. If this is Windows 2000 you will be asked to supply the driver.  Otherwise the driver will be installed automatically.
10. Shut down the VM
11. Through Vcenter edit the settings of the VM and highlight your SCSI controller that is currently configured for buslogic.
    1. Click Change Type and specify the appropriate LSI logic controller type for the OS (as discussed above).
12. Power on the VM and log in.
    1. If prompted to change the disks say yes to all prompts.  Sometimes you will not notice this popup, but you will see a message if you look at the summary page of the VM in VCenter.
13. Shut down the VM again.
14. Download the vmx file again and remove the two lines you added in step 6-2.  Save, and upload the vmx file.
15. Power on one final time.

Enable Powershell Remoting While Running VMWare Workstation in a Domain

Ugh…. What a headache.  I attempted to enable Powershell remoting on my workstation this morning in order to test out some deployment scripts I have ideas for.  Unfortunately while doing this I received the below message:

WinRM firewall exception will not work since one of the network connection types on this machine is
set to Public. Change the network connection type to either Domain or Private and try again.
At line:50 char:33
+             Set-WSManQuickConfig <<<<  -force
+ CategoryInfo          : InvalidOperation: (:) [Set-WSManQuickConfig], InvalidOperationException
+ FullyQualifiedErrorId : WsManError,Microsoft.WSMan.Management.SetWSManQuickConfigCommand
WinRM firewall exception will not work since one of the network connection types on this machine is set
to Public. Change the network connection type to either Domain or Private and try again.
At line:50 char:33+             Set-WSManQuickConfig <<<<  -force    + CategoryInfo          : InvalidOperation:
(:) [Set-WSManQuickConfig], InvalidOperationException    + FullyQualifiedErrorId :
WsManError,Microsoft.WSMan.Management.SetWSManQuickConfigCommand

It appears that the VMWare Workstation NICs that get added to your system take on the public network profile.  One would think this would be easy to fix…. load up control panel->network-> and make the switch.  It is that easy unless you are on a domain.  When you are on a domain the option is not available to you.  After some digging I found this article that discusses how to change these settings using a COM object through Powershell, but the script does a check to ensure that the computer is not on the domain before it makes the change.  I decided to go head first and try it out on the non-domain connected NICs, and I found that it worked like a charm.

Here is the script that will change all public profile NICs to become Work Network NICs:

$nlm = [Activator]::CreateInstance([Type]::GetTypeFromCLSID([Guid]"{DCB00C01-570F-4A9B-8D69-199FDBA5723B}"))
$connections = $nlm.getnetworkconnections()
$connections |foreach {
    if ($_.getnetwork().getcategory() -eq 0)
    {
        $_.getnetwork().setcategory(1)
    }
}

Afterwards I was able to enable remoting with no problem.

Powershell – Part 5 – Regular Expressions, oh my!

A lot of people get intimidated by Regular Expressions, and maybe you should.  However, if you plan on doing any type of text manipulation or searching you will need to learn the basics.  Administrator tasks are littered with getting dumps of data into text form and having to extract, reformat, or find a way to make the data useful.  Regular Expressions do just that.

Before I continue I must mention that the be-all-end-all source of information for regular expressions is “Chapter 5 – Pattern Matching” in the Perl Bible, “Programming Perl” (The Camel Book) from O’Reily.  I highly recommend reading this chapter a few times (God knows I must have read it over 50 times).  It contains everything you need to know about regular expressions.  I don’t plan on discussing how to form regular expressions in this tutorial, but I want to show how Powershell implements regular expressions.

Now let’s get dirty.  Powershell implements two different methods of regular expressions.  There is a built-in method using comparison operators and there is the method of using a .Net Regex.

Comparison Operator Implementation

get-help about_Comparison_Operators
get-help about_regular_expressions

Comparison operators are things like -eq (equal), -gt (greater than), -ne (not equal).  The regular expression operators are -match, -notmatch, and -replace.  There are also two additional operators not listed in the above helps that do the exact same thing as their counterparts, but they are case sensitive: -cmatch and -creplace.

'test' -match '^t'

The above returns true because ‘test’ contains a ‘t’ as its first character.  If we apply the -match above to the code from Part 2 of this series of tutorials that reads our dictionary file we can create a script that will list all words that start with the letter ‘t’:

Get-Content dictionary.txt | foreach {
	if ($_ -match '^t') {
		Write-Output $_
	}
}

In addition to using matches to determine if something is true you can also use them to capture groups within your matches.

'Powertoe is the best' -match '(\w+) is the (\w+)'
foreach ($match in $matches) {write-output $match}

The $matches special variable is used to display the contents of the match.  $match[0] will always show the entire regex match (as it does in a .net match capture), while $match[1] will show the contents captured by your first parenthesis pair, $match[2] will show the contents of the second pair, etc.

-replace is very similar to -match, but there is one very important thing to note.  -replace does not interpret a regular expression the way a normal regular expression works.  It will always do a greedy match (perl equivalent: s///g):

'test' -replace 't', 'w'

The output from the above operator is ‘wesw’ as opposed to ‘west’ like you would expect with a normal regular expression replace.  In order to have -replace work this way is to use the .Net implementation of regular expressions.

.Net Implementation

System.Text.RegularExpressions.Regex is the name of the class that implements regular expressions in .Net.  Here is how you would normally create an object of the Regex type:

$regex = new-object System.Text.RegularExpressions.Regex('^test$',[System.Text.RegularExpressions.RegexOptions]::MultiLine)

All methods and properties available to this class are available to powershell.  However Powershell also gives a nice shortcut to creating regex objects.  The above command can also be written as:

$regex = [regex] '(?m)^test$'

As mentioned we can now use this implementation of .Net’s Regex to do a non-greedy replace:

$regex = [regex]'t'
$regex.Replace("test","w",1)

That’s the down and dirty of it.  I apologize if this topic is a little bit more advanced than my others have been. I know it expects you to have an understanding of regular expressions.  I highly suggest reading both the pattern matching chapter of the Camel book as well as the supporting documentation about the Regex class to ensure that you learn all of the power available to you within regular expressions.  After that it’s just a matter of setting yourself to tasks that do string manipulation.  After the next couple of tutorials we should be able to start tackling real puzzles where you can challenge yourself to implement your own regular expressions so that you can start learning how to form them.

Powershell – Part 4 – Arrays and For Loops

Arrays

For those that have never worked with arrays here’s a great way to understand them:  If a variable is a piece of paper then the stack of papers is an array.  It’s a list of variables or objects, and every programming/scripting language has ways to store these variables or objects linearly so you can access them later via a number of different methods.

So let’s look at how we can create an array of string objects in powershell:

$array = @("test1", "test2", "test3")
Write-Output $array

You can also add an element to the end of an array:

$array = @("test1", "test2", "test3")
$array += "test4"
Write-Output $array

You can also add arrays together:

$array = @("test1", "test2", "test3")
$array2 = @("test4", "test5")
$array = $array + $array2
Write-Output $array

You can access an element of an array if you know the index number of the element you want.  Arrays are indexed by integers starting with 0.  This can be seen with the following code:

$array = @("test1", "test2", "test3")
Write-Output ("First array value: " + $array[0])
Write-Output ("Second array value: " + $array[1])
Write-Output ("Third array value: " + $array[2])

You can use that element as if it’s a regular variable at this point.  Since our array is an array of strings we can use string functions as if this was a regular variable set to that string value when we call the element of this array.  e.g.:

$array = @("test1", "test2", "test3")
Write-Output $array[1].ToUpper()

For Loops

Arrays can also be accessed linearly through the help of for loops.  A for loop generally has 3 bits of information in their declaration:

1. Initialization
2. Condition to continue the loop
3. A repeating occurrence – This is generally used to bring your loop closer to not meeting the condition for you loop so that the loop will eventually end.

This is best seen by looking at a simple loop.  The following will initialize by setting $i to 1.  It will then test that $i is less than 6 and increase $i by one on each pass of the loop:

for ($i=1;$i -lt 6; $i++) {
	Write-Output ("This is line number " + $i)
}

In the above example we use the -lt comparison operator to test that the value of $i is less than 6.  There are many other comparison operators available like -gt (greater than), -le (less than or equal to), -ge (greater than or equal to), -eq (equal), and -ne (not equal).  To learn about the comparison operators available to you can use the following command:

help about_comparison_operators

For Loops with Arrays

It’s easy to see how you can apply a loop to an array to iterate through each element of the array in order if only there was a way to test for how many elements are in the array.  We can do this by using a member of the array object called Length.  The thing to note here is that you want to initialize your index variable with 0 since that is the first element in your array, but you want to test that your index is less than the array length since the last element of the array will be one less than the number of elements in the array.   Here is the process in practice:

$array = @("test1", "test2", "test3")
for ($i=0; $i -lt $array.length; $i++) {
	Write-Output $array[$i]
}

An even faster way to do a for loop is by using a special foreach loop.  The foreach loop will set a variable to each item in a list with a much simpler construct:

$array = @("test1", "test2", "test3")
foreach ($element in $array) {
	Write-Output $element
}

The final thing to note with the foreach loop is that it can be accessed via the pipeline.  When this happens you are given a special variable to represent the element in your code block, $_:

$array = ("test1", "test2", "test3")
$array |foreach {
	Write-Output $_
}

We have already seen the foreach loop in action in the pipeline during Part 2 of this series when we looked at how to read files.  Here’s a snippit of that code so you can see how a cmdlet can pass a list through a foreach loop:

Get-Content dictionary.txt | foreach {Write-Output $_.toupper()}

Lists and arrays are very important to programming and scripting.  There is rarely a script that does not use an array or a list in some form or another.  Armed with this knowledge you can start looping away.  We’ll be touching on hashes or regular expressions in the next tutorial.  Until next time…

Powershell – Part 3 – Variables

Time to learn some more basics.  In our last tutorial we looked at some of the special variables given to us within Powershell, but now it’s time to discuss how user variables work.  Let’s start off simply.

$var1 = 'blahblahblah'
echo $var1

By using the $ before the word var1 we’ve created a variable.  Using the equal sign we are able to set this variable to the string ‘blahblahblahblah’.  Powershell is a very dynamic language in that it allows you to create and assign variables without casting them to a data type.  This can be a double-edged sword so it’s a good idea to understand how this can cause problems.  Take the following for example:

$var1 = 5
$var2 = "blah"
echo ($var1 + $var2)

Here we are trying to use a number (it’s a number because there are no quotes) and a string (it’s a string because it has quotes) with the + operator.  If they were both strings this would concatenate them.  If they were both Int32 they would add them together.  Because they are different we get an error that “blah” cannot be converted to Int32.  Powershell is extremely flexible with its data types.  Since the .net implementation of the Int32 class and the string class have Converto() functions the script will attempt to automatically convert the variable to the type expected by the operation.  In the above example the first variable is an Int32 so it assumes you are trying to add another number to this one.  If we reverse this and start with the string and use the + sign the compiler sees the first variable as a string and expects to concatenate.  Since an Int32 can easily be converted to a string reversing the operation works without failure:

$var1 = 5
$var2 = "blah"
echo ($var2 + $var1)

You can also call a conversion through the .Net methods available to the data type.  For example returning to our original problem code you can successfully do the following to ensure that the Int32 is interpreted as a string:

$var1 = 5
$var2 = "blah"
echo ($var1.tostring() + $var2)

Another way to get around this problem is to explicitly cast the variable to what you want it to be.  This is done with [] brackets before the variable:

[string]$var1 = 5
$var2 = "blah"
echo ($var1 + $var2)

Before we end this portion of the tutorial it’s important to discuss one more item, interpolation.  Powershell uses the same methods for interpolation that Perl uses.  If it’s in a double quote it will be interpolated.  If it’s in a single quote it is a string literal.  In both cases it is a string, but it’s how the string handles variables inside its quotes that makes them different.  The following illustrates this:

$var1 = 'blahblahblah'
echo "This is what my variable is set to: $var1"
echo 'This will not work: $var1'

We have now learned the basics of what Perl calls a scalar variable.  In the next tutorial we will discuss arrays.