Tuesday, June 08, 2010

Microsoft TechNet benefits

Microsoft TechNet subscription is a very handy tool for Windows based IT admins, or people breaking into the IT world. Let me first talk about my entry to the IT world and how TechNet would have helped me.

When I first got started with IT I was already involved with multimedia applications so I knew heavily about the problems of trial software ending before you really got a chance to play with it. The first book I bought was Windows NT Workstation for Dummies, came without Windows NT so I had to get my first real hands on by using an old Windows NT workstation at work. When I enrolled in the Microsoft classes at college the books came with Windows NT Server and Workstation demo keys, they were good for about 60 days.

Back then, it was difficult to find any other software besides the OS and my learning skills were limited to what software was included in the learning books, typically it was only the OS. Not to mention that downloading software on a 56K modem was not the fastest solution. It was after landing my first IT job that I started to work with applications like Microsoft Exchange and other difficult to play with applications.

Now I know people will say "those are available every where" but take for instance a few facts.

1. I didn't know anyone in the IT world.
2. Didn't want to bootleg a CD.
3. Limited to slow download speeds.

It's tough to get the applications to play with.

Now with faster Internet speeds and networking with friends there's many resources but I feel that going with a TechNet account still the best choice. Here's why.

1. Licenses do not expire! - The best thing I like about TechNet is you are given a license key that does not expire after a certain amount of time, now some of the beta software does end but the rest is just like buying a retail license.

2. Easy to find and download any Microsoft IT based application or server - This is a great reason to down load Exchange 2010 and get it working at home over the weekend. Nothing tells you more than getting it work on your own in your personal home lab.

3. Two free incident calls - Each Microsoft incident call is $265, so that is $530 dollars alone in support calls for a price of $300 (regular price, often cheaper with coupons) subscription price.

4. The extras - Microsoft eLearning, On-line chat, etc. I personally do not like the eLearning and just prefer watching simple videos but it's a nice benefit. I have not tested the on-line chat yet but honestly sounds good and I do have some low level nagging questions I haven't been able to debug yet.

The best part of having the applications available is the testing and since it's so easy to access makes you want to try them all out. I can say that even from my previous job I had a hard time accessing some applications even for work testing and learning. Either the media could not be found or the license manager did not have the keys. Here I am the manager and able to install and test all of the Microsoft server and business applications.

I highly recommend this for people just getting into IT. Where in the Linux world many of the applications are free and open source the Windows world is less open source and based upon Microsoft applications. If you have a robust workstation at home, with a good amount of physical member and VMware Workstation you will be set to do almost every possible IT server environment except SAN simulations.

Where to find it?


Friday, June 04, 2010

Mysterious Windows lockout

"Hello, my account keeps getting locked out"

Anyone who has admin for a Windows environment knows about accounts getting locked out. Typically when a user changes his or her password there are certain applications or processes that need to be manually updated with the new password. If you don't update these passwords, then each time they run, they will authentic your old password, which in turn will fail and register on the Active Directory as 1 failed password attempt.

If you think of a few more services and some mapped drives all trying to re-authentic at the same time, then you can easily be locked out if your Active Directory password policy is set to 5 failed passwords before account lockout. This problem while simple is often difficult to really track down where the problems are coming from.

I'm going to list the tools I use and a recent account lock out issue that I resolved. Please note I removed all of the details for security reasons.

What to look for

Before you make any changes, always gather the information of the user affected and what type of job role this user does. A data entry user will be much easier to resolve than someone who is an admin logging into many remote systems. Here's a listing of questions I usually ask.

1. When was the last time you changed your password?
2. Which account are you using?
3. Where do you login from, only this workstation or multiple computers?
4. Do you work from home on your personal computer?
5. Any special application or service you are using?

From here I can usually get an idea of the level for the user, let's say the user answered the following questions.

1. Two weeks ago, problem started today.
2. Jsmith
3. Only login from this laptop.
4. No, but I work from home on this laptop using VPN.
5. Google desktop search but nothing else.

Now let's check the user's desktop for any lock out messages. You can somewhat skip this part because the lock outs are from the Domain Controller but I like to just make sure I do this. As I checked the system, there is no indication of any security errors or lockouts. Strange, usually there's at least something to go on.

Ok, let's move to the Domain Controller. Checking the security logs I filter by the user name and again there is nothing listed for failed logins. I'm starting to wonder where is the lockouts coming from. The domain I am working on actually logs failed and successful from a group policy so with the amount of logs it could have been over written.

Time to install some tools

Before this I already installed the Windows Account Lockout and Management Toolset from Microsoft. I used the tools from our local Domain Controller and scanned the problem account with the LockOutStatus tool.

LockOutStatus shows a listing of the current status of the target account with details as where the lock outs are taking place and also what time they occur. Here's a screen shot of the tool and why each function is useful.


DC name - Shows which server the account is accessing
Site - Where the servers are located in the AD forest
User state - Is the account locked or unlocked
Bad pwd count - How many bad passwords were attempted
Last bad pwd - This is how long ago the last bad password was attempted
Pwd last set - When the password was last set
Lockout time - The time the accounted was locked out

What to look for?

So what I'm going to look for is this. First, I'm going to run the tool against the problem account, and the tool will automatically find all of the domain controllers. Here's what I look for in each value.

DC name - This is just the names of the DC, pair it up with the Site names if you are not sure where the servers are physically located.

Site - If you already know the DC name this is a bit non-important, unless you are not familiar with the DC's or do not know where a DC physical sits.

User state - This is handy to show the "wave" of lockouts getting updated as the account is locked out from it's primary DC. I alway focus on the DC that is first locked out, then if I can't find anything there move to the next closest DC.

Bad pwd count - This one is a bit tricky. In most cases you would see the number corresponding to your domain policy about bad password attempts. For example if your password count is 5 typically you will see "5" under this field. If you see 5 that would indicate that indeed there is an rogue application or process using an old password. Sometimes this does not mean that there were 5 attempts, but could mean that the DC saw 5 attempts. If you have some delay on your domain, 1 password attempt could register as 2 because of the delay between DC's.

Last bad pwd - This shows just when the account was locked out, for example if you know a user is working 8am to 5pm and the account lockout time shows 3am, then is must be a process running. If the user takes his or her laptop home and powers down at night, there might be a process running on a remote server.

Pwd last set - Of all of the values, I hold this second most important. Many times end users will change their passwords and that's when the problems will start. So I like to see the distance between when the password was set and when the password started to locking out. In most cases it's going to be very close, usually within a week from the password was last set. In some cases, it could be much longer because a service or application is not used every day.

Lockout time - Lockout time is when the DC reports the amount of failed password attempts to match the domain policy. It's a bit confusing and I'm not 100% but if an account is running on a service, and that service runs every 1 hour, it will register 1 lockout. But if the domain policy is set for 5 lockouts in one hour, this will not cause the account to be locked out.

So in this example, let's say we found the following values. We caught it just as the user's account was locked out so only one DC is showing locked.

DC name - DCSF
Site - San Francisco
User state - Locked
Bad pwd count - 5
Last bad pwd - 9:30AM
Pwd last set - 5/20/10 (two weeks ago)
Lockout time - 9:32AM

Ok, from this data I know the following. First, the account is contacting the DC in San Francisco, it's locking out from 5 attempts (5 is the setting in GPO), and it's locking out at 9:30AM. The user starts work at 9:15AM but he says the account is locked out before he can login to the workstation. This might be user error but let's dig further!

Checking the DC's security logs

Now we have a starting point, DCSF. From DCSF I filter out the logs and find this.

Lockout event

Here's the important information (removed real info).

Source: Security
Category: Account Management
Type: Success A
Event ID: 644
Computer: DCSF
User Account Locked Out
Target Account Name: jsmith
Target Account ID: BayareaCo\jsmith
Caller Machine Name: sfauth
Caller User Name: DCSF$
Caller Domain: BayareaCo

Now this is important, the value for "Caller Machine Name" is pointing to "sfauth". I double checked and the user does not access any system besides his own. I checked the event logs of "sfauth" and found this.

Source: Security
Category: Logon/Logoff
Event ID: 539
Computer: sfauth
Reason: Account locked out
User Name: jsmith
Domain: BayareaCo
Logon Type: 3
Logon Process: CISCO
Workstation Name: sfauth

We found the server, now what application is causing this

We now are focusing on the server sfauth since this last event log points to an process called "CISCO". Let's first see what is running on the sfauth. I'm going to use a handy tool from Sysinternals called TCPView. This shows me what processes and who they are talking to, as seen below.

TCP View

So we found that the process is running but let's look at the process details. From TCPView, it looks like a process called "CSAuth.exe" is running on the server talking to DCSF. I'm going to assume that since it's talking to DCSF, it must be doing some kinda of authentication for remote access. I used another Sysinternals tool called Process Explorer.


We found the application, now what?

Ok, so we found that it's an Cisco application called Cisco Secure ACS, but what do we do now? I snoped around and found that the Cisco application had a few GUI tools and a nice logging report tool. I opened the repoting tool and viewed the failed logins.

Below is one of the failed logins reports for user "jsmith".

Cisco Login

From here we have something to follow, a MAC address and an IP address. While the IP address can change, the MAC address is hard coded and unique for each network device. I first started by checking DHCP for where this IP address was located, it was not on the range of IP addresses. I used my favorite tool NMAP to do a port scan of the IP address listed in the failed event.

Below is the result, but there's something wrong with this picture, can you find it?


The MAC address listed here shows 00:13:19 while the logs shows 00:26:08, that's strange! Well if this device is a router then something else is accessing the network beyond what I could see. I did a quick search on a MAC address search tool and found this result.


Apple? We don't have any Apple devices here at work. Or do we? What is made by Apple and often times accessing the network AND would be accessing the network BEFORE the user would login to his computer?


Yes, the iPhone. The user has an iPhone which was accessing the WiFi network. In order to access the WiFi network you need to login with your AD account and this is held on the iPhone. Here's is the login screen for the WiFi access.


Solution and closure

Once I removed this from the user's phone access from the WiFi, the lockouts stopped. When I tried to recreate this problem I couldn't seem to get my iPhone to lockout but the end user did say he had a sync software running on his iPhone with his desktop. I'm glad I found the problem and resolved it. This by far was the most difficult account lockout I have battled.

I hope this will be helpful for you!

Wednesday, June 02, 2010

What version of Microsoft applications am I running?

I found this link to be very helpful, especially if you're throw in to support an server or app that you don't know much about.


Tuesday, June 01, 2010

Mounting CD-ROM in Ubuntu

I'm somewhat forgetful when I don't use a command every day. While trying to remember the exact command here I did a Google search and each answer I found was missing some step. So I am going to write out the exact steps here for anyone getting lost, and also for my self in the future. :)

This is from Ubuntu so your distribution might be slightly different.

1) sudo mkdir /mnt/cdrom

2) sudo mount /dev/cdrom /mnt/cdrom

3) ls /mnt/cdrom

4) Done!

It's really simple but from searching there were lots of very complex answers, and I prefer the simple as possible method.