Sunday, December 04, 2011

Install HTOP on FreeBSD

Working with command line I really miss some of the nice graphics you find with Windows based tools. The current Windows Resource Monitor included in Windows 7 and Windows Server 2008 is really my favorite. It's great to use for monitoring as it's a real time graph of the system performance. I know that Linux/UNIX tools like TOP can do the same but it's just visually easier to view a graph than numbers.

I recently started to use HTOP which is a more graphical version of TOP but is not as popular. The downside is that while it's more handy, you may need to have it installed manually, it's typically not included by default. Below is a screenshot of HTOP running.


Installing HTOP in CentOS is extremely simple.

#yum install htop

Done! For Debian based systems, use the following command.

$sudo apt-get install htop

Simple and easy, now run htop. But what if you want to install htop for FreeBSD? That's going to take a little bit more work.

Good news is that HTOP is available from the FreeBSD ports collection. This will make the process much easier that installing from source.

http://www.freebsd.org/cgi/cvsweb.cgi/ports/sysutils/htop/

For the actual installing, I following a few blogs and forum postings. One from The Technical Crib pointed me how to install the Linux emulators for FreeBSD. But after following the steps I received another error during the install process, which was covered in this FreeBSD forum posting. Finally, as suggested in the previous link, the fix was to install the source, found how to do that in this Cyberciti blog posting.

Below I will write out the full steps that worked for me, didn't discover this but it's complied between all the articles I found.

Installing the requirements 

1) Su or login as root

2) Run the command to enable Linux binarary compatibility for FreeBSD.

#kldload linux

3) Add to the startup

Edit /etc/rc.conf

Add following line

linux_enable="YES"

4) Install Linux base port f10.

#cd /usr/ports/emulators/linux_base-f10


#make install clean

5) Add linproc to fstab

Edit /etc/fstab

Add following line

linproc /compat/linux/proc   linprocfs   rw   0  0

6) Install Htop from the port

#cd /usr/ports/sysutils/htop

#make install clean

7) Htop should run!

$htop

Possible extra step

*note - htop should complete the install with no problems. If you see an error as the following, you may need to install the source files to your computer.

Error result ***


===>  Configuring for lsof-4.84A,5
Creating ./lockf_owner.h from /usr/src/sys/kern/kern_lockf.c
FATAL ERROR: can't read /usr/src/sys/kern/kern_lockf.c
FATAL ERROR: ./lockf_owner.h creation failed (see 00FAQ)
===>  Script "Configure" failed unexpectedly.
Please report the problem to ler@lerctr.org [maintainer] and attach the
"/usr/ports/sysutils/lsof/work/lsof_4.84A.freebsd/config.log" including the
output of the failure of your make command. Also, it might be a good idea to
provide an overview of all packages installed on your system (e.g. an `ls
/var/db/pkg`).
*** Error code 1


Stop in /usr/ports/sysutils/lsof.
*** Error code 1


Stop in /usr/ports/sysutils/htop.
*** Error code 1


Stop in /usr/ports/sysutils/htop.


Error result ***

I am using FreeBSD version 8.2 stable and saw this error, so I followed one of the links posted and installed the kernel source code. We're going to run through some settings with FreeBSD's sysinstaller, it's kinda of confusing but take your time and carefully tab around.

1) Su as root

2) Run the sysinstaller

#sysinstall

3) Select "Options", then tab to the "Select" button, press enter



4) Under the Options Editor, use the arrow keys to scroll down to "Release Name" value, we are going to change this by removing the "-p3" after "RELEASE". Press the space bar to open an edit window.


5) In the edit window, remove the "-P3" value. This is done because when we download the source files, it does not know where to find this source. Tab to "Ok" when done. Then press "q" to quit to the main screen.


6) Back at the main sysinstaller screen we are going to select "Configure", then tab to "Select" and press enter.


7) Use the arrow keys to select "Distributions" then then use the space bar to select to continue.


8) On the next page, scroll down again using the arrow keys and select "SRC - Sources for everything". Press the space bar to select.


9) Under the sub-components, you will need to select two source trees, Base and Sys. Scroll down again using the arrow keys and select the following with the space bar.

"Base - top-level files in /usr /src"
"Sys - /usr/src/sys (FreeBSD kernel)"

Tab to the "Ok" button again.


10) After this you will be prompted where to install the files from (CD/DVD/FTP, etc.). Since this is a smaller download, I would suggest FTP, and first starting with the main FTP site. Now if the FTP fails with the following error message

Warning: Can't find the '8.2-RELEASE-P3' distribution on this FTP server

Go back to step 5 and make sure you edited the name. 

Htop should finally run as normal! Yes, this was a pain but considering it's running a Linux based tool, wasn't that bad!

Rob

Wednesday, November 30, 2011

Alias, making quick custom commands

Recently I placed my home computer on the Internet for a remote access but also to run a personal web page. The problem I had is while I enabled ssh access using only private keys, I was still worried that I might leave a loop hole open on the system.

So here I was constantly checking the logs for any alerts to SSHD service. The command I would run is something simple, "tail /var/log/messages | grep sshd".

The results would typically something as simple as a few lines but it would be important to keep up on. Here's an example.



$ tail /var/log/messages | grep sshd
Nov 25 03:13:59 server1 sshd[927]: error: PAM: authentication error for user from 192.168.1.100
Nov 25 03:14:06 server1 sshd[927]: error: PAM: authentication error for user from 192.168.1.100
Nov 25 05:50:25 server1 sshd[1299]: error: PAM: authentication error for illegal user root from 192.168.1.100


Now this is kinda of a long command to type out each time, I was wondering if I could just type something shorter?

Here's where the alias command comes into play. From your terminal, you can type "alias" and find out what are your current alias commands. 

$ alias
g='egrep -i'
h='fc -l'
j=jobs
l='ls -l'
ll='ls -laFo'
m=more

We're now going to add our search command as listed above, but make an alias to "searchsshd". 

$ alias searchsshd='tail /var/log/messages | grep sshd'

Now, when we type "searchsshd" we get the following. 

$ tail /var/log/messages | grep sshd
Nov 25 03:13:59 server1 sshd[927]: error: PAM: authentication error for user from 192.168.1.100
Nov 25 03:14:06 server1 sshd[927]: error: PAM: authentication error for user from 192.168.1.100
Nov 25 05:50:25 server1 sshd[1299]: error: PAM: authentication error for illegal user root from 192.168.1.100

Pretty cool! 

Rob






Hard links vs Symbolic links

Working with Linux and UNIX offers some handy extras not typically found on Windows based systems. One great feature is hard and soft links, which is somewhat like a shortcut that you can create but looks just like a normal file.

This works great when you need to share a file but do not want to give permissions to the file directly. So how does this work? Every file on a Linux or UNIX based system is mapped to an inode. Inodes are the identification of every single file and object on the system. This includes users, files, rights, etc.

To see the inode values, you can run the command "ls -i".


rob@robmint ~/sandbox $ ls -li
total 4
2098382 -rw-r--r-- 1 rob rob 5 2011-11-30 17:29 testfile

The file "testfile" has a inode value of "2098382".

Now there are two different links we can create. One is a hard link, the other is a soft link. Here's the difference.

Hard link - These are files associated to the same inode. They are duplicate files and are independent from the original file. Also as you can see below, look the same but there is a slight hint.

Let's first create the hard link using the "ln" command.


rob@robmint ~/sandbox $ ln testfile testfilehardlink

Now let's see what we created. 



rob@robmint ~/sandbox $ ls -li
total 8
2098382 -rw-r--r-- 2 rob rob 5 2011-11-30 17:29 testfile
2098382 -rw-r--r-- 2 rob rob 5 2011-11-30 17:29 testfilehardlink

Again, we see the inode information is the same, but also note the value of "2". Which tells us that the original file "testfile" is linked two times, once to "testfile" and another to "testfilehardlink". 


So we mentioned that hard links are duplicates of the original file, but what happens when we delete the original file? Let's find out. We will delete the file "testfile" and leave behind "testfilehardlink".


rob@robmint ~/sandbox $ rm testfile
rob@robmint ~/sandbox $ ls -li
total 8
2098382 -rw-r--r-- 1 rob rob  5 2011-11-30 17:29 testfilehardlink

Now let's see if we can access "testfilehardlink". 

rob@robmint ~/sandbox $ cat testfilehardlink
1234

It works! Now, even after deleting the original, you can still access the data. But this looses the benefit of using link since now the file is no longer being updated from the original file. 

So now we know hard links, what about soft links? Let's create the soft links using the "ln -s" command. 

rob@robmint / $ ln -s testfile2 testfilesoftlink

Now let's look at the soft link we created. Notice how there is a visual reminder that it's a soft link. 

rob@robmint ~/sandbox $ ls -li
total 8
2097736 -rw-r--r-- 1 rob rob 10 2011-11-30 17:41 testfile2
2098382 -rw-r--r-- 1 rob rob  5 2011-11-30 17:29 testfilehardlink
2098638 lrwxrwxrwx 1 rob rob  9 2011-11-30 17:48 testfilesoftlink -> testfile2

Here we created the the soft link, but let's see how this works if we delete the original file. 

rob@robmint ~/sandbox $ rm testfile2
rob@robmint ~/sandbox $ ls -li
total 8
2098383 -rw-r--r-- 2 rob rob 5 2011-11-30 17:50 testfile
2098383 -rw-r--r-- 2 rob rob 5 2011-11-30 17:50 testfilehardlink
2098638 lrwxrwxrwx 1 rob rob 9 2011-11-30 17:48 testfilesoftlink -> testfile2


Now the original file is deleted, let's try to access the file "testfilesoftlink".


rob@robmint ~/sandbox $ cat testfilesoftlink
cat: testfilesoftlink: No such file or directory

As you can see, once the original file is gone, so it the soft link file. This is because the soft link is a link to the inode, not a duplicate file. 

Now, why would you want to use soft links over hard links? Soft links are a path back to the file, not to the inode, which means that you can access links across partitions. It's pretty cool feature and when working with longer path names this could be extremely handy. 

Rob








Friday, November 25, 2011

SSH access with keys and no password

One of the most handy things about SSH access is setting up the secure keys, allowing to bypass the password but keep a very secure access to the host. This allows a fast connection to the host, but also since you're using a security key, no one else will have access.

For a while I was having issues setting this up, I knew that you needed to create the keys but not sure how to setup ssh access further. So I setup a home server, using FreeBSD 8.2 at home and running a port forwarding from my home router. I wanted to keep this secure as possible, but also allow access from any location, so I could access my home system remotely. 

So here's the steps I used and hoping this will be helpful. 

First, if you're on a Windows system there might be some issues creating the keys. You can make keys from Putty but there is some differences, mostly you will need to rename the files and make some additional editing. In these examples I created the keys on a Ubuntu based system and accessing a FreeBSD host. Please note "user" is your account name and "hostname" is the machine name. 

Before starting, make sure you have your remote host setup if needed. Here I'm working on a home machine so I need to first edit the SSH service file located at /etc/ssh/sshd_config. I recommend using your favorite text editor, and will be listing only the important options below. 

First, make sure root SSH login is disabled, and only allowing your account to be listed. 


AllowUsers user


We are also going to disable password authentication. 

PasswordAuthentication no

Also allow RSA keys.

RSAAuthentication yes
PubkeyAuthentication yes
AuthorizedKeysFile      .ssh/authorized_keys

Save the file and restart sshd service. Now consider if you are accessing a company server the details will be different and these setup steps for the host might be already done. 

From your personal computer. 

1) Login to a host and access the terminal or console.
2) From you're home directory, run the following command.

user@hostname ~ $ ssh-keygen -t rsa

3) As seen below, you will be asked for the following details. You can take the defaults and press "enter" all the way through. If you would like added security, enter a passphrase when prompted. 

user@hostname ~ $ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/user/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/user/.ssh/id_rsa.
Your public key has been saved in /home/user/.ssh/id_rsa.pub.
The key fingerprint is:
[random letters and numbers] user@hostname
The key's randomart image is:
+--[ RSA 2048]----+
[random image]

4) Now check your directory and you should see the following files. The specific path is listed above as "/home/user/.ssh" directory. We are using the "a" option to view the .ssh directory since it is hidden. 

user@hostname ~ $ ls -la
drwx------  2 user  user       4096 2011-11-25 15:18 .ssh

5) So we have the directory there, now let's check for the files. We are going to change directory to .ssh and look for the files.

user@hostname ~/.ssh $ cd .ssh

6) We are looking for what files are in the directory. 

user@hostname ~/.ssh $ ls -l
total 20
drwx------  2 user user 4096 2011-11-25 15:18 .
drwxr-xr-x 57 user user 4096 2011-11-25 14:55 ..
-rw-------  1 user user 1675 2011-11-25 15:18 id_rsa
-rw-r--r--  1 user user  393 2011-11-25 15:18 id_rsa.pub
-rw-r--r--  1 user user  884 2011-11-25 15:01 known_hosts

7) We have two hosts that are important. id_rsa and id_rsa.pub, these are the two files which we created and will need to access remote hosts. 

id_rsa - This is your private key, you keep this on the computer that you are using to access other computers. This is a secure file not to be shared on remote systems but only on hosts you are using for clients. 

id_rsa.pub - This is the public key, you place this file on remote systems that you want to access remotely. 

8) Now we have the two files needed, we're going to copy the id_rsa.pub file to our remote system. This command will copy the files from your local id_rsa.pub to the remote host. 

user@hostname ~/.ssh $ cat id_rsa.pub | ssh user@192.168.1.1 'cat >> .ssh/authorized_keys'
Password:

There is a possibility this might fail. Make sure that under your home directory on the remote system you have .ssh directory and the file authorized_keys already created. If they are not there, then create them as needed. Below, run the commands from your home directory. 

user@hostname ~ $ mkdir .ssh

Now we are going to lock down the directory

user@hostname ~ $ chmod 700 .ssh

Now go inside the .ssh directory

user@hostname ~ $ cd .ssh

We are going to create a blank file to store the keys

user@hostname ~ $ touch authorized_keys

Again, we are going to lock this file down as well 

user@hostname ~ $ chmod 600 authorized_keys

Now try the command listed in step 8 again. This should work. 

9) Now try to ssh into the server again, you should not be prompted for the password and go straight to the terminal session. 

Hope this saves you time and keeps your system secure. 

Rob

Sunday, September 04, 2011

Windows Freeware applications list

The word free and Windows don't often go together. It's very common to find free open source applications in the Linux world but in Windows it's a bit harder to find, but they are there. 


Recently I found a few nice listing of free applications that are free or offer free trial versions. Really handy listing and some of the applications are just what you need for something small. 


So my personal list of free or open source applications for Windows is here. 


Filezilla - FTP client and server


Before I found Filezilla I was stuck using either the command line for FTP upload, Dreamweaver, or another paid FTP program. Granted FTP isn't something that requires a great deal of knowledge, but it just made things easier to use a mouse. Filezilla is basically everything you could want in an FTP client, I've been using it for years and never had any issues. 


AVG Free Client - Antivirus 


I have never compared anti-virus vendors but the interface of AVG is very simple and easy to use. AVG offers frequent updates and of course they ask that you upgrade to the paid version, which offers more options and protection. Personally it's a tie between AVG and other vendors that offer a free version but AVG seems cleaner.


Pidgin - IM and IRC client


There's many instant messenger clients and IRC clients out there but the interface of Pidgin is decently easy to use. The only issue is using some of the extra features that are included with the vendor's instant messenger client might not work 100% with Pidgin. For example Yahoo! Messenger conference does not appear to work correctly with Pidgin, but that's a minor issue. 


VLC - Media player


VLC is a handy media player that I often use over Windows Media Player. The best feature is this is a very universal player, can play pretty much any file you throw at it. 


FooBar2000 - Music player


My favorite music players have been Winamp and iTunes, but recently I have been using FooBar as it's lightweight and simple to look for files. There is many plugins available for FooBar and if you're looking for the simple music player interface this is highly recommended. 


CutePDF - Print to PDF


I have a love hate issue with Adobe's PDF format. I like how it's very simple to hold books but creating PDF's was an issue, if you didn't have the software there was little you could do. With CutePDF, you can print to PDF which is very handy and great for "recording" web pages or logging documents. I heavily use this for capturing e-mails or invoices that are in HTML format for archival.


Imgburn - CD DVD burner


Imgburn is very handy for create CD or DVD's plus creating ISO of physical media. The application is lightweight, offers a much simpler method than working with Nero. My only request is disabling the "successful" sound as it's annoying. 


TrueCrypt - Encryption 


TrueCrypt offers drive and directory encryption, plus ability to encrypt the hard drive upon bootup. Where this comes in handy is when you need to secure a directory from others, or when you need to have a laptop secured even if you loose it. It's also great to use for your USB memory sticks, and I highly recommend this be used since it locks down the file very securely. 


Notepad ++ - Text editor


Using Microsoft's Notepad is handy but doesn't offer correct formatting and other tools as does Notepad ++. I really like this application and even after buying a very expensive paid version of another text editor, still use Notepad ++.


Putty - SSH client


I honestly do not know of another SSH client for Windows that is free and highly used as is Putty. I just recently switched to using SecureCRT, a paid SSH client but only after I needed a feature not included with Putty. But I really liked Putty and used it for many years. 


Rich Copy - File copy utility


I've been using Microsoft's Robocopy and the add on TeraCopy but recently I found Rich Copy from Microsoft. I really like the interface of Rich Copy and the adjustments for the job which is handy for faster computers. It's also good for sync a directory or just auditing. 


For now these are my favorites and will add some more shortly. 


Rob

Searching for disk space on Linux UNIX

One of the most confusing issues I had moving to Linux was searching for large files. It's somewhat confusing on Windows but there are many graphical applications available to help with the process. On command line, it's a slightly different process but still similar steps taken. Here's an example of searching for the largest files on a host. 


First, run the command "df -h" 


-bash-4.2$ df -h
Filesystem              Size    Used   Avail Capacity  Mounted on
/dev/sda1              3.9G    927M    2.7G    25%    /
/dev/sda2               10G     10G      0G    99%    /home
-bash-4.2$ 


Now you know the largest files are in the /home directory. Run the command "cd /home" In the example below, there's only one user, "rob" so I will go straight to the path.



-bash-4.2$ cd home/rob
-bash-4.2$ sudo du -sh *
9.9G    logs
 17M    dev
4.0K    documents
 12K    tools
-bash-4.2$

We find that in the path /home/rob/logs there is 9.9G of files. One method is to move the files to another directory or you can move the files, compress them then move them back. Just a note, certain files that are already compressed (mp3, jpg, zip) will not compress that well. Others such as text and log files compress extremely well. 


For the same path /home/rob/logs we find there are many log files taking up the space. 




-bash-4.2$ ls -lh
total 5
-rw-r--r--  1 robertf  users   2.5G Aug 31 23:00 log_083111
-rw-r--r--  1 robertf  users   2.5G Sep  1 23:00 log_090111
-rw-r--r--  1 robertf  users   2.5G Sep  2 23:00 log_090211
-rw-r--r--  1 robertf  users   2.5G Sep  3 23:00 log_090311
-rw-r--r--  1 robertf  users     5M Sep  4 07:46 log_090411
-bash-4.2$ 

We do not have space on /home so let's copy to /tmp to run the archive command


First create the folder on your /tmp directory by running the command "mkdir /tmp/rob"

-bash-4.2$ mkdir /tmp/rob

Then let's move the files over using the command "mv * /tmp/rob"

-bash-4.2$ mv * /tmp/rob

Just make sure the files have been copied

-bash-4.2$ ls /tmp/rob
log_083111      log_090111      log_090211      log_090311      log_090411


Now, we can start the compress of the files using the command "tar -zcvf log.tar /tmp/rob"


-bash-4.2$ tar -zcvf log.tar /tmp/rob
tar: Removing leading '/' from member names
a tmp/robertf
a tmp/robertf/logs
a tmp/robertf/logs/log_090411
a tmp/robertf/logs/log_090311
a tmp/robertf/logs/log_090211
a tmp/robertf/logs/log_090111
a tmp/robertf/logs/log_083111

Let's check again if the tar file has been created. 


-bash-4.2$ ls
log.tar

Now, the log.tar was created in the original location /home/rob/logs so we don't need to move that back from the /tmp directory. 


This is just a small example of how to find and clear space on your system. There are many many other examples. 


Nixcraft - How to find large files

Rob












Thursday, August 04, 2011

New job

It's been a while since I updated this blog. For the past few months I've been busy with work, looking for a new job and finally starting a new job. I'm very excited to move away from the Windows world and work entirely with Linux and FreeBSD but it's also a big learning curve.

For now, I'll spend my time on the transition skills, things I know in Windows but not confident in Linux. Also I will be relearning networking skills as I haven't heavily used this knowledge in a long time since I was so OS focused.

How's that saying goes? Be careful what your wish for?

Rob