Setting up HAVP (with ClamAV) + Squid to work as Secure Proxy

There are two ways in which HAVP can be made to interact with Squid (obtained from http://kokikode.wordpress.com/2010/03/14/configuring-squid-havpclamav-in-ubuntu-reviews/).
1. First way is to have the HAVP as Parent proxy for squid.

               Transparent Proxy*
               192.168.0.253:3128
                      ||                    [eth1]
                      ||                      ||
 [Intranet]--------[Squid]-+-[HAVP]--------[Internet]
     ||                        ||
   [eth0]                  Parent Proxy
192.168.0.0/24            127.0.0.1:8080
                               ||
                               ||
                            [ClamAV]

2. Second is to make Squid Proxy as the Parent proxy for HAVP.

                 [ClamAV]
                     ||
                     ||
               Transparent Proxy*
               192.168.0.253:8080
                     ||                     [eth1]
                     ||                       ||
 [Intranet]--------[HAVP]-+-[Squid]--------[Internet]
     ||                        ||
   [eth0]                  Parent Proxy
192.168.0.0/24            127.0.0.1:3128

There are advantages and disadvantages for each (as described in http://kokikode.wordpress.com/2010/03/14/configuring-squid-havpclamav-in-ubuntu-reviews/)

I prefer Method-1, because the proxy only caches the content deemed safe.

Creating Method-1 Setup:
STEP1: Install Squid Proxy
The steps are outlined in my previous post.
STEP2: Install ClamAV.
I tried installing ClamAV using the command:

sudo apt-get install clamav

When performing signature update using

sudo freshclam

I obtained some warning saying ClamAV engine is out of date. So I decided to install ClamAV from the source code. The latest stable source code can be obtained from http://www.clamav.net/lang/en/download/sources/.

Before installing ClamAV, a new user and groupid needs to be created. The default userid and groupid are "clamav". They can be added using the commands:

groupadd clamav
useradd -g clamav clamav

Untar the downloaded source code using the command:

tar -xzvf clamav-0.97.3.tar.gz

Move into the newly created directory and try to perform "configure" with any needed options (No need to specify any options unless userid and groupid are different from default, clamav-milter needs to be installed or new experimental features need to be added). Before we actually install, it would be good to check for previous versions of clamav and remove them. The commands to install are:

cd clamav-0.97.3
./configure
make
sudo apt-get remove --purge clamav
sudo make uninstall
sudo make install

Before updating clamav, it is good to edit the clamd.conf freshclam.conf files located in /usr/local/etc folder. One line containing "Example" needs to be commented and any other entries need to be uncommented as per need. The config files are self explanatory. However, one issue could arise. The log files and signature directories used by freshclam and clamd, specified in the .conf files, might not have required file permissions to be created/edited. You can manually create the file and folder and edit their permissions using "chmod/chown" command.

Update the antivirus signatures and try scanning a simple file.

sudo freshclam
clamscan -v filename

STEP3: Install HAVP
The instructions given in http://kokikode.wordpress.com/2010/03/04/configuring-squid-havpclamav-in-ubuntu-example-1/ provide some guidance for the installation.

This command was installing an older version of HAVP.

sudo apt-get install havp

Hence, get the latest source code of HAVP from http://www.server-side.de/download.htm.

Install HAVP using the regular commands given below. They also add required userid and groupid "havp", allowing me to skip adding them manually.

tar -xvzf havp-0.92a.tar.gz
cd havp-0.92a
./configure
make
sudo make install

Add the below line in /usr/local/squid/etc/squid.conf

cache_peer 127.0.0.1 parent 8080 0 no-query no-digest no-netdb-exchange default

Next havp.config needs to be edited. This config file is located in /usr/local/etc/havp/havp.config. Most of the entries are self explanatory in the config file, however the configuration provided in http://kokikode.wordpress.com/2010/03/04/configuring-squid-havpclamav-in-ubuntu-example-1/ can be used as a reference.

Two major steps need to be performed before successfully running HAVP. The first one is set to set appropriate permissions for the files used by HAVP. In my case, I enumerated the files and folders needing permission change by looking at the error messages. The INSTALL file provided in the extracted havp folder (havp-0.92a) provides the following commands.

Make sure the directories you are using have correct permissions:

  # chown havp /var/tmp/havp /var/log/havp /var/run/havp
  # chmod 700 /var/tmp/havp /var/log/havp /var/run/havp

The second major step is to ensure the scan file system has mandatory locks enabled. The INSTALL file indicates how this could be done. The file /etc/fstab indicates the file systems which are mounted when booting. In my case I just had the root file system (/). So, as indicated in the INSTALL file, I used the following command to activate mandatory locks.

mount -o remount,mand /

The "mand" option could be similarly added to any other file system used by HAVP for scanning (maybe the file system specified for SCANTEMPFILE in havp.config?). To make the "mand" change permanent, we need to add the "mand" option in the fstab file, for the mount root file system entry. The entry in my case looks like:

# <file system> <mount point>   <type>  <options>       <dump>  <pass>
/host/ubuntu/disks/root.disk /               ext4    loop,mand,errors=remount-ro 0       1

We can then start havp using the command:

/usr/local/sbin/havp

We could also start HAVP as user "havp" using the command:

sudo -u havp /usr/local/sbin/havp

STEP 4: Testing the setup.
Try configuring the browser to the squid proxy ip and port 3128, and try browsing few webpages. To test if HAVP is working properly, try visiting this link http://www.eicar.org/download/eicarcom2.zip. It should through a HAVP-ACCESS DENIED PAGE (shown below).

References:
1. http://volatile-minds.blogspot.com/2008/06/installing-clamav-latest-from-source.html
2. http://wiki.clamav.net/bin/view/Main/InstallFromSource
3. http://informatix.or.id/willy/installing-clamav-from-source.php

Setting up Squid Proxy server

Squid could be installed from the Packet Manager, like Synaptic, on Ubuntu. However, most of the times, the latest squid version won't be available from packet managers. We can download the latest source code from http://www.squid-cache.org/Versions/.

I followed the instructions given here: http://www.technologytricks.com/install-squid-proxy-server/ to install Squid. While initializing the cache using the command:

/usr/local/squid/sbin/squid -z

I received the following error:

WARNING: Cannot write log file: /usr/local/squid/var/logs/cache.log/usr/local/squid/var/logs/cache.log: Permission denied

Following instructions given here:http://vonroeschlaub.com/kurt/server.html, I modified the permissions for the folder /usr/local/squid/var/logs, and all its contents using the commands

user@ubuntu:/usr/local/squid/var$ sudo chmod a+w logs
user@ubuntu:/usr/local/squid/var/logs$ sudo chmod a+w * 

You can then test Squid in debugging mode using the command:

sudo /usr/local/squid/sbin/squid -NCd1

"-NCd1" is used for starting it in debugging mode to look for any further errors. I had to use "sudo", because it threw an error as shown below:

2011/12/09 15:44:18| Accepting  HTTP connections at [::]:3128, FD 12.
2011/12/09 15:44:18| HTCP Disabled.
2011/12/09 15:44:18| /usr/local/squid/var/run/squid.pid: (13) Permission denied
FATAL: Could not write pid file
Aborted

Using "sudo" should make it run successfully in debug mode, like below:

2011/12/09 15:45:29| Accepting  HTTP connections at [::]:3128, FD 12.
2011/12/09 15:45:29| HTCP Disabled.
2011/12/09 15:45:29| Squid plugin modules loaded: 0
2011/12/09 15:45:29| Ready to serve requests.

You can then verify if squid is working from the browser. Just go to the connection settings and fill in the IP address of your machine and port 3128 for the proxy and try to browse a url. After browsing few urls, check the access logs in squid/var/logs folder to verify that squid did indeed see the url requests. If you are accessing the proxy from a subnet other than the one listed on your squid.conf file, you should get an "ACCESS DENIED" page. To allow proper browsing, probably the squid.conf needs to be edited further and squid needs to be restarted to make use of the updated configuration file.

Resetting wireless interface in Ubuntu

Most often in Ubuntu, the wireless stops working. Either it shows
Wireless Networks - Disconnected
and does not display any wireless connections available, or the wireless connection speed slows down drastically and resets quite frequently. This happens when the laptop wakes from a sleep or I try to unlock the screen after a brief period of inactivity.

One way to get out of this situation without rebooting is to restart the wireless lan interface (wlan0) using the commands:
sudo ifdown wlan0
sudo ifup wlan0

(or)

sudo ifconfig wlan0 down
sudo ifconfig wlan0 up

Reference:
1. http://askubuntu.com/questions/33818/lost-wireless-connection-and-detection

Retrieving deleted files from Ubuntu

If files are deleted from GUI by pressing the DELETE key, then those files can be recovered without the need for any additional recovery tools like Scalpel.

Linux version can be obtained by
uname -a

My Linux version is ubuntu 2.6.32-34-generic - Lucid (10.04).

In this linux version, when files/folders are deleted from GUI (by pressing the DELETE key - not sure about SHIFT+DELETE), the contents go to the Trash folder. The path for this trash folder is
~/.local/share/Trash/files

You should be able to find the deleted contents there.

Running shell commands in Python

You can run shell commands in Python using the "os" library:

import os
os.system("command") 

If we need to capture the stdout or the stderr outputs in the python code, we need to use subprocesses.

from subprocess import Popen, PIPE, STDOUT

cmd = 'echo "Hello World"'
p = Popen(cmd, shell=True, stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True)
output = p.stdout.read()
print output

To stream the output in realtime:

import sys
from subprocess import Popen, PIPE, STDOUT

cmd = 'echo "Hello World"'
p = Popen(cmd, shell=True, stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True)
for line in iter(p.stdout.readline, ''):
    sys.stdout.write(line)

In few cases, we might need to stop/kill the subprocess opened using Popen. Such cases arise when we need to execute some system command from python, but do not want that command to take more than 30 seconds/1 minute. One dumb way to make this stopwatch/timer work is to run a while loop in the main python program. The code looks like:

import time
from subprocess import Popen, PIPE, STDOUT

p = Popen('COMMAND', shell=True, stdin=PIPE, stdout=PIPE, stderr=STDOUT,close_fds=True)
time_start = time.time() 
seconds_passed = 0 
while(not p.poll() and seconds_passed<30):
    seconds_passed = time.time() - time_start 
if(seconds_passed>=30):    
    p.kill()   
else:
    output = p.stdout.read()

Reference:

1. http://www.saltycrane.com/blog/2008/09/how-get-stdout-and-stderr-using-python-subprocess-module/
2. https://stackoverflow.com/questions/11495783/redirect-subprocess-stderr-to-stdout

Running GUI applications in PlanetLab

I had to run GUI applications from PlanetLab for a Project. X11 forwarding (necessary to run GUI applications over SSH connections) is not enabled on the PlanetLab machines due to security reasons. Another option was to install a VNC server on each of the nodes, which provides a GUI enabled remote access to these machines.

I had setup a vnc-server on the planet lab nodes using the instructions given in http://www.g-loaded.eu/2005/11/10/configure-vnc-server-in-fedora/.

Step-1:
First you need to setup vnc-server on the planetlab node.

sudo yum install vnc-server

You need to have a remote user account that you can use with VNC. Your planetlab account should suffice.

Step-2:
Next you need to configure the vncservers config file. In Fedora Core or Red Hat based distros in general, all we have to do is define the VNC server instances in /etc/sysconfig/vncservers. These will be started by the vncserver initscript. This has to be done as root. Edit this file so that it contains the following lines. Below planetlabaccount is your slice name that you use for logging into planetlab.

VNCSERVERS="3:planetlabaccount"
VNCSERVERARGS[3]="-geometry 1024x768 -depth 16"

With these we define that a vnc server instance should be started as user 'planetlabaccount' on display 3 and we also set some options for this server such as resolution and color depth. Each VNC server instance listens on port 5900 plus the display number on which the server runs. In our case, planetlabaccount’s vnc server would listen on port 5903.

For multiple vnc instances /etc/sysconfig/vncservers would look like this:

VNCSERVERS="1:tiger 2:albatros 3:leopard"
VNCSERVERARGS[1]="-geometry 1024x768 -depth 16"
VNCSERVERARGS[2]="-geometry 800x600 -depth 8"
VNCSERVERARGS[3]="-geometry 1024x768 -depth 16"

These would listen on ports 5901, 5902, 5903 respectively.

Step-3:
You need to setup planetlabaccount’s vnc password. So, as user planetlabaccount give the command:

vncpasswd

We are prompted for a password. This is the password that we will use when we connect to planetlabaccount’s vnc server instance. This password is saved in /home/<planetlabaccount>/.vnc/passwd.

Step-4:
After the initial configuration is done we restart the vnc service. As root:

sudo /etc/init.d/vncserver restart

You can make VNC server to start on boot. The command looks like (I haven't tested it out):

chkconfig vncserver on

Step-5:
When vncserver is restarting you will see a message like

New '<planetlabmachine>:3 (planetlabaccount)' desktop is <planetlabmachine>:3

Here "planetlabmachine" is your planetlab node name, eg: "planetlab1.poly.edu"

You need to check if the DISPLAY environment variable is set using the command:

echo $DISPLAY

If you see a blank like, try setting it with the desktop name seen above "<planetlabmachine>:3"

export DISPLAY=<planetlabmachine>:3

Now any GUI applications like xterm or Firefox you run from the terminal will be active in your vncserver's desktop.

Step-6:

Now install a vncviewer like xvnc4viewer or xtightvncviewer on your local machine. If using windows, use realvnc's vnc viewer. Once you login to the planetlab's vnc server, you should be able to see all your GUI applications (like xclock and Firefox) running. Though the display is not so colorful and not so fast, it might serve the purpose.

Step-7 (OPTIONAL):
If working with Firefox, remember to install x11 fonts for Firefox to work. The command you need to give is:

sudo yum install xorg-x11-fonts-Type1

With regular VNC, you wont be able to move or resize the windows as there is no window manager running. You need to run "Metacity" a GNOME window manager. Run the command from the terminal:

metacity &

Reference:
1. http://www.g-loaded.eu/2005/11/10/configure-vnc-server-in-fedora/.
2. http://forums.fedoraforum.org/showthread.php?t=201885

How to start Dell Webcam after it stopped working

I have a Dell Inspiron 1440, bought two years back in 2009. Yesterday, my inbuilt webcam was not getting recognized by any application. When I launched Dell Webcam Central it showed a message "Please Plugin a Supported Device". I looked up Dell forums and people suggested to update the Webcam Driver or re-install the Webcam central application.

Re-installing the Webcam central software did not help. I tried looking up for drivers on http://support.dell.com/ website, but the website was down for some reason. Another way to update the driver was to launch Device Manager and look for webcam listing and update driver from there. However, I could not find the "Imaging Device" listing in the Device Manager (Right click computer -> Manage -> Device Manager). After digging through Dell Forums I landed on this page (http://en.community.dell.com/support-forums/laptop/f/3518/p/19271842/19476201.aspx). One of the last comments suggested to boot in safe mode (pressing F8 before windows OS starts loading once you boot) and look for Imaging Device listing in the Device Manager. I was able to find it in Safe Mode and tried updating the driver. I already had the latest driver. So, I just disabled and enabled the device and rebooted the machine. Now the webcam works fine and I am able to find it listed in Device Manager.