Showing posts with label Ubuntu. Show all posts
Showing posts with label Ubuntu. Show all posts

Friday, March 15, 2013

Listing files containing all the words

To find files which contain all words in a set, you can use the below awk script. Here I am searching for files containing three words (word1, word2 and word3). The script output all filenames which contain all three words.

 find  -type f -exec awk 'BEGIN{word1=0;word2=0;word3=0}/word1/{word1++}/word2/{word2++}/word3/{word3++}END{if(word1>0 && word2>0 && word3>0){print FILENAME}}' {} \;

Reference:
1. http://www.linuxquestions.org/questions/linux-newbie-8/grep-an-entire-file-but-must-contain-multiple-words-705681/

Tuesday, January 15, 2013

Google NativeClient on Ubuntu

Initially I tried installing NativeClient (NaCl) on a Ubuntu 32bit VM, but some NaCl tests failed saying specific CPU features needed for NaCl are not available. But later on when setting up a 64bit Ubuntu VM I had to enable Intel VT-X features in BIOS, and this seemed to enable 64bit VM to pass NaCl tests.

Installing NaCl on a 32bit Ubuntu:
Follow the instructions here to download NaCl source using gclient/depot tools: http://code.google.com/p/nativeclient/wiki/Source?tm=4

Once you download everything to nativeclient, you need to run:
make

I had to install g++ libraries too:
sudo apt-get install g++

make failed with some errors:
#include<gelf.h> was throwing an error saying gelf.h was not found.

Installing gelf and libelf did not help.
- sudo apt-get install gelf
- installing libelf from Software center

I had to install libelf-dev from the Ubuntu software center, then the make succeeded.

After make, go to native_client sub directory and run
./scons MODE=opt-linux,nacl
Compiling was successful.
When testing NaCl, just try small_tests, medium_tests and large_tests. run_all_tests argument has some tests specific for 64bit linux, and hence is supposed to fail.

Installing NaCl on a 64bit Ubuntu:
Most of the instructions are same as for 32 bit. When doing a make, I got this error:
make: *** [out/Debug/obj/gen/tc_newlib/lib32/crti.o] Error 1

As per this post: https://groups.google.com/forum/#!topic/native-client-discuss/NJZ2JKWCCCY
we need to install the capability to run x86_32 binaries on a x86_64 machine.

So I had to install :
sudo apt-get install ia32-libs

Then make threw another error:
bits/predefs.h: No such file or directory

This thread gives the solution: http://ubuntuforums.org/showthread.php?t=1877944
we need 32bit libc. We can get it using:
sudo apt-get install libc6-dev-i386

I then got this error:
/usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/4.7/libstdc++.so when searching for -lstdc++
/usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/4.7/libstdc++.a when searching for -lstdc++
/usr/bin/ld: cannot find -lstdc++

As per this link, i had to install "g++-multilib"
http://ubuntuforums.org/archive/index.php/t-1604953.html
sudo apt-get install g++-multilib


run_all_tests  does not work unless you put platform=x86-64 as argument for scons

I had to install python-dev package when giving platform=x86-64 for scons.

Sunday, June 24, 2012

apt doesnt work with system proxy


System proxy details are generally indicated in the environment variables "http_proxy" and "https_proxy". They can be set by applying System Wide Proxy in the GUI.

They can be set from shell or terminal using:
export http_proxy="http://proxy:port"
export https_proxy="https://proxy:port"

You can check using:
echo $http_proxy
echo $https_proxy

Sometimes apt-get doesn't not take in the system proxy information.
Check the file /etc/apt/apt.conf for proxy details. Sometimes, you might need to check /etc/apt/apt.conf.d and a separate proxy file. The proxy entries should be as below:

Acquire::http::proxy "http://proxy:port/";
Acquire::ftp::proxy "ftp://proxy:port/";
Acquire::https::proxy "https://proxy:port/";

You might need to add username and password information, as below:
Acquire::http::proxy "http://<username>:<password>@<proxy>:<port>/";
Acquire::ftp::proxy "ftp://<username>:<password>@<proxy>:<port>/";
Acquire::https::proxy "https://<username>:<password>@<proxy>:<port>/";

Save the file once you make the changes.

TIP: Add these lines in another file, /etc/apt/apt.conf.d/80proxy. This will ensure that after a version upgrade changes won't be lost.

References:
http://askubuntu.com/questions/89437/how-to-install-packages-with-apt-get-on-a-system-connected-via-proxy

Fedora - Ubuntu differences


When working with fedora16, I noticed there were significant differences in the commands used by Fedora and Ubuntu. In most cases, the trick lies in identifying the command equivalents between the two distributions. 
Fedora mostly uses "yum" and Ubuntu generally uses "apt-get".


gcc, g++ compilers, some other generic packages and utilities

In Ubuntu, we use build-essential.  

sudo apt-get install build-essential

Its equivalent in fedora with yum is

su
yum groupinstall "Development Tools"
yum install kernel-devel kernel-headers

References:
http://forums.fedoraforum.org/showthread.php?t=198819
http://blog.htbaa.com/news/yum-equivalent-for-apts-build-essential

Tuesday, December 13, 2011

Setting up HAVP (with ClamAV) + Squid to work as Secure Proxy

There are two ways in which HAVP can be made to interact with Squid (obtained from http://kokikode.wordpress.com/2010/03/14/configuring-squid-havpclamav-in-ubuntu-reviews/).
1. First way is to have the HAVP as Parent proxy for squid.
               Transparent Proxy*
               192.168.0.253:3128
                      ||                    [eth1]
                      ||                      ||
 [Intranet]--------[Squid]-+-[HAVP]--------[Internet]
     ||                        ||
   [eth0]                  Parent Proxy
192.168.0.0/24            127.0.0.1:8080
                               ||
                               ||
                            [ClamAV]
2. Second is to make Squid Proxy as the Parent proxy for HAVP.
                 [ClamAV]
                     ||
                     ||
               Transparent Proxy*
               192.168.0.253:8080
                     ||                     [eth1]
                     ||                       ||
 [Intranet]--------[HAVP]-+-[Squid]--------[Internet]
     ||                        ||
   [eth0]                  Parent Proxy
192.168.0.0/24            127.0.0.1:3128

There are advantages and disadvantages for each (as described in http://kokikode.wordpress.com/2010/03/14/configuring-squid-havpclamav-in-ubuntu-reviews/)

I prefer Method-1, because the proxy only caches the content deemed safe.

Creating Method-1 Setup:
STEP1: Install Squid Proxy
The steps are outlined in my previous post.
STEP2: Install ClamAV.
I tried installing ClamAV using the command:
sudo apt-get install clamav
When performing signature update using
sudo freshclam
I obtained some warning saying ClamAV engine is out of date. So I decided to install ClamAV from the source code. The latest stable source code can be obtained from http://www.clamav.net/lang/en/download/sources/.

Before installing ClamAV, a new user and groupid needs to be created. The default userid and groupid are "clamav". They can be added using the commands:
groupadd clamav
useradd -g clamav clamav
Untar the downloaded source code using the command:
tar -xzvf clamav-0.97.3.tar.gz

Move into the newly created directory and try to perform "configure" with any needed options (No need to specify any options unless userid and groupid are different from default, clamav-milter needs to be installed or new experimental features need to be added). Before we actually install, it would be good to check for previous versions of clamav and remove them. The commands to install are:
cd clamav-0.97.3
./configure
make
sudo apt-get remove --purge clamav
sudo make uninstall
sudo make install

Before updating clamav, it is good to edit the clamd.conf freshclam.conf files located in /usr/local/etc folder. One line containing "Example" needs to be commented and any other entries need to be uncommented as per need. The config files are self explanatory. However, one issue could arise. The log files and signature directories used by freshclam and clamd, specified in the .conf files, might not have required file permissions to be created/edited. You can manually create the file and folder and edit their permissions using "chmod/chown" command.

Update the antivirus signatures and try scanning a simple file.
sudo freshclam
clamscan -v filename

STEP3: Install HAVP
The instructions given in http://kokikode.wordpress.com/2010/03/04/configuring-squid-havpclamav-in-ubuntu-example-1/ provide some guidance for the installation.

This command was installing an older version of HAVP.
sudo apt-get install havp

Hence, get the latest source code of HAVP from http://www.server-side.de/download.htm.

Install HAVP using the regular commands given below. They also add required userid and groupid "havp", allowing me to skip adding them manually.
tar -xvzf havp-0.92a.tar.gz
cd havp-0.92a
./configure
make
sudo make install

Add the below line in /usr/local/squid/etc/squid.conf
cache_peer 127.0.0.1 parent 8080 0 no-query no-digest no-netdb-exchange default

Next havp.config needs to be edited. This config file is located in /usr/local/etc/havp/havp.config. Most of the entries are self explanatory in the config file, however the configuration provided in http://kokikode.wordpress.com/2010/03/04/configuring-squid-havpclamav-in-ubuntu-example-1/ can be used as a reference.

Two major steps need to be performed before successfully running HAVP. The first one is set to set appropriate permissions for the files used by HAVP. In my case, I enumerated the files and folders needing permission change by looking at the error messages. The INSTALL file provided in the extracted havp folder (havp-0.92a) provides the following commands.
Make sure the directories you are using have correct permissions:

  # chown havp /var/tmp/havp /var/log/havp /var/run/havp
  # chmod 700 /var/tmp/havp /var/log/havp /var/run/havp

The second major step is to ensure the scan file system has mandatory locks enabled. The INSTALL file indicates how this could be done. The file /etc/fstab indicates the file systems which are mounted when booting. In my case I just had the root file system (/). So, as indicated in the INSTALL file, I used the following command to activate mandatory locks.
mount -o remount,mand /
The "mand" option could be similarly added to any other file system used by HAVP for scanning (maybe the file system specified for SCANTEMPFILE in havp.config?). To make the "mand" change permanent, we need to add the "mand" option in the fstab file, for the mount root file system entry. The entry in my case looks like:
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
/host/ubuntu/disks/root.disk /               ext4    loop,mand,errors=remount-ro 0       1

We can then start havp using the command:
/usr/local/sbin/havp
We could also start HAVP as user "havp" using the command:
sudo -u havp /usr/local/sbin/havp

STEP 4: Testing the setup.
Try configuring the browser to the squid proxy ip and port 3128, and try browsing few webpages. To test if HAVP is working properly, try visiting this link http://www.eicar.org/download/eicarcom2.zip. It should through a HAVP-ACCESS DENIED PAGE (shown below).

References:
1. http://volatile-minds.blogspot.com/2008/06/installing-clamav-latest-from-source.html
2. http://wiki.clamav.net/bin/view/Main/InstallFromSource
3. http://informatix.or.id/willy/installing-clamav-from-source.php

Friday, December 9, 2011

Setting up Squid Proxy server

Squid could be installed from the Packet Manager, like Synaptic, on Ubuntu. However, most of the times, the latest squid version won't be available from packet managers. We can download the latest source code from http://www.squid-cache.org/Versions/.

I followed the instructions given here: http://www.technologytricks.com/install-squid-proxy-server/ to install Squid. While initializing the cache using the command:
/usr/local/squid/sbin/squid -z

I received the following error:
WARNING: Cannot write log file: /usr/local/squid/var/logs/cache.log/usr/local/squid/var/logs/cache.log: Permission denied

Following instructions given here:http://vonroeschlaub.com/kurt/server.html, I modified the permissions for the folder /usr/local/squid/var/logs, and all its contents using the commands
user@ubuntu:/usr/local/squid/var$ sudo chmod a+w logs
user@ubuntu:/usr/local/squid/var/logs$ sudo chmod a+w * 

You can then test Squid in debugging mode using the command:
sudo /usr/local/squid/sbin/squid -NCd1

"-NCd1" is used for starting it in debugging mode to look for any further errors. I had to use "sudo", because it threw an error as shown below:
2011/12/09 15:44:18| Accepting  HTTP connections at [::]:3128, FD 12.
2011/12/09 15:44:18| HTCP Disabled.
2011/12/09 15:44:18| /usr/local/squid/var/run/squid.pid: (13) Permission denied
FATAL: Could not write pid file
Aborted

Using "sudo" should make it run successfully in debug mode, like below:
2011/12/09 15:45:29| Accepting  HTTP connections at [::]:3128, FD 12.
2011/12/09 15:45:29| HTCP Disabled.
2011/12/09 15:45:29| Squid plugin modules loaded: 0
2011/12/09 15:45:29| Ready to serve requests.

You can then verify if squid is working from the browser. Just go to the connection settings and fill in the IP address of your machine and port 3128 for the proxy and try to browse a url. After browsing few urls, check the access logs in squid/var/logs folder to verify that squid did indeed see the url requests. If you are accessing the proxy from a subnet other than the one listed on your squid.conf file, you should get an "ACCESS DENIED" page. To allow proper browsing, probably the squid.conf needs to be edited further and squid needs to be restarted to make use of the updated configuration file.

Saturday, November 12, 2011

Resetting wireless interface in Ubuntu

Most often in Ubuntu, the wireless stops working. Either it shows
Wireless Networks - Disconnected
and does not display any wireless connections available, or the wireless connection speed slows down drastically and resets quite frequently. This happens when the laptop wakes from a sleep or I try to unlock the screen after a brief period of inactivity.

One way to get out of this situation without rebooting is to restart the wireless lan interface (wlan0) using the commands:
sudo ifdown wlan0
sudo ifup wlan0

(or)

sudo ifconfig wlan0 down
sudo ifconfig wlan0 up

Reference:
1. http://askubuntu.com/questions/33818/lost-wireless-connection-and-detection

Retrieving deleted files from Ubuntu

If files are deleted from GUI by pressing the DELETE key, then those files can be recovered without the need for any additional recovery tools like Scalpel.

Linux version can be obtained by
uname -a

My Linux version is ubuntu 2.6.32-34-generic - Lucid (10.04).

In this linux version, when files/folders are deleted from GUI (by pressing the DELETE key - not sure about SHIFT+DELETE), the contents go to the Trash folder. The path for this trash folder is
~/.local/share/Trash/files

You should be able to find the deleted contents there.