Category Archives: Linux

Howto’s, scripts and tips for Linux

FTP a folder with wget

You can easily use wget to FTP your files. The command below FTP’s a complete directory structure to your local machine.

wget -r -nH ==cut-dirs=4 ftp://[username]:[password]ftp.svr.nl/webspace/httpdocs/site1.nl

 

The above command retrieves all files with the folder webspace/httpdocs/site1.nl . The -nH  options makes sure that there is no folder ftp.svr.nl is created. The –cut-dirs=3  options removes 3 (webspace, httpdocs and site1.nl) folders from the remote directory structure and re-creates the structure local (excluding the 3 levels) local.

Share
GIT logo

Mount NFS share under Ubuntu

On the server:

Edit the file /etc/exports  file and add the line:

/share hosts(rw,nohide,insecure,no_subtree_check,async,all_squash,anonuid=idofshare,anonguid=guidofshare]

Example:

/home *(rw,nohide,insecure,no_subtree_check,async,all_squash,anonuid=1008,anongid=1008)

Explanation:

/share  is the location you want to share
hosts  is the specification of hosts you allow access
all_squash  to translate all anonymous id’s (not known on server) to the give anonuid and anongid
no_subtree_check does no checking on the complete subtree of filepermissions (see also here)

After adding or changing an export to the /etc/exports file don’t forget to restart the NFS server:

sudo service nfs-kernel-server restart

On the client:

To view the list of exported shares on the server execute the command:

showmount -e [ip_of_server]

 

Example of output:

Export list for 192.168.2.200:
/home *

To mount the NFS share local create a new subfolder and execute the command:

mount -v [host]:/home ~/home/

This will mount the folder /home on the server local on your /home/ folder.

 

 

Share

Analyze IP addresses accessing your Apache server

The awk command below retrieves the first column of your apache log file which containsscript the IP address of the browser accessing your host (if you have a virtual host setup with the vhost_combined CustomLog you should retrieve column 2 instead).

After retrieving the column it is sorted and all unique values are determined and counted. After that the list of unique values and there count is sorted (reverse) to get the top list of IP’s.

awk '{ print $1}' $LOGFILE | sort | uniq -c | sort -nr > $DEST

Output of this statement:

606 188.203.177.225
502 84.87.80.237
376 86.81.89.200
365 24.132.181.21
295 94.212.84.128
279 86.81.89.250
251 94.212.208.48
235 85.150.175.72
226 94.212.194.142

 

 

Share

Setup proxy ignore list on Ubuntu

Follow these steps to change the proxy ignore list on your Ubuntu installation:

1. Install the dconf-editor:

sudo apt-get install dconf-tools

2. Start the dconf-editor and navigate to System -> Proxy; add your hosts to ignore to the ignore-hosts value.

Logoff and logon for changes to take effect

Your can inspect your current settings with:

env | grep proxy

 

Share

apt-get through socks5 proxy

Enable temporary proxy for apt-get by editing (create it if not exist /etc/apt/apt.conf  and add the line (change username, password, host and port):

Acquire::http::Proxy "http://uname:upwd@yourhost:port/";

Install the tsocks application:

sudo apt-get install tsocks

Remove the line added in the step before from from /etc/apt/apt.conf

Edit the /etc/socks.conf  file and change the default server to the IP address of your socks server (a domain name does not work!); if applicable also change the port number for your socks connection.

Now to use this new socks proxy go ahead and execute:

sudo tsocks apt-get update

Even though you have removed the proxy settings from the /etc/apt/apt.conf  file this should still update your apt list.

 

Share

MySQL automated database backup

This article describes a way to setup a regular backup for your MySQL databases. This method uses the crontab to schedule backup jobs (you could also use the logrotate method).

The database backups are stored in separate files. Once database backup is completed the file is zipped and password encrypted with openssl.

First create a .my.cnf file in your home directory with the following contents:

[client]
user=backup
password=1234567890

Make sure the file is only readable by your self:

chown 600 ~/.my.cnf

Next create a SQL user as shown below:

mysql> grant lock tables,select,reload,show databases,event on *.* to 'backup'@'localhost' identified by '1234567890';
mysql> flush privileges;

Create a folder in for example /var/backups/ and name it mysql

mkdir /var/backups/mysql

Create a script called backup_mysql_dbs.sh at a suitable location, for example your $HOME/cron/scripts folder (make it NOT world readable / writeable):

#!/bin/bash
BACKUPDEST=/var/backups/mysql

if [ ! -d "$BACKUPDEST" ]; then
echo "$BACKUPDEST does not exist, exiting"
exit 1
fi

# Remove backup files older dan 7 days
/usr/bin/find $BACKUPDEST/*.sql.gz.enc -type f -mtime +5 -exec rm {} \;

# Backup all mysql databases to separate files. The files are gzipped and encoded with a password.
umask 077
/usr/bin/mysql -Ne "show databases" | grep -v schema |
while read db; do
CDATE=`date '+%Y-%m-%d-%H%M%S'`
/usr/bin/mysqldump --skip-events --events $db | /bin/gzip | /usr/bin/openssl aes-256-cbc -salt -k $CDATE > $BACKUPDEST/$db.$CDATE.sql.gz.enc;
done

Now it is time to add a cron.d file; go to /etc/cron.d and add a crontab file, e.g. backup_mysql, with the following contents:

# /etc/cron.d/anacron: crontab entries for the anacron package
MAILTO=user@host.ext
M H * * * root /var/www/backup_mysql_dbs.sh

Replace the M and H with minute and hour you want the backup to be performed.

That’s it; your mysql databases are backedup at the sepcified interval.

 

Share

Setup Chroot SFTP in Ubuntu Linux

Chroot keeps the user in his login / home directory when logging in. This is not default behavior. In the default setup of SFTP you can CD anywhere on the system!

1. Create a new group

groupadd sftpusers

2. Create users who want to SFTP

useradd -g sftpusers -s /usr/sbin/nologin guestuser
passwd guestuser

3. Modify the the /etc/ssh/sshd_config file and comment out the following line:

Subsystem sftp /usr/libexec/openssh/sftp-server

4. Add the following line directly below the line you have commented out:

Subsystem sftp internal-sftp

5. Add the following lines at the end of /etc/ssh/sshd_config:

Match Group sftpusers
        ChrootDirectory /var/www/%u
        ForceCommand internal-sftp

6. Users start in the /var/www directory; this directory should be owned by root (see below) with chmod 755:

ll -ld /var/www
drwxr-xr-x 11 root root 4096 jun 30 13:39 /var/www/

7. Create a new directory with the same name as the username you just added:

mkdir /var/www/guestuser

8. Make sure that this new directory is also owned by root (see below):

ll -ld /var/www/guestuser
drwxr-xr-x 3 root www-data 4096 aug 14 18:18 /var/www/guestuser

10. Create a subfolder below /var/www/guestuser for example www and set user ownership to guestuser and groupowner ship to www-data. Set access right two 2755 ( 2=set group id, 7=rwx for owner (guestuser), 5=rx for group (www-data), 5=rx for world (including apache www-data user):

ll -ld /var/www/guestuser
drwxr-sr-x 2 guestuser www-data 4096 okt  1 17:02 /var/www/guestuser/www/

9. Restart the ssh server:

service sshd restart

If the directories have not the correct ownership a message will appear in /var/log/auth.log:

Aug 14 18:18:29 hostname sshd[24996]: fatal: bad ownership or modes for chroot directory "/var/www/guestuser"

 

Share

This article describes how to setup a basic firewall configuration on your linux box with iptables.

Check your current setup with:

sudo iptables -L

Reset the default firewall rules:

sudo iptables -F

Setup the most basic firewall with the script below. This script only allows ssh connections (which are being logged in /var/log/syslog); adjust the script to allow communications through other ports.

Both the OUTPUT and INPUT chain have a default log rule at the end of the chain.

#!bin/sh
IPTABLES=/sbin/iptables
$IPTABLES -F
### Set default policies for INPUT, OUTPUT and FORWARD chain to DROP
$IPTABLES -P INPUT DROP
$IPTABLES -P OUTPUT DROP
$IPTABLES -P FORWARD DROP
####### INPUT chain ######
### State tracking rules
$IPTABLES -A INPUT -m state --state INVALID -j LOG --log-prefix "DROP INVALID" --log-ip-options --log-tcp-options
$IPTABLES -A INPUT -m state --state INVALID -j DROP
$IPTABLES -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

### ACCEPT rules for connections in
$IPTABLES -A INPUT -p tcp --dport 22 -j LOG --log-prefix "SSH CONNECTION"
$IPTABLES -A INPUT -p tcp --dport 22 --syn -m state --state NEW -j ACCEPT

### default INPUT LOG rule
$IPTABLES -A INPUT -j LOG --log-prefix "LOG " --log-ip-options --log-tcp-options

####### OUTPUT chain ######
### State tracking rules
$IPTABLES -A OUTPUT -m state --state INVALID -j LOG --log-prefix "DROP INVALID" --log-ip-options --log-tcp-options
$IPTABLES -A OUTPUT -m state --state INVALID -j DROP
$IPTABLES -A OUTPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

### ACCEPT rules for connections out
$IPTABLES -A OUTPUT -p tcp --dport 22 --syn -m state --state NEW -j ACCEPT

### default OUTPUT LOG rule
$IPTABLES -A OUTPUT -j LOG --log-prefix "LOG " --log-ip-options --log-tcp-options

Saving your iptables configuration (answer Yes to both questions):

sudo apt-get install iptables-persistent

Start the persistency service:

sudo service iptables-persistent start

Changes to your configuration can be stored by using the command

sudo service iptables-persistent save

Or reload the current configuration:

sudo service iptables-persistent reload

More detailed information can be found here.

Share

Add SSL to localhost on apache / linux

Generate a Certificate Signing Request

1. Generate the keys for the Certificate Signing Request (CSR)

openssl genrsa -des3 -out server.key.secure 1024

2. Create the insecure key.

openssl rsa -in server.key.secure -out server.key.insecure

3. Create the CSR.

openssl req -new -key server.key.insecure -out server.csr

Fill in the appropriate information.

4. Create the self-signed certificate

openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt

This creates server.crt

5. Install the self-signed certificate

sudo cp server.crt /etc/ssl/certs
sudo cp server.key /etc/ssl/private

Now you can configure apache with the ability to use public-key cryptography to use the certificate and key files.

Configure Apache to use SSL on local host

6. Enable ssl

sudo a2enmod ssl

7. Edit your default-ssl site (make backup copy)

Change:

SSLCertificateFile /etc/ssl/certs/ssl-cert-snakeoil.pem
SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key

To:

SSLCertificateFile /etc/ssl/certs/server.crt
SSLCertificateKeyFile /etc/ssl/private/server.key

8. Enable the default-ssl site.

sudo a2ensite default-ssl

8. Restart the server.

sudo service apache2 restart

Navigate to https://localhost and examine your certificate details

Share