Setup git-http-backend on ubuntu / apache

Description

GIT

GIT

Since version 1.6.6 GIT is able to tunnel its native protocol through HTTP or HTTPS. In this post I describe how to set things up so you can use GIT over HTTP(s). As always it is best to make use of HTTPS for security reasons. In this setup we use Basic authentication so you better use HTTP

I also use virtualmin to keep my hosting business running but that should not be a problem when following along with the steps in this post.

Preparation

Ok first of all create a subdirectory in your public_html  (document root) directory. This is where we are going to store the repositories. I suggest you call this directory….git (lowercase). Change directory to your new folder.

Apache htaccess and htpasswd

We are going to create a couple of CGI scripts to have more control over the way the backend is executed. First create a .htaccess file with the following contents:

This tells apache that it is ok to execute a CGI script from this folder (line 1, 2). It also tells apache to require a “valid-user”; this user can be found in the .htpasswd file (see below). Now we have to create a password file for the user authentication:

For testing purpose you could create an index.html  file and try to open that in the browser. The browser should ask your username and password now.

CGI Scripts

Now create a CGI script that will initialise a new bare repository for us to use. Create an init.cgi  script with the following contents (extend parameter checking if you wish).

init.cgi

When you execute this script via the browser ( https://yourdomain/git/init.cgi?reponame=[yourreponame] ) a new bare repository is created. The actual repositories are created in the subdirectory repos below the git folder.

The next script will startup the actual GIT http backend. I have wrapped this in an additional script so I could perform some logging. Create a script called git.cgi  in your git directory with the following contents.

git.cgi

Finally you need a little configuration script, named config.sh , which sets some general parameters. Source is shown below.

config.sh

Now you can clone a repository by sending your browser to the url

Example workflow

In your browser: https://[yourdomain]/git/init.cgi?reponame=first
In your shell (local): git clone https://[yourdomain]/git/git.cgi/git/repos/first.git
Apply your changes
Add all items to the staging area: git add --all
Commit all changes in the staging area: git commit -am "My commit message"
Push the changes back to the server: git push

That’s all; happy GITing

Share

Add admin user to WordPress with SQL statements

Step 1 Add the user who is going to be an Administrator

INSERT INTO wp_users (ID, user_login, user_pass, user_nicename, user_email, user_url, user_registered, user_activation_key, user_status, display_name) VALUES ('4', 'demo', MD5('demo'), 'Your Name', 'test@yourdomain.com', 'http://www.test.com/','2011-06-07 00:00:00', '', '0', 'Your Name');

Step 2 Give the user the appropriate rights

INSERT INTO wp_usermeta (umeta_id, user_id, meta_key, meta_value) VALUES (NULL, '4','wp_capabilities', 'a:1:{s:13:"administrator";s:1:"1";}');

INSERT INTO wp_usermeta (umeta_id, user_id, meta_key, meta_value) VALUES (NULL, '4','wp_user_level', '10');

Share

SPF and DNS configuration

I recommend that you create a Sender Policy Framework (SPF) record for your domain. An SPF record is a type of Domain Name Service (DNS) record that identifies which mail servers are permitted to send email on behalf of your domain.SPF configuration

The purpose of an SPF record is to prevent spammers from sending messages with forged From addresses at your domain. Recipients can refer to the SPF record to determine whether a message purporting to be from your domain comes from an authorized mail server.

You can also lookup and test your SPF settings here.

So how do you add a SPF record to your domain. Well that is easy. A SPF record is nothing more than a TXT record in your DNS administration.

For a mail server the following DNS entries are defined:

SPF record in DNS

As you can see there is a SPF record on the subdomain mail. A mail server that is receiving email from your domain retrieves the SPF record for your domain and verifies that the sending IP is auhtorized (by the SPF record) for this action.

The headers below of an email message show a domain without a SPF record:

As you can see there is no SPF record defined. You can verify this with the host command:

Now after adding an SPF record to the domain and sending a new message (wait a while for the changes to take effect in the global DNS systems) headers are as shown below (send to a hotmail account):

The SPF check is now passed. Also the output of the host -t txt mail.bjdejong.nl command  should now give you the TXT SPF record.

Also be sure to edit your /etc/mailname  to reflect the name of your server on an Ubuntu distribution.

Share

OpenELEC installation on the Raspberry PI

A short blog post about setting up your Raspberry PI and openElec. Nothing difficult just some basic instructions to get the PI up and running.

openELec on the Pi

First download the OpenELEC Raspbian image here. Then place your SD card in your computer; unmount it with (determine the name of the SD card with the command diskutil list )

Use the raw version of the disk to speed up writing. So instead of using /dev/diskN use /dev/rdiskN

Then unzip the zip file you have just downloaded and write the included img file to the SD card with the command:

This takes a long while to complete. On Mac OS X you can type Ctrl+T in the terminal window where you executed the dd command to check progress.

After the image is written to the SD card place it in your PI and go ahead and fire up your Raspberry PI.

Share

Sublime and SFTP

Sublime Text

Sublime

Ever wanted to directly edit your files on your remote server. This can be done with Sublime and the SFTP plugin. Follow the steps below to setup your SFTP client with Sublime.

First install Sublime, you can find it here. After installation startup sublime and install the package manager. Follow the instructions that you can find here.

Okay, now we have sublime and its package manager installed. Next install the SFTP sublime plugin. Start the package manager in Sublime; type Cmd + Shift + P. Type Install Package and then type SFTP.

Now we have to create an account on the remote server. Setting up an account on your FTP server is not part of this post.

Create a new server setup by choosing File -> SFTP/FTP -> Setup server.

Change the correct items in the example shown and save this file

Now you can browse your server. Goto File -> SFTP/FTP -> Browse server. Choose the server you want to browse. If everything is correctly setup a list of files will appear. You can now edit these files as were they local files.

Share

ownCloud: upgrading your installation

ownCloud logo

ownCloud logo

Upgrade your ownCloud software with this step by step instruction. This post describes the procedure to upgrade the software. Follow the steps for a save and secure updating of your server. For a more verbose description of this procedure you can also check the upgrade page.


This step by step guide assumes you have a data folder outside your ownCloud installation. Check this in your ~/public_html/owncloud/config/config.php  file. The setting 'datadirectory'  should point to a directory outside your ownCloud installation.

Check your version in the admin panel (admin -> Personal info):

Open a (ssh) shell on your linux box. Then su to the user that runs the ownCloud installation and execute the command (in the installation folder; normally this is on ~/public_html/owncloud):

When you navigate to the site you will see the message below. It indicates that your installation is in maintenance mode:

ownCloud maintenance message

ownCloud maintenance message

Check your version with (execute in installation folder):

Goto the download page and copy the link; then execute the command below in your terminal session

Stop the Apache webserver:

Rename your current installation directory and unpack the archive in a new folder outside of your ownCloud directory with the command (a subdirectory ownCloud will be created by the tar command):

Copy the old configuration file to your new ownCloud folder:

Also copy any apps you have installed in your ownCloud instance!

Now start the actual upgrade by executing the command below (again execute this in the installation folder):

If everything went fine you should see something like this:

As you can see maintenance mode is automatically turned off and your server is ready to serve files again after you have re-started the Apache webserver:

Navigate to your site and check that things are still working as expected. In some cases you have to press the update button in your browser that appears when navigating to the site. If everything is working fine don’t forget to remove the old installation files and the archive you downloaded.

Remember: this step by step guide assumes you have a data folder outside your ownCloud installation. Check this in your ~/public_html/owncloud/config/config.php. The setting ‘datadirectory’ should point to a directory outside your ownCloud installation.

Happy ownClouding!

Share

Move to a WordPress https website from http

Wordpress https

WordPress https

Google recently announced that it has started using HTTPS as a ranking signal. So to improve your SEO results you can choose to ONLY use HTTPS for your WordPress site. For this to work you have to have a valid certificate in place (obviously).

Setting up a “SSL only” blog takes two steps.

1. Update your .htaccess file

Go to your WordPress installation folder and edit the .htaccess file in there. Below the line

add the following two lines:

The two rules above will take care for the redirection (in case the user visited your http site) to the https site.

2. Update the WordPress blog settings

Go to the dashboard of your WordPress site and navigate to Settings -> General . Change the WordPress Address and Site Address to use the htpps URL:

Wordpress https

WordPress https

Ok that is all. Your visitors will now always be redirected to the https version of your website.

Share

Test a WCF service with SOAPUI

With help of SOAP UI you can easily test your webservices. In this pos tI’m going to test a WCF web service (yes MS WCF; not my favourite company and technology but anyway….)

First create your webservice. Within Visual Studio goto File -> New Project . Choose Visual C# and select  WCF Service Application

Create new project

Now start your project by pressing F5; the service, together with the WCF Test Client, will start and there are two methods available on this service:

GetData and  GetDataUsingDataContract

The WCF Test client

Play around with the WCF Test Client to get the idea of how this works. Next we will install SOAP UI. Download your copy here. Install SOAP UI using the default settings.

After installation start SOAP UI and create a new project. Goto File -> New soapui Project . Fill out the dialog as shown below (don’t forget to add the ?wsdl ):

 Press Ok; a new soapui project will be created. See the image below:

Now you can execute the same request as the WCF Test Client did; but there is more; much more!

Right click on the WCFTestService1 and choose New TestSuite.

Press OK. Right click on TestSuite 1 and choose New TestCase:

Press OK. Right click on Test Steps choose Add step and then Test request.

Press OK. Choose the operation you want to test:

Press OK. On the next screen leave things as suggested

Press OK. Your first test request is added to the TestCase.  By double clicking the TestCase 1 item and pressing the play button the test will be excuted. As you expected the test will succeed and a green icon is shown right before the test request entry:

Now lets add some testdata to the request. Place the following XML in the test request XML window. First select the xml view:

Now paste the XML below into the request:

Now you can add assertions to your test script to make sure the operations works the way you want. For example lets assert that the GetDataResult field contains the text “You entered: 3”. Press the plus sign next to the play button:

Select the SOAP response assertion:

Give the assertiona unique name:

Type the text you want to look for in the response.

Now if you execute the request it will still succeed because the request contains the number three. Change this number to, for example 4, and you will see that the assertion fails.

Share

Get certificate information with openssl

To display certificate information of a certificate issue the command below:

Information about the certificate is displayed. Some important items are:

The Issuer is a CA that signed this certificate.

The validity period of the certificate. Remember to renew your certificate before it expires!

The subject for this certificate. This certificate can be used for a website for the given CN.

Share

Setup SSH backup on Ubuntu with VirtualMin

Imagine the scenario in which you want to ssh backup your (VirtualMin) server to another (VirtualMin) server. This post describes in detail the steps to take.

First create your backup user on the server where you want to store the backup; lets name it remotebackup for example:

Give a strong password to this user (although we will be using public / private  key authentication).

Then move to the server that creates the backups and create a RSA private / public key pair for the root user:

Do not specify a passphrase (as the backup will run without user intervention). Leave the default filename for private and public key ( id_rsa  and id_rsa.pub).

Now copy the public key (contents of id_rsa.pub) id to the user on the server where the backups have to be stored:

If the copy fails because public key authentication is required over ssh you have to disable this first. To do this edit your /etc/ssh/sshd_config file and set the value for passwordauthentication to yes. Now execute the ssh-copy-id again and reset the value for passwordauthentication.

After the ssh-copy-id  has succeeded a new directory and file is created on the backup server. The directory is in the home folder of the remotebackup user and is called .ssh . In this directory a file is created, authorized_keys , which contains the public key from the root user of the server that wants to store the backups on this server.

To make things more secure it is advisable to make use of the rssh  shell for the remotebackup user. The rssh  shell is a restricted ssh shell. Only the commands that you specify are allowed in a rssh shell. To install it execute the following command on the backup server:

After rssh is installed successfully edit the file /etc/rssh.config . Uncomment the line containing the text allowscp  (VirtualMin executes backups with the scp  command).

Now edit the password file stored at /etc/passwd . Search for the line containing your backup user (remotebackup) at the start of the line and change the shell to /usr/bin/rssh.

Now if you try to login interactively over ssh you get a message saying that it is not allowed:

Okay; this is good; VirtualMin only executes scp commands over your ssh connection.

Now goto your VirtualMin interface on the server that creates the backups and add the required backup schedules.

Create a full backup that runs every first of the month and create an incremental backup that runs every day.

Settings for the full backup. Mind the File on server setting which stores the files in a subfolder full .

Setup ssh backup to external server

 

Setup ssh backup to external server

The incremental backup uses the same settings except for the Backup level which you have to set to, guess what, Incremental.

There is one extra consideration if you create a backup user like this and that is User Disk Quota. When a user is created in a VirtualMin installation with quotas enabled the default maximum quota is 2Gb. That probably is not enough for your backup user. To change this goto your VirtualMin interface and adjust the quota:

VirtualMin -> System -> Disk Quotas; select Groups and select your backupuser.

Screen Shot 2015-02-21 at 15.13.29

Because VirtualMin cannot delete backups on the remote machine in this setup (only scp command is allowed for the remotebackup user) you can make use of the crontab to cleanup old backups. First create a script called cleanup.sh in your remotebackup home folder, place the following contents in it:

Next edit the crontab for the remotebackup user and execute this script, lets say, every day at 0300:

Add this line to the crontab of the user and save it:

Thats all there is to it. Your backups are save now!

Share