nodejs – https and selfsigned certificate

In this article I will talk about nodejs and listening to a ssl (https) port. To make a selfsigned certificate execute the command below:

openssl req -nodes -new -x509 -keyout server.key -out server.cert -days 3650

Remember that browsers will complain about an invalid certificate. For most browsrs you can add a security exception for the certificate.

Now we have to tell nodejs to make use of this certificate when starting the https server. We have to create an option object with two properties: ‘key’ and ‘cert’. When we create the https server we pass in this option object:

var options = {
	key: fs.readFileSync('server.key'),
	cert: fs.readFileSync('server.cert')
};
https.createServer(options, app).listen(3030);

The complete code for a https server with nodejs is shown below

const https = require('https');
const fs = require('fs')
const express = require('express')
const app = express()
const port = 3030

app.use(express.json())
app.use(express.urlencoded({ extended: true }))
app.set('view engine', 'pug')

app.get('/', (req, res) =>  {
        res.render('index')
    }
)

var options = {
	key: fs.readFileSync('server.key'),
	cert: fs.readFileSync('server.cert')
};
https.createServer(options, app).listen(3030);

console.log(`Listening on port ${port}`);

The pug template that is served:

html
  head
  body
    p SSL site
Share

nodejs – asynchronous explained

nodejs – asynchronous explained

In this article i will talk about asynchronous nodejs functions. I use express as the routing middleware and pug as the template engine. The code below shows the setup for the nodejs snippets used later on in this post:

/*
 * Setup:
 * - use 'express' for routing
 * - use 'fs' for reading file from filesystem
 * - use 'pug' as the templating engine
 */
const express = require('express')
const fs = require('fs')
const app = express()
const port = 3000

app.use(express.json());
app.use(express.urlencoded({ extended: true }))
app.set('view engine', 'pug')

We have setup the nodejs app with the code above. Now we are going to add the routes that we want to publish in our example app, see the code below:

// render input form which posts to /read route
app.get('/', (req, res) =>  {

        res.render('index')

    }
)

// read the file
app.post('/read', async (req, res, next) => {

    fs.readFile('./test.txt', (err, data) =>{

        if (err) {
            next(err)
        }

        next(data)
    })

});

app.listen(port, () => console.log(`Listening on port ${port}`))

When we navigate to ‘http://localhost:3000’ the route ‘/’ is processed. This will render our index.pug file which has the following contents:

html
  head
  body
    form(method='POST' action='/read')
      div
        p Your email address
        input#name.form-control(type='text', placeholder='Email address' name='email')
        p 
          button.btn.btn-primary(type='submit') Sign up

Pressing the ‘Sign up’ button will take us to the ‘http://localhost:3000/read’ route. As you can see this route will try to read the ‘./test.txt’ file, The fs.readFile is an asynchronous function. When the readFile operation is complete it will execute the callback that is provided as the second parameter.

When an error occurs fs.readFile will execute the callback with an error object (the data parameter will be ‘undefined’). The error object contains, as you expect, a description of the error:

Error: ENOENT: no such file or directory, open './does_not_exist.txt'

If you would like to have a complete stacktrace in your error object you should change the line ‘next(err)’ to next(new Error(err)’, the resulting error after this change:

Error: Error: ENOENT: no such file or directory, open './does_not_exist.txt'
  at ReadFileContext.fs.readFile [as callback] (/home/uname/node/app.js:21:18)
  at FSReqWrap.readFileAfterOpen [as oncomplete] (fs.js:420:13)

The code above makes use of the callback function of the readFile method on the fs object. An alternative way of accomplishing the same result is to use promises. Lets have a look at the code below:

app.post('/read', (req, res, next) => {

    new Promise((resolve,reject) => {

        fs.readFile('./test.txt', 'utf-8', (err, data) => {

            if (err) {
                // in the case of error, reject the promise, executing the catch code block
                reject(err); 
            }
            else {
                // in the case of success, resolve the promise, exectuing the then code block
                resolve(data);  
            }

        });

    })
    .then((data) => {

        res.send(data)

    })
    .catch((err) => {

        res.send(err)
        
    })
});

As you can see we wrap our fs.readFile in a Promise object. A promise will be rejected on error (the file could not be found for example) and will be resolved on success. When a promise is rejected the catch code block will be executed. If the promise is resolved the then code block will be executed.

The advantage of using a promise is that we can wait on the result if we want to. The code below will wait for the readFile function to complete and then continues the /read handle after the wait promise line

app.post('/read', async (req, res, next) => {

    let fileContent = null

    let promise = new Promise((resolve,reject) => {

        fs.readFile('./test.txt', 'utf-8', (err, data) => {

            if (err) {
                // in the case of error, reject the promise, executing the catch code block
                reject(err); 
            }
            else {
                // in the case of success, resolve the promise, exectuing the then code block
                resolve(data);  
            }

        });

    })
    .then((data) => {

        fileContent = data

    })
    .catch((err) => {

        res.send(err)

    })

    await promise

    if (fileContent) {
        // the catch of the promise did not send(..) any text
        res.send(fileContent)
    }

});
Share

Setup shared folder for Ubuntu guest

First of all install VirtualBox guest additions (goto “Devices” -> “Insert Guest Additions CD Image…”) and reboot your machine. In VirtualBox goto “Devices” -> “Shared Folders” -> “Shared folder settings..”. Press the “plus” button to the right of the (empty) folder list. Choose your path and type a folder name (select auto mount and make permanent for convenience). Press OK

Now the shared folder is created but not active yet in your guest. Reboot your guest machine and go to the folder /media/sf_Sharename  (virtualbox creates a folder in /media with the name of your share with a sf_ prefix.

When you cd into this directory you probable get an access denied error.
To solve this add your account to the vboxsf group with this command:
sudo usermod -aG vboxsf [username] 

To mount the shared folder with specific gid  (change gid to your desired value):

sudo mount -t vboxsf Downloads /mnt/shared -o umask=0022,gid=999

When you get the infamous “protocol error” double check your shared folder name.
Be sure to use the uid and gid assigned to your account (check /etc/passwd and /etc/groups for your uid and gid). If you do not use uid and gid the shared folder may have the wrong access rights.

Share

XAMPP: Installing and using it on Windows 2012 R2

What is XAMPP

XAMPP stands for Cross-Platform (X), Apache (A), MariaDB (M), PHP (P) and Perl (P). It is a simple, lightweight Apache distribution that makes it extremely easy for developers to create a local web server for testing and deployment purposes.

XAMPP Installation

Download your XAMPP installation here. After installation start the XAMPP Control panel (right click on tray icon for XAMPP and choose show/hide).

XAMPP Control Panel

XAMPP Control Panel

Start the Apache and MySQL service. PID(s) and Port(s) should show a number now indicating the services are listening at the ports shown.

Apache

After installation navigate to http://localhost . If Apache is started the XAMPP dashboard is shown in your browser.

MySQL (MariaDB)

To check if MySQL is up and running choose the phpMyAdmin link on the dashboard; if everything is ok the phpMyAdmin dashboard shows up. First thing todo is change your root password for the MySQL instance. Change directory to c:\xampp\mysql\bin and execute the command:

mysqladmin.exe –user=root password “<newpwd>”

To change the MySQL password execute the command:

mysqladmin.exe –user=root –password=<oldpwd> password newpwd

Now phpMyAdmin will stop working because you just changed the root password. To solve this open the phpMyadmin configuration file at c:\xampp\phpMyAdmin\config.inc.php . Change the blowfish_secret to some random value not being xampp. Next set a value of “cookie” for auth_type. Next time you navigate to the phpMyadmin site phpMyAdmin will ask for a username and password.

Setup your first Apache web-site

If both Apache and MySQL are running you can go ahead and setup your first site. In XAMPP sites are stored at c:\xampp\htdocs . Create a directory called xamp.test.tld . Inside this directroy create a document index.php with the following contents:

<?php
phpinfo();

Ok; now on to the Apache configuration. Open the file c:\xampp\apache\conf\extra\httpd-vhosts.conf  and add the following text to the bottom of this file:

<VirtualHost *:80> 
DocumentRoot C:/xampp/htdocs/ 
ServerName localhost 
</VirtualHost> 

<VirtualHost *:80>
   DocumentRoot C:/xampp/htdocs/xampp.test.tld
   ServerName xampp.test.tld
   <Directory "C:/xampp/htdocs/xampp.test.tld">
      Require all granted
      AllowOverride All
   </Directory>
</VirtualHost>

The first virtualhost is the primary or default virtualhost. Hosts that have an unknown ServerName (ie there is no virtualhost definition with this ServerName attribute) are served from this virtual host. In out setup this is also localhost.

That’s all for the Apache configuration. One more thing left to configure and that is the windows host file at c:\windows\system32\drivers\etc\hosts . Add the following line to this file:

127.0.0.1   xampp.test.tld

Your configuration is now complete. Restart your apache server and send your browser to http://xamp.test.tld . The phpinfo page should appear.

Create SSL Website

Open SSL logo

Open SSL

For a site to use SSL we have to create a certificate first which has to be referenced in our Apache virtual host definition. To create your own certificates check this great tutorial. It will guide you step by step through creating a root CA, intermediate CA, certificates and revocation lists. For future reference a short transcript can be found here.

The root certificate you create has to be installed in the Trusted root user certificate store.

After you have create and installed the root CA you can start using the new certificates in your Apache configuration. Lets create a new SSL website for  xampp.test.tld . Open the file c:\xampp\apache\conf\extra\httpd-vhosts.conf  and add the following lines to the end of this file:

<VirtualHost *:443>
DocumentRoot C:/xampp/htdocs/xampp.test.tld
ServerName xampp.test.tld
SSLEngine on
SSLCertificateFile "conf/mycerts/ca/intermediate/certs/xampp.test.tld.cert.pem"
SSLCertificateKeyFile "conf/mycerts/ca/intermediate/private/xampp.test.tld.key.pem"
SSLCertificateChainFile "conf/mycerts/ca/intermediate/certs/ca-chain.cert.pem"
</VirtualHost>

This virtual host defines the SSL site. As you can see there are references to the certificates you created before with OpenSSL.

Now send your browser to https://xampp.test.tld  (mind the s in https) and you should see the PHP information page.

Share

Setup wget proxy on ubuntu

Setup wget to use proxy

When you are behind a proxy server you have to tell wget to use that proxy server. To do this create a .wgetrc  file in your home directory with the contents below (of course change username, password and proxy url).

use_proxy = on
http_proxy = http://username:password@proxy.location.tld:80/
https_proxy = http://username:password@proxy.location.tld:80/

If you want to disable certificate checking add the line below to your .wgetrc

check_certificate = off

 

Share

Linux ls – first directories than files

If you would like to list directories first and than the files with the linux ls command you would have to execute the following ls commandGIT logo
ls -l –group-directories-first

The –group-directories-first  takes care of showing the directories first and than the files.

My favorite ls  alias is (add / change it in your .bashrc ):

ls -AlF --group-directories-first --color=auto
# -A => do not show the implied . and .. directories
# -l => use a long listing
# -F => appends an indicator to entries (a / for a directory)
Share

Setup git-http-backend on ubuntu / apache

Description

GIT

GIT

Since version 1.6.6 GIT is able to tunnel its native protocol through HTTP or HTTPS. In this post I describe how to set things up so you can use GIT over HTTP(s). As always it is best to make use of HTTPS for security reasons. In this setup we use Basic authentication so you better use HTTP

I also use virtualmin to keep my hosting business running but that should not be a problem when following along with the steps in this post.

Preparation

Ok first of all create a subdirectory in your public_html  (document root) directory. This is where we are going to store the repositories. I suggest you call this directory….git (lowercase). Change directory to your new folder.

mkdir /home/[username]/public_html/git
chown [username]:[username] /home/[username]/public_html/git
cd /home/[username]/public_html/git

Apache htaccess and htpasswd

We are going to create a couple of CGI scripts to have more control over the way the backend is executed. First create a .htaccess file with the following contents:

Options +ExecCGI
AddHandler cgi-script cgi
AuthUserFile /home/[username]/public_html/git/.htpasswd
AuthType Basic
AuthName "Git Private"
Require valid-user

This tells apache that it is ok to execute a CGI script from this folder (line 1, 2). It also tells apache to require a “valid-user”; this user can be found in the .htpasswd file (see below). Now we have to create a password file for the user authentication:

htpasswd -c .htpasswd username
New password:
Re-type password for user username

For testing purpose you could create an index.html  file and try to open that in the browser. The browser should ask your username and password now.

CGI Scripts

Now create a CGI script that will initialise a new bare repository for us to use. Create an init.cgi  script with the following contents (extend parameter checking if you wish).

init.cgi

#!/bin/bash                                                                                                   

# init.cgi
#       Initialise a new git repository. 
# Example:
#       https://[yourdomain]/git/init.cgi?reponame=mynewrepo
# Params:
#       reponame        - the name of the git repository to create
# Remarks
# - The name of the repository may only contain the letters a-z and A-Z
# - The repository should not exist already
echo 'Content-type: text/plain'
echo

source config.sh

saveIFS=$IFS
IFS='=&'
parm=($QUERY_STRING)
IFS=$saveIFS

if [ "${#parm[@]}" -ne "2" ]; then
        echo "Invalid number of parameters"
        exit 1
fi

if [ "${parm[0]}" != "reponame" ]; then
        echo "Invalid parameter name ${parm[0]}"
        exit 2
fi

if ! [[ ${parm[1]} =~ ^[a-zA-Z]*$ ]]; then
        echo "Invalid parameter value ${parm[1]}"
        exit 3
fi

if [ -d "$GIT_PROJECT_ROOT/${parm[1]}.git" ]; then
        echo "Git repository already exists"
        exit 4
fi

mkdir -p "$GIT_PROJECT_ROOT"

git init --bare "$GIT_PROJECT_ROOT/${parm[1]}.git/" 2>&1

echo "Repository created at `date` from $REMOTE_ADDR" > "$GIT_PROJECT_ROOT/${parm[1]}.git/description"
echo "`date` : \"${parm[1]}\" repository created from ip $REMOTE_ADDR" >> "$logfile"
echo >> "$logfile"

exit 0

When you execute this script via the browser (https://yourdomain/git/init.cgi?reponame=[yourreponame] ) a new bare repository is created. The actual repositories are created in the subdirectory repos below the git folder.

The next script will startup the actual GIT http backend. I have wrapped this in an additional script so I could perform some logging. Create a script called git.cgi  in your git directory with the following contents.

git.cgi

#!/bin/bash                                                                                                   

# git.cgi
#       Execute the git http-backend command
# Params
#       As handed by the git client command
# Example:
#       git clone https://[yourdomain]/git/git.cgi/myrepo.git
source config.sh

git http-backend "$@" 2>> "$logfile" || echo failed >> "$logfile"

echo "`date` : git command executed $@" >> "$logfile"

Finally you need a little configuration script, named config.sh , which sets some general parameters. Source is shown below.

config.sh

#!/bin/bash
# When not setting the variable below every repo has to have the magic file
# git-daemon-export-ok. If both are not present a message "Repository not exported"
# shows iup in the log file
export GIT_HTTP_EXPORT_ALL=1
export GIT_PROJECT_ROOT=~/public_html/git/repos
logfile=~/tmp/git_log.txt

Now you can clone a repository by sending your browser to the url

https://yourdomain/git/git.cgi/git/repos/[repo].git

Example workflow

In your browser: https://[yourdomain]/git/init.cgi?reponame=first
In your shell (local): git clone https://[yourdomain]/git/git.cgi/git/repos/first.git
Apply your changes
Add all items to the staging area: git add –all
Commit all changes in the staging area: git commit -am “My commit message”
Push the changes back to the server: git push

That’s all; happy GITing

Share

Add admin user to WordPress with SQL statements

Step 1 Add the user who is going to be an Administrator

INSERT INTO `wp_users` (`ID`, `user_login`, `user_pass`, `user_nicename`, `user_email`, `user_url`, `user_registered`, `user_activation_key`, `user_status`, `display_name`) VALUES ('4', 'demo', MD5('demo'), 'Your Name', 'test@yourdomain.com', 'http://www.test.com/','2011-06-07 00:00:00', '', '0', 'Your Name');

Step 2 Give the user the appropriate rights

INSERT INTO `wp_usermeta` (`umeta_id`, `user_id`, `meta_key`, `meta_value`) VALUES (NULL, '4','wp_capabilities', 'a:1:{s:13:"administrator";s:1:"1";}');

INSERT INTO `wp_usermeta` (`umeta_id`, `user_id`, `meta_key`, `meta_value`) VALUES (NULL, '4','wp_user_level', '10');

Share

SPF and DNS configuration

I recommend that you create a Sender Policy Framework (SPF) record for your domain. An SPF record is a type of Domain Name Service (DNS) record that identifies which mail servers are permitted to send email on behalf of your domain.SPF configuration

The purpose of an SPF record is to prevent spammers from sending messages with forged From addresses at your domain. Recipients can refer to the SPF record to determine whether a message purporting to be from your domain comes from an authorized mail server.

You can also lookup and test your SPF settings here.

So how do you add a SPF record to your domain. Well that is easy. A SPF record is nothing more than a TXT record in your DNS administration.

For a mail server the following DNS entries are defined:

SPF record in DNS

As you can see there is a SPF record on the subdomain mail. A mail server that is receiving email from your domain retrieves the SPF record for your domain and verifies that the sending IP is auhtorized (by the SPF record) for this action.

The headers below of an email message show a domain without a SPF record:

x-store-info:w5JOV+GpEg16Hd3Liu8PdV6w7ot2s5vN
Authentication-Results: hotmail.com; spf=none (sender IP is 149.210.153.91) smtp.mailfrom=pkn-bnn-iw@mail.bjdejong.nl; dkim=none header.d=example.nl; x-hmca=none header.id=example@example.nl
X-SID-PRA: example@example.nl
X-AUTH-Result: NONE
X-SID-Result: NONE
X-Message-Status: n:n
X-Message-Delivery: Vj0xLjE7dXM9MDtsPTA7YT0xO0Q9MjtHRD0xO1NDTD0z
X-Message-Info: /3z5fcrqMMUC9h0cjR+uvxBFbZLKdSdtWqbr6MInznDp3iFPY3dfZtYCX/McjjMtj/elcEu8E8GWPTQfME0U8yM95EcDOlBGfpGY3FVu+zhLC/YR0apj5gaojeU+XWKSFMn5xpzk3h7mi1Zrgx5MoCPw09osDoKH5afXmmKDtnxMmXlqQKv2juplilatuJpfG2/eYbQkFt2x+JC9NXiXJelaZ42f/aaw0S73pWX0oijyKeKhtLa1cA==
Received: from mail.bjdejong.nl ([149.210.153.91]) by COL004-MC4F4.hotmail.com with Microsoft SMTPSVC(7.5.7601.23143);
	 Fri, 11 Dec 2015 03:05:49 -0800
Received: by mail.bjdejong.nl (Postfix, from userid 1053)
	id 2D6A93C2F6B; Fri, 11 Dec 2015 12:05:40 +0100 (CET)
To: berendjdejong@gmail.com, berendjdejong@hotmail.com
Subject: Your subject goes here
X-PHP-Originating-Script: 1053:class-phpmailer.php
Date: Fri, 11 Dec 2015 11:05:40 +0000
From: example <example@example.nl>
Message-ID: <421364d92aacfc35ba32861d8d20d90a@pkn-bnn-iw.nl>
X-Priority: 3
X-Mailer: PHPMailer 5.2.10 (https://github.com/PHPMailer/PHPMailer/)
Reply-To: <example@example.nl>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Return-Path: pkn-bnn-iw@mail.bjdejong.nl
X-OriginalArrivalTime: 11 Dec 2015 11:05:49.0453 (UTC) FILETIME=[E48E43D0:01D13403]

Message contents

As you can see there is no SPF record defined. You can verify this with the host command:

host -t txt mail.bjdejong.nl

Now after adding an SPF record to the domain and sending a new message (wait a while for the changes to take effect in the global DNS systems) headers are as shown below (send to a hotmail account):

x-store-info:w5JOV+GpEg16Hd3Liu8PdV6w7ot2s5vN
Authentication-Results: hotmail.com; spf=pass (sender IP is 149.210.153.91) smtp.mailfrom=example@mail.bjdejong.nl; dkim=none header.d=example.nl; x-hmca=none header.id=example@example.nl
X-SID-PRA: example@example.nl
X-AUTH-Result: NONE
X-SID-Result: NONE
X-Message-Status: n:n
X-Message-Delivery: Vj0xLjE7dXM9MDtsPTA7YT0xO0Q9MjtHRD0xO1NDTD0z
X-Message-Info: /3z5fcrqMMUC9h0cjR+uvxBFbZLKdSdtWqbr6MInznDp3iFPY3dfZtYCX/McjjMtj/elcEu8E8GWPTQfME0U8yM95EcDOlBGfpGY3FVu+zhLC/YR0apj5gaojeU+XWKSFMn5xpzk3h7mi1Zrgx5MoCPw09osDoKH5afXmmKDtnxMmXlqQKv2juplilatuJpfG2/eYbQkFt2x+JC9NXiXJelaZ42f/aaw0S73pWX0oijyKeKhtLa1cA==
Received: from mail.bjdejong.nl ([149.210.153.91]) by COL004-MC4F4.hotmail.com with Microsoft SMTPSVC(7.5.7601.23143);
	 Fri, 11 Dec 2015 03:05:49 -0800
Received: by mail.bjdejong.nl (Postfix, from userid 1053)
	id 2D6A93C2F6B; Fri, 11 Dec 2015 12:05:40 +0100 (CET)
To: berendjdejong@gmail.com, berendjdejong@hotmail.com
Subject: Your subject goes here
X-PHP-Originating-Script: 1053:class-phpmailer.php
Date: Fri, 11 Dec 2015 11:05:40 +0000
From: example <example@example.nl>
Message-ID: <421364d92aacfc35ba32861d8d20d90a@pkn-bnn-iw.nl>
X-Priority: 3
X-Mailer: PHPMailer 5.2.10 (https://github.com/PHPMailer/PHPMailer/)
Reply-To: <example@example.nl>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Return-Path: example@mail.bjdejong.nl
X-OriginalArrivalTime: 11 Dec 2015 11:05:49.0453 (UTC) FILETIME=[E48E43D0:01D13403]

Message contents

The SPF check is now passed. Also the output of the host -t txt mail.bjdejong.nl command  should now give you the TXT SPF record.

Also be sure to edit your /etc/mailname  to reflect the name of your server on an Ubuntu distribution.

Share

OpenELEC installation on the Raspberry PI

A short blog post about setting up your Raspberry PI and openElec. Nothing difficult just some basic instructions to get the PI up and running.

openELec on the Pi

First download the OpenELEC Raspbian image here. Then place your SD card in your computer; unmount it with (determine the name of the SD card with the command diskutil list )

Use the raw version of the disk to speed up writing. So instead of using /dev/diskN use /dev/rdiskN

diskutil unmountDisk /dev/<disk>

Then unzip the zip file you have just downloaded and write the included img file to the SD card with the command:

sudo dd bs=1m if=path_of_your_image.img of=/dev/diskn

This takes a long while to complete. On Mac OS X you can type Ctrl+T in the terminal window where you executed the dd command to check progress.

After the image is written to the SD card place it in your PI and go ahead and fire up your Raspberry PI.

Share