Create Windows 10 Kernel-based Virtual Machine (KVM) on Linux Debian server

There are many tutorials out there about how to create KVM, but not many go in to details on how to do it if your guest OS is Windows and also scares info on the problems you may encounter. This is why I decided to put in brief the steps how to do it and describe the problems I had to deal with.

This guide assumes you already have a working Debian 9 installation /the steps should in general work for Debian 10 as well/. Commands has to be executed as root.

Before you start is a good idea to check some hardware capabilities of the host computer, especially if it is not relatively new.

First you need to check if your CPU supports virtualisation, to do this run the command:

 

egrep -c ‘(vmx|svm)’ /proc/cpuinfo

 

If the output of the command is 1 or 2 you are good, 0 means no go for you.

Then you need to check if the hardware virtualisation is enabled and if not you need to change the settings in your BIOS. Run this command and read the output:

 

dmesg | grep “disabled by bios”

 

If the output is “kvm: disabled by bios” you need to alter the BIOS settings, otherwise you are fine.

When ready you need to install quite a lot of packages, so in your terminal type the command:

 

apt-get install –no-install-recommends qemu-kvm libvirt-daemon-system libvirt-clients libvirt-daemon-system libvirt-dev libguestfs-tools genisoimage virtinst libosinfo-bin virt-viewer virt-manager acpid

 

Be patient, it will take some time. The reason of –no-install-recommends switch is to avoid installation on packages related to GUI, which I personally avoid on server installations.

When the installation is finished you will have to change your network interfaces so the virtual machine becomes part of your LAN. For this you will have to create a bridge interface, make sure you have the bridge-utils package installed. If it happens to have a OpenVPN server already running on the Debian server as I did, make sure you change your VPN interface to TAP and add it to the bridge, otherwise you won’t be able to browse your network.

This is how my /etc/network/interfaces file looked like after the configuration change:

 

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
# allow-hotplug enp2s0
# iface enp2s0 inet static
# address 192.168.1.2
# netmask 255.255.255.0
# gateway 192.168.1.1

######################################
# bridge part, coment above int conf                      #
######################################

auto enp2s0
iface enp2s0 inet manual

auto tap0
iface tap0 inet manual

auto br0
iface br0 inet static
address 192.168.1.2
broadcast 192.168.1.255
netmask 255.255.255.0
gateway 192.168.1.1
bridge_ports enp2s0 tap0
bridge_stp off
bridge_waitport 0
bridge_fd 0
dns-nameservers 8.8.8.8
dns-nameservers 8.8.4.4
dns-nameservers 192.168.1.2

 

Restart the networking service (or reboot) and test your connectivity is working as intended.

Next step is to let regular user to manage the VM:

adduser your_user libvirt
adduser your_user libvirt-qemu

Then reload group membership:

newgrp libvirt
newgrp libvirt-qemu

Verify your group membership with id command:

id

 

Check virtual machine network and state

virsh net-list –all
virsh list –all

 

Bridge networking has to be configured for the VM as well, so do the following:

Create a file using a text editor

 

nano /root/bridged.xml

 

Append the following config in the file, make sure bridge interface name is the same as in your interfaces configuration:

 

<network>
<name>br0</name>
<forward mode=”bridge”/>
<bridge name=”br0″/>
</network>

 

Save and close the file. Then run the following commands to apply this configuration to the VM:

 

virsh net-define –file /root/bridged.xml
virsh net-autostart br0
virsh net-start br0

 

Veryfy bridged network

 

virsh net-list –all

 

In order Windows 10 to recognise your virtual hard disk during its installation you will need to get the vertio drivers. Create “virtio” (or some other name) folder in /var/lib/libvirt/boot/ and download the drivers (checking for the latest version is probably a good idea):

 

cd /var/lib/libvirt/boot/virtio/

wget https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/archive-virtio/virtio-win-0.1.173-2/virtio-win-0.1.173.iso

 

Next create the VM’s virtual disk, adjust to your needs:

qemu-img create -f qcow2 /var/lib/libvirt/images/windows_10_x64.qcow2 80G

 

With the disk set it is time to create the VM:

 

virt-install \
–virt-type=kvm \
–hvm \
–name=windows10 \
–ram=2048 \
–cpu=host \
–vcpus=2 \
–os-type=windows \
–os-variant=win10 \
–disk path=/var/lib/libvirt/images/windows_10_x64.qcow2,format=qcow2,bus=virtio \
–disk /var/lib/libvirt/boot/Win10_1909_EnglishInternational_x64.iso,device=cdrom,bus=ide \
–disk /var/lib/libvirt/boot/virtio/virtio-win-0.1.173.iso,device=cdrom,bus=ide \
–network=bridge=br0,model=virtio \
–graphics vnc

 

If stumble upon error stating that the access to the KVM kernel module is denied do the following:

Open the file qemu.conf and edit the following:

 

nano /etc/libvirt/qemu.conf

#user = root -> user = root
#group = “root” – > group = “kvm”

 

After a successful VM creation you will need a VNC client to connect to the VM and start the installation of Windows. In the installation process you will have to navigate to the folder containing the virtio disk driver, mind in my case it was drive E:, but may not be the same for you. The path for me was: E:\viostor\w10\amd64. After the installation you can run RDC on Windows and set users to connect to your VM directly without third party software.

To have your VM starting automatically when the host starts or is rebooted run the commands:

First make sure libvirtd service is stared on boot:

 

systemctl enable libvirtd

 

Then run:

 

virsh autostart Your_VM_Name_Here

 

If you  try to shutdown your VM from host terminal with the virsh shutdown VM_Name_Here without success this might be due to not working acpi event handler. To fix this edit the content of powerbtn file (create the file if does not exist):

nano /etc/acpi/events/powerbtn

 

Then type these lines (delete anything in the file if the file exists)

 

event=button/power
action=/sbin/poweroff

 

and restart the acpid service

 

service acpid restart

 

And this is the end of it in general, next is to post the script to shutdown, backup and then start the KVM when time is available.

Have fun!

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Advertisement

Bash script for complete system backup on external hard drive using tar

Some time ago I posted a script to backup Linux system to external drive using rsync. As the task this time was to create an archive of the whole system and the old backup files to be removed the script had to be changed a bit. Here is the version of it with the use of tar command. As before your external drive’s mount point has to be defined in fstab file.

And this is the script, it does not need much explanation:

 

#!/bin/bash

# Script to create full system archived backup to external USB drive.

# Specify the mount point, backup path and other stuff here (DO NOT end mount_point with a forward-slash).

mount_point=’/mnt/WD’
current_year=`date +%Y`
current_day=`date +’%Y-%m-%d_%H:%M:%S’`
backup_path=${mount_point}’/backup/’${current_year}
files_to_keep=2

echo “#####”
echo “”

# Check whether target volume is mounted, and mount it if not.

if ! mountpoint -q ${mount_point}/; then
   echo “Mounting the external USB drive.”
   echo “Mountpoint is ${mount_point}”
      if ! mount ${mount_point}; then
         echo “An error code was returned by mount command!”
         exit 5
      else echo “Mounted successfully.”
      fi
else echo “${mount_point} is already mounted.”;
fi

# Target volume **must** be mounted by this point.

if ! mountpoint -q ${mount_point}/; then
   echo “Mounting failed! Cannot run backup without backup volume!”
   exit 1
fi

echo “Preparing the archive using tar.”

mkdir –parents ${backup_path}/

tar czf ${backup_path}/${current_day}_backup.tar.gz –exclude=${backup_path}/${current_day}_backup.tar.gz –exclude=/dev/* –exclude=/mnt/* –exclude=/proc/* –exclude=/sys/* –exclude=/run/* –exclude=/media/* –exclude=/tmp/* –exclude=/lost+found /

# Delete old backup files

cd ${backup_path}/

rm -f `ls -t *_backup.tar.gz | awk “NR>$files_to_keep”`

cd /

# Dismounting target volume

if ! umount ${mount_point}; then
   echo “An error code was returned by command!”
   exit 5
else
echo “Dismounted successfully.”;
fi

 

And you are good to go! 🙂

Configure BT Echolife/Huawai HG612 as ADSL modem only device

Quite a lot of information on this little, but rather useful device (which in fact is a VDSL/ADSL modem router) can be found at kitz.co.uk, but a step by step guide of how to use this device as a ADSL modem only is not easy to find, so I decided to put my adventures in writing.

First thing first – to get a hold on a device like this you have to visit eBay, as these are made for BT and are the cheapest possible option to get a hold on a device capable of being VDSL/ADSL modem, compared to NETGEAR DM200 with its bad reputation or the pricey DrayTek Vigor 130.

What basically has to happen is to set the WAN interface of the device in bridging mode, so it just passes on the frames to your cable router. The device is by default configured to be a VDSL bridge for the BT network in UK, so it has to be unlocked first, which is a fairly easy procedure, which I have only performed with HG612 type 3B device.

Navigate to https://mega.co.nz/#F!LdJFDIJL!e_E1twsIg2kTet8mPjrb4w and go to B030SP08 folder, where you will need to download the firmware. The file you would most likely want is called bcm96368MVWG_fs_kernel_HG612V100R001C01B030SP08_webgui, which you could probably guess by the name has the web GUI, unless you would like to try the hard way of course…

When ready, do the following:

Configure your computer’s Ethernet port with the IP address 192.168.1.2 and subnet mask 255.255.255.0

With the HG612 powered off, connect a cable between the LAN2 port (you may need to remove the “not in use” sticker) and the Ethernet port on your computer.

Hold down the reset button on the back of the HG612 whilst inserting the power cable. Keep the reset button held for about 5–10 seconds after powering up, then release.

After about 10 seconds you should be able to access the HG612’s “Update Software” web page at the address http://192.168.1.1

Follow the on-screen instructions to upload the latest firmware, you will be notified the process will take 2 minutes – please be patient, give it 3 minutes, go to the toilet, make yourself a coffee, but do not interrupt or power off.

When this is done navigate again to http://192.168.1.1 and you should be greeted by the login page, use default username and password which are both “admin”. Navigate to BASIC -> DSL, tick ALL and then Submit.

 

 

Next step is important, configuring the ATM interface. It has to be configured as Ethernet over ATM type so it can translate the PPPoA frames to PPPoE and pass it on to your cable router to do the authentication, this is a handy function you won’t easily find on many devices, especially cheap once. More about the technology behind can be found at https://en.wikipedia.org/wiki/Point-to-Point_Protocol_over_Ethernet#PPPoEoE-to-PPPoA_converting_modem

Navigate to BASIC -> ATM and choose these options: for the virtual identifiers use 0/38, these are for UK, if it happens so you are somewhere else round the globe check these with your ISP. Leave DSL latency as it is – Path1. For the DSL link type pick the correct one, EoA, this is kind of important. Encapsulation mode – pick VCMUX, again check with your ISP if not sure or see what it is on the router you are supplied, if available. Service type is UBR without PCR. When ready press Submit.

 

 

We are almost ready… Now we have to assign the ATM interface to be the WAN interface. You have few things to do before that though, as by default the device is set to be a VDSL modem, not a ADSL. You have to either remove the PTM interface from the WAN setup, or disable it. In my case I just disabled it, thinking it might one day save me a few clicks.

Go to BASIC -> WAN and select the PTM interface, there untick the WAN connection option and also untick the Port binding LAN1. Click submit when ready.

 

 

Now while you are still there click on the New tab so you can assign the ATM to WAN. From the Layer 2 interface drop-down menu pick atm1/(0_0_38), thick WAN connection so it enables the interface on the WAN, Service list live as it is – INTERNET, Port binding – tick LAN1, this will be the port you connect to your cable router, LAN2 will remain the port you can connect to HG612 if you want to do some configuration changes in the future. For Connection mode select Bridge. And final, but important bit, set Bridge type as PPPoE_Bridged. Click Submit button.

 

 

And this is it! You have a pure ADSL modem only device which passes the WAN IP to the WAN port on your cable router as nature intended! 🙂

On your cable router under the WAN interface configuration choose Connection type PPPoE, add your username and password and you should be good to go, enjoy!

Install Pydio web file manager on Debian 9 with MySQL 8

If you came across my previous post regarding setting up Nginx for user access control you might be thinking that the setup is okay for two or just a few users, but what if the users are more? Yes, it gets a bit inconvenient and it lacks flexibility. In the previous work I have already installed Nginx, MySQL and PHP, so I will not be explaining that again here and now is the time to use them with, in my case, a good web file management platform. As I have not used such before and I went looking around, reading and picking. On the end I picked Pydio as it is a very well documented, there are lots of tutorials on how to install it on Debian, even a very good Nginx configuration file on Pydio’s web site itself – https://pydio.com/en/docs/kb/system/installing-debiannginx and also Pydio is in active development as well.

But as usual I had a slightly bumpy way and I thought I better write it down, you never know who you gonna help! 😉

I changed my Nginx configuration a bit, so people are not allowed to “wonder” on the server, especially on Pydio directories, though that did not make the software to not show its warning message about that on install.

I installed Pydio the “wget” way, as finding out it can be done via apt came later, never mind… 🙂

wget https://download.pydio.com/pub/core/archives/pydio-core-8.0.2.zip

unzip pydio-core-8.0.2.zip

mv <extracted directory name> /var/www/html/pydio

chown -R www-data:www-data /var/www/html/pydio

If your workspaces will be located somewhere else make sure they are owed by www-data, so the software can manipulate the files and the folders.

Adjust your php.ini and make sure those are covered, in my case:

file_uploads = On

post_max_size = 1G

upload_max_filesize = 1G

max_file_uploads = 200

output_buffering = Off

In /var/www/html/pydio/conf/bootstrap_conf.php add setlocale(LC_ALL, “en_US.UTF-8”); and uncomment define(“AJXP_LOCALE”, “en_EN.UTF-8”);

Now it is time to set up database, user and password in MySQL. This is where I had a funny surprise, after a have done it and started the installation Pydio could not connect to my MySQL server and I got the message that the server does not accept the client authentication method. MySQL 8 by default now accepts better SHA-256 Pluggable Authentication /which I did not know/, which seemingly my client was not able to support /or I could not figure out how to change that from client side/. To go around that add in /etc/mysql/my.cnf:

[mysqld]

default_authentication_plugin=mysql_native_password

and restart MySQL server.

Now you can create Pydio database and user.

mysql -u root -p

Enter password:

Welcome to the MySQL monitor. Commands end with ; or \g.

Your MySQL connection id is 8

Server version: 8.0.11 MySQL Community Server – GPL

Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its

affiliates. Other names may be trademarks of their respective

owners.

Type ‘help;’ or ‘\h’ for help. Type ‘\c’ to clear the current input statement.

mysql> CREATE DATABASE pydio;

Query OK, 1 row affected (0.13 sec)

mysql> CREATE USER ‘pydio_user’@’localhost’ IDENTIFIED BY ‘your_password’;

mysql> GRANT ALL PRIVILEGES ON pydio.* TO ‘pydio_user’@’localhost’;

mysql> FLUSH PRIVILEGES;

Now navigate to Pydio and start the installation process, it is very easy to follow.

And that’s pretty much it.

Install and configure Nginx web sever, MySQL and PHP support with Let’s Encrypt and DDNS on Debian 9

Without going in too much details, I have been asked to build a server with certain requirements witch let me to that project and I thought it might be a good idea to describe the process as a reference as well as something to help others.

The assumption here is that you know how to install Debian, beware there are some changes if this is your first try on Debian 9. Make sure you know your network interfaces names, use the ip command or dmesg to find them out. Also as this setup is with DDNS make sure before you sign up for a DDNS service to check if the domain you want is included in the DNS Public Suffix List, otherwise you will not be able to use Let’s Encrypt services. You can check it here. I have used one of noip.com listed once.

Before everything else I installed the NO-IP’s DUC software. There is a good bash script to help you with that, which you can find here.

It did not quite work for me, I was missing the killall command, which can be installed with apt install psmisc and also on restart the service was not active, so what I did is to add a symbolic link in my rc.3 folder pointing to the script in /etc/init.d folder. You can also follow the guide on NO-IP’s web site and see how it goes.

Then comes Nginx, to install it run

apt install nginx,

in /etc/nginx/nginx.conf I changed the keep alive parameter down to 25.

Once the installation is finished next comes MySQL. MySQL is not Debian’s default database any more, so to install it you have to have the sources in your apt repository list. Navigate to https://dev.mysql.com/downloads/repo/apt/ to check on the latest version and correct download link, at the time of writing 0.8.10-1. Go to your download folder or create one and run

wget https://dev.mysql.com/get/mysql-apt-config_0.8.10-1_all.deb

and install it using dpkg:

dpkg -i mysql-apt-config_0.8.10-1_all.deb

Now having the right information in sources.list file we can install MySQL:

apt update

apt install mysql-community-client mysql-community-server

To have PHP installed run:

apt install php7.0-fpm php7.0-gd php-pear php7.0-mysqlnd php7.0-curl php7.0-intl php-imagick php7.0-imap php7.0-mcrypt php-memcache php7.0-intl php7.0-pspell php7.0-recode php7.0-tidy php7.0-xmlrpc php7.0-xsl

Have a look at the file /etc/php/7.0/fpm/php.ini if you need to change something, probably at least date.timezone to match your location.

Reload PHP: systemctl reload php7.0-fpm.service

Not much will be discussed here on MySQL and PHP. I wanted them installed for future development on multi level user login and file management, but in this instance I have only used the built in basic authentication in Nginx. (P.S. After finishing reading you can now check the next post where this is discussed.)

Before that though we will generate Let’s Encrypt certificates and this will be done using the default Nginx configuration. With your preferred editor open the file /etc/nginx/sites-available/default and set your domain name in the server_name line as well as the web root folder if different from default. The line should look like this after editing:

server_name yourhost.provider.com;

Replace yourhost.provider.com with your own domain name and then install Certbot:

apt install certbot python-certbot-nginx

Now we request an SSL certificate from Let’s Encrypt:

certbot certonly –webroot -d yourhost.provider.com

Read carefully the questions and give the appropriate answers. The newly generated SSL certificate is in a subfolder of /etc/letsencrypt/live/ folder. The exact path is shown in the Certbot output. Now it is time to create your own Nginx web server file. In mine I have the HTTP traffic redirected to the HTTPS, the web root denied for everyone so no one sees its content, two folders for two users and autoindex option on so users can see the file content in the folders. This is the configuration file:

server {

listen [::]:80;

listen 80;

server_name yourhost.provider.com;

# redirect http to https

return 301 https://yourhost.provider.com$request_uri;

}

server {

listen [::]:443 ssl http2;

listen 443 ssl http2;

server_name yourhost.provider.com;

root /var/www/html;

index index.php index.html index.htm;

ssl on;

ssl_certificate_key /etc/letsencrypt/live/yourhost.provider.com/privkey.pem;

ssl_certificate /etc/letsencrypt/live/yourhost.provider.com/fullchain.pem;

ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

ssl_ciphers ‘ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:!DSS’;

ssl_prefer_server_ciphers on;

# Rest of the config

location = / {

deny all;

}

location ^~ /test1/ {

auth_basic “Restricted Access”;

auth_basic_user_file /etc/nginx/users/.test1;

try_files $uri $uri/ =404;

autoindex on;

}

location ^~ /test2/ {

auth_basic “Restricted Access”;

auth_basic_user_file /etc/nginx/users/.test2;

try_files $uri $uri/ =404;

autoindex on;

}

# pass PHP scripts to FastCGI server

location ~ \.php$ {

include snippets/fastcgi-php.conf;

# With php-fpm (or other unix sockets):

fastcgi_pass unix:/var/run/php/php7.0-fpm.sock;

# With php-cgi (or other tcp sockets):

# fastcgi_pass 127.0.0.1:9000;

}

# Deny all attempts to access hidden files such as .htaccess, .htpasswd, .DS_Store (Mac).

location ~ /\. {

deny all;

access_log off;

log_not_found off;

}

}

It is useful to run nginx -t before reloading the service with systemctl restart nginx.service.

To create the files with the Nginx users and passwords I used the handy python script htpasswd.py, but if you are having troubles running it you are very likely to be missing the Trac package, which you can easily install:

apt install trac

To actually get the script just do:

cd /usr/local/bin

wget https://trac.edgewall.org/export/16537/trunk/contrib/htpasswd.py

chmod 755 /usr/local/bin/htpasswd.py

though check the Trac website for the latest download link.

To create a file /in my case/ do:

htpasswd.py -c -b /etc/nginx/users/.test1 user password

Do not use -c option if you want to add more users to the same file.

Finally it is not a bad idea to add firewall rules, these are my set as a bash script, but you can always change it the way it suits your needs. I use iptables-persistent package to save the rules and load them at boot:

#!/bin/bash

# Flush previous rules, delete chains and reset counters

iptables -F

iptables -X

iptables -Z

iptables -t nat -F

# INPUT

# Allows all loopback (lo0) traffic and drop all traffic to 127/8 that doesn’t use lo0

iptables -A INPUT -i lo -j ACCEPT

iptables -A INPUT ! -i lo -d 127.0.0.0/8 -j REJECT

# Accepts all established inbound connections

iptables -A INPUT -m conntrack –ctstate ESTABLISHED,RELATED -j ACCEPT

# Allows HTTP and HTTPS connections from anywhere (the normal ports for websites)

iptables -A INPUT -p tcp –dport 80 -j ACCEPT

iptables -A INPUT -p tcp –dport 443 -j ACCEPT

# Allows SSH connections

# The –dport number is the same as in /etc/ssh/sshd_config

iptables -A INPUT -p tcp -s 192.168.0.0/16 -m conntrack –ctstate NEW,ESTABLISHED –dport 22 -j ACCEPT

# Allow Samba connections

iptables -A INPUT -p tcp -s 192.168.0.0/16 –dport 445 -j ACCEPT

iptables -A INPUT -p tcp -s 192.168.0.0/16 –dport 139 -j ACCEPT

iptables -A INPUT -p udp -s 192.168.0.0/16 –dport 137 -j ACCEPT

iptables -A INPUT -p udp -s 192.168.0.0/16 –dport 138 -j ACCEPT

# Allow ping

# note that blocking other types of icmp packets is considered a bad idea by some

# remove -m icmp –icmp-type 8 from this line to allow all kinds of icmp:

# https://security.stackexchange.com/questions/22711

iptables -A INPUT -p icmp -j ACCEPT

# log iptables denied calls (access via ‘dmesg’ command)

iptables -A INPUT -m limit –limit 5/min -j LOG –log-prefix “iptables denied: ” –log-level 7

# REST

# Drop invalid state packets

iptables -A INPUT -m conntrack –ctstate INVALID -j DROP

iptables -A OUTPUT -m conntrack –ctstate INVALID -j DROP

iptables -A FORWARD -m conntrack –ctstate INVALID -j DROP

# Reject all other inbound – default deny unless explicitly allowed policy:

iptables -A INPUT -j REJECT

iptables -A FORWARD -j REJECT

# Allows all outbound traffic

# You could modify this to only allow certain traffic

iptables -A OUTPUT -j ACCEPT

And that’s it, if I haven’t forgotten something… 🙂

Configure Roundcube Webmail to “remember” username in login form in Debian 8

Roundcube is very well made webmail client written in PHP and all the aspects of the program experiences are taken very seriously, like the security as an example.

If installed by default Roundcube does not allow the browser to remember your username details. If you are like me a bit unhappy to type it every time – this is the article for you. Of course when I first decided to change this behaviour I went to see if there is anything in the official documentation, but I could not find the answer there nor I managed to find a decent answer to that online. So I had to take the hard way and do it myself digging the content of the program’s files… 🙂

My Roundcube installation is on a Debian system, but in general the steps should be the same for any Linux installation.

First of all find your installation folder, on a Debian system it should be /usr/share/roundcube.

 

Navigate to include folder:

cd /usr/share/roundcube/program/include/

Open the file rcmail_output_html.php with your favourite text editor.

nano rcmail_output_html.php

 

In this file find the lines with this content /line 1792 in my case/:

// set atocomplete attribute

$user_attrib = $autocomplete > 0 ? array() : array(‘autocomplete’ => ‘off’);

 

Now at the end of the line change ‘off’ to ‘on’ so it looks like this:

// set atocomplete attribute

$user_attrib = $autocomplete > 0 ? array() : array(‘autocomplete’ => ‘on’);

 

And that’s all, just reload Roundcube’s login page so your change takes effect. Keep in mind after update you may /most likely will/ need to change this again.

GRUB bootloader repair with chroot

I had to do this quite often lately, so I thought it deserves going to the blog… 🙂 If you are having computers with multiple OS to boot sooner or later you will end up in a situation of having GRUB corrupt.

My latest adventure was after Windows 10 Anniversary update. The update messed up the hard drive partition table and on boot the computer ends up to GRUB’s rescue console. You can try the steps from this post and fix GRUB from its rescue console, though it did not work for me and you will notice at the end you still need to reinstall grub and recreate the boot menu:

http://www.legendiary.at/2016/01/04/windows-10-update-changes-partition-table-and-breaks-grub/

So I got one of my flash drives with live Linux on it and went for the well-known classic approach 🙂 Please mind your flash and hard disk Linux kernel architecture /32 or 64 bit/, they must match.

Boot your computer of the flash drive, run a terminal, make sure you are root /or use sudo/ when execute the commands and follow these steps:

1. List your drives and partitions with a command of your choice and make a note of the Linux partition

 

          fdisk –l

 

2. Mount your Linux partition to a folder and link required system folders to their alternatives in that folder. Substitute sdXY below with the correct Linux partition for your system.*

 

          mount /dev/sdXY /mnt

 

          mount –bind /dev /mnt/dev

          mount –bind /dev/pts /mnt/dev/pts

          mount –bind /proc /mnt/proc

          mount –bind /sys /mnt/sys

 

3. Use chroot to change your root directory.

 

          chroot /mnt

 

4. Recreate GRUB menu and install it on the boot drive. This commands are for Debian and its family, for other distributions you have to use the correct commands.

 

          update-grub

 

          grub-install /dev/sdX

 

Reboot your computer and enjoy the boot menu you are used to. 🙂

*You can substitute the multiple mount commands with a single line if you wish to:

          for i in /dev /dev/pts /proc /sys; do mount -B $i /mnt$i; done

** Just a quick addition to the post to bring it up to current times. If your happen to have a EFI bootloader you will have to mount the EFI partition as well, like this:

mount /dev/sdXZ /mnt/boot/efi

Debian 7 Wheezy – L2TP VPN Server behind NAT with strongSwan and self-signed certificate authentication

As usual before everything else a few good and must read articles on the subject: https://www.zeitgeist.se/2013/11/22/strongswan-howto-create-your-own-vpn/ – a very good tutorial on setting up strongSwan, http://www.manabii.info/2014/08/debian-wheezy-strongswan-l2tp-ipsec.html – another very good tutorial on L2TP strongSwan configuration /you will have to translate it from Japan unless you are a ninja :-)/ and of course the very important strongSwan wiki documentation at https://wiki.strongswan.org/projects/strongswan.

Why strongSwan and L2TP? Well after some research I have done on the available IPsec implementations for Linux I have decided to go for strongSwan because of its active development. As for L2TP I decided that a bit of overhead, which any modern hardware can handle the capability to use other protocols as well as IP over the VPN connection is worth the hassle.

 

STEP 1 – GENERAL PREPARATION

The assumption is you already have control over a DNS server, local network in range of 192.168.0.0/24, a server running Debian 7 Wheezy  with IP address 192.168.0.10 and as a good practice advises – first make sure your system is up to date, just run /do not forget to be a root user or use sudo when necessary/:

#apt-get update && apt-get upgrade

Normally Debian 7 will install strongSwan 4, but I wanted version 5 because it only runs the charon daemon which handles everything for you and you do not have to configure NAT-T – it is triggered for you automatically if needed. To do this you will have to alter the /etc/apt/sources.list file adding those lines in it:

deb http://ftp.debian.org/debian/ wheezy-backports main
deb-src http://ftp.debian.org/debian/ wheezy-backports main

After that you can run the set of commands which will update the sources, install the xl2tpd daemon /this is the software responsible for L2TP/ and install strongSwan:

#apt-get update
#apt-get install xl2tpd
#apt-get -t wheezy-backports install strongswan

Just to check you have the version 5 of strongSwan:

#dpkg -l | grep strong

The next important thing is to change the kernel parameter so it can route the VPN traffic /later on NAT will be configured/. To achieve this in a permanent way some parameters in /etc/sysctrl.conf must be set properly, you will need to have this:

net.ipv4.ip_forward = 1
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.all.accept_redirects = 0

To apply the changes run the following command:

#sysctl –p

 

STEP 2 – CERTIFICATES PREPARATION

Before generating certificates find out the fully qualified domain name of your VPN server, make sure an A record for it exist in your DNS server and use it as CN when generating certificates. To find your FQDN run the fallowing command:

#hostname –f

Now you are ready to generate the certificate authority key and certificate:

#cd /etc/ipsec.d/
#ipsec pki –gen –type rsa –size 4096 \
–outform pem \
> private/CA_Key.pem

#chmod 600 private/CA_Key.pem

#ipsec pki –self –ca –lifetime 3650 \
–in private/CA_Key.pem –type rsa \
–dn “C=GB, O=SomeName, CN=server.example.com” \
–outform pem \
> cacerts/CA_Cert.pem

 

Create the VPN Server Key and Certificate

#ipsec pki –gen –type rsa –size 2048 \
–outform pem \
> private/VPN_Server_Key.pem

#chmod 600 private/VPN_Server_Key.pem

#ipsec pki –pub –in private/VPN_Server_Key.pem –type rsa | \
ipsec pki –issue –lifetime 3650 \
–cacert cacerts/CA_Cert.pem \
–cakey private/CA_Key.pem \
–dn ” C=GB, O=SomeName, CN=server.example.com ” \
–san server.example.com \
–flag serverAuth –flag ikeIntermediate \
–outform pem > certs/VPN_Server_Cert.pem

 

Create Client Certificate

#ipsec pki –gen –type rsa –size 2048 \
–outform pem \
> private/ClientKey.pem

#chmod 600 private/ClientKey.pem

#ipsec pki –pub –in private/ClientKey.pem –type rsa | \
ipsec pki –issue –lifetime 3650 \
–cacert cacerts/CA_Cert.pem \
–cakey private/CA_Key.pem \
–dn ” C=GB, O=SomeName, CN=server.example.com ” \
–san user@example.com \
–outform pem > certs/ClientCert.pem

 

Export Certificate for the Client as PKCS#12

#openssl pkcs12 -export -inkey private/ClientKey.pem \
-in certs/ClientCert.pem -name “Client VPN Certificate” \
-certfile cacerts/CA_Cert.pem \
-caname “server.example.com” \
-out Client.p12

 

STEP 3 – SETTING UP srongSwan

There are few files used to configure strongSwan all located in /etc folder. The file /etc/strongswan.conf I left unchanged as it probably is set correctly for most users, but of course you can check its content and if you are sure you know what you are doing adjust it to your needs.
Next is the file /etc/ipsec.conf, which you can backup first before changing:

#cp /etc/ipsec.conf /etc/ipsec.conf.bac

Then you can create the configuration you need, mine looks like:

# ipsec.conf – strongSwan IPsec configuration file

# basic configuration

config setup
#       strictcrlpolicy=yes
#       uniqueids = no
charondebug=”cfg 2, dmn 2, ike 2, net 2″

# Add connections here.
conn L2TP
forceencaps=yes
auto=add
keyexchange=ikev1
keyingtries=3
rekey=no
ikelifetime=8h
lifetime=1h
type=transport
left=192.168.0.10
leftsubnet=0.0.0.0/0[udp/1701]
leftauth=pubkey
leftcert=VPN_Server_Cert.pem
right=%any
rightsubnet=0.0.0.0/0[udp/%any]
rightauth=pubkey
rightcert=ClientCert.pem
dpddelay=40
dpdtimeout=130
dpdaction=clear

include /var/lib/strongswan/ipsec.conf.inc

And finally the file /etc/ipsec.secrets has to be set so the client can log in. Mine is set as follows:

: RSA VPN_Server_Key.pem
#user1 : EAP “BigSecret”
#user2 : XAUTH “AnotherBigSecret”

It is not a bad idea to set the permissions:

#chmod 600 /etc/ipsec.secrets

To make sure ipsec service loads on boot:

#insserv ipsec

 

STEP 4 – SETTING UP L2TP

Three files must be configured so xl2tpd daemon to work properly. First one is /etc/xl2tpd/xl2tpd.conf where the global parameters are set and in my case it is like that:

[global]
[lns default]
ip range = 192.168.111.2 – 192.168.111.254
local ip = 192.168.111.1
length bit = yes
refuse chap = yes
require authentication = yes
name = l2tp
pppoptfile = /etc/ppp/l2tpd-options

The next one is obviously the file /etc/ppp/l2tpd-options where you set some PPP options and mine are set as:

name l2tp
refuse-pap
refuse-chap
refuse-mschap
require-mschap-v2
ms-dns 8.8.8.8
ms-dns 8.8.4.4
nodefaultroute
lock
nobsdcomp
mtu 1280
mru 1280

And finally you set the PPP user in the file /etc/ppp/chap-secrets:

# client    server  secret      IP addresses
vpnuser    l2tp    “aPassword”  *

 

STEP 5 – SETTING UP NAT

We are almost there now. The final bit is to set up the NAT, which is done with iptables, but before that I recon it is not a bad idea to install the iptables-persistent package and have your rules saved and loaded properly on reboot. To do this you will have to:

#apt-get install iptables-persistent

Then you can create the rule and have it saved. To go for SNAT or Masquerading is a personal choice, but as you most likely will have a static IP address for your server probably SNAT is better:

#iptables -t nat -A POSTROUTING -j SNAT –to-source 192.168.0.10 -o eth+
#iptables-save > /etc/iptables/rules.v4

To check if your services are running you can see their status or/and network activity:

#service –status-all

or

#netstat -lnput | grep charon
#netstat -lnput | grep xl2tpd

Do not forget to forward UDP ports 500, 1701 and 4500 on your router.

And now the funny part…  Connecting your clients. Android devices connect effortlessly, just install your client certificate using the p12 file, put your PPP username and password as well as the server IP address /which will be the public address of your router/ or its DNS name /should resolve to your router’s public address/ and you are done.
Have not tried iOS device yet, but as Mac OS X connects I would assume there will not be a problem, though it was a bit tricky with the Mac. Make sure you install the client certificate in your System key chain and then make it trusted. Make sure you have a record in your DNS server with exactly the same name of your VPN server as it is in the certificate and when setting the connection do not use the IP address, use the domain name. If you do not do it this way you will not be able to connect.
Windows… well I could not manage to connect even a single Windows 7 client PC regardless of what I tried – switching off firewall and antivirus software, changing the configuration of the VPN server, had the certificates properly installed with MMC, and before you ask – yes, I have had the AssumeUDPEncapsulationContextOnSendRule in the registry set to 2. Nothing worked. Had a brief look at third party VPN clients for Windows, but could not find a good enough to support L2TP VPN with certificates. So if anyone have experience with this I will be happy to see what is wrong and where, I could not figure it out from the server logs.

“And that’s all I’ve got to say about that” – Forrest Gump 🙂

For now. 🙂

Linux – Bash script for complete system backup on external hard drive using rsync

*2022 Update. As this was written some time ago and time flies, some minor changes to the script are needed. Our old friend udisks is now depreciated so instead it’s successor udisksctl has to be used. You can comment the line and add a new one /I corrected the script here this way/ or just replace it with the new one.*

Now before you read this article I strongly advise you to read those once first if you think you are not too familiar with this subject.  The first one is located HERE and is a very good explanation on how to use rsync for full OS backup. The second one is a detailed explanation on backing up data on to external USB drive by Bobulous, which you can find HERE. The script in this article is a modified version of the original script created by Bobulous as I needed to achieve few other things as well.

It is kind of important to get your fstab file organized first. Do not forget to modify it so your external hard drive gets the same mount point every time it is mounted. For this you will have to find out what your drive UUID is. As an example, the line in my fstab file look like:

UUID=ff0ba5cf-dc79-4b96-a86b-a64f26154232 /mnt/sync ext4 user,noauto,noatime     0     0

As I wanted the drive to be used only for the purpose of system backup I did not want it working all the time, so I wanted not just to unmount, but eject it as well and switch it off after use. This is where udisks utility comes handy, if it is not installed on your system – you will have to install it so the script works as it should. But udisks does not use the UUID to handle the drive, so the correct device file mapping has to be found.

The first thing is to extract the UUID, this is done in variable declaration:

uuid=`cat /etc/fstab | grep ${mount_point} | cut -c 6-41`

Before using the script make sure the result of the commands is the UUID of your drive, take care of cut command and adjust it if the output is not exactly what you are expecting. Or instead of cut you can use sed command, which might be a bit more universal:

uuid=`cat /etc/fstab | grep ${mount_point} | sed -r ‘s_.*=(.*) ${mount_point}.*_\1_’`

though I haven’t tried it myself.

After the disk is successfully mounted the declaration of HDID takes care of finding the correct device file:

HDID=`ls -l /dev/disk/by-uuid/${uuid} | sed -r ‘ s_^.* -> ../../(…).$_\1_’`

If you are like me and find sed command often confusing – there is a great tutorial about it by Bruce Barnett.

The actual backup work is done by rsync command, but you may want to adjust it depending on your Linux distribution, I run it on Debian 7 and it works fine for me.

And this is the script itself, you can modify it to suit your needs:

#!/bin/bash

# Script to backup full ystem to the external USB drive.

# Specify the mount point here (DO NOT end mount_point with a forward-slash).

mount_point=’/mnt/sync’

 

# Set the backup path here (DO NOT end mount_point with a forward-slash).

backup_path=${mount_point}

 

#Find UUID by mount point

uuid=`cat /etc/fstab | grep ${mount_point} | cut -c 6-41`

 

echo “#####”

echo “”

# Check whether target volume is mounted, and mount it if not.

if ! mountpoint -q ${mount_point}/; then

                echo “Mounting the external USB drive.”

                echo “Mountpoint is ${mount_point}”

                if ! mount ${mount_point}; then

                                echo “An error code was returned by mount command!”

                                exit 5

                else echo “Mounted successfully.”

                     HDID=`ls -l /dev/disk/by-uuid/${uuid} | sed -r ‘ s_^.* -> ../../(…).$_\1_’`

                     echo “${HDID} is the backup drive ID”;

                fi

else echo “${mount_point} is already mounted.”;

fi

# Target volume **must** be mounted by this point. If not, die screaming.

if ! mountpoint -q ${mount_point}/; then

                echo “Mounting failed! Cannot run backup without backup volume!”

                exit 1

fi

 

echo “Preparing to transfer differences using rsync.”

 

echo “Backup storage directory path is ${backup_path}”

 

echo “Starting backup of / . . . “

 

rsync -aAXHhv /* ${backup_path}/ –progress –delete –exclude={/dev/*,/proc/*,/sys/*,/tmp/*,/run/*,/mnt/*,/media/*,/lost+found}

 

# Ask user whether target volume should be unmounted.

echo -n “Do you want to unmount ${mount_point} and eject the drive (no)”

read -p “: ” unmount_answer

unmount_answer=${unmount_answer,,}  # make lowercase

if [ “$unmount_answer” == “y” ] || [ “$unmount_answer” == “yes” ]; then

                if ! umount ${mount_point}; then

                                echo “An error code was returned by command!”

                                exit 5

                else echo “Dismounted successfully.”

#                     udisks –detach /dev/${HDID}

udisksctl power-off -b /dev/${HDID}

                     echo “Disk ejected.”;

                fi

else echo “Volume remains mounted.”;

fi

 

echo “”

echo “####”