Tag Archives: Linux

Create Windows 10 Kernel-based Virtual Machine (KVM) on Linux Debian server

There are many tutorials out there about how to create KVM, but not many go in to details on how to do it if your guest OS is Windows and also scares info on the problems you may encounter. This is why I decided to put in brief the steps how to do it and describe the problems I had to deal with.

This guide assumes you already have a working Debian 9 installation /the steps should in general work for Debian 10 as well/. Commands has to be executed as root.

Before you start is a good idea to check some hardware capabilities of the host computer, especially if it is not relatively new.

First you need to check if your CPU supports virtualisation, to do this run the command:

 

egrep -c ‘(vmx|svm)’ /proc/cpuinfo

 

If the output of the command is 1 or 2 you are good, 0 means no go for you.

Then you need to check if the hardware virtualisation is enabled and if not you need to change the settings in your BIOS. Run this command and read the output:

 

dmesg | grep “disabled by bios”

 

If the output is “kvm: disabled by bios” you need to alter the BIOS settings, otherwise you are fine.

When ready you need to install quite a lot of packages, so in your terminal type the command:

 

apt-get install –no-install-recommends qemu-kvm libvirt-daemon-system libvirt-clients libvirt-daemon-system libvirt-dev libguestfs-tools genisoimage virtinst libosinfo-bin virt-viewer virt-manager acpid

 

Be patient, it will take some time. The reason of –no-install-recommends switch is to avoid installation on packages related to GUI, which I personally avoid on server installations.

When the installation is finished you will have to change your network interfaces so the virtual machine becomes part of your LAN. For this you will have to create a bridge interface, make sure you have the bridge-utils package installed. If it happens to have a OpenVPN server already running on the Debian server as I did, make sure you change your VPN interface to TAP and add it to the bridge, otherwise you won’t be able to browse your network.

This is how my /etc/network/interfaces file looked like after the configuration change:

 

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
# allow-hotplug enp2s0
# iface enp2s0 inet static
# address 192.168.1.2
# netmask 255.255.255.0
# gateway 192.168.1.1

######################################
# bridge part, coment above int conf                      #
######################################

auto enp2s0
iface enp2s0 inet manual

auto tap0
iface tap0 inet manual

auto br0
iface br0 inet static
address 192.168.1.2
broadcast 192.168.1.255
netmask 255.255.255.0
gateway 192.168.1.1
bridge_ports enp2s0 tap0
bridge_stp off
bridge_waitport 0
bridge_fd 0
dns-nameservers 8.8.8.8
dns-nameservers 8.8.4.4
dns-nameservers 192.168.1.2

 

Restart the networking service (or reboot) and test your connectivity is working as intended.

Next step is to let regular user to manage the VM:

adduser your_user libvirt
adduser your_user libvirt-qemu

Then reload group membership:

newgrp libvirt
newgrp libvirt-qemu

Verify your group membership with id command:

id

 

Check virtual machine network and state

virsh net-list –all
virsh list –all

 

Bridge networking has to be configured for the VM as well, so do the following:

Create a file using a text editor

 

nano /root/bridged.xml

 

Append the following config in the file, make sure bridge interface name is the same as in your interfaces configuration:

 

<network>
<name>br0</name>
<forward mode=”bridge”/>
<bridge name=”br0″/>
</network>

 

Save and close the file. Then run the following commands to apply this configuration to the VM:

 

virsh net-define –file /root/bridged.xml
virsh net-autostart br0
virsh net-start br0

 

Veryfy bridged network

 

virsh net-list –all

 

In order Windows 10 to recognise your virtual hard disk during its installation you will need to get the vertio drivers. Create “virtio” (or some other name) folder in /var/lib/libvirt/boot/ and download the drivers (checking for the latest version is probably a good idea):

 

cd /var/lib/libvirt/boot/virtio/

wget https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/archive-virtio/virtio-win-0.1.173-2/virtio-win-0.1.173.iso

 

Next create the VM’s virtual disk, adjust to your needs:

qemu-img create -f qcow2 /var/lib/libvirt/images/windows_10_x64.qcow2 80G

 

With the disk set it is time to create the VM:

 

virt-install \
–virt-type=kvm \
–hvm \
–name=windows10 \
–ram=2048 \
–cpu=host \
–vcpus=2 \
–os-type=windows \
–os-variant=win10 \
–disk path=/var/lib/libvirt/images/windows_10_x64.qcow2,format=qcow2,bus=virtio \
–disk /var/lib/libvirt/boot/Win10_1909_EnglishInternational_x64.iso,device=cdrom,bus=ide \
–disk /var/lib/libvirt/boot/virtio/virtio-win-0.1.173.iso,device=cdrom,bus=ide \
–network=bridge=br0,model=virtio \
–graphics vnc

 

If stumble upon error stating that the access to the KVM kernel module is denied do the following:

Open the file qemu.conf and edit the following:

 

nano /etc/libvirt/qemu.conf

#user = root -> user = root
#group = “root” – > group = “kvm”

 

After a successful VM creation you will need a VNC client to connect to the VM and start the installation of Windows. In the installation process you will have to navigate to the folder containing the virtio disk driver, mind in my case it was drive E:, but may not be the same for you. The path for me was: E:\viostor\w10\amd64. After the installation you can run RDC on Windows and set users to connect to your VM directly without third party software.

To have your VM starting automatically when the host starts or is rebooted run the commands:

First make sure libvirtd service is stared on boot:

 

systemctl enable libvirtd

 

Then run:

 

virsh autostart Your_VM_Name_Here

 

If you  try to shutdown your VM from host terminal with the virsh shutdown VM_Name_Here without success this might be due to not working acpi event handler. To fix this edit the content of powerbtn file (create the file if does not exist):

nano /etc/acpi/events/powerbtn

 

Then type these lines (delete anything in the file if the file exists)

 

event=button/power
action=/sbin/poweroff

 

and restart the acpid service

 

service acpid restart

 

And this is the end of it in general, next is to post the script to shutdown, backup and then start the KVM when time is available.

Have fun!

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Advertisement

Bash script for complete system backup on external hard drive using tar

Some time ago I posted a script to backup Linux system to external drive using rsync. As the task this time was to create an archive of the whole system and the old backup files to be removed the script had to be changed a bit. Here is the version of it with the use of tar command. As before your external drive’s mount point has to be defined in fstab file.

And this is the script, it does not need much explanation:

 

#!/bin/bash

# Script to create full system archived backup to external USB drive.

# Specify the mount point, backup path and other stuff here (DO NOT end mount_point with a forward-slash).

mount_point=’/mnt/WD’
current_year=`date +%Y`
current_day=`date +’%Y-%m-%d_%H:%M:%S’`
backup_path=${mount_point}’/backup/’${current_year}
files_to_keep=2

echo “#####”
echo “”

# Check whether target volume is mounted, and mount it if not.

if ! mountpoint -q ${mount_point}/; then
   echo “Mounting the external USB drive.”
   echo “Mountpoint is ${mount_point}”
      if ! mount ${mount_point}; then
         echo “An error code was returned by mount command!”
         exit 5
      else echo “Mounted successfully.”
      fi
else echo “${mount_point} is already mounted.”;
fi

# Target volume **must** be mounted by this point.

if ! mountpoint -q ${mount_point}/; then
   echo “Mounting failed! Cannot run backup without backup volume!”
   exit 1
fi

echo “Preparing the archive using tar.”

mkdir –parents ${backup_path}/

tar czf ${backup_path}/${current_day}_backup.tar.gz –exclude=${backup_path}/${current_day}_backup.tar.gz –exclude=/dev/* –exclude=/mnt/* –exclude=/proc/* –exclude=/sys/* –exclude=/run/* –exclude=/media/* –exclude=/tmp/* –exclude=/lost+found /

# Delete old backup files

cd ${backup_path}/

rm -f `ls -t *_backup.tar.gz | awk “NR>$files_to_keep”`

cd /

# Dismounting target volume

if ! umount ${mount_point}; then
   echo “An error code was returned by command!”
   exit 5
else
echo “Dismounted successfully.”;
fi

 

And you are good to go! 🙂


GRUB bootloader repair with chroot

I had to do this quite often lately, so I thought it deserves going to the blog… 🙂 If you are having computers with multiple OS to boot sooner or later you will end up in a situation of having GRUB corrupt.

My latest adventure was after Windows 10 Anniversary update. The update messed up the hard drive partition table and on boot the computer ends up to GRUB’s rescue console. You can try the steps from this post and fix GRUB from its rescue console, though it did not work for me and you will notice at the end you still need to reinstall grub and recreate the boot menu:

http://www.legendiary.at/2016/01/04/windows-10-update-changes-partition-table-and-breaks-grub/

So I got one of my flash drives with live Linux on it and went for the well-known classic approach 🙂 Please mind your flash and hard disk Linux kernel architecture /32 or 64 bit/, they must match.

Boot your computer of the flash drive, run a terminal, make sure you are root /or use sudo/ when execute the commands and follow these steps:

  1. List your drives and partitions with a command of your choice and make a note of the Linux partition

 

          fdisk –l

 

  1. Mount your Linux partition to a folder and link required system folders to their alternatives in that folder. Substitute sdXY below with the correct Linux partition for your system.*

 

          mount /dev/sdXY /mnt

 

          mount –bind /dev /mnt/dev

          mount –bind /dev/pts /mnt/dev/pts

          mount –bind /proc /mnt/proc

          mount –bind /sys /mnt/sys

 

  1. Use chroot to change your root directory.

 

          chroot /mnt

 

  1. Recreate GRUB menu and install it on the boot drive. This commands are for Debian and its family, for other distributions you have to use the correct commands.

 

          update-grub

 

          grub-install /dev/sdX

 

Reboot your computer and enjoy the boot menu you are used to. 🙂

*You can substitute the multiple mount commands with a single line if you wish to:

          for i in /dev /dev/pts /proc /sys; do mount -B $i /mnt$i; done

** Just a quick addition to the post to bring it up to current times. If your happen to have a EFI bootloader you will have to mount the EFI partition as well, like this:

mount /dev/sdXZ /mnt/boot/efi


Debian 7 Wheezy – L2TP VPN Server behind NAT with strongSwan and self-signed certificate authentication

As usual before everything else a few good and must read articles on the subject: https://www.zeitgeist.se/2013/11/22/strongswan-howto-create-your-own-vpn/ – a very good tutorial on setting up strongSwan, http://www.manabii.info/2014/08/debian-wheezy-strongswan-l2tp-ipsec.html – another very good tutorial on L2TP strongSwan configuration /you will have to translate it from Japan unless you are a ninja :-)/ and of course the very important strongSwan wiki documentation at https://wiki.strongswan.org/projects/strongswan.

Why strongSwan and L2TP? Well after some research I have done on the available IPsec implementations for Linux I have decided to go for strongSwan because of its active development. As for L2TP I decided that a bit of overhead, which any modern hardware can handle the capability to use other protocols as well as IP over the VPN connection is worth the hassle.

 

STEP 1 – GENERAL PREPARATION

The assumption is you already have control over a DNS server, local network in range of 192.168.0.0/24, a server running Debian 7 Wheezy  with IP address 192.168.0.10 and as a good practice advises – first make sure your system is up to date, just run /do not forget to be a root user or use sudo when necessary/:

#apt-get update && apt-get upgrade

Normally Debian 7 will install strongSwan 4, but I wanted version 5 because it only runs the charon daemon which handles everything for you and you do not have to configure NAT-T – it is triggered for you automatically if needed. To do this you will have to alter the /etc/apt/sources.list file adding those lines in it:

deb http://ftp.debian.org/debian/ wheezy-backports main
deb-src http://ftp.debian.org/debian/ wheezy-backports main

After that you can run the set of commands which will update the sources, install the xl2tpd daemon /this is the software responsible for L2TP/ and install strongSwan:

#apt-get update
#apt-get install xl2tpd
#apt-get -t wheezy-backports install strongswan

Just to check you have the version 5 of strongSwan:

#dpkg -l | grep strong

The next important thing is to change the kernel parameter so it can route the VPN traffic /later on NAT will be configured/. To achieve this in a permanent way some parameters in /etc/sysctrl.conf must be set properly, you will need to have this:

net.ipv4.ip_forward = 1
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.all.accept_redirects = 0

To apply the changes run the following command:

#sysctl –p

 

STEP 2 – CERTIFICATES PREPARATION

Before generating certificates find out the fully qualified domain name of your VPN server, make sure an A record for it exist in your DNS server and use it as CN when generating certificates. To find your FQDN run the fallowing command:

#hostname –f

Now you are ready to generate the certificate authority key and certificate:

#cd /etc/ipsec.d/
#ipsec pki –gen –type rsa –size 4096 \
–outform pem \
> private/CA_Key.pem

#chmod 600 private/CA_Key.pem

#ipsec pki –self –ca –lifetime 3650 \
–in private/CA_Key.pem –type rsa \
–dn “C=GB, O=SomeName, CN=server.example.com” \
–outform pem \
> cacerts/CA_Cert.pem

 

Create the VPN Server Key and Certificate

#ipsec pki –gen –type rsa –size 2048 \
–outform pem \
> private/VPN_Server_Key.pem

#chmod 600 private/VPN_Server_Key.pem

#ipsec pki –pub –in private/VPN_Server_Key.pem –type rsa | \
ipsec pki –issue –lifetime 3650 \
–cacert cacerts/CA_Cert.pem \
–cakey private/CA_Key.pem \
–dn ” C=GB, O=SomeName, CN=server.example.com ” \
–san server.example.com \
–flag serverAuth –flag ikeIntermediate \
–outform pem > certs/VPN_Server_Cert.pem

 

Create Client Certificate

#ipsec pki –gen –type rsa –size 2048 \
–outform pem \
> private/ClientKey.pem

#chmod 600 private/ClientKey.pem

#ipsec pki –pub –in private/ClientKey.pem –type rsa | \
ipsec pki –issue –lifetime 3650 \
–cacert cacerts/CA_Cert.pem \
–cakey private/CA_Key.pem \
–dn ” C=GB, O=SomeName, CN=server.example.com ” \
–san user@example.com \
–outform pem > certs/ClientCert.pem

 

Export Certificate for the Client as PKCS#12

#openssl pkcs12 -export -inkey private/ClientKey.pem \
-in certs/ClientCert.pem -name “Client VPN Certificate” \
-certfile cacerts/CA_Cert.pem \
-caname “server.example.com” \
-out Client.p12

 

STEP 3 – SETTING UP srongSwan

There are few files used to configure strongSwan all located in /etc folder. The file /etc/strongswan.conf I left unchanged as it probably is set correctly for most users, but of course you can check its content and if you are sure you know what you are doing adjust it to your needs.
Next is the file /etc/ipsec.conf, which you can backup first before changing:

#cp /etc/ipsec.conf /etc/ipsec.conf.bac

Then you can create the configuration you need, mine looks like:

# ipsec.conf – strongSwan IPsec configuration file

# basic configuration

config setup
#       strictcrlpolicy=yes
#       uniqueids = no
charondebug=”cfg 2, dmn 2, ike 2, net 2″

# Add connections here.
conn L2TP
forceencaps=yes
auto=add
keyexchange=ikev1
keyingtries=3
rekey=no
ikelifetime=8h
lifetime=1h
type=transport
left=192.168.0.10
leftsubnet=0.0.0.0/0[udp/1701]
leftauth=pubkey
leftcert=VPN_Server_Cert.pem
right=%any
rightsubnet=0.0.0.0/0[udp/%any]
rightauth=pubkey
rightcert=ClientCert.pem
dpddelay=40
dpdtimeout=130
dpdaction=clear

include /var/lib/strongswan/ipsec.conf.inc

And finally the file /etc/ipsec.secrets has to be set so the client can log in. Mine is set as follows:

: RSA VPN_Server_Key.pem
#user1 : EAP “BigSecret”
#user2 : XAUTH “AnotherBigSecret”

It is not a bad idea to set the permissions:

#chmod 600 /etc/ipsec.secrets

To make sure ipsec service loads on boot:

#insserv ipsec

 

STEP 4 – SETTING UP L2TP

Three files must be configured so xl2tpd daemon to work properly. First one is /etc/xl2tpd/xl2tpd.conf where the global parameters are set and in my case it is like that:

[global]
[lns default]
ip range = 192.168.111.2 – 192.168.111.254
local ip = 192.168.111.1
length bit = yes
refuse chap = yes
require authentication = yes
name = l2tp
pppoptfile = /etc/ppp/l2tpd-options

The next one is obviously the file /etc/ppp/l2tpd-options where you set some PPP options and mine are set as:

name l2tp
refuse-pap
refuse-chap
refuse-mschap
require-mschap-v2
ms-dns 8.8.8.8
ms-dns 8.8.4.4
nodefaultroute
lock
nobsdcomp
mtu 1280
mru 1280

And finally you set the PPP user in the file /etc/ppp/chap-secrets:

# client    server  secret      IP addresses
vpnuser    l2tp    “aPassword”  *

 

STEP 5 – SETTING UP NAT

We are almost there now. The final bit is to set up the NAT, which is done with iptables, but before that I recon it is not a bad idea to install the iptables-persistent package and have your rules saved and loaded properly on reboot. To do this you will have to:

#apt-get install iptables-persistent

Then you can create the rule and have it saved. To go for SNAT or Masquerading is a personal choice, but as you most likely will have a static IP address for your server probably SNAT is better:

#iptables -t nat -A POSTROUTING -j SNAT –to-source 192.168.0.10 -o eth+
#iptables-save > /etc/iptables/rules.v4

To check if your services are running you can see their status or/and network activity:

#service –status-all

or

#netstat -lnput | grep charon
#netstat -lnput | grep xl2tpd

Do not forget to forward UDP ports 500, 1701 and 4500 on your router.

And now the funny part…  Connecting your clients. Android devices connect effortlessly, just install your client certificate using the p12 file, put your PPP username and password as well as the server IP address /which will be the public address of your router/ or its DNS name /should resolve to your router’s public address/ and you are done.
Have not tried iOS device yet, but as Mac OS X connects I would assume there will not be a problem, though it was a bit tricky with the Mac. Make sure you install the client certificate in your System key chain and then make it trusted. Make sure you have a record in your DNS server with exactly the same name of your VPN server as it is in the certificate and when setting the connection do not use the IP address, use the domain name. If you do not do it this way you will not be able to connect.
Windows… well I could not manage to connect even a single Windows 7 client PC regardless of what I tried – switching off firewall and antivirus software, changing the configuration of the VPN server, had the certificates properly installed with MMC, and before you ask – yes, I have had the AssumeUDPEncapsulationContextOnSendRule in the registry set to 2. Nothing worked. Had a brief look at third party VPN clients for Windows, but could not find a good enough to support L2TP VPN with certificates. So if anyone have experience with this I will be happy to see what is wrong and where, I could not figure it out from the server logs.

“And that’s all I’ve got to say about that” – Forrest Gump 🙂

For now. 🙂


Linux – Bash script for complete system backup on external hard drive using rsync

*2022 Update. As this was written some time ago and time flies, some minor changes to the script are needed. Our old friend udisks is now depreciated so instead it’s successor udisksctl has to be used. You can comment the line and add a new one /I corrected the script here this way/ or just replace it with the new one.*

Now before you read this article I strongly advise you to read those once first if you think you are not too familiar with this subject.  The first one is located HERE and is a very good explanation on how to use rsync for full OS backup. The second one is a detailed explanation on backing up data on to external USB drive by Bobulous, which you can find HERE. The script in this article is a modified version of the original script created by Bobulous as I needed to achieve few other things as well.

It is kind of important to get your fstab file organized first. Do not forget to modify it so your external hard drive gets the same mount point every time it is mounted. For this you will have to find out what your drive UUID is. As an example, the line in my fstab file look like:

UUID=ff0ba5cf-dc79-4b96-a86b-a64f26154232 /mnt/sync ext4 user,noauto,noatime     0     0

As I wanted the drive to be used only for the purpose of system backup I did not want it working all the time, so I wanted not just to unmount, but eject it as well and switch it off after use. This is where udisks utility comes handy, if it is not installed on your system – you will have to install it so the script works as it should. But udisks does not use the UUID to handle the drive, so the correct device file mapping has to be found.

The first thing is to extract the UUID, this is done in variable declaration:

uuid=`cat /etc/fstab | grep ${mount_point} | cut -c 6-41`

Before using the script make sure the result of the commands is the UUID of your drive, take care of cut command and adjust it if the output is not exactly what you are expecting. Or instead of cut you can use sed command, which might be a bit more universal:

uuid=`cat /etc/fstab | grep ${mount_point} | sed -r ‘s_.*=(.*) ${mount_point}.*_\1_’`

though I haven’t tried it myself.

After the disk is successfully mounted the declaration of HDID takes care of finding the correct device file:

HDID=`ls -l /dev/disk/by-uuid/${uuid} | sed -r ‘ s_^.* -> ../../(…).$_\1_’`

If you are like me and find sed command often confusing – there is a great tutorial about it by Bruce Barnett.

The actual backup work is done by rsync command, but you may want to adjust it depending on your Linux distribution, I run it on Debian 7 and it works fine for me.

And this is the script itself, you can modify it to suit your needs:

#!/bin/bash

# Script to backup full ystem to the external USB drive.

# Specify the mount point here (DO NOT end mount_point with a forward-slash).

mount_point=’/mnt/sync’

 

# Set the backup path here (DO NOT end mount_point with a forward-slash).

backup_path=${mount_point}

 

#Find UUID by mount point

uuid=`cat /etc/fstab | grep ${mount_point} | cut -c 6-41`

 

echo “#####”

echo “”

# Check whether target volume is mounted, and mount it if not.

if ! mountpoint -q ${mount_point}/; then

                echo “Mounting the external USB drive.”

                echo “Mountpoint is ${mount_point}”

                if ! mount ${mount_point}; then

                                echo “An error code was returned by mount command!”

                                exit 5

                else echo “Mounted successfully.”

                     HDID=`ls -l /dev/disk/by-uuid/${uuid} | sed -r ‘ s_^.* -> ../../(…).$_\1_’`

                     echo “${HDID} is the backup drive ID”;

                fi

else echo “${mount_point} is already mounted.”;

fi

# Target volume **must** be mounted by this point. If not, die screaming.

if ! mountpoint -q ${mount_point}/; then

                echo “Mounting failed! Cannot run backup without backup volume!”

                exit 1

fi

 

echo “Preparing to transfer differences using rsync.”

 

echo “Backup storage directory path is ${backup_path}”

 

echo “Starting backup of / . . . “

 

rsync -aAXHhv /* ${backup_path}/ –progress –delete –exclude={/dev/*,/proc/*,/sys/*,/tmp/*,/run/*,/mnt/*,/media/*,/lost+found}

 

# Ask user whether target volume should be unmounted.

echo -n “Do you want to unmount ${mount_point} and eject the drive (no)”

read -p “: ” unmount_answer

unmount_answer=${unmount_answer,,}  # make lowercase

if [ “$unmount_answer” == “y” ] || [ “$unmount_answer” == “yes” ]; then

                if ! umount ${mount_point}; then

                                echo “An error code was returned by command!”

                                exit 5

                else echo “Dismounted successfully.”

#                     udisks –detach /dev/${HDID}

udisksctl power-off -b /dev/${HDID}

                     echo “Disk ejected.”;

                fi

else echo “Volume remains mounted.”;

fi

 

echo “”

echo “####”