Virtualisation

From Things and Stuff Wiki
Revision as of 07:17, 20 February 2016 by Milk (talk | contribs) (→‎Xen)
Jump to navigation Jump to search


General

Hardware level

QEMU

KVM

  • KVM (for Kernel-based Virtual Machine) is a full virtualization solution for Linux on x86 hardware containing virtualization extensions (Intel VT or AMD-V). It consists of a loadable kernel module, kvm.ko, that provides the core virtualization infrastructure and a processor specific module, kvm-intel.ko or kvm-amd.ko. KVM also requires a modified QEMU although work is underway to get the required changes upstream. Using KVM, one can run multiple virtual machines running unmodified Linux or Windows images. Each virtual machine has private virtualized hardware: a network card, disk, graphics adapter, etc. 
grep -E "(vmx|svm)" --color=always /proc/cpuinfo

If nothing is displayed after running that command, then your processor does not support hardware virtualization, and you will not be able to use KVM.

Xen

  • Xen Hypervisor is an open source virtualization platform that powers the world's largest clouds in production and is the foundation of many commercial products. Xen powers public clouds such as Amazon Web Services, Rackspace Public Cloud and many others. Examples of Xen based server products include Huawei UVP, Oracle VM and XenServer. Examples of client products and appliances include QubesOS, XenClient and Netscaler. Xen is 9 years old, mature and its stability and versatility is second to none.

VMWare

VirtualBox

VirtualBox config;

  • Enable host Linux modules: vboxnetflt and vboxnetadp
  • Enable host-only adapter 'virtualbox0' in VB Preferences > Network if not previously enabled, then select in guest preferences
  • Keep VM names lowercase and no spaces for easier command-line stuff

Install guest additions for mouse, etc.

Provisioning VirtualBoxes for local dev work;

Vargrant

  • Vagrant uses Oracle’s VirtualBox to build configurable, lightweight, and portable virtual machines dynamically. The first couple pages serve to introduce you to Vagrant and what it has to offer while the rest of the guide is a technical walkthrough for building a fully functional web development environment. The getting started guide concludes by explaining how to package the newly created vagrant environment so other developers can get up and running in just a couple commands.
vagrant package --vagrantfile Vagrantfile.pkg --include README.txt
  stored in ~/.vagrant.d/boxes
vagrant box add lucid32 http://files.vagrantup.com/lucid32.box
vagrant box remove lucid32
vagrant box list
  • Veewee is a tool for easily (and repeatedly) building custom Vagrant base boxes, KVMs, and virtual machine images.
  • PuPHPet - A simple GUI to set up virtual machines for PHP Web development.

Other

  • bhyve - the "BSD hypervisor" is a legacy-free hypervisor/virtual machine manager developed on FreeBSD and relies on modern CPU features such as Extended Page Tables (EPT) and VirtIO network and storage drivers. bhyve was officially released on January 20th, 2014 as part of FreeBSD 10.0.

Operating system level

  • Operating system-level virtualization is a server virtualization method where the kernel of an operating system allows for multiple isolated user-space instances, instead of just one. Such instances (often called containers, VEs, VPSs or jails) may look and feel like a real server, from the point of view of its owner. On Unix systems, this technology can be thought of as an advanced implementation of the standard chroot mechanism. In addition to isolation mechanisms, the kernel often provides resource management features to limit the impact of one container's activities on the other containers.
  • http://en.wikipedia.org/wiki/Comparison_of_platform_virtual_machines

chroot

FreeBSD jail

User-mode Linux

LinuX Containers

  • lmctfy is the open source version of Google’s container stack, which provides Linux application containers.
  • warden - Manages isolated, ephemeral, and resource controlled environments.
  • wsh - execute command in a Linux Container through unix socket
  • Pipework lets you connect together containers in arbitrarily complex scenarios. Pipework works with "plain" LXC containers (created with lxc-start), and therefore, it also works with the awesome Docker.


Docker

Setup

sysctl net.ipv4.ip_forward=1
  enable network forwarding

net.ipv4.ip_forward=1
  persistent network forwarding, goes in /etc/sysctl.d/docker.conf

sudo <path to>/docker -d &
   start docker in daemon mode

sudo systemctl enable docker
  start docker as service

sudo systemctl start docker
  start on system boot

ls -lah /var/run/docker.sock
  exists when docker is running

Notes

Add user to docker group to avoid sudo.

Docker isn't screen/tmux friendly due to tty handling. Easy fix is to ssh in to the instance.

sshd probably doesn't start by default. tis in root path. 

/usr/sbin/sshd

If you are behind a proxy you can launch docker with this command 

sudo HTTP_PROXY=http://youruserproxy:yourpassword@theaddressproxy:theportproxy/ docker -d &

Commands

# Download an ubuntu image
docker pull [name]

docker run [OPTIONS] IMAGE[:TAG] [COMMAND] [ARG...]
docker run ubuntu uname -a

# Run an interactive shell in the ubuntu image,
# allocate a tty, attach stdin and stdout
# To detach the tty without exiting the shell,
# use the escape sequence Ctrl-p + Ctrl-q
docker run -i -t ubuntu /bin/bash

# Bind TCP port 8080 of the container to TCP port 80 on 127.0.0.1 of the host machine.
docker run -p 127.0.0.1:80:8080 <image> <cmd>

# Bind TCP port 8080 of the container to a dynamically allocated TCP port on 127.0.0.1 of the host machine.
docker run -p 127.0.0.1::8080 <image> <cmd>

# Bind TCP port 8080 of the container to TCP port 80 on all available interfaces of the host machine.
docker run -p 80:8080 <image> <cmd>

# Bind TCP port 8080 of the container to a dynamically allocated TCP port on all available interfaces of the host machine.
docker run -p 8080 <image> <cmd>

# List your containers
docker images

# Listing all running containers
docker ps

Running docker command returns container ID 

# run a process which echoes 'hello world' in every second
CONTAINER_ID=$(sudo docker run -d ubuntu /bin/sh -c "while true; do echo hello world; sleep 1; done")

# attach the console to the container
docker attach $CONTAINER_ID

# stop an active container
docker stop $CONTAINER_ID

# kill an active container
docker kill $CONTAINER_ID

# Commit your container to a new named image
docker commit <container_id> <some_name>

Guides

Extra

  • boot2docker is a lightweight Linux distribution based on Tiny Core Linux made specifically to run Docker containers. It runs completely from RAM, weights ~38mb and boots in ~5-6s (YMMV).
  • dockerlite lets you run Linux apps in lightweight, isolated environments, using LXC (Linux Containers). Using BTRFS snapshots, dockerlite can save the state of a given environment in a frozen "image", and later, create more environments ("containers") out of that image.
  • DockerUI is a web interface to interact with the Remote API. The goal is to provide a pure client side implementation so it is effortless to connect and manage docker.
  • Docker Desktop - Dockerfile that creates a docker image which, once executed, creates a container that runs X11 and SSH services. The ssh is used to forward X11 and provide you encrypted data communication between the docker container and your local machine. Xpra + Xephyr allows to display the applications running inside of the container such as Firefox, LibreOffice, xterm, etc. with recovery connection capabilities.
  • Autodock is a docker automation tool. It automatically sorts servers in your Docker cluster by lowest load. It then distributes the containers you want to create among them. After bootstrapping the containers with Paramiko and Salt it saves this information to the ETCD cluster. Autodock was designed to use saltstack and etcd for configuration management and replication. [14]
  • Fig - Fast, isolated development environments using Docker. [15]
  • Drone is a Continuous Integration platform built on Docker [16]


Toolbox

Linux-VServer

  • Linux-VServer provides virtualization for GNU/Linux systems. This is accomplished by kernel level isolation. It allows to run multiple virtual units at once. Those units are sufficiently isolated to guarantee the required security, but utilize available resources efficiently, as they run on the same kernel.

OpenVZ

  • OpenVZ is container-based virtualization for Linux. OpenVZ creates multiple secure, isolated Linux containers (otherwise known as VEs or VPSs) on a single physical server enabling better server utilization and ensuring that applications do not conflict. Each container performs and executes exactly like a stand-alone server; a container can be rebooted independently and have root access, users, IP addresses, memory, processes, files, applications, system libraries and configuration files.

CoreOS

  • CoreOS is Linux for the container world. Linux kernel + systemd. That's about it. CoreOS has just enough bits to run containers, but does not ship a package manager itself. In fact, the root partition is completely read-only, to guarantee consistency and make updates reliable. Use docker as a package manager to build and push your app. The primitive of an application is a container, not a one-off package. Build containers using docker, by hand, or however you see fit! [21]

Lguest

Solaris Containers / Zones

systemd-nspawm

Other

  • bhyve - the "BSD hypervisor" is a hypervisor/virtual machine manager developed on FreeBSD and relies on modern CPU features such as Extended Page Tables (EPT) and VirtIO network and storage drivers. [22]
  • runC - a CLI tool for spawning and running containers according to the OCP specification. The code can be found on Github. Containers are started as a child process of runC and can be embedded into various other systems without having to run a Docker daemon. runC is built on libcontainer, the same container technology powering millions of Docker Engine installations. Docker images can be run with runC. [23]
  • ZeroVM is an open source virtualization technology that is based on the Chromium Native Client (NaCl) project. ZeroVM creates a secure and isolated execution environment which can run a single thread or application. ZeroVM is designed to be lightweight, portable, and can easily be embedded inside of existing storage systems. This functionality allows developers to push their application to their data instead of having to pull their data to their application, as is the case with more traditional architectures.
  • Jailhouse is a partitioning Hypervisor based on Linux. It is able to run bare-metal applications or (adapted) operating systems besides Linux. For this purpose it configures CPU and device virtualization features of the hardware platform in a way that none of these domains, called "cells" here, can interfere with each other in an unacceptable way. Jailhouse is optimized for simplicity rather than feature richness. Unlike full-featured Linux-based hypervisors like KVM or Xen, Jailhouse does not support overcommitment of resources like CPUs, RAM or devices. It performs no scheduling and only virtualizes those resources in software, that are essential for a platform and cannot be partitioned in hardware.


  • Rump kernels enable you to build the software stack you need without forcing you to reinvent the wheels. The key observation is that a software stack needs driver-like components which are conventionally tightly-knit into operating systems — even if you do not desire the limitations and infrastructure overhead of a given OS, you do need drivers.

We solve the problem by providing free, reusable, componentized, kernel quality drivers such as file systems, POSIX system calls, PCI device drivers and TCP/IP and SCSI protocol stacks. As a production-ready example, we offer the Rumprun unikernel, which clocks in at a few thousand lines of code plus rump kernel components, and supports POSIX'y software directly on both raw hardware and cloud hypervisors such as KVM and Xen.

Desktop level

Virtual machines

  • Packer is a tool for creating identical machine images for multiple platforms from a single source configuration.

Disk images

Bitnami

ZeroVM

Networking

See also Network

Retrieved from "https://wiki.thingsandstuff.org/index.php?title=Virtualisation&oldid=34191"