====== LXC - Linux Container Tools ======
[[https://linuxcontainers.org/lxc/|LXC]] is a tool to create and manage containers. It contains a full featured container with the isolation / virtualization of the pids, the ipc, the utsname, the mount points, /proc, /sys, the network and it takes into account the control groups. It is very light, flexible, and provides a set of tools around the container like the monitoring with asynchronous events notification, or the freeze of the container. This package is useful to create Virtual Private Server, or to run isolated applications like bash or sshd.
LXC is pretty low level, very flexible and covers just about every containment feature supported by the upstream kernel. For a completely fresh and intuitive user experience with a single command line tool to manage your containers see [[LXD]].
**Resources**
* [[https://www.stgraber.org/2013/12/20/lxc-1-0-blog-post-series/|LXC 1.0 blog post series]] - must read to get quick overview what's out there
* [[http://lists.linuxfoundation.org/mailman/listinfo/containers|Linux Containers mailing list]]
* [[http://wiki.gentoo.org/wiki/LXC|Gentoo wiki about LXC]]
* [[http://www.funtoo.org/Linux_Containers|Funtoo LXC pages]]
* [[https://wiki.archlinux.org/index.php/Linux_Containers#Container_setup|Arch Linux documentation]]
* [[http://en.opensuse.org/LXC#Container_Setup|OpenSUSE documentation]]
* [[https://help.ubuntu.com/12.04/serverguide/lxc.html|Ubuntu Server Guide]]
* [[http://wiki.alpinelinux.org/wiki/LXC|Alpine Linux Docs]]
* [[http://libvirt.org/drvlxc.html|libvirt LXC driver]]
* [[http://www.freedesktop.org/wiki/Software/systemd/ContainerInterface/|Systemd Container Interface]]
===== Prerequisites =====
* 3.8+ kernel [[http://docs.docker.io/en/latest/installation/kernel/|according]] to docker devs
* [[package>lxc]] package
* cgroups mounted, use [[package>systemd]] or [[package>libcgroup]] for that (edit and enable most groups in ///etc/cgconfig.conf// except debug)
Kernels with vserver support compiled in, do not work correctly with LXC
* [[http://lists.pld-linux.org/mailman/pipermail/pld-devel-en/2014-January/thread.html#23786]]
* http://www.paul.sladen.org/vserver/archives/201402/0015.html
===== Guest creation =====
Build the guest container.
==== Bare minimum, no template ====
# lxc-create -n test
lxc-create: No config file specified, using the default config /etc/lxc/default.conf
'test' created
# lxc-ls --fancy (install python3-lxc for lxc-ls)
NAME STATE IPV4 IPV6
-----------------------------------
test STOPPED -
this just creates ''$LXC_ROOT/test'' directory with ''config'' copied from ''/etc/lxc/default.conf'' inside.
==== PLD Linux from template ====
create guest with default config using pld template:
# lxc-create -n test -t pld
NOTYET:
There are two versions of PLD available for guest systems:
* ac - [[:ac|PLD 2.0 (Ac)]]
* th - [[:th|PLD 3.0 (Th)]]
You may choose one using ''-R'' option:
# lxc-create -t pld-test -p pld -f network-configuration-file -- -R th
!!! WARNING: pld template for LXC is yet to be written !!!
===== Common problems / Useful tricks =====
==== lxc-start has no output ====
In case ''lxc-start -n test'' produces no output, ensure /dev/console is present in guest filesystem.
==== lxc-stop is not graceful ====
Currently ''lxc-stop -n test'' sends ''SIGPWR'' to init inside container, but ''rc-scripts'' fails to shutdown things properly (shutdown scripts are not invoked). For workaround, stop services manually before issueing ''lxc-stop'' or run ''poweroff''/''halt''/''reboot'' from container.
Details: In process table is only this process runrning, no further actions from ''rc-scripts'':
/sbin/shutdown -f -h +2 Power Failure; System Shutting Down
==== loginuid ====
having ''audit_control'' dropped:
lxc.cap.drop = audit_control
''pam_loginuid.so'' does not allow ''sshd'' to login:
Nov 24 16:02:10 test sshd[2694]: error: PAM: pam_open_session(): Cannot make/remove an entry for the specified session
You can either [[http://kb.parallels.com/en/112597|workaround]] to disable ''pam_loginuid.so'' in the authentication rules:
# sed '/pam_loginuid.so/s/^/#/g' -i /etc/pam.d/*
Or just **do not** drop the capability.
==== syslog ====
[[package>syslog-ng]] gives following on startup:
# service syslog-ng restart
syslog-ng: Error setting capabilities, capability management disabled; error='Operation not permitted'
Stopping syslog-ng service.............................................................[ DONE ]
Starting syslog-ng service.............................................................[ DONE ]
**FIXME:** no solution yet
===== Vserver comparision =====
When in Vserver, guest processes are not visible in host, then in LXC all guest processes are visible. Beware when running ''killall(1)'' commands on host.
Also, unfortunately ''/proc/PID/root'' points to ''/'' for LXC guests as well, so ''rc-scripts'' ''filter_chroot()'' can't differentiate between host and guest processes.
Also, ''dmesg(1)'' in guest sees hosts' dmesg by default, you can turn this off by setting ''kernel.dmesg_restrict=1'' sysctl param, available since ''2.6.37'' kernel.
**Commands:**
^ Vserver ^ LXC ^ Notes ^
| vserver test enter | lxc-attach -n test | Add ''-e'' to enter with elevated privileges (ignoring ''lxc.cap.drop'') |
| vserver test start | lxc-start -n test -d |
| vserver test stop | lxc-stop -n test |
| vserver-stat | %%lxc-ls --fancy --running%% | you need ''python3-lxc'' installed for this tool |
===== Network configs =====
==== general ====
static networking, set ''VSERVER=yes'' and ''VSERVER_ISOLATION_NET=yes'' in guest ''/etc/sysconfig/system'' to disable all network configuration by guest, set RC_PROMPT=no to avoid hanging startup scripts, in general it's good idea to turn off there most of things
==== network using macvlan in bridge mode ====
- traffic from host to guest (and vice-versa) is NOT passed. external trafic works
- guest interface is NOT visible on host
- you can't filter guest straffic from host's firewall
- host can use seme default interface with and without guests running.
- one have better to set static MAC address. If not - on every container start you'll have different MAC generated and your router may have problems with passing traffic.
- iptables is initialized from lxc.hook.pre-mount hook (ran in the container's namespace and having guest macvlan interface visible)
first boot with ''hwaddr'' line disabled, look what the random address was assigned, set it in config.
also you may use some generation techniques like these: using last three ip numbers and [[http://xenbits.xen.org/docs/4.3-testing/misc/xl-network-configuration.html|Xen's OUI (00:16:3e)]] address space. If IP is ''192.168.2.160'', then:
$ printf "00:16:3e:%x:%x:%x" 168 2 160
00:16:3e:a8:2:a0
lxc.network.type = macvlan
lxc.network.flags = up
#lxc.network.hwaddr = 00:16:c0:a8:3:34
lxc.network.link = eth0
lxc.network.macvlan.mode = bridge
lxc.network.name = eth0
lxc.network.ipv4 = 192.168.2.160/23
lxc.network.ipv4.gateway = 192.168.2.1
lxc.hook.pre-mount = /sbin/service iptables start
lxc.cap.drop = net_admin
==== network using bridged veth interfaces ====
==== More raeding about network ====
[[http://containerops.org/2013/11/19/lxc-networking/|Elaborate article about configuring different types of network ]]
===== Sample configs =====
==== full config ====
# lxc for test
lxc.network.type = macvlan
lxc.network.flags = up
#lxc.network.hwaddr = 00:16:c0:a8:3:34
lxc.network.link = eth0
lxc.network.macvlan.mode = bridge
lxc.network.name = eth0
lxc.network.ipv4 = 192.168.2.160/23
lxc.network.ipv4.gateway = 192.168.2.1
lxc.rootfs = /srv/test
lxc.utsname = pldmachine.local
lxc.tty = 4
lxc.pts = 1024
# load iptables, if you want to setup firewall when container is already up
# you should run 'lxc-attach -e -n test -- service iptables start'
lxc.hook.pre-mount = /sbin/service iptables start
# lxc.mount.entry is prefered, because it supports relative paths
lxc.mount = /var/lib/lxc/test/fstab
lxc.cap.drop = linux_immutable
#lxc.cap.drop = sys_boot # works as expected in newer kernels (3.4+)
lxc.cap.drop = syslog
# don't drop net_admin, allows firewall to be configured from inside
lxc.cap.drop = net_admin
# http://www.funtoo.org/Linux_Containers
## Capabilities, see capabilities(7) what is available
#lxc.cap.drop = audit_control
lxc.cap.drop = audit_write
lxc.cap.drop = mac_admin
lxc.cap.drop = mac_override
lxc.cap.drop = mknod
lxc.cap.drop = setfcap
lxc.cap.drop = setpcap
lxc.cap.drop = sys_admin
#lxc.cap.drop = sys_boot
#lxc.cap.drop = sys_chroot # required by SSH
lxc.cap.drop = sys_module
#lxc.cap.drop = sys_nice
lxc.cap.drop = sys_pacct
lxc.cap.drop = sys_rawio
lxc.cap.drop = sys_resource
lxc.cap.drop = sys_time
#lxc.cap.drop = sys_tty_config # required by getty
lxc.autodev = 0
# When using LXC with apparmor, uncomment the next line to run unconfined:
lxc.aa_profile = unconfined
# cgroups
# Devices
lxc.cgroup.devices.deny = a # Deny access to all devices
# /dev/null and zero
lxc.cgroup.devices.allow = c 1:3 rwm
lxc.cgroup.devices.allow = c 1:5 rwm
# consoles
lxc.cgroup.devices.allow = c 5:1 rwm
lxc.cgroup.devices.allow = c 5:0 rwm
lxc.cgroup.devices.allow = c 4:0 rwm
lxc.cgroup.devices.allow = c 4:1 rwm
# /dev/{,u}random
lxc.cgroup.devices.allow = c 1:9 rwm
lxc.cgroup.devices.allow = c 1:8 rwm
lxc.cgroup.devices.allow = c 136:* rwm
lxc.cgroup.devices.allow = c 5:2 rwm
# rtc
lxc.cgroup.devices.allow = c 254:0 rm