Table of Contents
LXC - Linux Container Tools
LXC is a tool to create and manage containers. It contains a full featured container with the isolation / virtualization of the pids, the ipc, the utsname, the mount points, /proc, /sys, the network and it takes into account the control groups. It is very light, flexible, and provides a set of tools around the container like the monitoring with asynchronous events notification, or the freeze of the container. This package is useful to create Virtual Private Server, or to run isolated applications like bash or sshd.
LXC is pretty low level, very flexible and covers just about every containment feature supported by the upstream kernel. For a completely fresh and intuitive user experience with a single command line tool to manage your containers see LXD.
Resources
- LXC 1.0 blog post series - must read to get quick overview what's out there
Prerequisites
Guest creation
Build the guest container.
Bare minimum, no template
# lxc-create -n test lxc-create: No config file specified, using the default config /etc/lxc/default.conf 'test' created # lxc-ls --fancy (install python3-lxc for lxc-ls) NAME STATE IPV4 IPV6 ----------------------------------- test STOPPED -
this just creates $LXC_ROOT/test directory with config copied from /etc/lxc/default.conf inside.
PLD Linux from template
create guest with default config using pld template:
# lxc-create -n test -t pld
NOTYET:
There are two versions of PLD available for guest systems:
- ac - PLD 2.0 (Ac)
- th - PLD 3.0 (Th)
You may choose one using -R option:
# lxc-create -t pld-test -p pld -f network-configuration-file -- -R th
!!! WARNING: pld template for LXC is yet to be written !!!
Common problems / Useful tricks
lxc-start has no output
In case lxc-start -n test produces no output, ensure /dev/console is present in guest filesystem.
lxc-stop is not graceful
Currently lxc-stop -n test sends SIGPWR to init inside container, but rc-scripts fails to shutdown things properly (shutdown scripts are not invoked). For workaround, stop services manually before issueing lxc-stop or run poweroff/halt/reboot from container.
Details: In process table is only this process runrning, no further actions from rc-scripts:
/sbin/shutdown -f -h +2 Power Failure; System Shutting Down
loginuid
having audit_control dropped:
- config
lxc.cap.drop = audit_control
pam_loginuid.so does not allow sshd to login:
Nov 24 16:02:10 test sshd[2694]: error: PAM: pam_open_session(): Cannot make/remove an entry for the specified session
You can either workaround to disable pam_loginuid.so in the authentication rules:
# sed '/pam_loginuid.so/s/^/#/g' -i /etc/pam.d/*
Or just do not drop the capability.
syslog
syslog-ng gives following on startup:
# service syslog-ng restart syslog-ng: Error setting capabilities, capability management disabled; error='Operation not permitted' Stopping syslog-ng service.............................................................[ DONE ] Starting syslog-ng service.............................................................[ DONE ]
: no solution yet
Vserver comparision
When in Vserver, guest processes are not visible in host, then in LXC all guest processes are visible. Beware when running killall(1) commands on host.
Also, unfortunately /proc/PID/root points to / for LXC guests as well, so rc-scripts filter_chroot() can't differentiate between host and guest processes.
Also, dmesg(1) in guest sees hosts' dmesg by default, you can turn this off by setting kernel.dmesg_restrict=1 sysctl param, available since 2.6.37 kernel.
Commands:
| Vserver | LXC | Notes |
|---|---|---|
| vserver test enter | lxc-attach -n test | Add -e to enter with elevated privileges (ignoring lxc.cap.drop) |
| vserver test start | lxc-start -n test -d | |
| vserver test stop | lxc-stop -n test | |
| vserver-stat | lxc-ls --fancy --running | you need python3-lxc installed for this tool |
Network configs
general
static networking, set VSERVER=yes and VSERVER_ISOLATION_NET=yes in guest /etc/sysconfig/system to disable all network configuration by guest, set RC_PROMPT=no to avoid hanging startup scripts, in general it's good idea to turn off there most of things
network using macvlan in bridge mode
- traffic from host to guest (and vice-versa) is NOT passed. external trafic works
- guest interface is NOT visible on host
- you can't filter guest straffic from host's firewall
- host can use seme default interface with and without guests running.
- one have better to set static MAC address. If not - on every container start you'll have different MAC generated and your router may have problems with passing traffic.
- iptables is initialized from lxc.hook.pre-mount hook (ran in the container's namespace and having guest macvlan interface visible)
first boot with hwaddr line disabled, look what the random address was assigned, set it in config.
also you may use some generation techniques like these: using last three ip numbers and Xen's OUI (00:16:3e) address space. If IP is 192.168.2.160, then:
$ printf "00:16:3e:%x:%x:%x" 168 2 160 00:16:3e:a8:2:a0
lxc.network.type = macvlan lxc.network.flags = up #lxc.network.hwaddr = 00:16:c0:a8:3:34 lxc.network.link = eth0 lxc.network.macvlan.mode = bridge lxc.network.name = eth0 lxc.network.ipv4 = 192.168.2.160/23 lxc.network.ipv4.gateway = 192.168.2.1 lxc.hook.pre-mount = /sbin/service iptables start lxc.cap.drop = net_admin
network using bridged veth interfaces
More raeding about network
Sample configs
full config
# lxc for test lxc.network.type = macvlan lxc.network.flags = up #lxc.network.hwaddr = 00:16:c0:a8:3:34 lxc.network.link = eth0 lxc.network.macvlan.mode = bridge lxc.network.name = eth0 lxc.network.ipv4 = 192.168.2.160/23 lxc.network.ipv4.gateway = 192.168.2.1 lxc.rootfs = /srv/test lxc.utsname = pldmachine.local lxc.tty = 4 lxc.pts = 1024 # load iptables, if you want to setup firewall when container is already up # you should run 'lxc-attach -e -n test -- service iptables start' lxc.hook.pre-mount = /sbin/service iptables start # lxc.mount.entry is prefered, because it supports relative paths lxc.mount = /var/lib/lxc/test/fstab lxc.cap.drop = linux_immutable #lxc.cap.drop = sys_boot # works as expected in newer kernels (3.4+) lxc.cap.drop = syslog # don't drop net_admin, allows firewall to be configured from inside lxc.cap.drop = net_admin # http://www.funtoo.org/Linux_Containers ## Capabilities, see capabilities(7) what is available #lxc.cap.drop = audit_control lxc.cap.drop = audit_write lxc.cap.drop = mac_admin lxc.cap.drop = mac_override lxc.cap.drop = mknod lxc.cap.drop = setfcap lxc.cap.drop = setpcap lxc.cap.drop = sys_admin #lxc.cap.drop = sys_boot #lxc.cap.drop = sys_chroot # required by SSH lxc.cap.drop = sys_module #lxc.cap.drop = sys_nice lxc.cap.drop = sys_pacct lxc.cap.drop = sys_rawio lxc.cap.drop = sys_resource lxc.cap.drop = sys_time #lxc.cap.drop = sys_tty_config # required by getty lxc.autodev = 0 # When using LXC with apparmor, uncomment the next line to run unconfined: lxc.aa_profile = unconfined # cgroups # Devices lxc.cgroup.devices.deny = a # Deny access to all devices # /dev/null and zero lxc.cgroup.devices.allow = c 1:3 rwm lxc.cgroup.devices.allow = c 1:5 rwm # consoles lxc.cgroup.devices.allow = c 5:1 rwm lxc.cgroup.devices.allow = c 5:0 rwm lxc.cgroup.devices.allow = c 4:0 rwm lxc.cgroup.devices.allow = c 4:1 rwm # /dev/{,u}random lxc.cgroup.devices.allow = c 1:9 rwm lxc.cgroup.devices.allow = c 1:8 rwm lxc.cgroup.devices.allow = c 136:* rwm lxc.cgroup.devices.allow = c 5:2 rwm # rtc lxc.cgroup.devices.allow = c 254:0 rm