Table of Contents

LXC - Linux Container Tools

LXC is a tool to create and manage containers. It contains a full featured container with the isolation / virtualization of the pids, the ipc, the utsname, the mount points, /proc, /sys, the network and it takes into account the control groups. It is very light, flexible, and provides a set of tools around the container like the monitoring with asynchronous events notification, or the freeze of the container. This package is useful to create Virtual Private Server, or to run isolated applications like bash or sshd.

LXC is pretty low level, very flexible and covers just about every containment feature supported by the upstream kernel. For a completely fresh and intuitive user experience with a single command line tool to manage your containers see LXD.

Resources

Prerequisites

Guest creation

Build the guest container.

Bare minimum, no template

# lxc-create -n test
lxc-create: No config file specified, using the default config /etc/lxc/default.conf
'test' created
 
# lxc-ls --fancy            (install python3-lxc for lxc-ls) 
NAME   STATE    IPV4           IPV6  
-----------------------------------
test   STOPPED  -          

this just creates $LXC_ROOT/test directory with config copied from /etc/lxc/default.conf inside.

PLD Linux from template

create guest with default config using pld template:

# lxc-create -n test -t pld

NOTYET:

There are two versions of PLD available for guest systems:

You may choose one using -R option:

# lxc-create -t pld-test -p pld -f network-configuration-file -- -R th

!!! WARNING: pld template for LXC is yet to be written !!!

Common problems / Useful tricks

lxc-start has no output

In case lxc-start -n test produces no output, ensure /dev/console is present in guest filesystem.

lxc-stop is not graceful

Currently lxc-stop -n test sends SIGPWR to init inside container, but rc-scripts fails to shutdown things properly (shutdown scripts are not invoked). For workaround, stop services manually before issueing lxc-stop or run poweroff/halt/reboot from container.

Details: In process table is only this process runrning, no further actions from rc-scripts:

/sbin/shutdown -f -h +2 Power Failure; System Shutting Down

loginuid

having audit_control dropped:

config
lxc.cap.drop = audit_control

pam_loginuid.so does not allow sshd to login:

Nov 24 16:02:10 test sshd[2694]: error: PAM: pam_open_session(): Cannot make/remove an entry for the specified session

You can either workaround to disable pam_loginuid.so in the authentication rules:

# sed '/pam_loginuid.so/s/^/#/g' -i  /etc/pam.d/*

Or just do not drop the capability.

syslog

syslog-ng gives following on startup:

# service syslog-ng restart
syslog-ng: Error setting capabilities, capability management disabled; error='Operation not permitted'
Stopping syslog-ng service.............................................................[ DONE ]
Starting syslog-ng service.............................................................[ DONE ]

FIXME: no solution yet

Vserver comparision

When in Vserver, guest processes are not visible in host, then in LXC all guest processes are visible. Beware when running killall(1) commands on host.

Also, unfortunately /proc/PID/root points to / for LXC guests as well, so rc-scripts filter_chroot() can't differentiate between host and guest processes.

Also, dmesg(1) in guest sees hosts' dmesg by default, you can turn this off by setting kernel.dmesg_restrict=1 sysctl param, available since 2.6.37 kernel.

Commands:

Vserver LXC Notes
vserver test enter lxc-attach -n test Add -e to enter with elevated privileges (ignoring lxc.cap.drop)
vserver test start lxc-start -n test -d
vserver test stop lxc-stop -n test
vserver-stat lxc-ls --fancy --running you need python3-lxc installed for this tool

Network configs

general

static networking, set VSERVER=yes and VSERVER_ISOLATION_NET=yes in guest /etc/sysconfig/system to disable all network configuration by guest, set RC_PROMPT=no to avoid hanging startup scripts, in general it's good idea to turn off there most of things

network using macvlan in bridge mode

  1. traffic from host to guest (and vice-versa) is NOT passed. external trafic works
  2. guest interface is NOT visible on host
  3. you can't filter guest straffic from host's firewall
  4. host can use seme default interface with and without guests running.
  5. one have better to set static MAC address. If not - on every container start you'll have different MAC generated and your router may have problems with passing traffic.
  6. iptables is initialized from lxc.hook.pre-mount hook (ran in the container's namespace and having guest macvlan interface visible)

first boot with hwaddr line disabled, look what the random address was assigned, set it in config.

also you may use some generation techniques like these: using last three ip numbers and Xen's OUI (00:16:3e) address space. If IP is 192.168.2.160, then:

$ printf "00:16:3e:%x:%x:%x" 168 2 160
00:16:3e:a8:2:a0
lxc.network.type = macvlan
lxc.network.flags = up
#lxc.network.hwaddr = 00:16:c0:a8:3:34
lxc.network.link = eth0
lxc.network.macvlan.mode = bridge
lxc.network.name = eth0
lxc.network.ipv4 = 192.168.2.160/23
lxc.network.ipv4.gateway = 192.168.2.1
 
lxc.hook.pre-mount = /sbin/service iptables start
lxc.cap.drop       = net_admin

network using bridged veth interfaces

More raeding about network

Elaborate article about configuring different types of network

Sample configs

full config

# lxc for test
 
lxc.network.type = macvlan
lxc.network.flags = up
#lxc.network.hwaddr = 00:16:c0:a8:3:34
lxc.network.link = eth0
lxc.network.macvlan.mode = bridge
lxc.network.name = eth0
lxc.network.ipv4 = 192.168.2.160/23
lxc.network.ipv4.gateway = 192.168.2.1
 
lxc.rootfs = /srv/test
lxc.utsname = pldmachine.local
lxc.tty = 4
lxc.pts = 1024
 
# load iptables, if you want to setup firewall when container is already up
# you should  run 'lxc-attach -e -n test -- service iptables start'
lxc.hook.pre-mount = /sbin/service iptables start
 
# lxc.mount.entry is prefered, because it supports relative paths
lxc.mount = /var/lib/lxc/test/fstab
 
lxc.cap.drop                            = linux_immutable
#lxc.cap.drop                            = sys_boot # works as expected in newer kernels (3.4+)
lxc.cap.drop                            = syslog
 
# don't drop net_admin, allows firewall to be configured from inside
lxc.cap.drop                            = net_admin
 
# http://www.funtoo.org/Linux_Containers
## Capabilities, see capabilities(7) what is available
#lxc.cap.drop                            = audit_control
lxc.cap.drop                            = audit_write
lxc.cap.drop                            = mac_admin
lxc.cap.drop                            = mac_override
lxc.cap.drop                            = mknod
lxc.cap.drop                            = setfcap
lxc.cap.drop                            = setpcap
lxc.cap.drop                            = sys_admin
#lxc.cap.drop                            = sys_boot
#lxc.cap.drop                            = sys_chroot # required by SSH
lxc.cap.drop                            = sys_module
#lxc.cap.drop                            = sys_nice
lxc.cap.drop                            = sys_pacct
lxc.cap.drop                            = sys_rawio
lxc.cap.drop                            = sys_resource
lxc.cap.drop                            = sys_time
#lxc.cap.drop                            = sys_tty_config # required by getty
 
lxc.autodev = 0
 
# When using LXC with apparmor, uncomment the next line to run unconfined:
lxc.aa_profile = unconfined
 
# cgroups
# Devices
lxc.cgroup.devices.deny = a # Deny access to all devices
 
# /dev/null and zero
lxc.cgroup.devices.allow = c 1:3 rwm
lxc.cgroup.devices.allow = c 1:5 rwm
# consoles
lxc.cgroup.devices.allow = c 5:1 rwm
lxc.cgroup.devices.allow = c 5:0 rwm
lxc.cgroup.devices.allow = c 4:0 rwm
lxc.cgroup.devices.allow = c 4:1 rwm
# /dev/{,u}random
lxc.cgroup.devices.allow = c 1:9 rwm
lxc.cgroup.devices.allow = c 1:8 rwm
lxc.cgroup.devices.allow = c 136:* rwm
lxc.cgroup.devices.allow = c 5:2 rwm
# rtc
lxc.cgroup.devices.allow = c 254:0 rm