Compare commits

..

No commits in common. "master" and "v0.1.0" have entirely different histories.

15 changed files with 1682 additions and 4867 deletions

1
.gitignore vendored
View File

@ -10,4 +10,3 @@ build
*.egg-info
src/*.service
.mypy_cache
.tox

8
Jenkinsfile vendored
View File

@ -2,4 +2,10 @@
@Library('shared-jenkins-libraries@master') _
genericFedoraRPMPipeline(null, null, null, null, TestStrategySkipTests())
def test_step() {
return {
println "Tests disabled"
}
}
genericFedoraRPMPipeline(null, null, null, null, test_step())

View File

@ -11,7 +11,7 @@ src/qubes-routing-manager.service: src/qubes-routing-manager.service.in
ROOT_DIR := $(shell dirname $(realpath $(firstword $(MAKEFILE_LIST))))
.PHONY: clean dist rpm srpm install-template install-dom0 test
.PHONY: clean dist rpm srpm install-template install-dom0
clean:
cd $(ROOT_DIR) || exit $$? ; find -name '*.pyc' -o -name '*~' -print0 | xargs -0 rm -f
@ -42,6 +42,3 @@ install-dom0:
PYTHONDONTWRITEBYTECODE=1 python3 networkserversetup.py install $(PYTHON_PREFIX_ARG) -O0 --root $(DESTDIR)
install: install-dom0 install-template
test:
tox --current-env

View File

@ -8,11 +8,6 @@ of setting up your own Xen server.
This release is only intended for use with Qubes OS 4.2. Older Qubes OS releases
will not support it. For Qubes OS 4.1, check branch `r4.1`.
**Important note about upgrades**: when you upgrade your system from Qubes OS 4.1 to
Qubes OS 4.2, if you have this package installed in your template, the template will
likely **fail to update**. Please consult [our upgrade instructions](doc/distupgrade.md)
for information on how to proceed.
## Why?
Qubes OS is a magnificent operating system. That said, there are many use cases its networking
@ -43,18 +38,15 @@ connectivity for ProxyVMs and other user VMs alike.
No provision is made for running a server in a virtualized environment,
such that the server's ports are accessible by (a) other VMs (b) machines
beyond the perimeter of the NetVM. By default, firewall rules in NetVMs
prevent traffic from reaching any VM attached to them. Furthermore, even
with custom, permissive firewall rules, the IP addresses of VMs attached
to any NetVM are not visible "on the other side of the NetVM", so firewall
rules can only help with something like DNAT. Finally, such custom firewalls
require careful thought not to open the system to unexpected attack vectors.
The Qubes OS user interface provides no means to set this up either.
beyond the perimeter of the NetVM. To the extent that such a thing is
possible, it is only possible by painstakingly maintaining firewall rules
for multiple VMs, which need to carefully override the existing firewall
rules, and require careful thought not to open the system to unexpected
attack vectors. The Qubes OS user interface provides no help either.
### The Qubes network server networking model
Qubes network server builds on the Qubes security model and enhances it
to optionally permit traffic to user VMs.
Qubes network server builds on that.
![Qubes network server model](./doc/Qubes%20network%20server%20model.png)
@ -63,8 +55,7 @@ network servers in user VMs available to other machines, be them
peer VMs in the same Qubes OS system or machines connected to
a physical link shared by a NetVM. Those network server VMs also
obey the Qubes OS outbound firewall rules controls, letting you run
services with outbound connections restricted using the standard Qubes OS
firewall system.
services with outbound connections restricted.
This is all, of course, opt-in, so the standard Qubes OS network security
model remains in effect until you decide to enable the feature on any
@ -82,55 +73,43 @@ the software is straightforward.
These sample instructions assume:
* the software is properly installed (see below),
* you understand the distinction between dom0 and qubes,
* you already have an AppVM VM set up, named `testvm`, and
* you understand the distinction between dom0 and qubes
* you already have an AppVM VM set up, named `testvm`,
* your `sys-net` VM is attached to a network with subnet `192.168.16.0/24`
— this, of course, may vary depending on your local router configuration.
*Do not proceed any further if you do not yet meet these requirements.*
First, **attach** the VM you want to expose to the network
First, attach the VM you want to expose to the network
to a NetVM that has an active network connection:
`qvm-prefs -s testvm netvm sys-net`
Set an **IP** address on `testvm` belonging to the same LAN as `sys-net`:
Then, set an IP address on the VM:
`qvm-prefs -s testvm ip 192.168.16.25`
**Restart** the `testvm` VM if it was already running.
(The step above requires you restart the `testvm` VM if it was running.)
**Configure** routing method; to enable the network server feature for
your `testvm` VM, all you have to do in your AdminVM (`dom0`) is run
the following command:
Then, to enable the network server feature for your `testvm` VM, all you have
to do in your AdminVM (`dom0`) is run the following command:
`qvm-features testvm routing-method forward`
Now `testvm` is exposed to the network with address `192.168.16.25`, as well
as to other VMs attached to `NetVM`.
Finally, adjust **input firewall rules** on `testvm` to permit traffic coming from
machines in your LAN. `testvm` will have the standard Qubes OS firewall
rules stopping inbound traffic. To solve that issue, you can use a sample
rule in `testvm`:
```
sudo nft add rule qubes custom-input ip saddr 192.168.16.0/24 ct state new,established,related counter accept
```
You can make these rules persistent by [following instructions on the Qubes
OS firewall documentation page](https://www.qubes-os.org/doc/firewall/#enabling-networking-between-two-qubes).
Note that you do not need to change the `custom-forward` chain at all
on any qube -- Qubes network server manages that for you transparently
in your `NetVM`.
Do note that `testvm` will have the standard Qubes OS firewall rules stopping
inbound traffic. To solve that issue, you can
[use the standard `rc.local` Qubes OS mechanism to alter the firewall rules](https://www.qubes-os.org/doc/firewall/#where-to-put-firewall-rules)
in your `testvm` AppVM.
Here are documents that will help you take advantage of Qubes network server:
* [Setting up your first server](doc/Setting up your first server.md)
* [Setting up an SSH server](doc/Setting up an SSH server.md)
## Setup
## Installation of packages
Package installation consists of two steps (**the package creation instructions are below**):
@ -151,38 +130,30 @@ a terminal in your NetVM, then typing the following:
systemctl status qubes-routing-manager.service
```
The routing manager should show as `enabled` and `active` in the terminal
output, with no errors. You can now follow the usage instructions above.
The routing manager should show as `enabled` and `active` in the terminal output.
### How to build the packages to install
You will first build the `qubes-core-admin-addon-network-server` RPM.
To build this package, you will need to use a `chroot` jail containing
a Fedora installation of the exact same release as your `dom0` (Fedora 37
for Qubes 4.2). You can do this using `toolbox` (for maximum safety
within a disposable qube):
a Fedora installation of the exact same release as your `dom0` (Fedora 25
for Qubes release 4.0, Fedora 32 for Qubes release 4.1).
```
dnf install -y toolbox
toolbox create -r 37
toolbox enter fedora-toolbox-37
# Bam! You have a shell in an isolated Fedora 37 instance now.
```
Copy the source of the package to your `chroot`. Then start a shell in
your `chroot`, and type `make rpm`. You may have to install some packages
in your `chroot` -- use `dnf install git rpm-build make coreutils tar gawk findutils systemd systemd-rpm-macros`
to get the minimum dependency set installed.
Within the toolbox, all your normal files from your home directory are
visible. Change into the directory that contains this source code,
then type `make rpm`. You may have to install some packages within your
toolbox -- use `dnf install git rpm-build make coreutils tar gawk findutils systemd systemd-rpm-macros`
to get the minimum dependency set installed. Don't worry -- nothing within
the toolbox affects
Once built, in the source directory you will find the RPM built for the
exact release of Qubes you need.
Once built, in the source directory you will find the
`qubes-core-admin-addon-network-server` RPM built for your dom0.
Alternatively, you may first create a source RPM using `make srpm` on your
regular workstation, then use `mock` to rebuild the source RPM produced
in the source directory, using a Fedora release compatible with your `dom0`.
To build the `qubes-network-server` RPM that goes in your template, you
can simply use a DisposableVM running the same Fedora release as your NetVM.
Build said package as follows:
To build the `qubes-network-server` RPM, you can use a DisposableVM running
the same Fedora release as your NetVM. Build said package as follows:
```
# Dependencies
@ -194,8 +165,8 @@ make rpm
```
The process will output a `qubes-network-server-*.noarch.rpm` in the
directory where it ran. Fish it out and copy it into the template where
you'll install it.
directory where it ran. Fish it out and save it into the VM where you'll
install it.
You can power off the DisposableVM now.

View File

@ -1 +1 @@
["QUBES_RELEASES": "4.2"]
["RELEASE": "q4.2 37 38 39"]

View File

@ -1,64 +0,0 @@
# How to upgrade a Qubes network server from Qubes OS 4.1 to Qubes OS 4.2
The [standard instructions to upgrade Qubes OS systems](https://www.qubes-os.org/doc/upgrade/4.2/)
will fail to work. The instructions tell you to run something to the effect of:
```
qubes-dist-upgrade --all-pre-reboot <other flags>
```
then reboot, then run:
```
qubes-dist-upgrade --all-post-reboot <other flags>
```
The pre-reboot phase will fail if run without the following precautions.
## Step by step instructions
First, build a `qubes-network-server` RPM with the instructions provided
by this package's [README.md](../README.md) file. Then, for each template
where `qubes-network-server` is installed, deposit your build of the
`qubes-network-server` RPM in a folder `/root/update` of the template,
and run the command `createrepo_c /root/update` (you may have to install
package `createrepo_c` via `dnf` to run it).
Now build a `qubes-core-admin-addon-network-server` package for your dom0,
then copy the file to your profile directory into dom0. Remember this
package has to be built *in the same Fedora release (37)* as the Qubes OS
4.2 dom0 (the `toolbox` command in a disposable qube is handy for this!).
Now open the file `/etc/dnf/dnf.conf` on every template qube where you
did the above, then add an `exclude=qubes-network-server` setting under
its `[main]` section.
Remove the currently-installed `qubes-core-admin-addon-network-server`
package from your dom0 (using `dnf remove`).
Run the pre-reboot phase.
Install the recently-built `qubes-core-admin-addon-network-server` package
into dom0 (using `dnf install` with the path to the RPM file).
Reboot.
Before running the post-reboot phase, remove the setting you added to the
`dnf.conf` file of each template you modified. Finally, add the file
`/etc/yum.repos.d/local.repo` with the following contents:
```
[local]
name=Local packages
baseurl=file:///root/update
enabled=1
gpgcheck=0
metadata_expire=15
```
Now run the post-reboot phase. The template upgrade should succeed now.
To finalize, delete folder `/root/update` and file `/etc/yum.repos.d/local.repo`
from every template that has it.
You are now updated to Qubes OS 4.2 and `qubes-network-server` is ready.

View File

@ -3,7 +3,7 @@
%define mybuildnumber %{?build_number}%{?!build_number:1}
Name: qubes-network-server
Version: 0.1.6
Version: 0.1.0
Release: %{mybuildnumber}%{?dist}
Summary: Turn your Qubes OS into a network server
BuildArch: noarch
@ -19,12 +19,8 @@ BuildRequires: findutils
BuildRequires: python3
BuildRequires: python3-rpm-macros
BuildRequires: systemd-rpm-macros
BuildRequires: python3-tox-current-env
BuildRequires: python3-mypy
BuildRequires: python3-pytest
Requires: qubes-core-agent-networking >= 4.2
Conflicts: qubes-core-agent < 4.2
Requires: python3
Requires: python3-qubesdb
Requires: nftables
@ -50,7 +46,6 @@ BuildRequires: python3-setuptools
Requires: python3
Requires: qubes-core-dom0 >= 4.2
Conflicts: qubes-core-dom0 < 4.2
%description -n qubes-core-admin-addon-network-server
This package lets you turn your Qubes OS into a network server. Install this
@ -74,9 +69,6 @@ make install DESTDIR=$RPM_BUILD_ROOT SBINDIR=%{_sbindir} UNITDIR=%{_unitdir} PYT
mkdir -p "$RPM_BUILD_ROOT"/%{_presetdir}
echo 'enable qubes-routing-manager.service' > "$RPM_BUILD_ROOT"/%{_presetdir}/75-%{name}.preset
%check
tox --current-env
%files
%attr(0755, root, root) %{_sbindir}/qubes-routing-manager
%attr(0644, root, root) %{python3_sitelib}/qubesroutingmanager/*
@ -92,26 +84,6 @@ tox --current-env
%post
%systemd_post qubes-routing-manager.service
%posttrans
# Remove old unit enablement paths.
reenable=0
if [ -h %{_sysconfdir}/systemd/system/multi-user.target.wants/qubes-routing-manager.service ]
then
reenable=1
rm -f %{_sysconfdir}/systemd/system/multi-user.target.wants/qubes-routing-manager.service
fi
if [ -h %{_sysconfdir}/systemd/system/qubes-iptables.service.wants/qubes-routing-manager.service ]
then
reenable=1
rm -f %{_sysconfdir}/systemd/system/qubes-iptables.service.wants/qubes-routing-manager.service
fi
if [ $reenable = 1 ]
then
mkdir -p %{_sysconfdir}/systemd/system/qubes-network.service.wants
ln -sf %{_unitdir}/qubes-routing-manager.service %{_sysconfdir}/systemd/system/qubes-network.service.wants/qubes-routing-manager.service
fi
exit 0
%preun
%systemd_preun qubes-routing-manager.service

View File

@ -4,10 +4,10 @@ import json
import logging
import subprocess
from typing import TypedDict, Any, cast, Literal, Union
from typing import TypedDict, Any, cast, Literal
ADDRESS_FAMILIES = Union[Literal["ip"], Literal["ip6"]]
ADDRESS_FAMILIES = Literal["ip"] | Literal["ip6"]
class Chain(TypedDict):
@ -65,10 +65,9 @@ ADDRESS_FAMILY_IPV6 = "ip6"
ADDRESS_FAMILY_IPV4 = "ip"
TABLE_NAME = "qubes"
FORWARD_CHAIN_NAME = "forward"
POSTROUTING_CHAIN_NAME = "postrouting"
ROUTING_MANAGER_CHAIN_NAME = "qubes-routing-manager"
ROUTING_MANAGER_POSTROUTING_CHAIN_NAME = "qubes-routing-manager-postrouting"
NFTABLES_CMD = "nft"
ADD_RULE_AFTER_THIS_RULE = "custom-forward"
def get_table(address_family: ADDRESS_FAMILIES, table: str) -> NFTablesOutput:
@ -138,20 +137,15 @@ def append_counter_at_end(
)
def _append_or_insert_rule(
where: Literal["add"] | Literal["insert"],
address_family: ADDRESS_FAMILIES,
table: str,
chain: str,
handle: int,
*rest: str,
def append_rule_after(
address_family: ADDRESS_FAMILIES, table: str, chain: str, handle: int, *rest: str
) -> None:
subprocess.check_output(
[
NFTABLES_CMD,
"-n",
"-j",
where,
"add",
"rule",
address_family,
table,
@ -164,18 +158,6 @@ def _append_or_insert_rule(
)
def append_rule_after(
address_family: ADDRESS_FAMILIES, table: str, chain: str, handle: int, *rest: str
) -> None:
_append_or_insert_rule("add", address_family, table, chain, handle, *rest)
def insert_rule_before(
address_family: ADDRESS_FAMILIES, table: str, chain: str, handle: int, *rest: str
) -> None:
_append_or_insert_rule("insert", address_family, table, chain, handle, *rest)
def delete_rule(
address_family: ADDRESS_FAMILIES, table: str, chain: str, handle: int
) -> None:
@ -220,160 +202,109 @@ def setup_plain_forwarding_for_address(source: str, enable: bool, family: int) -
forward_chain = [x for x in existing_chains if x["name"] == FORWARD_CHAIN_NAME][
0
]
postrouting_chain = [
x for x in existing_chains if x["name"] == POSTROUTING_CHAIN_NAME
][0]
except IndexError:
logging.warn(
"No forward or postrouting chains in table %s, not setting up forwarding",
TABLE_NAME,
"No forward chain in table %s, not setting up forwarding", TABLE_NAME
)
return
for chain_name in [
ROUTING_MANAGER_CHAIN_NAME,
ROUTING_MANAGER_POSTROUTING_CHAIN_NAME,
]:
chain: None | Chain = None
try:
chain = [x for x in existing_chains if x["name"] == chain_name].pop()
except IndexError:
pass
qubes_routing_manager_chain: None | Chain = None
try:
qubes_routing_manager_chain = [
x for x in existing_chains if x["name"] == ROUTING_MANAGER_CHAIN_NAME
].pop()
except IndexError:
pass
if not chain:
logging.info(
"Adding %s chain to table %s and counter to chain",
chain_name,
TABLE_NAME,
)
add_chain(af, TABLE_NAME, chain_name)
append_counter_at_end(
af,
TABLE_NAME,
chain_name,
)
def is_oifgroup_2(rule):
return (
rule["chain"] == forward_chain["name"]
and len(rule["expr"]) == 3
and (
rule["expr"][0].get("match", {}).get("op") == "=="
and rule["expr"][0]
.get("match", {})
.get("left", {})
.get("meta", {})
.get("key")
== "oifgroup"
and rule["expr"][0].get("match", {}).get("right") == 2
)
and (rule["expr"][-1].get("drop", "not none") is None)
if not qubes_routing_manager_chain:
logging.info(
"Adding %s chain to table %s", ROUTING_MANAGER_CHAIN_NAME, TABLE_NAME
)
add_chain(af, TABLE_NAME, ROUTING_MANAGER_CHAIN_NAME)
def is_postrouting_masquerade(rule):
return (
rule["chain"] == postrouting_chain["name"]
and len(rule["expr"]) == 1
and "masquerade" in rule["expr"][0]
)
qubes_routing_manager_rule: None | Rule = None
try:
qubes_routing_manager_rule = [
x
for x in existing_rules
if x["chain"] == forward_chain["name"]
and x["family"] == af
and len(x["expr"]) == 1
and x["expr"][0].get("jump", {}).get("target") == ROUTING_MANAGER_CHAIN_NAME
].pop()
except IndexError:
pass
for parent_chain, child_chain_name, previous_rule_detector, insertor in [
(
forward_chain,
ROUTING_MANAGER_CHAIN_NAME,
is_oifgroup_2,
insert_rule_before,
),
(
postrouting_chain,
ROUTING_MANAGER_POSTROUTING_CHAIN_NAME,
is_postrouting_masquerade,
insert_rule_before,
),
]:
jump_rule: None | Rule = None
if not qubes_routing_manager_rule:
try:
jump_rule = [
custom_forwarding_rule = [
x
for x in existing_rules
if x["chain"] == parent_chain["name"]
and x["family"] == af
if x["chain"] == forward_chain["name"]
and len(x["expr"]) == 1
and x["expr"][0].get("jump", {}).get("target") == child_chain_name
].pop()
and x["expr"][0].get("jump", {}).get("target")
== ADD_RULE_AFTER_THIS_RULE
][0]
except IndexError:
pass
if not jump_rule:
try:
previous_rule = [
x for x in existing_rules if previous_rule_detector(x)
][0]
except IndexError:
logging.warn(
"Cannot find appropriate previous rule in chain %s of table %s, not setting up forwarding",
parent_chain["name"],
TABLE_NAME,
)
logging.info(
"Adding rule to jump from chain %s to chain %s in table %s",
parent_chain["name"],
child_chain_name,
logging.warn(
"No state forwarding rule in chain %s of table %s, not setting up forwarding",
forward_chain["name"],
TABLE_NAME,
)
insertor(
af,
TABLE_NAME,
parent_chain["name"],
previous_rule["handle"],
"jump",
child_chain_name,
)
def detect_ip_rule(rule: Rule, chain_name: str, ip: str, mode: str):
return (
rule["chain"] == chain_name
and len(rule["expr"]) == 2
and rule["expr"][0].get("match", {}).get("op", {}) == "=="
and rule["expr"][0]["match"]
.get("left", {})
.get("payload", {})
.get("protocol", "")
== af
and rule["expr"][0]["match"]["left"]["payload"].get("field", "") == mode
and rule["expr"][0].get("match", {}).get("right", []) == ip
and "accept" in rule["expr"][1]
logging.info(
"Adding rule to jump to %s to table %s after jump to %s",
ROUTING_MANAGER_CHAIN_NAME,
TABLE_NAME,
ADD_RULE_AFTER_THIS_RULE,
)
append_rule_after(
af,
TABLE_NAME,
forward_chain["name"],
custom_forwarding_rule["handle"],
"jump",
ROUTING_MANAGER_CHAIN_NAME,
)
append_counter_at_end(
af,
TABLE_NAME,
ROUTING_MANAGER_CHAIN_NAME,
)
for chain_name, mode in [
(ROUTING_MANAGER_CHAIN_NAME, "daddr"),
(ROUTING_MANAGER_POSTROUTING_CHAIN_NAME, "saddr"),
]:
address_rules = [
x for x in existing_rules if detect_ip_rule(x, chain_name, source, mode)
]
address_rules = [
x
for x in existing_rules
if x["chain"] == ROUTING_MANAGER_CHAIN_NAME
and len(x["expr"]) == 2
and x["expr"][0].get("match", {}).get("op", {}) == "=="
and x["expr"][0]["match"].get("left", {}).get("payload", {}).get("protocol", "")
== af
and x["expr"][0]["match"]["left"]["payload"].get("field", "") == "daddr"
and x["expr"][0].get("match", {}).get("right", []) == source
and "accept" in x["expr"][1]
]
if enable and not address_rules:
logging.info(
"Adding accept rule on chain %s for %s.",
chain_name,
source,
)
append_rule_at_end(
af,
TABLE_NAME,
chain_name,
af,
mode,
source,
"accept",
)
elif not enable and address_rules:
logging.info(
"Removing %s accept rules from chain %s for %s.",
len(address_rules),
chain_name,
source,
)
for rule in reversed(sorted(address_rules, key=lambda r: r["handle"])):
delete_rule(af, TABLE_NAME, chain_name, rule["handle"])
if enable and not address_rules:
logging.info(
"Adding accept rule on chain %s to allow traffic to %s.",
ROUTING_MANAGER_CHAIN_NAME,
source,
)
append_rule_at_end(
af,
TABLE_NAME,
ROUTING_MANAGER_CHAIN_NAME,
af,
"daddr",
source,
"accept",
)
elif not enable and address_rules:
logging.info(
"Removing %s accept rules from chain %s to stop traffic to %s.",
len(address_rules),
ROUTING_MANAGER_CHAIN_NAME,
source,
)
for rule in reversed(sorted(address_rules, key=lambda r: r["handle"])):
delete_rule(af, TABLE_NAME, ROUTING_MANAGER_CHAIN_NAME, rule["handle"])

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@ -14,7 +14,7 @@ import logging
import os
import socket
import qubesdb # type: ignore
import qubesdb
from qubesroutingmanager import setup_plain_forwarding_for_address

View File

@ -1,13 +1,12 @@
[Unit]
Description=Configure the network to allow network server VMs
Documentation=https://github.com/Rudd-O/qubes-network-server
After=qubes-network.service qubes-iptables.service
After=qubes-iptables.service
BindsTo=qubes-iptables.service
ConditionPathExists=/var/run/qubes-service/qubes-network
[Service]
Type=notify
ExecStart=@SBINDIR@/qubes-routing-manager
[Install]
WantedBy=qubes-network.service
WantedBy=qubes-iptables.service

10
tox.ini
View File

@ -1,10 +0,0 @@
[tox]
envlist = basepython
[testenv]
deps =
pytest
mypy
commands =
pytest -vv
mypy -p qubesroutingmanager