Orgs
An org holds its people's and agents' sandboxes, stacks, volumes and secrets. Everything in an org is administered by its members, and nothing crosses orgs. That makes the org the unit you hand to a team, a customer or an agent: give them an org, and they can do anything inside it and nothing outside it.
Every command, daemon tool and stack takes --org ORG (or $ISB_ORG).
Platform commands (stack, app, project, secret, ...) without it work
in the default org; plain sandbox commands (isb create, isb up) without
it work outside every org (below).
isb org create NAME [--cpus N] [--memory 16GiB] [--disk 100GiB] [--instances N]
[--default-cpus N] [--default-memory 512MiB]
[--bind-root DIR]... [--allow-egress DEST]...
[--allow-domain SUFFIX]... [--ingress caddy|cloudflare-tunnel]
[--cloudflare-account ID] [--cloudflare-zone ID]
[--server SERVER | --vm [--vm-cpus N] [--vm-memory 4GiB] [--vm-disk 40GiB]]
isb org ls [--json]
isb org show NAME [--json]
isb org rm NAME [--force] [--delete-vm]
isb org nesting NAME [on|off]
sudo isb host setup [--uplink IFACE] [--user USER] [--dry-run] [--public-ingress]
isb org create on an existing org updates it to the flags given. system
is not an org name: the incus project isb-system holds isb's own services
(the local registry, dedicated VMs).
Where an org runs is chosen once, when it is created (Placement):
- by default, on this host;
--server SERVERcreates it on another host instead, one the local daemon (a control plane) added withisb server add(Servers and dedicated VMs). The org's project, network and workloads then live on that server, and every call for it goes there;isb org showandisb org rmfind it through the daemon;--vmruns it in a dedicated VM: the local daemon makes a VM on this host for the org alone, with its own kernel and its own incus, registers it as servervm-NAMEand places the org there.isb org rm NAME --delete-vmdeletes the VM with the org.
An org does not move between placements afterwards (Moving an org).
An org's builds (Builds and the local registry) run in
its own project too, as ordinary unprivileged containers (or VMs for
untrusted source) that count against its quota, and its images are the
registry repositories <org>/*, which only its own stacks and sandboxes can
name.
From the API and the web UI#
The same operations are daemon tools (MCP tools), and the web UI's org Settings and Platform pages use them:
| Tool | Who | Does |
|---|---|---|
org_get |
the org's members | limits, defaults, network, egress, bind roots, service-name domain (<org>.isb), counts, placement |
org_list |
platform admins | every org, each with the server it runs on (local for this daemon) and its placement |
org_create |
platform admins | isb org create without --bind-root; placement puts it on a server or in a dedicated VM |
org_update |
platform admins | limits, per-instance defaults, egress exceptions (a different server or placement is refused) |
org_delete |
platform admins | isb org rm, refused while stacks are deployed in the org, and while it has sandboxes unless force; delete_vm also deletes a dedicated VM |
org_nesting |
superadmins | whether the org's workspace may run Docker (isb org nesting ORG on|off); org_get shows it as allow_nesting |
Limits and egress exceptions are what keep one org from the others and from
the host's networks, so changing them is for platform admins, not the org's
own owners and admins, who see them read-only. Bind roots are host paths and
are set only on the host (isb org create --bind-root): an update through
the API keeps them, as it keeps any field it is not given. A limit, once
set, can be changed but not lifted, as with the CLI. Creating an org through
the API also adds it to the identity store, and deleting one removes its
memberships, invitations and tokens; its secrets stay under the state
directory.
What an org is in incus#
An org on this host shares the host's kernel with every other local org: the
project, bridge and ACL below keep them apart, and the kernel is what they
all trust. org_get says so as placement.isolation: shared-kernel here,
own-host on a server, own-kernel in a dedicated VM
(Placement).
Org acme is the incus project isb-acme (config user.isb.org=acme), its
bridge isbbr<hash> and its network ACL isb-acme.
The default org is an org like any other: the incus project
isb-default, with its own bridge, ACL and service names, settings, limits,
egress rules, workspaces and, if you set one, a Cloudflare Tunnel
(Domains). Multi-app templates and workspaces work in
it as in any org. isb serve makes sure it exists on every start, so
isb serve install, which starts the daemon, leaves a host with one, and it
cannot be removed. isb org ls shows it as default -> isb-default.
incus' own default project is never an org. Plain isb create and isb up
without --org put sandboxes there, outside every org, and the TUI and the
superadmin's Host page show them; --org default means the default org,
isb-default.
If a default-org stack still has instances in incus' default project,
isb serve logs a warning when it starts: the controller recreates that
stack in isb-default with new, empty volumes, and the old instances and
volumes keep running in default until you delete them. isb has no tool to
move data between the two projects.
The project is restricted, so incus itself refuses what would reach the host:
- unprivileged containers only; no nesting, no
raw.lxc, noraw.idmapof root, no proxy devices (a superadmin can allow nesting for the org's workspace alone: The Docker exception); - disks are managed volumes only, or bind mounts from the org's
--bind-rootdirectories; - the org's own bridge is the only network;
- each instance gets its own uid range (
security.idmap.isolated), and only the daemon's own uid may be mapped 1:1 (soidmap: autokeeps bind-mounted files writable); - snapshots and exports are allowed (
restricted.snapshots,restricted.backups): isb takes them for volumes.
--cpus, --memory, --disk and --instances limit the org as a whole.
Once a project has limits, incus wants limits on every instance, so the
org's default profile carries --default-cpus (1) and --default-memory
(512MiB) for instances whose spec sets none. The host's images are shared
with every org. When an instance or volume would pass a limit, isb says
which (org lab is at its CPU quota (limits.cpu 2, 2 in use)) and how to
raise it: isb org create lab --cpus N on the host, or org_update.
The project's bind paths are the --bind-root directories plus the host
folders of the org's workspace homes, which are recorded on the project, so
rewriting the bind roots never takes a home away.
The network#
Each org has its own bridge with a /24 that incus picks, NAT to the internet,
no IPv6, and DNS domain <org>.isb: an instance web is web.acme.isb to
the others in the org (and plain web, through the search domain).
The org's ACL decides where its instances may go:
- within the org: anything;
- the internet: anything;
- private ranges (10/8, 172.16/12, 192.168/16, 100.64/10, 169.254/16): rejected, apart from the org's own subnet (and the exceptions below). This covers other orgs' bridges, the host's other networks, the LAN and the tailnet.
Traffic into an org's instances is not filtered by the ACL: the host, and the daemon's load balancer, reach them.
A host with a default-deny firewall (ufw) needs sudo isb host setup once,
so org bridges get DHCP, DNS and egress while staying apart from each other:
see Host firewall.
Egress exceptions#
--allow-egress DEST lets the org reach a private destination anyway, such
as a tailnet host or another org's published service. DEST is
CIDR[:PORTS[/tcp|udp]]:
DEST |
Allows |
|---|---|
10.20.0.0/16 |
everything to that network |
100.79.171.47 |
everything to that address (a bare address is a /32) |
100.79.171.47/32:1080/tcp |
TCP port 1080 only (tcp is the default) |
10.1.2.3:53/udp |
UDP port 53 only |
10.1.2.3:8000-8100,9000 |
those TCP ports |
The flag repeats. Giving it replaces the org's exceptions; leaving it out
keeps them; --allow-egress none clears them. They are stored in the
project's user.isb.egress and shown by isb org show. Two exceptions for
different networks may not overlap; the same network may be listed more than
once (its ports add up, and a whole-network exception wins).
How they are rendered: incus evaluates an ACL's drop rules first, then
reject, then allow, whatever the order they were written in, so an allow
rule can never punch through the private-range reject. Instead, the
exception's network is carved out of the rejected ranges, as the org's own
subnet is. A port-limited exception then adds reject rules for the rest of
that network: its other TCP ports, its other UDP ports (all of them if none
were allowed) and ICMP. Other IP protocols (GRE, SCTP, ...) to a port-limited
destination are not filtered. An exception outside the private ranges
changes nothing, since the internet is allowed anyway.
An exception only lifts the org's ACL. The host's firewall still applies: a
destination routed out of an interface other than the uplink (a tailnet host
through tailscale0, say) also needs the host to forward to it.
Domains#
The hostnames an org's stacks may serve through the ingress (Domains and ingress) are the platform's to decide:
--allow-domain SUFFIX(repeatable) limits them to names at or under each suffix:example.comallowsexample.comandshop.example.com.*.example.comalso allows wildcard hosts (*.example.com,*.team.example.com). Giving the flag replaces the list;noneclears it. Stored in the project'suser.isb.domains.- Without a list, any concrete name is allowed and no wildcard.
- Generated names (
host: auto, under sslip.io) are always allowed. - Whatever the lists say, a name one org serves is refused to every other: the first to claim it keeps it.
--ingress picks how the org's domains are reached: caddy (default) on
the server's public listeners, or cloudflare-tunnel through the org's own
Cloudflare Tunnel, whose token the org keeps in its secret
cloudflare-tunnel-token. With an API token in cloudflare-api-token as
well, isb manages the tunnel's ingress rules and the hostnames' DNS records;
--cloudflare-account and --cloudflare-zone name the account (default:
the tunnel token's) and zone (default: looked up per hostname). Stored in
user.isb.ingress and user.isb.ingress.cloudflare.*.
Service names#
Inside an org, a stack's service is reachable at a stable name:
<service>.<stack>.<org>.isb, and <service>.<stack> for short. It
resolves to every replica in rotation and follows rollouts; see
Service discovery.
How it works: the org's bridge runs incus' dnsmasq with raw.dnsmasq set to
hostsdir=/var/lib/isb/dns/<org>. dnsmasq watches that directory and
re-reads a file as soon as one is renamed into it, so the daemon keeps one
hosts file per service there and DNS follows within milliseconds, with no
dnsmasq restart. raw.dnsmasq is set once, when the org is created:
changing it later restarts the org's dnsmasq.
sudo isb host setupcreates/var/lib/isb/dns, owned by the user isb runs as (--user, default the one who ran sudo) with groupincus(dnsmasq's) and the setgid bit, mode 2750: the daemon writes, dnsmasq reads, nobody else can. A host without anincusgroup gets a world-readable 0755 directory.isb org createmakes<org>/in it. On a host without the directory the org is created without service names and says so. Oncesudo isb host setuphas made the directory, a runningisb servesetsraw.dnsmasqon every org that lacks it (restarting that org's dnsmasq once), thedefaultorg included, at its next start or within a minute; with no daemon, runisb org create ORGagain.isb org rmdeletes the org's directory.ISB_DNS_DIRmoves the directory, forisb organdisb servealike.- When the directory is inside the daemon's state directory (a daemon
running as root keeps its state in
/var/lib/isb, as does a server's agent),isb servemakes the state directories on the way traversable (mode 0711: others may pass through, not list) so dnsmasq can reach the hosts files; everything in them stays 0600/0700.
Trade-off: incus runs a bridge's dnsmasq unconfined (no AppArmor profile)
once raw.dnsmasq is set, and it is the only way to point dnsmasq at a
directory. dnsmasq still drops to the incus user.
Removing an org#
isb org rm NAME refuses an org that has instances; --force deletes them,
then the project (with its volumes, profiles and buckets), the bridge, the
ACL and the service-name directory. For an org in a dedicated VM,
--delete-vm deletes the VM too.