luaebpf

Lua to eBPF by example

Experimental, as Compiling Lua to eBPF is.

What a compiled function computes, construct by construct. The table below says what Lua computes, what the eBPF instruction alone would compute and what the compiler emits between the two. Each section after it compiles a program file whose operands come from the packet, so that lunatic folds none of them, lists the object with each instruction annotated, and traces one input to the result. The rows of The Lua a program may use link to these sections.

What the compiler emits

A construct the compiler lowers computes what the kernel’s Lua computes, stops the program where Lua would raise or where a for outruns the budget the kernel gives it, with XDP_ABORTED in XDP and TC_ACT_SHOT in TC, logging the error Lua raises there, or is refused at its line. Where the eBPF instruction alone computes something else, the compiler emits what lies between:

Lua Lua computes The instruction alone The compiler emits
-7 // 2 -4, the floor -3: BPF_SDIV truncates BPF_DIV of the magnitudes, the quotient negated where the signs differ, as C’s, and less one where the signs differ and the remainder is not zero, as luaV_idiv
-7 % 2 1, the divisor’s sign -1: BPF_SMOD truncates BPF_MOD of the magnitudes, the remainder given the dividend’s sign, as C’s, and the divisor added where the remainder is not zero and its sign is not the divisor’s, as luaV_mod
x // -1, x % -1 -x and 0, so math.mininteger // -1 is math.mininteger for math.mininteger, a divide error on x86-64 before Linux 6.12, whose verifier patches the instruction to give math.mininteger and 0, as arm64 always gave nothing of its own: the magnitude of math.mininteger is itself, which BPF_DIV reads as 263, so its quotient by 1 reads back as math.mininteger, and every remainder by 1 is 0
x // 0, x % 0 raises 0 and x a test of the divisor, which stops the program
1 << 64 0 1: the count is masked to 63 the count tested against 64 and -64, past which the result is zero
1 << -1 0: a negative count shifts the other way math.mininteger: -1 masked is 63 a negative count negated into the other shift; x >> n is x << -n, as in Lua
x & 0xffffffff the low 32 bits of x x: a 32-bit immediate, sign-extended, reads -1 the constant in a register, by a MOV of its immediate where it fits 32 bits and by LD_IMM64 otherwise
x + y, x - y, x * y, -x, ~x, the bitwise operators the result wrapped to 64 bits the same one ALU64 instruction; -x a subtraction from zero, ~x an exclusive or with -1
x < 1 with x at -1 true false under BPF_JLT, which reads -1 as the largest value the signed jump, BPF_JSLT, and its kin for <=, > and >=
if n then, n a number at 0 taken: 0 is true not taken: the word is zero the jump or nothing, from the register’s type (Branches)
for i = 1, n, s the iterations counted up front, so the loop never passes n by overflowing a back edge, which the verifier walks iteration by iteration or refuses the count as forprep computes it, with an unsigned division; a bounded loop where the compiler holds the three values, and otherwise the kernel’s iterator, handed the count, i the start plus the step times the pass it answers, and a may_goto header where the kernel has it
for with a step of 0 raises BPF_DIV by zero gives a count of 0, one pass through the body a step the compiler holds refused, “‘for’ step is zero”, and one the program computes tested, which stops the program
for i = 1, n past the kernel’s budget every pass bpf_iter_num_new takes the low 32 bits of the count, and past 8,388,608 answers an error and counts none, so the first pass alone runs; may_goto jumps once the budget runs out, which past the loop goes on with what the loop held so far the count tested against INT_MAX, the answer tested, and the header’s jump, each of which stops the program
break, return or a stop in such a loop leaves the loop a jump or an exit with the iterator held, which the verifier refuses: “Unreleased reference” bpf_iter_num_destroy on each way out, of each loop it leaves
goto continue a jump to the label the same jump, BPF_JA; to a label above it, a back edge the verifier walks pass by pass or refuses one JA to the label, and nothing for the CLOSE Lua’s parser leaves behind the jump, which no path reaches; a label above the goto refused, “‘while’, ‘repeat’ and a ‘goto’ backwards are not compiled yet”
f(x) the callee’s value, and its raise a call, which answers R0 and nothing else the arguments in R1 to R4, the address of the caller’s flag after them, and the flag read after the call
p:getint16(n) two bytes at n, signed; raises past the packet a load the verifier refuses until the program tests the end the offset tested unsigned against 65535 less the width, the end against data_end, and the sign extended by a shift left and an arithmetic shift right, where BPF_MEMSX would extend it
p:getint16(n) on TC the same, of the whole packet, past the skb’s linear head too a load stops at data_end, the end of the linear head the offset tested unsigned against INT_MAX, the helper’s own bound, since it takes the low 32 bits, then bpf_skb_load_bytes into the frame, which copies the bytes wherever the skb holds them, a stop where it answers non-zero, and the sign extended alike
h.field, h a view the field codec:view reads at the view’s offset plus the field’s, most significant byte first where the layout marks it bigendian, a bitfield out of its load; raises past the packet a load the verifier refuses until the program tests the end, in the kernel’s byte order the view’s offset, which its register holds, plus the field’s, read as p:getint16(n) reads, of the field’s width or of the load that covers a bitfield, through the parameter that holds the context or a packet; then what v.field of a struct value emits after its load
#p the bytes the data object views no instruction: the context holds the two bounds data_end less data, both read from the context; on TC len, as #skb
skb:data() on an skb that is not linear the whole packet, in a tc callback, read from a copy of it data_end ends the linear head, short of the packet the context register, which the reads hand to bpf_skb_load_bytes
skb:priority(v) raises outside 0 to U32_MAX a four-byte store keeps the low 32 bits a value the compiler holds checked against the field’s width, and one the program computes tested, which stops the program
m[k] the value, or nil the helper answers a pointer, or NULL, which the verifier lets the program read only past a test the helper, and a value typed “value or nil” until if v then reads it, on the branch that found it
limits:run(a, b) the callback’s verdict, or nil the kfunc’s int in the low 32 bits of R0, the only ones the verifier marks defined, and -1 for nil the name and the arguments in the frame, the context in R3, R0 sign-extended from 32 bits, and an answer typed “runtime answer” until a test reads it
if v then, v == nil on an answer false and true where the call answered nil a test of zero reads -1 as true a comparison with -1, the branch that finds it storing the zero word nil is everywhere else
m[k] = v, m[k] = nil the kernel’s bpf.map raises when the map refuses the update, or refuses the delete for any reason but an absent key the helper’s return code the helper, and a stop on any code but zero, or -ENOENT from a delete
m[k] with k outside the key’s spec string.pack raises a store that keeps the low bytes a key the compiler holds packed when the function is compiled, and one the program computes tested, which stops the program; a value alike
v.field of a struct value the field the codec decodes, most significant byte first where the layout marks it bigendian, and a bitfield as the codec’s view takes it out of its load a load the verifier refuses until the program tests the lookup, in the kernel’s byte order a load at the field’s offset and width, a BPF_END to big-endian where the layout marks a field wider than a byte bigendian, and the sign extended where the field is signed; for a bitfield, the load that covers it shifted right and masked, and the weight of its sign bit taken twice where it is signed, as the view computes it
return v, the verdict the kernel’s Lua answers a value outside its type’s actions, or a boolean, with the hook’s fallback r0, whose low 32 bits the kernel reads as an action, a boolean as 1 or 0 v tested against the first and the last action, which stops the program outside them; a boolean refused, and so is a known v outside them
any stop the error, <file>:<line>: <reason>, which the hook logs, rate limited, in luakfunc_invoke nothing: a program writes no log a block per check, after the instructions where the function stops, that adds one to this CPU’s count of the line’s place in the map luaebpf_errors, then hands bpf_trace_printk the line, a constant of the object’s .data.luaebpf_errors, under a window of bpf_jiffies64 that lets ten lines through every five seconds, and goes on to them; a callee’s stop is counted and logged by the callee alone

The shift pair stands for BPF_MEMSX, which the s390 JIT of 6.6 does not take, and the division of the magnitudes for BPF_SDIV and BPF_SMOD, which that JIT and others of the range read as the unsigned BPF_DIV and BPF_MOD. A read is of the kernel’s byte order on both sides: the kernel’s data object reads one so, and a map spec of another order is refused when the body declares it.

The sources are lua/lvm.c, luaV_idiv, luaV_mod, luaV_shiftl and forprep; the eBPF instruction set, for BPF_SDIV, BPF_SMOD, the shift mask, the zero divisor and BPF_MEMSX; bpf: Fix a sdiv overflow issue, for math.mininteger over -1; and kernel-notes.md, Rules that shape the subset and The packet a network program reads.

Reading a listing

A listing is llvm-objdump -d of the object lunatik bpf writes, with the hexadecimal left out, each jump given its target, and the Lua line each instruction came from. That line is in the object’s .BTF.ext, which llvm-objdump does not read: the verifier’s log quotes the same lines, and so does bpftool prog dump xlated of a loaded program, after the kernel has rewritten its context reads. An llvm-objdump older than LLVM 15, LLVM 14’s among them, prints a modulo as <unknown>; the listings write it %=. LLVM 14 prints may_goto as <unknown> too, which the listings write may_goto and its target.

The registers:

Every listing ends with the instructions where the function stops: XDP_ABORTED or TC_ACT_SHOT, and the exit, under the line that closes the function, whatever default verdict it names. After them comes a block for each check that stops it, under the check’s line, which counts the stop at the place of the line the check stands for, <file>:<line>: <reason>, logs that line and jumps to them. The packet shows one whole, and the other listings one line each, whose “logs” stands for the count and the line; each section’s program file is named for it, packet.bpf.lua for the packet, the name the line carries. A program that returns a value it computes tests it first against the actions of its type, two jumps to the block that logs “invalid action”, as The verdict shows.

The packet

A packet read, p:getuint16(16), held to the bounds of the packet:

local xdp    = require("bpf.xdp")
local action = require("linux.xdp").action

return xdp.program(function(ctx)
    local p = ctx:packet()
    local length = p:getuint16(16)
    return length
end, {default = action.PASS})

Byte 16 of an Ethernet frame that carries IPv4 is the total length of the IPv4 packet, two bytes in network order.

      line 4: return xdp.program(function(ctx)
   0  r6 = r1                       ctx
      line 5: local p = ctx:packet()
   1  r8 = r6                       the packet: the context register
   2  r7 = r8                       p
      line 6: local length = p:getuint16(16)
   3  r9 = r7
   4  r1 = 16
   5  *(u64 *)(r10 -40) = r1
   6  r4 = *(u64 *)(r10 -40)        the offset
   7  r1 = *(u32 *)(r9 +0)          xdp_md.data
   8  r2 = *(u32 *)(r9 +4)          xdp_md.data_end
   9  if r4 > 65533 goto 22         past 65535 less the width: stops
  10  r1 += r4                      data + 16
  11  r4 = r1
  12  r4 += 2
  13  if r4 > r2 goto 22            past the end of the packet: stops
  14  r1 = *(u16 *)(r1 +0)          two bytes, in the kernel byte order
  15  r8 = r1                       length
      line 7: return length
  16  r0 = r8
  17  if r0 s< 0 goto 53            not an XDP action: stops
  18  if r0 s> 4 goto 53
  19  exit
      line 8: end, {default = action.PASS})
  20  r0 = 0                        XDP_ABORTED, where the program stops
  21  exit
      line 6: local length = p:getuint16(16)
  22  r1 = 0                        the place of the line, the object's first
  23  *(u32 *)(r10 -24) = r1        the key, the place's index
  24  r1 = 0 ll                     the map luaebpf_errors, relocated by libbpf
  26  r2 = r10
  27  r2 += -24                     the key's address
  28  call 1                        bpf_map_lookup_elem: this CPU's count of the place
  29  if r0 == 0 goto 33
  30  r1 = *(u64 *)(r0 +0)
  31  r1 += 1                       one more stop, logged or not
  32  *(u64 *)(r0 +0) = r1
  33  call 118                      bpf_jiffies64: now
  34  r1 = 0 ll                     the window, in .data: two slots, relocated by libbpf
  36  r2 = *(u64 *)(r1 +0)          the jiffies it opened at
  37  r3 = r0
  38  r3 -= r2
  39  if r3 <= 5000 goto 42         DEFAULT_RATELIMIT_INTERVAL, at HZ 1000: still open
  40  *(u64 *)(r1 +0) = r0          a window opens now
  41  *(u64 *)(r1 +8) = 0           and has logged no line
  42  r2 = *(u64 *)(r1 +8)
  43  if r2 >= 10 goto 20           DEFAULT_RATELIMIT_BURST lines logged: stops, logging none
  44  r2 += 1
  45  *(u64 *)(r1 +8) = r2
  46  r1 = 0 ll                     the format "%s", in .data
  48  r2 = 3                        its bytes, the NUL among them
  49  r3 = 0 ll                     "packet.bpf.lua:6: out of bounds", in .data.luaebpf_errors
  51  call 6                        bpf_trace_printk: the line, to the trace ring
  52  goto 20
      line 7: return length
  53  to 83: logs "packet.bpf.lua:7: invalid action", then goto 20

Instruction 13 is the test the verifier asks for before instruction 14 reads: it takes the load only on the path where data + 16 + 2 is not past data_end. Instruction 9 stops the program at an offset no packet reaches, and at a negative one too, since the test is unsigned. Both jump to 22, which counts the stop in this CPU’s entry of the map luaebpf_errors at the place of the read’s line, the object’s first, then logs the line unless the window of the object, whose two words every program of it shares, has logged ten in the last five seconds, and then goes on to 20. A signed read, p:getint16(n), adds a shift left and an arithmetic shift right after the load, as instructions 15 and 16 of Floor division and modulo do for a byte. #p is data_end less data, both read from the context.

Frame Instructions What they compute The program returns
34 bytes, IPv4, total length 00 1c, 28 13, 14, 18, 53 18 is within 34; 00 1c read little-endian is 7168, 0x1c00, past XDP_REDIRECT 0, XDP_ABORTED, and “packet.bpf.lua:7: invalid action” counted and logged
14 bytes, an Ethernet header alone 13, 22 18 is past 14 0, XDP_ABORTED, and “packet.bpf.lua:6: out of bounds” counted and logged

The kernel’s Lua reads the same 7168 from p:getuint16(16) on arm64 and x86. Read through a view, as Views reads it, the length is 28 on every kernel.

Views

A view of the kernel’s IPv4 header reads its fields by name, here to drop a header that carries options or a packet longer than 1500 bytes:

local struct  = require("struct")
local vmlinux = require("bpf.vmlinux")
local xdp     = require("bpf.xdp")
local action  = require("linux.xdp").action
local eth     = require("linux.eth")

local LONGEST <const> = 1500

local iphdr = struct(vmlinux.layout("iphdr"))

return xdp.program(function(ctx)
    local p = ctx:packet()
    local ip = iphdr:view(p, eth.HLEN)
    if ip.ihl > 5 or ip.tot_len > LONGEST then
        return action.DROP
    end
    return action.PASS
end, {default = action.PASS})

bpf.vmlinux.layout reads iphdr from the kernel’s BTF: ihl is the low four bits of byte 0 on a little-endian kernel, and tot_len two bytes at 2, a __be16 the layout marks bigendian. The view is one register, r8, which holds the offset it starts at; a field is read through the context, r6, as p:getbyte(n) reads, at that offset plus the field’s.

      line 11: return xdp.program(function(ctx)
   0  r6 = r1                       ctx
      line 12: local p = ctx:packet()
   1  r8 = r6                       the packet: the context register
   2  r7 = r8                       p
      line 13: local ip = iphdr:view(p, eth.HLEN)
   3  *(u64 *)(r10 -40) = r7        p, the argument
   4  r1 = 14                       eth.HLEN, read at compile time
   5  *(u64 *)(r10 -48) = r1
   6  r1 = *(u64 *)(r10 -48)
   7  r8 = r1                       ip: the offset it starts at
      line 14: if ip.ihl > 5 or ip.tot_len > LONGEST then
   8  r4 = r8                       14, plus ihl's offset, 0
   9  r1 = *(u32 *)(r6 +0)          xdp_md.data, through ctx
  10  r2 = *(u32 *)(r6 +4)          xdp_md.data_end
  11  if r4 > 65534 goto 47         past 65535 less the width: stops
  12  r1 += r4
  13  r4 = r1
  14  r4 += 1
  15  if r4 > r2 goto 47            past the end of the packet: stops
  16  r1 = *(u8 *)(r1 +0)           the byte that holds ihl
  17  r5 = 15                       ihl's mask; its shift is 0
  18  r1 &= r5
  19  r9 = r1                       ip.ihl
  20  r1 = r9
  21  r2 = 5
  22  if r1 s> r2 goto 39
  23  r4 = r8
  24  r4 += 2                       14, plus tot_len's offset, 2
  25  to 31: the tests of 9 to 15, for two bytes, jumping to 78
  32  r1 = *(u16 *)(r1 +0)          two bytes, in the kernel byte order
  33  r1 = be16 r1                  BPF_END: tot_len is marked bigendian
  34  r9 = r1                       ip.tot_len
  35  r1 = 1500                     LONGEST
  36  *(u64 *)(r10 -40) = r1
  37  r1 = *(u64 *)(r10 -40)
  38  if r1 s>= r9 goto 42          tot_len is not past LONGEST
      line 15: return action.DROP
  39  r9 = 1                        XDP_DROP
  40  r0 = r9
  41  exit
      line 17: return action.PASS
  42  r9 = 2                        XDP_PASS
  43  r0 = r9
  44  exit
      line 18: end, {default = action.PASS})
  45  r0 = 0                        XDP_ABORTED, where the program stops
  46  exit
      line 14: if ip.ihl > 5 or ip.tot_len > LONGEST then
  47  to 77: logs "views.bpf.lua:14: out of bounds", then goto 45
  78  to 108: the same, for the read of tot_len

Instructions 8 to 16 are the read The packet lists, at the view’s offset plus the field’s, so a field past the end stops the program as a read does, at that field. Instruction 33 swaps the two bytes on a little-endian kernel and keeps them on a big-endian one, where they are already in order. A bitfield at a shift past 0, ip.version, is shifted right before the mask, and a signed one has the weight of its sign bit taken twice after it, an exclusive or and a subtraction. On TC the read is bpf_skb_load_bytes, as The context lists it.

Frame Instructions What they compute The program returns
34 bytes, IPv4, 45, total length 00 1c 16 to 22, 32 to 38 ihl is 5, not past 5; 00 1c swapped is 28, not past 1500 2, XDP_PASS
the same, 46 16 to 22 ihl is 6, past 5 1, XDP_DROP
14 bytes, an Ethernet header alone 15, 47 15 is past 14 0, XDP_ABORTED, and “views.bpf.lua:14: out of bounds” logged

A view holds no packet of its own: it is read through the parameter that holds the function’s context or a packet, and refused where the function has assigned it, since a copy in another register may be one a single path wrote; it is not passed to a function nor returned.

Arithmetic

a + b, a - b, a * b and the bitwise operators are one ALU64 instruction each, as instruction 30 of Loops adds a byte to s; -a is a subtraction from zero and ~a an exclusive or with -1. A constant that does not fit 32 bits signed is loaded whole, by LD_IMM64. //, % and the shifts compute something else than the instruction alone, and the compiler emits what lies between.

Floor division and modulo

local xdp    = require("bpf.xdp")
local action = require("linux.xdp").action

return xdp.program(function(ctx)
    local p = ctx:packet()
    local a = p:getint8(14)
    local b = p:getint8(15)
    return a // b
end, {default = action.PASS})

a and b are read from the packet, so lunatic folds nothing and the division is compiled. In the kernel’s Lua a / b is a // b, and it compiles to the same instructions.

      line 4: return xdp.program(function(ctx)
   0  r6 = r1                       ctx
      line 5: local p = ctx:packet()
   1  r8 = r6
   2  r7 = r8                       p
      line 6: local a = p:getint8(14)
   3  r9 = r7
   4  r1 = 14
   5  *(u64 *)(r10 -40) = r1
   6  r4 = *(u64 *)(r10 -40)        the offset
   7  r1 = *(u32 *)(r9 +0)          xdp_md.data
   8  r2 = *(u32 *)(r9 +4)          xdp_md.data_end
   9  if r4 > 65534 goto 58         past 65535 less the width: stops
  10  r1 += r4
  11  r4 = r1
  12  r4 += 1
  13  if r4 > r2 goto 58            past the end of the packet: stops
  14  r1 = *(u8 *)(r1 +0)           the byte
  15  r1 <<= 56                     its sign extended by a shift pair
  16  r1 s>>= 56
  17  r8 = r1                       a
      line 7: local b = p:getint8(15)
  18  to 32: the same read at 15, jumping to 89
  33  r9 = r1                       b
      line 8: return a // b
  34  r1 = r8                       a
  35  r2 = r9                       b
  36  if r2 == 0 goto 120           Lua raises: stops
  37  r3 = r1
  38  r3 ^= r2                      negative where the signs differ
  39  if r1 s>= 0 goto 41
  40  r1 = -r1                      |a|
  41  if r2 s>= 0 goto 43
  42  r2 = -r2                      |b|
  43  r4 = r1
  44  r4 %= r2                      BPF_MOD: |a| % |b|, unsigned
  45  r1 /= r2                      BPF_DIV: |a| / |b|, unsigned
  46  if r3 s>= 0 goto 50           the same signs: the quotient is |a| / |b|
  47  r1 = -r1                      the quotient truncated, as C's
  48  if r4 == 0 goto 50            exact: nothing to correct
  49  r1 += -1                      the floor: one less
  50  *(u64 *)(r10 -40) = r1        the result, in Lua register 4
  51  r0 = *(u64 *)(r10 -40)
  52  r0 = r0
  53  if r0 s< 0 goto 151           not an XDP action: stops
  54  if r0 s> 4 goto 151
  55  exit
      line 9: end, {default = action.PASS})
  56  r0 = 0                        XDP_ABORTED, where the program stops
  57  exit
      line 6: local a = p:getint8(14)
  58  to 88: logs "division.bpf.lua:6: out of bounds", then goto 56
      line 7: local b = p:getint8(15)
  89  to 119: logs "division.bpf.lua:7: out of bounds", then goto 56
      line 8: return a // b
 120  to 150: logs "division.bpf.lua:8: attempt to divide by zero", then goto 56
 151  to 181: logs "division.bpf.lua:8: invalid action", then goto 56

The magnitudes are divided unsigned, and the quotient takes the sign C’s division gives it. The correction is then Lua’s luaV_idiv: the quotient less one where the remainder is not zero and the signs differ. Bytes 14 and 15 at f9 02, so a is -7 and b is 2:

Instructions What they compute Value
9, 13 14 is under 65535, and 15 is within the 16 bytes no jump
14 to 17 the byte f9, its sign extended a is -7
33 the byte 02 b is 2
36 b is not 0 no jump
37, 38 -7 ^ 2 r3 is -5, negative: the signs differ
39 to 42 the magnitudes r1 is 7, r2 is 2
44 7 % 2 r4 is 1
45 7 / 2 r1 is 3
46, 47 the signs differ: 0 - 3 r1 is -3, what BPF_SDIV alone gives
48 the remainder is not 0 no jump
49 -3 - 1 r1 is -4
50 to 52 r0 is -4
53 -4 is below XDP_ABORTED, 0 jump to 151, which logs “division.bpf.lua:8: invalid action”, then to 56: the program stops and returns 0

Lua computes -7 // 2 as -4, which is no XDP action, so the program stops and returns 0, XDP_ABORTED (The verdict). With b at 0, instruction 36 jumps to 120, which logs “division.bpf.lua:8: attempt to divide by zero”, and the program returns 0 too.

a % b is the same program with return a % b; instructions 0 to 33 read a and b as above:

      line 8: return a % b
  34  r1 = r8                       a
  35  r2 = r9                       b
  36  if r2 == 0 goto 120           Lua raises: stops
  37  r3 = r1                       a, whose sign the remainder takes
  38  r4 = r2
  39  if r1 s>= 0 goto 41
  40  r1 = -r1                      |a|
  41  if r4 s>= 0 goto 43
  42  r4 = -r4                      |b|
  43  r1 %= r4                      BPF_MOD: |a| % |b|, unsigned
  44  if r3 s>= 0 goto 46
  45  r1 = -r1                      the remainder truncated, as C's
  46  if r1 == 0 goto 50            exact: 0
  47  r3 ^= r2                      negative where the signs differ
  48  if r3 s>= 0 goto 50           the same signs: nothing to correct
  49  r1 += r2                      the remainder takes the sign of b
  50  *(u64 *)(r10 -40) = r1
  51  r0 = *(u64 *)(r10 -40)
  52  r0 = r0
  53  if r0 s< 0 goto 151           not an XDP action: stops
  54  if r0 s> 4 goto 151
  55  exit
      line 9: end, {default = action.PASS})
  56  r0 = 0                        XDP_ABORTED, where the program stops
  57  exit
      line 8: return a % b
 120  to 150: logs "division.bpf.lua:8: attempt to perform 'n%0'", then goto 56
 151  to 181: logs "division.bpf.lua:8: invalid action", then goto 56

With -7 and 2, instruction 43 gives 1, the remainder of 7 by 2, and instruction 45 gives -1, what BPF_SMOD alone gives. The correction is Lua’s luaV_mod, which reads the remainder’s sign, and that is the sign of a: -7 ^ 2 is negative, so instruction 49 adds 2, and the program returns 1, as Lua computes -7 % 2.

Shifts

The same program with return a << n, n read from byte 15; instructions 0 to 33 read a and n:

      line 8: return a << n
  34  r1 = r8                       a
  35  r2 = r9                       n
  36  if r2 s< 0 goto 40            a negative count shifts right
  37  if r2 s>= 64 goto 44          64 or more: every bit shifted out
  38  r1 <<= r2
  39  goto 45
  40  if r2 s<= -64 goto 44         -64 or less: every bit shifted out
  41  r2 = -r2
  42  r1 >>= r2                     logical: Lua fills with zeros
  43  goto 45
  44  r1 = 0
  45  *(u64 *)(r10 -40) = r1
  46  r0 = *(u64 *)(r10 -40)
  47  r0 = r0
  48  if r0 s< 0 goto 115           not an XDP action: stops
  49  if r0 s> 4 goto 115
  50  exit
      line 9: end, {default = action.PASS})
  51  r0 = 0                        XDP_ABORTED, where the program stops
  52  exit
      lines 6 and 7: the reads of a and n
  53  to 114: their blocks, as in Floor division and modulo
      line 8: return a << n
 115  to 145: logs "division.bpf.lua:8: invalid action", then goto 51

This is Lua’s luaV_shiftl. BPF_LSH alone masks the count to its low six bits, so 1 << 64 would give 1, and 3 << -1, a shift by 63, math.mininteger. Bytes 14 and 15 at 03 ff, so a is 3 and n is -1:

Instructions What they compute Value
36 -1 is negative jump to 40
40 -1 is above -64 no jump
41 the count negated r2 is 1
42 3 >> 1 r1 is 1
45 to 50 r0 is 1, XDP_DROP the program returns 1

Lua computes 3 << -1 as 1. With 1 and 64, instruction 37 jumps to 44, and the program returns 0. a >> n is a << -n, as in Lua: the count is negated first, and the same tests follow.

Comparisons and truth

local xdp    = require("bpf.xdp")
local action = require("linux.xdp").action

return xdp.program(function(ctx)
    local p = ctx:packet()
    local a = p:getint8(14)
    local small = a < 1
    local n = p:getbyte(15)
    if n and small then
        return action.DROP
    end
    return action.PASS
end, {default = action.PASS})

a < 1 is signed, as every comparison of Lua integers is. n is a number, and a number is true in Lua, 0 included, so n and small is small: the compiler knows it from the type of n, and tests small alone. Instructions 0 to 17 read a into r8.

      line 7: local small = a < 1
  18  r1 = r8                       a
  19  r2 = 1
  20  if r1 s< r2 goto 23           BPF_JSLT, the signed jump
  21  r9 = 0                        small is false
  22  goto 24
  23  r9 = 1                        small is true
      line 8: local n = p:getbyte(15)
  24  to 36: the read of byte 15, unsigned, so with no shift pair, jumping to 84
  37  *(u64 *)(r10 -40) = r1        n, in Lua register 4
      line 9: if n and small then
                                    nothing for n: a number is true
  38  r1 = r9                       small
  39  r2 = 0
  40  if r1 == r2 goto 46           false: to the PASS
      line 10: return action.DROP
  41  r1 = 1                        XDP_DROP, read from linux.xdp
  42  to 45: r0 is 1, and the exit
      line 12: return action.PASS
  46  r1 = 2                        XDP_PASS
  47  to 50: r0 is 2, and the exit
      line 13: end, {default = action.PASS})
  51  r0 = 0                        XDP_ABORTED, where Lua would raise
  52  exit
      line 6: local a = p:getint8(14)
  53  to 83: logs "comparisons.bpf.lua:6: out of bounds", then goto 51
      line 8: local n = p:getbyte(15)
  84  to 114: logs "comparisons.bpf.lua:8: out of bounds", then goto 51

Bytes 14 and 15 at ff 00, so a is -1 and n is 0:

Instructions What they compute Value
20 -1 s< 1 jump to 23
23 small is true
37 the byte 00 n is 0
line 9 n is a number, so true, 0 included no instruction
40 small is not 0 no jump
41 to 45 r0 is 1 the program returns XDP_DROP

Lua takes both branches the same way. An unsigned BPF_JLT would read -1 as the largest value and leave small false, and a test of n as C makes it would read 0 as false: either one would pass the packet. A variable that holds a number on one path and nil on another has no such reading, since the kernel holds nil, false and 0 as the same word, and a test or an == of it is refused.

Values of the body

local struct  = require("struct")
local vmlinux = require("bpf.vmlinux")
local xdp     = require("bpf.xdp")
local action  = require("linux.xdp").action
local eth     = require("linux.eth")

local protocols = require("linux.socket").ipproto

local iphdr = struct(vmlinux.layout("iphdr"))

return xdp.program(function(ctx)
    local p = ctx:packet()
    local protocol = iphdr:view(p, eth.HLEN).protocol
    if protocol == protocols.UDP then
        return action.DROP
    end
    return action.PASS
end, {default = action.PASS})

protocols is a table the body required, linux.socket.ipproto, which autogen generates from the kernel’s IPPROTO_ names. The function reads protocols.UDP when it is compiled, and the program holds its value, 17. Instructions 0 to 18 read the IPv4 header’s protocol into r8, as Views reads a field.

      line 14: if protocol == protocols.UDP then
  19  r9 = 17                       protocols.UDP, read at compile time
  20  r1 = r8                       protocol
  21  if r1 != r9 goto 25
      line 15: return action.DROP
  22  r9 = 1                        XDP_DROP
  23  r0 = r9
  24  exit
      line 17: return action.PASS
  25  r9 = 2                        XDP_PASS
  26  r0 = r9
  27  exit
      line 18: end, {default = action.PASS})
  28  r0 = 0                        XDP_ABORTED, where Lua would raise
  29  exit
      line 13: local protocol = iphdr:view(p, eth.HLEN).protocol
  30  to 60: logs "values.bpf.lua:13: out of bounds", then goto 28

An IPv4 frame whose protocol is 17, UDP, returns 1, XDP_DROP; one whose protocol is 6, TCP, returns 2, and a 14-byte frame stops at the read, logs “values.bpf.lua:13: out of bounds” and returns 0, XDP_ABORTED. An index the program computes, protocols[name] with name read from the packet, has no value the compiler can read, and is refused; so are #protocols and a write to the table.

The verdict

local xdp    = require("bpf.xdp")
local action = require("linux.xdp").action

return xdp.program(function(ctx)
    local p = ctx:packet()
    local verdict = p:getbyte(14)
    return verdict
end, {default = action.PASS})

Instructions 0 to 15 read byte 14 into r8, as The packet reads two bytes, their bounds tests jumping to 22.

      line 7: return verdict
  16  r0 = r8
  17  if r0 s< 0 goto 53            below XDP_ABORTED: stops
  18  if r0 s> 4 goto 53            past XDP_REDIRECT: stops
  19  exit
      line 8: end, {default = action.PASS})
  20  r0 = 0                        XDP_ABORTED, where the program stops
  21  exit
      line 6: local verdict = p:getbyte(14)
  22  to 52: logs "verdict.bpf.lua:6: out of bounds", then goto 20
      line 7: return verdict
  53  to 83: logs "verdict.bpf.lua:7: invalid action", then goto 20

A verdict is one of the actions of its program type: linux.xdp.action, from 0, XDP_ABORTED, to 4, XDP_REDIRECT, and linux.tc.action, from -1, TC_ACT_UNSPEC, to 8, TC_ACT_TRAP, which a TC program’s two jumps test instead. The kernel’s Lua holds a kernel script’s callback to the same range, in luakfunc_action, and answers any other value with the hook’s fallback, as it answers a callback that raised and logs “invalid action”: the compiled program stops there, and logs the same, as it does where Lua would raise. The eBPF instruction alone would hand the kernel the low 32 bits of r0, which it reads as an action whatever they are, so the -4 of Floor division and modulo would reach it as 4294967292, and an XDP value that is no action drops the packet.

Byte 14 Instructions The program returns
01 17, 18: 1 is within 0 to 4 1, XDP_DROP
09 18: 9 is past 4 0, XDP_ABORTED, and “verdict.bpf.lua:7: invalid action” logged

A verdict the compiler knows takes no test, and one outside the actions is refused: return 11 gives “the verdict 11 is not an XDP action”. So is a verdict that may be a boolean, which no action is: return p:getbyte(14) == 1 gives “the verdict may be a boolean here; return an action”, and is written if p:getbyte(14) == 1 then return action.DROP end, then return action.PASS.

The context

local tc     = require("bpf.tc")
local action = require("linux.tc").action

return tc.program(function(skb)
    local p = skb:data()
    local class = p:getint8(15)
    skb:priority(class)
    return action.OK
end, {default = action.OK})

The context of a TC program is the kernel’s __sk_buff, read at the offsets its BTF gives: len at 0 and priority at 32. The packet is the context register under another name, and a read of it is bpf_skb_load_bytes, helper 26, which copies the bytes into the frame at r10 - 32 wherever the skb holds them, past its linear head too, and answers non-zero past the packet: the kernel’s skb:data() reads the same bytes from a copy skb_copy_bits makes, which the helper calls for the bytes past the head.

      line 4: return tc.program(function(skb)
   0  r6 = r1                       skb
      line 5: local p = skb:data()
   1  r8 = r6
   2  r7 = r8                       p
      line 6: local class = p:getint8(15)
   3  r9 = r7
   4  r1 = 15
   5  *(u64 *)(r10 -40) = r1
   6  r2 = *(u64 *)(r10 -40)        the offset, 15
   7  r1 = r9                       the context
   8  if r2 > 2147483647 goto 34    past INT_MAX: stops
   9  r3 = r10
  10  r3 += -32                     the frame's value slot
  11  r4 = 1                        the width
  12  call 26                       bpf_skb_load_bytes
  13  if r0 != 0 goto 34            past the end of the packet: stops
  14  r1 = *(u8 *)(r10 -32)         byte 15
  15  r1 <<= 56
  16  r1 s>>= 56                    its sign extended
  17  r8 = r1                       class
      line 7: skb:priority(class)
  18  *(u64 *)(r10 -40) = r6
  19  *(u64 *)(r10 -48) = r8
  20  r2 = *(u64 *)(r10 -48)        class
  21  r5 = r2
  22  r5 >>= 32
  23  if r5 != 0 goto 65            outside 0 to 4294967295: stops
  24  r1 = *(u64 *)(r10 -40)
  25  *(u32 *)(r1 +32) = r2         __sk_buff.priority
  26  r1 = *(u64 *)(r10 -40)
  27  r2 = *(u32 *)(r1 +32)         what skb:priority returns
  28  r9 = r2
      line 8: return action.OK
  29  r9 = 0                        TC_ACT_OK
  30  r0 = r9
  31  exit
      line 9: end, {default = action.OK})
  32  r0 = 2                        TC_ACT_SHOT, where Lua would raise
  33  exit
      line 6: local class = p:getint8(15)
  34  to 64: logs "context.bpf.lua:6: out of bounds", then goto 32
      line 7: skb:priority(class)
  65  to 95: logs "context.bpf.lua:7: out of bounds", then goto 32

A frame whose byte 15 is 3 leaves its priority at 3 and returns 0, TC_ACT_OK. One whose byte 15 is ff gives -1, which instruction 23 finds outside the range: the program stops before the store, logs “context.bpf.lua:7: out of bounds” and returns 2, TC_ACT_SHOT, so the packet is dropped and its priority stays where it was.

Maps

local map     = require("bpf.map")
local struct  = require("struct")
local vmlinux = require("bpf.vmlinux")
local xdp     = require("bpf.xdp")
local action  = require("linux.xdp").action
local eth     = require("linux.eth")

local iphdr = struct(vmlinux.layout("iphdr"))

local seen = map.hash("seen", {key = "I4", value = "I8", entries = 256})

return xdp.program(function(ctx)
    local p = ctx:packet()
    local proto = iphdr:view(p, eth.HLEN).protocol
    local n = seen[proto]
    if n then
        seen[proto] = n + 1
    else
        seen[proto] = 1
    end
    return action.PASS
end, {default = action.PASS})

seen[proto] is a lookup, whose value is a number or nil until if n then tests it. The helpers are called by number: 1 is bpf_map_lookup_elem, 2 bpf_map_update_elem. llvm-objdump -r prints the map as a relocation, which libbpf fills in when it loads the object. Instructions 0 to 18 read the IPv4 header’s protocol into r8, as Views reads a field.

      line 15: local n = seen[proto]
  19  r5 = r8                       proto
  20  r5 >>= 32
  21  if r5 != 0 goto 112           wider than I4: stops
  22  *(u64 *)(r10 -24) = 0
  23  *(u32 *)(r10 -24) = r8        the key, four bytes
  24  r1 = map seen                 two slots, relocated by libbpf
  26  r2 = r10
  27  r2 += -24                     the key, by address
  28  call 1                        bpf_map_lookup_elem
  29  r9 = r0                       a pointer to the value, or NULL
      line 16: if n then
  30  if r9 == 0 goto 34            NULL: n is nil
  31  r1 = *(u64 *)(r9 +0)          found: the value, eight bytes
  32  r9 = r1                       n
  33  goto 35
  34  goto 57                       the else branch
      line 17: seen[proto] = n + 1
  35  to 38: n + 1, kept in Lua register 5
  39  to 41: the key held to I4 again, jumping to 143
  42  to 46: the key at r10 -24, the value at r10 -32
  47  to 52: r1 the map, r2 and r3 the key and the value, by address
  53  r4 = 0                        BPF_ANY: insert or replace
  54  call 2                        bpf_map_update_elem
  55  if r0 != 0 goto 174           the map refuses: stops
  56  goto 74
      line 19: seen[proto] = 1
  57  to 73: the same update, with the value 1, its tests jumping to 205 and 236
      line 21: return action.PASS
  74  to 78: r0 is 2, and the exit
      line 22: end, {default = action.PASS})
  79  r0 = 0                        XDP_ABORTED, where Lua would raise
  80  exit
      line 14: local proto = iphdr:view(p, eth.HLEN).protocol
  81  to 111: logs "maps.bpf.lua:14: out of bounds", then goto 79
      line 15: local n = seen[proto]
 112  to 142: logs "maps.bpf.lua:15: unsigned overflow", then goto 79
      line 17: seen[proto] = n + 1
 143  to 173: logs "maps.bpf.lua:17: unsigned overflow", then goto 79
 174  to 204: logs "maps.bpf.lua:17: the map 'seen' refused the update", then goto 79
      line 19: seen[proto] = 1
 205  to 266: the same two, at line 19

Instruction 31 reads the value only past instruction 30: the verifier refuses a read through a pointer that may be NULL, and the compiler refuses to use n before if n then tests it. Four frames, one after the other, over a map that starts empty:

Frame proto Path seen after
IPv4, TCP 6 30 jumps: nil, so 57 to 73 store 1 {[6] = 1}
the same again 6 30 goes on: 1, so 35 to 55 store 2 {[6] = 2}
ARP, 42 bytes 0, the byte where IPv4 keeps its protocol nil, so 1 is stored {[6] = 2, [0] = 1}
14 bytes none the read of the protocol jumps to 81, which logs “maps.bpf.lua:14: out of bounds”, before the lookup unchanged

Each run returns 2 but the last, which stops and returns 0, XDP_ABORTED. A kernel script reads the same map with the kernel’s bpf.map, as map.hash("/sys/fs/bpf/lunatik/<script>/seen", "I4", "I8")<code>. A </code>struct value is read a field at a time, by a load at the field’s offset and width, which a field wider than a byte that the layout marks bigendian follows with BPF_END: a byte swap on a little-endian kernel, so the field reads as the codec decodes it. A bitfield is the load that covers it, shifted right by its shift and masked by its mask, ihl of an IPv4 header the low nibble of its first byte on a little-endian kernel.

Loops

local xdp    = require("bpf.xdp")
local action = require("linux.xdp").action

return xdp.program(function(ctx)
    local p = ctx:packet()
    local s = 0
    for i = 14, 17 do
        s = s + p:getbyte(i)
    end
    return s
end, {default = action.PASS})

The compiler holds the three values, 14, 17 and 1, so it counts the passes when it compiles, as Lua’s forprep counts them when it runs: (17 - 14) // 1 is 3 more after the first. The loop is a back edge, which the verifier walks pass by pass. In Lua 5.5 a loop takes three registers, the count, the step and i: here r9, r10 - 40 and r10 - 48. Instructions 0 to 2 take ctx and p.

      line 6: local s = 0
   3  r8 = 0                        s
      line 7: for i = 14, 17 do
   4  to 8: the three values, as the bytecode loads them
   9  r9 = 3                        the count, known when compiled
  10  to 11: r10 -40 is 1           the step
  12  to 13: r10 -48 is 14          i
      line 8: s = s + p:getbyte(i)
  14  to 26: the read of byte i, held to the bounds of the packet; past them, goto 47
  27  to 29: s in r1, the byte in r2
  30  r1 += r2                      s + the byte
  31  r8 = r1                       s
      line 7: for i = 14, 17 do
  32  r1 = r9
  33  if r1 == 0 goto 41            no pass left
  34  r1 += -1
  35  r9 = r1
  36  to 39: i + the step
  40  goto 14                       the back edge
      line 10: return s
  41  r0 = r8
  42  if r0 s< 0 goto 78            not an XDP action: stops
  43  if r0 s> 4 goto 78
  44  exit
      line 11: end, {default = action.PASS})
  45  r0 = 0                        XDP_ABORTED, where the program stops
  46  exit
      line 8: s = s + p:getbyte(i)
  47  to 77: logs "loops.bpf.lua:8: out of bounds", then goto 45
      line 10: return s
  78  to 108: logs "loops.bpf.lua:10: invalid action", then goto 45

Bytes 14 to 17 at 01 02 03 04 make four passes, i at 14, 15, 16 and 17, and s is 10, which is no XDP action, so the program stops and returns 0, XDP_ABORTED; at 00 01 01 02, s is 4, and the program returns 4, XDP_REDIRECT. A 16-byte frame stops at the third pass, whose read of byte 16 is past the packet, logs “loops.bpf.lua:8: out of bounds” and returns 0 too.

With a bound the program computes, the count is computed when the loop starts, with an unsigned division as forprep does, and handed to the kernel’s iterator, bpf_iter_num_new, which counts the passes after the first and answers each with its number. The verifier walks a pass or two of such a loop rather than all of them, and i is the start plus the step times that number: an i each pass carried to the next would differ from every pass the verifier walked once the body compared it with a number or read the packet at it. Each pass goes through may_goto too, which the kernel takes from Linux 6.9. Compiled for 6.9 or later:

local xdp    = require("bpf.xdp")
local action = require("linux.xdp").action

return xdp.program(function(ctx)
    local p = ctx:packet()
    local s = 0
    for i = 14, #p - 1 do
        s = s + p:getbyte(i)
    end
    if s > 100 then
        return action.DROP
    end
    return action.PASS
end, {default = action.PASS})
      line 6: local s = 0
   3  r8 = 0                        s
      line 7: for i = 14, #p - 1 do
   4  r9 = 14                       the start
   5  to 12: #p less 1 at r10 -40   the bound
  13  to 14: r10 -48 is 1           the step
  15  to 18: the start in r1, the bound in r2, the step in r3 and r5
  19  if r3 s< 0 goto 23            a step below 0 counts down
  20  if r1 s> r2 goto 75           no pass at all
  21  r2 -= r1                      the bound less the start
  22  goto 28
  23  to 27: the same, counted down
  28  if r3 == 1 goto 30
  29  r2 /= r3                      over the step, unsigned: the count
  30  r9 = r1                       the start
  31  to 32: r10 -40 is the step, r10 -48 is i
  33  if r2 >= 2147483647 goto 90   a count no int holds: stops
  34  to 38: the count + 1 in r3, 1 in r2, the iterator at r10 -80 in r1
  39  call bpf_iter_num_new         passes 1 to the count, after the first
  40  to 41: r0 sign-extended from 32 bits
  42  if r0 != 0 goto 121           more than it counts: stops
      line 8: s = s + p:getbyte(i)
  43  to 55: the read of byte i, held to the bounds of the packet; past them, goto 152
  56  to 60: r8 = r8 + the byte     s
      line 7: for i = 14, #p - 1 do
  61  to 62: r1 = r10 - 80          the iterator
  63  call bpf_iter_num_next        the number of the next pass, or NULL
  64  if r0 == 0 goto 72            no pass left
  65  may_goto 183                  the budget spent: stops
  66  r1 = *(u32 *)(r0 + 0)         the pass
  67  to 69: r1 * the step + r9
  70  *(u64 *)(r10 -48) = r1        i
  71  goto 43                       the back edge
  72  to 74: bpf_iter_num_destroy   past the loop
      line 10: if s > 100 then
  75  to 77: if s s<= 100 goto 81
      line 11: return action.DROP
  78  to 80: r0 = 1, exit           XDP_DROP
      line 13: return action.PASS
  81  to 83: r0 = 2, exit           XDP_PASS
      line 7: for i = 14, #p - 1 do
  84  to 86: bpf_iter_num_destroy   where the loop stops
  87  goto 88
      line 14: end, {default = action.PASS})
  88  r0 = 0                        XDP_ABORTED, where the program stops
  89  exit
      line 7: for i = 14, #p - 1 do
  90  to 120: logs "loops.bpf.lua:7: 'for' outruns the budget the kernel gives it", then goto 88
 121  to 151: the same line, then goto 84
      line 8: s = s + p:getbyte(i)
 152  to 182: logs "loops.bpf.lua:8: out of bounds", then goto 84
      line 7: for i = 14, #p - 1 do
 183  to 213: the line of 90, then goto 84

A 17-byte frame whose last three bytes are 10 20 30 makes three passes: the count is 2, the iterator answers 1 and 2, i is 14, 15 and 16, s is 96, and the program returns 2, XDP_PASS; one byte more, 40, makes s 160, and the program returns 1, XDP_DROP. A 14-byte frame makes no pass, the start past the bound at 20. No packet is long enough to reach 33 or 42, which a bound read from the packet reaches: a count of 2,147,483,647 or more, which no int holds, stops at 33, and one past 8,388,608, which bpf_iter_num_new answers with an error, at 42, both before the first pass. The kernel takes the jump of instruction 65 once the budget it gives the loops of a call of the function runs out, about a quarter of a second where its JIT times them and 8,388,608 passes after the first where it counts them, which one loop alone never spends. Each way out of the loop destroys the iterator, or the verifier refuses the program: 72 when no pass is left, 84 when the program stops in the loop, and the same call before the jump of a break or a goto out of it, and before a return in it, the blocks of 121, 152 and 183 going on to 84. Compiled for a kernel before 6.9, the listing has no instruction 65 and no block 183, and every instruction after 65 comes one earlier.

Goto

local xdp    = require("bpf.xdp")
local action = require("linux.xdp").action

return xdp.program(function(ctx)
    local p = ctx:packet()
    local n = 0
    for i = 14, 17 do
        if p:getbyte(i) == 0 then goto continue end
        n = n + 1
        ::continue::
    end
    return n
end, {default = action.PASS})

The label closes the body, so goto continue jumps to the loop’s step, as continue does in C. Lua’s parser writes a CLOSE after each goto, and moves it before the jump only when the jump leaves the scope of a local a closure captured or of a to-be-closed variable, both refused in a compiled function: the CLOSE stays behind the jump, no path reaches it, and it emits nothing. Instructions 0 to 13 take ctx and p, set n to 0 and start the loop as the one above: the count in r9, the step in r10 - 40 and i in r10 - 48.

      line 8: if p:getbyte(i) == 0 then goto continue end
  14  to 27: the read of byte i, held to the bounds of the packet; past them, goto 51
  28  r1 = *(u64 *)(r10 -56)        the byte
  29  r2 = 0
  30  if r1 != r2 goto 32           not 0: on to n = n + 1
  31  goto 36                       goto continue: to the loop's step
      line 9: n = n + 1
  32  to 34: n + 1 in r1
  35  r8 = r1                       n
      line 7: for i = 14, 17 do
  36  r1 = r9
  37  if r1 == 0 goto 45            no pass left
  38  to 43: the count less one, i + the step
  44  goto 14                       the back edge
      line 12: return n
  45  r0 = r8
  46  if r0 s< 0 goto 82            not an XDP action: stops
  47  if r0 s> 4 goto 82
  48  exit
      line 13: end, {default = action.PASS})
  49  r0 = 0                        XDP_ABORTED, where the program stops
  50  exit
      line 8: if p:getbyte(i) == 0 then goto continue end
  51  to 81: logs "goto.bpf.lua:8: out of bounds", then goto 49
      line 12: return n
  82  to 112: logs "goto.bpf.lua:12: invalid action", then goto 49

Bytes 14 to 17 at 01 00 03 04:

Pass Byte Path n after
i at 14 01 30 jumps to 32 1
i at 15 00 31 jumps to 36, past n = n + 1 1
i at 16 03 30 jumps to 32 2
i at 17 04 30 jumps to 32 3

At 37 no pass is left, and the program returns 3, XDP_TX. A 16-byte frame stops at the third pass, whose read of byte 16 is past the packet, logs “goto.bpf.lua:8: out of bounds” and returns 0. A goto out of the loop is the same JA, to the instructions after the loop. A goto to a label above it makes a loop the compiler counts no pass of, and is refused: “‘while’, ‘repeat’ and a ‘goto’ backwards are not compiled yet”.

Calls

local xdp    = require("bpf.xdp")
local action = require("linux.xdp").action

local function half(n, d)
    return n // d
end

return xdp.program(function(ctx)
    local p = ctx:packet()
    local a = p:getint8(14)
    local b = p:getint8(15)
    local v = half(a, b)
    return v
end, {default = action.PASS})

half is compiled once, as a BPF-to-BPF function in the section .text. Its arguments go in r1 and r2, and the address of the caller’s flag in the register after them, r3. Instructions 0 to 33 of the program read a and b, their tests jumping to 52 and 83.

   the program, in the section xdp
      line 12: local v = half(a, b)
  34  to 37: r1 is a, r2 is b
  38  *(u64 *)(r10 -8) = 0          the flag, lowered
  39  r3 = r10
  40  r3 += -8                      its address, the third argument
  41  call half                     relocated by libbpf
  42  r1 = *(u64 *)(r10 -8)
  43  if r1 != 0 goto 50            raised: stops
  44  *(u64 *)(r10 -40) = r0        v
      line 13: return v
  45  to 49: r0 is v, tested against the actions, jumping to 114 outside them, and the exit
      line 14: end, {default = action.PASS})
  50  r0 = 0                        XDP_ABORTED, where the program stops
  51  exit
      line 10: local a = p:getint8(14)
  52  to 82: logs "calls.bpf.lua:10: out of bounds", then goto 50
      line 11: local b = p:getint8(15)
  83  to 113: logs "calls.bpf.lua:11: out of bounds", then goto 50
      line 13: return v
 114  to 144: logs "calls.bpf.lua:13: invalid action", then goto 50

   half, in the section .text
      line 4: local function half(n, d)
   0  *(u64 *)(r10 -16) = r3        the address of the flag
   1  r6 = r1                       n
   2  r7 = r2                       d
      line 5: return n // d
   3  to 18: the floor division above, its zero test jumping to 26
  19  to 21: r0 is n // d, and the exit
      line 6: end
  22  r1 = *(u64 *)(r10 -16)
  23  *(u64 *)(r1 +0) = 1           the flag, raised
  24  r0 = 0                        the verifier reads r0 on every exit
  25  exit
      line 5: return n // d
  26  to 56: logs "calls.bpf.lua:5: attempt to divide by zero", then goto 22

Bytes 14 and 15 at 07 00: a is 7 and b is 0. In half the zero test jumps to 26, which logs “calls.bpf.lua:5: attempt to divide by zero”, and 22 raises the flag and returns; at 43 the program reads the flag and jumps to 50, logging nothing, so it returns 0, XDP_ABORTED, as if the division had been in the program’s own function. With -7 and 2, half returns -4, which half may, since only the program’s own return is a verdict; -4 is no XDP action, so the program stops and returns 0 too.

The call returns to the program, which reads the flag before it goes on. return half(a, b) would be a tail call, which Lua runs without coming back, and is refused: “a tail call cannot be compiled”.

Calls into a kernel Lua runtime

local struct  = require("struct")
local vmlinux = require("bpf.vmlinux")
local xdp     = require("bpf.xdp")
local lunatik = require("lunatik")
local action  = require("linux.xdp").action
local eth     = require("linux.eth")

local WORD <const> = 4 -- bytes in a unit of ihl

local iphdr  = struct(vmlinux.layout("iphdr"))
local udphdr = struct(vmlinux.layout("udphdr"))

local limits = lunatik.runtime("limits")

return xdp.program(function(ctx)
    local p = ctx:packet()
    local ip = iphdr:view(p, eth.HLEN)
    local port = udphdr:view(p, eth.HLEN + ip.ihl * WORD).dest
    local verdict = limits:run(port, #p)
    if verdict then
        return verdict
    end
    return action.TX
end, {default = action.PASS})

limits is the runtime the body named: the kernel script limits of Calls into a kernel Lua runtime, whose callback drops a frame to port 53 longer than 512 bytes and returns nothing for any other. limits:run reads the runtime’s one method, which emits nothing: the runtime is a value of the body, and its register holds no word. The call lowers to the kfunc of an XDP program, bpf_luaxdp_run: r1 the address of the name, r2 its length with the NUL, r3 the context, r4 the address of the arguments and r5 their size. The name and the arguments go below the frame’s deepest word, r10 - 64 here, where nothing outlives the call. TX on the nil branch tells that branch from the callback’s verdict and from the XDP_ABORTED a stop returns. Instructions 0 to 44 take ctx and p, and read ip.ihl and the UDP header’s dest, past the IPv4 header by ihl, into port, in r9, as Views reads a field, their tests jumping to 86 and 117.

      line 19: local verdict = limits:run(port, #p)
  45  *(u64 *)(r10 -56) = r9        port, in Lua register 6, past the method and limits
  46  r1 = *(u32 *)(r7 +0)          xdp_md.data
  47  r2 = *(u32 *)(r7 +4)          xdp_md.data_end
  48  r2 -= r1                      #p
  49  *(u64 *)(r10 -64) = r2        #p, in Lua register 7
  50  r1 = 126943822178668 ll       the name: two slots, decoded below
  52  *(u64 *)(r10 -88) = r1        the name, below the frame
  53  to 56: port at r10 -80 and #p at r10 -72, the arguments in order
  57  r1 = r10
  58  r1 += -88                     r1: the name, by address
  59  r2 = 7                        r2: "limits" and its NUL
  60  r3 = r6                       r3: the context
  61  r4 = r10
  62  r4 += -80                     r4: the arguments, by address
  63  r5 = 16                       r5: their size, two words
  64  call -1                       bpf_luaxdp_run: decoded below
  65  r0 <<= 32                     the int's 32 bits, the only ones defined
  66  r0 s>>= 32                    sign-extended, so -1 stays -1
  67  *(u64 *)(r10 -40) = r0        verdict, in Lua register 4
      line 20: if verdict then
  68  r1 = *(u64 *)(r10 -40)
  69  if r1 == -1 goto 71           the kfunc's -1: nil
  70  goto 74                       a verdict: the number branch
  71  r1 = 0                        nil, the zero word it is everywhere else
  72  *(u64 *)(r10 -40) = r1
  73  goto 79                       the nil branch
      line 21: return verdict
  74  r0 = *(u64 *)(r10 -40)
  75  r0 = r0
  76  if r0 s< 0 goto 148           not an XDP action: stops
  77  if r0 s> 4 goto 148
  78  exit
      line 23: return action.TX
  79  r1 = 3                        XDP_TX, read from linux.xdp
  80  to 83: r0 is 3, and the exit
      line 24: end, {default = action.PASS})
  84  r0 = 0                        XDP_ABORTED, where the program stops
  85  exit
      line 18: local port = udphdr:view(p, eth.HLEN + ip.ihl * WORD).dest
  86  to 116: logs "runtime.bpf.lua:18: out of bounds", then goto 84
 117  to 147: the same, for the read of dest
      line 21: return verdict
 148  to 178: logs "runtime.bpf.lua:21: invalid action", then goto 84

llvm-objdump prints two of these instructions by what their bytes hold before libbpf has read the object. Instruction 50 is 18 01 00 00 6c 69 6d 69 00 00 00 00 74 73 00 00, an LD_IMM64 whose two immediates are the bytes 6c 69 6d 69 and 74 73 00 00: limits and its NUL, padded to a word, in the kernel’s byte order, which LLVM prints as the integer they make. Instruction 64 is 85 00 00 00 ff ff ff ff, a call whose source register is 0 and whose immediate is -1, on which llvm-objdump -r prints the relocation R_BPF_64_32 bpf_luaxdp_run: libbpf, as it loads the object, writes there the source register 2, BPF_PSEUDO_KFUNC_CALL, the kfunc’s id in luaxdp’s BTF as the immediate, and which BTF that is as the offset. bpftool prog dump xlated of the loaded program prints it call bpf_luaxdp_run.

A frame of 600 bytes, an IPv4 header of 20 and a UDP one whose destination port is 00 35, with the runtime limits running:

Instructions What they compute Value
3 to 44 ihl, 5, and the UDP header’s dest, the two bytes at 14 + 5 * 4 + 2, 00 35, swapped port is 53
46 to 48 data_end less data #p is 600
50 to 63 the name, 53 and 600 below the frame, and their addresses and sizes
64 the callback reads 53 with getint64(0) and 600 with getint64(8): 600 is over 512 r0 is 1, XDP_DROP
65 to 67 1, sign-extended verdict is 1
68 to 70 1 is not -1 jump to 74
74 to 78 r0 is 1, within 0 to 4 the program returns 1, XDP_DROP

A frame of 100 bytes to port 53, or one of 600 to port 80, makes the callback return nothing: the kfunc answers -1, instruction 69 jumps to 71, and the program returns 3, XDP_TX, as it does for every frame that reaches the call while no runtime limits runs. A frame of 30 bytes stops at the read of the port, past its end, logs “runtime.bpf.lua:18: out of bounds” and returns 0, XDP_ABORTED. bpftool prog run returned these for the object loaded on Linux 6.8, arm64, with the runtime started by lunatik run --context=softirq limits. In a TC program the call is the same over bpf_luatc_run, and r3 holds the __sk_buff.

Edit this page on GitHub