Crate pixelflux 

pixelflux

A high-performance screen capture and encoding pipeline exposed as a Python extension via PyO3. It supports two independent backends — X11 (XShm + XFixes) and Wayland (a headless Smithay compositor) — and a shared encoding layer that dispatches to software (x264/JPEG striping, OpenH264) or hardware (NVENC, VA-API) encoders based on the available GPU and operator settings.

Crate structure

Module	Purpose
encoders	Encoder backends: software x264/JPEG, OpenH264, NVENC, VA-API, watermark overlay
wayland	Headless Smithay compositor, cursor rendering
x11	X11/XShm capture loop and stripe dispatch
pipeline	Frame-processing policy shared by both backends (send/QP/keyframe decisions)
recording_sink	Unix-socket H.264 fan-out for external recording
computer_use	HTTP API for AI-agent desktop control (screenshots, input injection)
nvgpufilter	Multi-GPU NVENC device filtering via ioctl

Data flow

Python  ──►  CaptureSettings  ──►  X11 / Wayland backend
                                        │
                                   frame pixels
                                        │
                                   ┌────┴────┐
                                   │ Encoder │  (NVENC / VAAPI / x264 / OpenH264 / JPEG)
                                   └────┬────┘
                                        │
                                  EncodedStripe(s)
                                        │
                                   Python callback

Re-exports

pub use encoders::nvenc;

pub use encoders::software::StripeState;

pub use encoders::vaapi;

Modules

computer_use

HTTP server implementing the Anthropic Computer Use spec for AI agent desktop control. HTTP server implementing the Anthropic Computer Use specification.

encoders

nvgpufilter

Multi-GPU NVENC device filtering via kernel ioctl. Multi-GPU NVENC GET_ATTACHED_IDS / GET_PROBED_IDS ioctl filter — it exists so a container handed only a subset of the host’s GPUs can still open an NVENC session.

pipeline

Frame-processing policy shared by the X11 and Wayland backends. Frame-processing policy shared by the two capture backends. It lives in its own module for one reason: the Wayland path (dmabuf, compositor damage) and the X11 path (host-ARGB, stripe-hash damage) capture pixels in completely different ways, but a viewer must never be able to tell which one produced a frame — a paint-over refresh or a recovery keyframe has to behave identically either way. Keeping the decision logic here, source-agnostic, is what guarantees it.

recording_sink

Unix-socket H.264 recording sink for external capture tools. Out-of-band H.264 recording sink. Its reason to exist is separation: a recorder needs the exact encoded stream the viewers see, but neither side should be able to disturb the other — so this fans the encoder’s raw Annex-B elementary stream to clients on a private Unix domain socket, entirely apart from the live viewer transport (WebSocket / WebRTC). A recorder attaching, stalling, or detaching cannot perturb what viewers receive, and recording works even with no viewer connected at all. Recording is also strictly optional: with no socket path configured, RecordingSink::try_bind returns None and there is no sink for the caller to write frames to.

wayland

Headless Wayland compositor and cursor rendering. Wayland backend: a headless Smithay compositor that stands in for a real display server.

x11

X11/XShm capture loop, stripe dispatch, and per-stripe change detection. X11 host capture: grab the root window into host memory as BGRA, composite the XFixes hardware cursor and the watermark on the CPU, and feed each frame to pipeline::X11Pipeline, which owns damage/stripe/encode. The grab goes through a shared-memory segment (XShm via x11rb) rather than a plain GetImage for one reason: a full-screen frame is far too large to copy through the X protocol socket every tick, so XShm has the server write the pixels straight into memory this process already has mapped.

Structs

LiveTunables

The per-frame decision/quality knobs every encoder re-reads from the settings on each tick, so they retune a running capture with no encoder re-init: x264 reconfigures, NVENC CQP retargets, VAAPI re-opens only its codec ctx, JPEG is stateless. Applied on the thread that owns the settings copy. Structural switches (encoder, chroma, RC mode, device) still need a capture restart.

RustCaptureSettings

The full set of capture + encode parameters the Python layer hands to the Rust backend.

WlEncodeControls

Cross-thread controls for the Wayland encode thread, the X11 Controls scheme: the UpdateRate handler stores the current values then flips rate_dirty with Release; the encode thread swaps it with Acquire and re-reads the payload, never seeing it half-applied. force_idr is swapped just before each encode, so an on-demand keyframe lands on the frame ALREADY in flight instead of waiting one pipeline stage for the next publish.

WlEncodeStats

Shared capture stats: whichever thread owns the encoders counts frames/stripes and composes desc + n_stripes (the encoder half of the 1 s debug log line); the calloop log loads, prints and resets the counters.

WlFrame

One captured host-pixel frame in flight from the calloop (render/readback) to the Wayland encode thread: the pixels plus the per-frame inputs of the encode dispatch (damage, overlay animation); the IDR request travels separately via the controls atomic.

WlFramePool

Render->encode handoff for the Wayland readback paths, mirroring the X11 FramePool’s single-slot non-dropping design with one deliberate difference: the calloop thread is also the compositor + input dispatcher, so it must NEVER block on the pool. try_begin hands out a buffer only while the publish slot is empty, so publish cannot block, and a saturated encoder throttles capture by SKIPPING ticks (compositor damage accumulates via buffer age, so nothing is lost). Every published frame is encoded, in order: the H.264 reference chain stays contiguous exactly as on X11.

Enums

CompositionElements

ThreadCommand

Control messages sent from the Python-facing methods to the capture thread.