It would, of course, be preferable to place everything within a single process,
under a unified structure. This would be elegant. It is not how things are.
@(define-runtime-path img-path ".")
@(define img (path->string (build-path img-path
"rktwebview-shared-memory-diagram-simple.svg"
)))
@;@(displayln (format "image: '~a'" img))
@centered{
@image[#:scale 0.45]{@img}
}
Qt WebEngine establishes its own order: threads, event loops, internal state.
Once set in motion, it does not easily yield. It persists, and it expects its
environment to adapt accordingly. These conditions are accepted.
The Racket process, however, is of a different nature. It is light, precise,
capable of starting and stopping without residue. It must remain so.
So a boundary is drawn.
On one side, Qt: a large, immovable instrument—something like an organ. Once it
begins to sound, it fills the space, and it is not easily silenced. On the other,
Racket: a violin, agile and expressive, able to begin and end a phrase at will.
They do not become the same instrument. They are allowed to play together.
Communication is arranged accordingly. A shared memory region, containing three
queues: commands, results, and events. A command is issued. It crosses the boundary. It is taken up and executed. A result returns.
Events also arise, independently, and must be handled when they appear.
Within this structure, the violin may move freely—provided it does not attempt to
reconfigure the organ. No attempt is made to unify the instruments. Such efforts would not improve the music. Instead, the composition is written so that each plays its part.
From the outside, one hears only a simple exchange: a call, a response. Internally, the balance is carefully maintained. For now, this is sufficient. And it holds.
@section{Overview}
This backend provides a webview implementation by delegating all GUI and browser
functionality to a separate Qt process.
The runtime consists of two processes. The Racket process owns the public API,
contexts used by the local HTTPS server, handle wrappers, and user callbacks.
The helper process @tt{rktwebview_prg} owns @tt{QApplication}, Qt widgets,
@tt{QWebEngineProfile} objects, and all browser state.
The embedding Racket process does not manipulate Qt widgets directly. Instead,
it communicates with a helper process that owns the Qt event loop and all
@tt{QWebEngine} objects.
The processes communicate through standard pipes. There is no shared library,
shared-memory queue, or callback from native code into the Racket runtime.
This design exists to work around limitations of Qt WebEngine in combination with
the lifecycle model of the DrRacket environment.
@section{Standard Streams}
@section{Execution Model}
The streams have fixed roles:
The runtime consists of two processes: the embedding Racket process and a helper
process running Qt and Qt WebEngine.
@itemlist[#:style 'compact
@item{child stdin carries commands from Racket}
@item{child stdout carries the handshake, command results, and events}
@item{child stderr carries diagnostic logging}]
All GUI state lives in the helper process. The embedding side holds no direct
references to Qt objects. Communication is explicit and happens through shared
memory.
Messages on stdin and stdout are compact UTF-8 JSON objects, one object per
line. Stdout must not contain ordinary logging because the Racket protocol
reader treats every value there as a protocol record.
@section{Shared Memory and Queues}
@section{Protocol Compatibility}
A shared memory region is created during initialization. Inside that region,
three FIFO queues are established: a command queue, a result queue, and an event
queue.
The command identifiers in @tt{rkt_protocol.h} and the JSON objects used as
command payloads are unchanged. The process transport adds an outer message
object with a type and, for commands and results, a request identifier.
Each message consists of a numeric code and a payload, typically JSON:
The queues have distinct roles. The @italic{command queue} carries requests from the embedding process to the Qt process, for example creating a window, loading a URL, or executing JavaScript. The @italic{result queue} carries direct replies to those commands. A synchronous call on the embedding side blocks until a corresponding result is available. The @italic{event queue} carries asynchronous notifications generated by the Qt side, such as page load completion, navigation requests, window movement, or events
Each event contains a name, an identifier, and optional fields depending on its
type. Events are delivered in FIFO order.
The stdout reader captures the callback associated with the handle and places
the callback plus event string on an asynchronous Racket channel. A separate
event thread invokes user callbacks. Therefore a slow user callback cannot block
protocol parsing or prevent command results from being received.
@section{Contexts and Webviews}
Capturing the callback while the event record is read is important for window
closure. Qt writes the @tt{"closed"} event before the result of the close command.
The handle can be removed immediately after the result reaches its caller, but
the already captured callback still receives the event.
The backend uses a two-level model consisting of contexts and webviews.
@section{Contexts and Handles}
A context represents a browser environment and corresponds to a
@tt{QWebEngineProfile}. It defines how pages run, including injected scripts and optional trust configuration using explicitly trusted self-signed certificates.
A context corresponds to a @tt{QWebEngineProfile} and is identified by an
integer. A webview or tray icon is also identified by an integer handle. Only
these numbers cross the process boundary; Qt object pointers never do.
Each context is identified by an integer handle.
The existing Racket @tt{rkt-wv} structure is retained so the higher layers do
not need to change.
Within a context, one or more webviews can be created. A webview represents a
window containing a browser view. Webviews are also identified by integer
handles.
@section{Lifecycle and Failure Handling}
A webview always belongs to exactly one context. When creating a webview, the
context handle must be provided.
The backend writes a protocol-version handshake after initialization. Racket
waits for this handshake before sending application commands.
Webviews may optionally have a parent webview. If a parent is specified, the
resulting window is created as a modal child of that parent; otherwise it is
created as a top-level window.
For normal shutdown, Racket closes open handles, sends the existing quit
command, waits for its result, closes the child's stdin, and waits for the child
process and reader threads.
From the Racket side, this means that a context must be created first. That
context handle is then used to create webviews, which are subsequently addressed
through their own handles.
If Racket terminates unexpectedly, the operating system closes the stdin pipe.
The command-reader thread sees EOF, closes all Qt windows, and quits the Qt event
loop. This replaces the former shared-memory alive and alive-ack queues.
All Qt objects remain internal to the helper process; only these integer handles
cross the process boundary.
If stdout closes unexpectedly, Racket marks all handles invalid and delivers an
exception to every command still waiting for a result.
@section{JavaScript Bridge}
@section{Logging}
Each context installs a small JavaScript bridge into every page, allowing
JavaScript code to send structured data to the host via:
The Qt backend writes its existing log messages to stderr. Racket forwards that
stream to its current error port. The @racket[rkt-webview-info] result reports
@tt{"stderr"} as the backend log destination and retains zero-valued
shared-memory metrics for compatibility with callers that display those fields.
@centerline{@tt{window.rkt_send_event(obj)}}
@section{Development Override}
The objects are collected and forwarded to the event queue.
@section{Navigation and Window Behavior}
User actions are not always executed immediately; navigation initiated by the
user may result in a @tt{"navigation-request"} event instead of being followed
automatically, and closing a window may result in a @tt{"can-close?"} event. The
Racket side is expected to decide how to handle these situations.
@section{Design Considerations}
@bold{Qt WebEngine lifecycle.}
Qt WebEngine cannot be safely reinitialized within a single process.
In practice, once a @tt{QApplication} using WebEngine has been started and shut
down, the WebEngine runtime cannot be started again. This is a known and
documented limitation (see for example QTBUG-70519, QTBUG-87460,
QTBUG-145033). The underlying cause is that WebEngine starts internal threads
and resources that are not fully released, even after application shutdown.
Attempts to reinitialize WebEngine in the same process result in undefined
behavior, including crashes, hangs, or inconsistent state.
In the DrRacket environment, where components may be restarted under a
custodian, this makes an in-process design fundamentally unsuitable. Even when
libraries are loaded and unloaded using @tt{#:custodian}, the WebEngine runtime
cannot be reset to a clean state.
By moving Qt and WebEngine into a separate process, this limitation is avoided
entirely: each start of the backend creates a fresh runtime, and terminating the
helper process guarantees that all associated threads and resources are released
by the operating system.
@bold{Event loop and threading.}
Qt requires that GUI operations are performed on the Qt GUI thread.
Instead of attempting to integrate Qt’s event loop with Racket, the design
isolates Qt completely and runs it in its own process.
Within that process, a worker thread receives commands and forwards them to the
GUI thread using Qt’s event mechanism (via @tt{postEvent}). The Racket side never
interacts with Qt objects directly.
@bold{Failure isolation.}
Qt WebEngine is a large subsystem with its own internal processes (including the
Chromium-based @tt{QtWebEngineProcess}) and is generally stable in practice.
Running the Qt side in a separate process provides isolation: if the helper
process terminates, the embedding Racket process remains unaffected and can
decide how to recover.
@bold{Shared memory communication.}
The communication pattern consists of commands, results, and events, mapped onto
shared memory FIFO queues, keeping the model simple and explicit.
@bold{JSON encoding.}
All payloads are encoded as JSON, providing a natural bridge between JavaScript,
Qt/C++, and Racket: JavaScript produces JSON natively, Qt maps it to variant
types, and Racket can decode it easily.
For control commands, payload sizes are small and infrequent, so serialization
cost is negligible compared to GUI-thread execution and WebEngine processing; for
dynamic data such as JavaScript results and custom events, JSON is the
appropriate representation. A binary protocol would reduce overhead but increase
complexity and reduce inspectability.
@section{Shared Memory Architecture}
Communication between the Racket process and @tt{rktwebview_prg} is implemented
using a shared memory region. This region serves three purposes at once: it
stores shared data structures, it provides a simple allocator, and it hosts the
FIFO queues used for message passing.
At the start of the shared memory block, a small administration area is stored,
including a pointer to the current end of allocated memory, a list of active
allocations, a free list, and a fixed slot table. The slot table acts as a
directory of shared objects; queues are created once, stored in slots, and can be
retrieved by both processes using only the slot number.
Memory allocation inside the shared block is intentionally simple. Each
allocation is preceded by a small header containing size information and links
for a double-linked list. Allocation first attempts to reuse a block from the
free list; if no suitable block is available, memory is taken from the unused
tail of the region. Freed blocks are returned to the free list and may later be
reused. Blocks are not compacted or coalesced. This is not a general-purpose
heap; it is a small, predictable allocator for queue items and payload strings.
Shared objects are referenced primarily through offsets (via @tt{ShmPlace})
rather than raw pointers. This makes the layout independent of the virtual
address at which the shared memory is mapped in each process.
Queues are built directly on top of this allocator. Each queue consists of a
small header containing the first item, the last item, and a count, followed by a
linked list of queue items. Each item stores a numeric command or event code, a
pointer to its payload in shared memory, and links to neighboring items.
Synchronization is split into two layers. A shared lock protects allocator and
queue metadata, while each queue has its own semaphore indicating whether items
are available. One mechanism protects the structure; the other tells you whether
there is anything worth reading.
On POSIX systems such as Linux, shared memory is implemented using
@tt{shm_open}, @tt{ftruncate}, and @tt{mmap}, with synchronization via named
POSIX semaphores created using @tt{sem_open}. The owner process initializes these
objects and removes them again using @tt{shm_unlink} and @tt{sem_unlink}.
On Windows, the same model is implemented using @tt{CreateFileMappingA} and
@tt{MapViewOfFile} for shared memory, and @tt{CreateSemaphoreA} or
@tt{OpenSemaphoreA} for synchronization. The design is identical, but the kernel
objects follow the Windows lifetime model and are released when the last handle
is closed.
The shared memory region has a fixed size (currently 10MB) and is not resized at
runtime. Although the use of @tt{ShmPlace} offsets would in principle allow
relocation, resizing would require coordinated remapping in both processes while
all activity is paused. The current design therefore treats the region as
fixed-size and relies on reuse of freed blocks.
This implies that the communication channel is bounded. Payloads such as
@tt{set_html} or large JavaScript results must fit within the available free
space in the shared memory block. In practice, the usable limit is somewhat below
the nominal 10MB due to allocator overhead, queue administration, and concurrent
messages.
This is a message transport, not an infinite sack of HTML.
Setting @tt{RKT_WEBVIEW_PRG} to a backend executable bypasses the installed
release directory. This makes it possible to test a newly compiled backend
without repackaging or changing the downloader first.
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
