Documentation and some other stuff.

This commit is contained in:
2026-03-30 11:03:56 +02:00
parent 8349b65a83
commit 9dd8b895ae
13 changed files with 2619 additions and 1607 deletions

View File

@@ -1,346 +1,256 @@
#lang scribble/manual
@title{Structure and behavior of @tt{rktwebview_qt}}
@title{Qt WebView Backend Architecture}
@author[@author+email["Hans Dijkema" "hans@dijkewijk.nl"]]
@section{Introduction}
It would, of course, be preferable to place everything within a single process,
under a unified structure. This would be elegant. It is not how things are.
@centered{
@image[#:scale 0.45]{rktwebview-shared-memory-diagram-simple.svg}
}
Qt WebEngine establishes its own order: threads, event loops, internal state.
Once set in motion, it does not easily yield. It persists, and it expects its
environment to adapt accordingly. These conditions are accepted.
The Racket process, however, is of a different nature. It is light, precise,
capable of starting and stopping without residue. It must remain so.
So a boundary is drawn.
On one side, Qt: a large, immovable instrument—something like an organ. Once it
begins to sound, it fills the space, and it is not easily silenced. On the other,
Racket: a violin, agile and expressive, able to begin and end a phrase at will.
They do not become the same instrument. They are allowed to play together.
Communication is arranged accordingly. A shared memory region, containing three
queues: commands, results, and events. A command is issued. It crosses the boundary. It is taken up and executed. A result returns.
Events also arise, independently, and must be handled when they appear.
Within this structure, the violin may move freely—provided it does not attempt to
reconfigure the organ. No attempt is made to unify the instruments. Such efforts would not improve the music. Instead, the composition is written so that each plays its part.
From the outside, one hears only a simple exchange: a call, a response. Internally, the balance is carefully maintained. For now, this is sufficient. And it holds.
@section{Overview}
@tt{rktwebview_qt} is a Qt-based backend that provides embedded webviews for
applications written in Racket. The library exposes a small C-compatible API
intended to be used through the Racket FFI. Internally, the implementation is
written in C++ and uses Qt WebEngine.
This backend provides a webview implementation by delegating all GUI and browser
functionality to a separate Qt process.
The backend provides:
The embedding Racket process does not manipulate Qt widgets directly. Instead,
it communicates with a helper process that owns the Qt event loop and all
@tt{QWebEngine} objects.
@itemlist[#:style 'compact
@item{creation of browser windows}
@item{navigation to URLs or HTML content}
@item{execution of JavaScript}
@item{return values from JavaScript}
@item{delivery of browser and window events}
@item{native dialogs such as file choosers and message boxes}
]
This design exists to work around limitations of Qt WebEngine in combination with
the lifecycle model of the DrRacket environment.
The implementation is intentionally organized so that the external interface
remains simple and language-neutral while the Qt-specific logic is hidden
inside the runtime controller.
@section{Execution Model}
@section{Requirements}
The runtime consists of two processes: the embedding Racket process and a helper
process running Qt and Qt WebEngine.
The @tt{rktwebview_qt} backend requires Qt version @tt{6.10.2} or newer.
All GUI state lives in the helper process. The embedding side holds no direct
references to Qt objects. Communication is explicit and happens through shared
memory.
The backend is built against Qt WebEngine and depends on the runtime
components provided by Qt. Earlier Qt versions are not supported.
@section{Shared Memory and Queues}
Applications using the backend must therefore ensure that a compatible
Qt runtime (≥ @tt{6.10.2}) is available.
A shared memory region is created during initialization. Inside that region,
three FIFO queues are established: a command queue, a result queue, and an event
queue.
In packaged builds the required Qt runtime is normally bundled together
with the backend library.
Each message consists of a numeric code and a payload, typically JSON:
@subsection{Compatibility}
@centerline{@tt{(code, payload)}}
The backend requires Qt version @tt{6.10.2} or newer.
The queues have distinct roles. The @italic{command queue} carries requests from the embedding process to the Qt process, for example creating a window, loading a URL, or executing JavaScript. The @italic{result queue} carries direct replies to those commands. A synchronous call on the embedding side blocks until a corresponding result is available. The @italic{event queue} carries asynchronous notifications generated by the Qt side, such as page load completion, navigation requests, window movement, or events
originating from JavaScript.
This requirement is due to the use of the method
@tt{setAdditionalTrustedCertificates} provided by
@tt{QWebEngineProfileBuilder}. This functionality allows the backend to
install additional trusted certificates for a context.
@section{Command Execution}
The mechanism is used to support trusted self-signed certificates for
local services. Earlier Qt versions do not provide this functionality
and are therefore not supported.
A function call on the embedding side is translated into a command and written to
the command queue.
Packaged builds typically include the required Qt runtime components.
From there the flow is fixed: (1) the command is read by a worker thread in the
helper process, (2) it is reposted onto the Qt GUI thread, (3) the GUI thread
executes the operation, and (4) the result is written back to the result queue.
@section{Layers}
The worker thread never manipulates Qt objects. All GUI work happens on the GUI
thread.
The system consists of several layers.
From the callers perspective, a synchronous call returns only after the GUI
thread has completed the action.
@subsection{C ABI Layer}
@section{Event Delivery}
The file @tt{rktwebview.h} defines the public API. This layer provides a stable
C-compatible interface that can easily be consumed from Racket through FFI.
Many relevant events are not tied to a specific command. Page loading, navigation
attempts, window movement, and JavaScript-originated events are delivered through
the event queue.
Important characteristics of the ABI include:
Events are retrieved explicitly by polling.
@itemlist[#:style 'compact
@item{numeric handles represent windows and browser contexts}
@item{result codes indicate success or failure}
@item{data returned from the library is represented as @tt{rkt_data_t}}
@item{event callbacks deliver JSON strings describing events}
]
Each event contains a name, an identifier, and optional fields depending on its
type. Events are delivered in FIFO order.
The C layer is implemented in @tt{rktwebview.cpp}. These functions mainly
forward calls to a singleton instance of the internal runtime class
@tt{Rktwebview_qt}.
@section{Contexts and Webviews}
@section{Runtime Controller}
The backend uses a two-level model consisting of contexts and webviews.
The class @tt{Rktwebview_qt} acts as the central runtime controller.
A context represents a browser environment and corresponds to a
@tt{QWebEngineProfile}. It defines how pages run, including injected scripts and optional trust configuration using explicitly trusted self-signed certificates.
Its responsibilities include:
Each context is identified by an integer handle.
@itemlist[#:style 'compact
@item{owning the Qt application instance}
@item{managing https contexts}
@item{managing open windows}
@item{dispatching commands}
@item{forwarding events to the host application}
]
Within a context, one or more webviews can be created. A webview represents a
window containing a browser view. Webviews are also identified by integer
handles.
Internally, it maintains several registries:
A webview always belongs to exactly one context. When creating a webview, the
context handle must be provided.
@itemlist[#:style 'compact
@item{a map from window handles to @tt{WebviewWindow} instances}
@item{a map from context identifiers to @tt{QWebEngineProfile} objects}
@item{a map from window handles to callback functions}
]
Webviews may optionally have a parent webview. If a parent is specified, the
resulting window is created as a modal child of that parent; otherwise it is
created as a top-level window.
Each API call that manipulates a window or browser state is forwarded through
this controller.
From the Racket side, this means that a context must be created first. That
context handle is then used to create webviews, which are subsequently addressed
through their own handles.
@section{HTTP(s) Contexts}
All Qt objects remain internal to the helper process; only these integer handles
cross the process boundary.
A http(s) context represents a shared WebEngine profile.
@section{JavaScript Bridge}
Contexts are created using @tt{rkt_webview_new_context}. Each context contains:
Each context installs a small JavaScript bridge into every page, allowing
JavaScript code to send structured data to the host via:
@itemlist[#:style 'compact
@item{a @tt{QWebEngineProfile}}
@item{optional injected JavaScript code}
@item{optional trusted certificates}
]
@centerline{@tt{window.rkt_send_event(obj)}}
Multiple windows normally share the same context. This allows cookies,
configuration, and injected scripts to be reused.
The objects are collected and forwarded to the event queue.
A context normally represents a https server that can be reached on a certain (local) Url.
@section{Navigation and Window Behavior}
@section{Window and View Classes}
User actions are not always executed immediately; navigation initiated by the
user may result in a @tt{"navigation-request"} event instead of being followed
automatically, and closing a window may result in a @tt{"can-close?"} event. The
Racket side is expected to decide how to handle these situations.
Two Qt classes implement the actual browser window.
@section{Design Considerations}
@subsection{@tt{WebviewWindow}}
@bold{Qt WebEngine lifecycle.}
Qt WebEngine cannot be safely reinitialized within a single process.
@tt{WebviewWindow} derives from @tt{QMainWindow}. It represents the native
top-level window.
In practice, once a @tt{QApplication} using WebEngine has been started and shut
down, the WebEngine runtime cannot be started again. This is a known and
documented limitation (see for example QTBUG-70519, QTBUG-87460,
QTBUG-145033). The underlying cause is that WebEngine starts internal threads
and resources that are not fully released, even after application shutdown.
Responsibilities include:
Attempts to reinitialize WebEngine in the same process result in undefined
behavior, including crashes, hangs, or inconsistent state.
@itemlist[#:style 'compact
@item{hosting the browser widget}
@item{reporting window events such as show, hide, move, and resize}
@item{handling close requests}
@item{reporting navigation events}
@item{forwarding JavaScript events}
]
In the DrRacket environment, where components may be restarted under a
custodian, this makes an in-process design fundamentally unsuitable. Even when
libraries are loaded and unloaded using @tt{#:custodian}, the WebEngine runtime
cannot be reset to a clean state.
Window geometry changes are slightly delayed through timers so that the host
application receives fewer intermediate events.
By moving Qt and WebEngine into a separate process, this limitation is avoided
entirely: each start of the backend creates a fresh runtime, and terminating the
helper process guarantees that all associated threads and resources are released
by the operating system.
@subsection{@tt{WebViewQt}}
@bold{Event loop and threading.}
Qt requires that GUI operations are performed on the Qt GUI thread.
@tt{WebViewQt} derives from @tt{QWebEngineView}. This class mainly associates
the Qt widget with a numeric handle used by the external API.
Instead of attempting to integrate Qts event loop with Racket, the design
isolates Qt completely and runs it in its own process.
@section{Command Dispatch}
Within that process, a worker thread receives commands and forwards them to the
GUI thread using Qts event mechanism (via @tt{postEvent}). The Racket side never
interacts with Qt objects directly.
Many API functions are implemented using a small command-dispatch mechanism.
@bold{Failure isolation.}
Qt WebEngine is a large subsystem with its own internal processes (including the
Chromium-based @tt{QtWebEngineProcess}) and is generally stable in practice.
A command object represents a pending request. The request is delivered to the
Qt event system using a custom Qt event. The runtime controller receives the
event and executes the corresponding operation.
Running the Qt side in a separate process provides isolation: if the helper
process terminates, the embedding Racket process remains unaffected and can
decide how to recover.
This design ensures that GUI operations occur inside the Qt event loop while
the external API remains synchronous.
@bold{Shared memory communication.}
The communication pattern consists of commands, results, and events, mapped onto
shared memory FIFO queues, keeping the model simple and explicit.
@section{Event Processing}
@bold{JSON encoding.}
All payloads are encoded as JSON, providing a natural bridge between JavaScript,
Qt/C++, and Racket: JavaScript produces JSON natively, Qt maps it to variant
types, and Racket can decode it easily.
Qt events are processed through repeated calls to
@tt{rkt_webview_process_events}, which internally calls
@tt{QApplication::processEvents}. This allows the host runtime to control how
frequently GUI events are processed.
For control commands, payload sizes are small and infrequent, so serialization
cost is negligible compared to GUI-thread execution and WebEngine processing; for
dynamic data such as JavaScript results and custom events, JSON is the
appropriate representation. A binary protocol would reduce overhead but increase
complexity and reduce inspectability.
@section{Navigation}
@section{Shared Memory Architecture}
Two main navigation operations are provided.
Communication between the Racket process and @tt{rktwebview_prg} is implemented
using a shared memory region. This region serves three purposes at once: it
stores shared data structures, it provides a simple allocator, and it hosts the
FIFO queues used for message passing.
@itemlist[#:style 'compact
@item{@tt{rkt_webview_set_url} loads a URL}
@item{@tt{rkt_webview_set_html} loads raw HTML}
]
At the start of the shared memory block, a small administration area is stored,
including a pointer to the current end of allocated memory, a list of active
allocations, a free list, and a fixed slot table. The slot table acts as a
directory of shared objects; queues are created once, stored in slots, and can be
retrieved by both processes using only the slot number.
When page loading finishes, a @tt{"page-loaded"} event is emitted. The event
contains a boolean field indicating whether loading succeeded.
@section{JavaScript Execution}
JavaScript execution is provided through two functions.
@subsection{Fire-and-Forget Execution}
@tt{rkt_webview_run_js} executes JavaScript without returning a result.
This is useful for DOM manipulation or triggering page behavior.
@subsection{Synchronous Execution with Result}
@tt{rkt_webview_call_js} evaluates JavaScript and returns a result object.
The JavaScript is wrapped in a small function that captures either the result
value or an exception. The result is returned as a JSON object containing:
@itemlist[#:style 'compact
@item{@tt{oke}}
@item{@tt{result}}
@item{@tt{exn}}
]
The native side waits for the asynchronous Qt callback before returning the
value to the host application.
@section{JavaScript Event Bridge}
Communication from JavaScript to the host application uses a small event queue
implemented in injected JavaScript.
When a browser context is created, the runtime injects support code defining
the following objects:
@itemlist[#:style 'compact
@item{@tt{window.rkt_event_queue}}
@item{@tt{window.rkt_send_event(obj)}}
@item{@tt{window.rkt_get_events()}}
]
The queue stores JavaScript-generated events until the native side retrieves
them.
@subsection{Event Queue}
When page code wants to emit an event, it calls:
@verbatim|{
window.rkt_send_event(obj)
}|
The object is appended to the queue.
The queue can be retrieved using:
@verbatim|{
window.rkt_get_events()
}|
This function returns a JSON array and clears the queue.
@subsection{Native Notification}
To notify the native side that events are waiting, the injected code creates a
hidden iframe named @tt{rkt-evt-frame}. The iframe is used only as a signalling
mechanism.
After adding an event to the queue, the script attempts to call:
@verbatim|{
frameWindow.print()
}|
Qt receives this through the signal
@tt{QWebEnginePage::printRequestedByFrame}.
The window implementation checks whether the frame name is
@tt{"rkt-evt-frame"}. If so, it calls @tt{processJsEvents} to retrieve the
queued events.
This results in the following sequence:
@itemlist[#:style 'compact
@item{JavaScript calls @tt{rkt_send_event}.}
@item{The event is added to the queue.}
@item{The hidden iframe triggers a print request.}
@item{Qt receives the request.}
@item{The native side calls @tt{rkt_get_events}.}
@item{Each event object is delivered to the host.}
]
The mechanism is therefore not a simple native polling loop. It is better
described as a queued JavaScript event bridge with a lightweight native
wake-up signal.
@subsection{Delivered Event Format}
JavaScript-originated events are forwarded to the host as structured JSON.
Each event uses a generic envelope:
@verbatim|{
{
"evt": "js-evt",
"js-evt": { ... }
}
}|
The inner object is the original JavaScript event payload.
This design keeps the native API simple while allowing arbitrary JavaScript
objects to be delivered to the host application.
@section{Window Lifecycle}
Windows are created using @tt{rkt_webview_create}. Each window receives a
callback function used to deliver events.
Important lifecycle events include:
@itemlist[#:style 'compact
@item{@tt{"show"}}
@item{@tt{"hide"}}
@item{@tt{"move"}}
@item{@tt{"resize"}}
@item{@tt{"closed"}}
]
If the user attempts to close a window directly, the close request is converted
into a @tt{"can-close?"} event so that the host application can decide whether
the window should actually close.
@section{Native Dialogs}
The backend also exposes native dialogs including:
@itemlist[#:style 'compact
@item{directory selection}
@item{file open}
@item{file save}
@item{message boxes}
]
The results of these dialogs are delivered as events so that the host
application remains in control of the user interface flow.
@section{Certificates}
Contexts may optionally install trusted certificates. Additionally, a window
may specify a special Organizational Unit token used to automatically trust
certain self-signed certificates.
This mechanism is useful when communicating with local development services.
@section{Memory Ownership}
Data returned from the library and (javascript) events originating from the library
are allocated on the native side and must be released by the host using
@tt{rkt_webview_free_data}.
This avoids exposing C++ ownership semantics across the FFI boundary.
@section{Summary}
The @tt{rktwebview_qt} backend provides a compact architecture for embedding
Qt WebEngine inside Racket applications.
The design allows Racket programs to implement desktop applications using
HTML and JavaScript for the user interface while keeping application logic in
the host runtime.
Memory allocation inside the shared block is intentionally simple. Each
allocation is preceded by a small header containing size information and links
for a double-linked list. Allocation first attempts to reuse a block from the
free list; if no suitable block is available, memory is taken from the unused
tail of the region. Freed blocks are returned to the free list and may later be
reused. Blocks are not compacted or coalesced. This is not a general-purpose
heap; it is a small, predictable allocator for queue items and payload strings.
Shared objects are referenced primarily through offsets (via @tt{ShmPlace})
rather than raw pointers. This makes the layout independent of the virtual
address at which the shared memory is mapped in each process.
Queues are built directly on top of this allocator. Each queue consists of a
small header containing the first item, the last item, and a count, followed by a
linked list of queue items. Each item stores a numeric command or event code, a
pointer to its payload in shared memory, and links to neighboring items.
Synchronization is split into two layers. A shared lock protects allocator and
queue metadata, while each queue has its own semaphore indicating whether items
are available. One mechanism protects the structure; the other tells you whether
there is anything worth reading.
On POSIX systems such as Linux, shared memory is implemented using
@tt{shm_open}, @tt{ftruncate}, and @tt{mmap}, with synchronization via named
POSIX semaphores created using @tt{sem_open}. The owner process initializes these
objects and removes them again using @tt{shm_unlink} and @tt{sem_unlink}.
On Windows, the same model is implemented using @tt{CreateFileMappingA} and
@tt{MapViewOfFile} for shared memory, and @tt{CreateSemaphoreA} or
@tt{OpenSemaphoreA} for synchronization. The design is identical, but the kernel
objects follow the Windows lifetime model and are released when the last handle
is closed.
The shared memory region has a fixed size (currently 10MB) and is not resized at
runtime. Although the use of @tt{ShmPlace} offsets would in principle allow
relocation, resizing would require coordinated remapping in both processes while
all activity is paused. The current design therefore treats the region as
fixed-size and relies on reuse of freed blocks.
This implies that the communication channel is bounded. Payloads such as
@tt{set_html} or large JavaScript results must fit within the available free
space in the shared memory block. In practice, the usable limit is somewhat below
the nominal 10MB due to allocator overhead, queue administration, and concurrent
messages.
This is a message transport, not an infinite sack of HTML.