Omzetting naar stdio.
This commit is contained in:
@@ -1,265 +1,114 @@
|
||||
#lang scribble/manual
|
||||
@(require racket/runtime-path)
|
||||
|
||||
@defmodule{racket-webview/internals}
|
||||
|
||||
@title{Qt WebView Backend Architecture}
|
||||
@author[@author+email["Hans Dijkema" "hans@dijkewijk.nl"]]
|
||||
|
||||
@section{Introduction}
|
||||
|
||||
It would, of course, be preferable to place everything within a single process,
|
||||
under a unified structure. This would be elegant. It is not how things are.
|
||||
|
||||
@(define-runtime-path img-path ".")
|
||||
@(define img (path->string (build-path img-path
|
||||
"rktwebview-shared-memory-diagram-simple.svg"
|
||||
)))
|
||||
@;@(displayln (format "image: '~a'" img))
|
||||
|
||||
@centered{
|
||||
@image[#:scale 0.45]{@img}
|
||||
}
|
||||
|
||||
Qt WebEngine establishes its own order: threads, event loops, internal state.
|
||||
Once set in motion, it does not easily yield. It persists, and it expects its
|
||||
environment to adapt accordingly. These conditions are accepted.
|
||||
|
||||
The Racket process, however, is of a different nature. It is light, precise,
|
||||
capable of starting and stopping without residue. It must remain so.
|
||||
|
||||
So a boundary is drawn.
|
||||
|
||||
On one side, Qt: a large, immovable instrument—something like an organ. Once it
|
||||
begins to sound, it fills the space, and it is not easily silenced. On the other,
|
||||
Racket: a violin, agile and expressive, able to begin and end a phrase at will.
|
||||
|
||||
They do not become the same instrument. They are allowed to play together.
|
||||
Communication is arranged accordingly. A shared memory region, containing three
|
||||
queues: commands, results, and events. A command is issued. It crosses the boundary. It is taken up and executed. A result returns.
|
||||
Events also arise, independently, and must be handled when they appear.
|
||||
|
||||
Within this structure, the violin may move freely—provided it does not attempt to
|
||||
reconfigure the organ. No attempt is made to unify the instruments. Such efforts would not improve the music. Instead, the composition is written so that each plays its part.
|
||||
|
||||
From the outside, one hears only a simple exchange: a call, a response. Internally, the balance is carefully maintained. For now, this is sufficient. And it holds.
|
||||
|
||||
@section{Overview}
|
||||
|
||||
This backend provides a webview implementation by delegating all GUI and browser
|
||||
functionality to a separate Qt process.
|
||||
The runtime consists of two processes. The Racket process owns the public API,
|
||||
contexts used by the local HTTPS server, handle wrappers, and user callbacks.
|
||||
The helper process @tt{rktwebview_prg} owns @tt{QApplication}, Qt widgets,
|
||||
@tt{QWebEngineProfile} objects, and all browser state.
|
||||
|
||||
The embedding Racket process does not manipulate Qt widgets directly. Instead,
|
||||
it communicates with a helper process that owns the Qt event loop and all
|
||||
@tt{QWebEngine} objects.
|
||||
The processes communicate through standard pipes. There is no shared library,
|
||||
shared-memory queue, or callback from native code into the Racket runtime.
|
||||
|
||||
This design exists to work around limitations of Qt WebEngine in combination with
|
||||
the lifecycle model of the DrRacket environment.
|
||||
@section{Standard Streams}
|
||||
|
||||
@section{Execution Model}
|
||||
The streams have fixed roles:
|
||||
|
||||
The runtime consists of two processes: the embedding Racket process and a helper
|
||||
process running Qt and Qt WebEngine.
|
||||
@itemlist[#:style 'compact
|
||||
@item{child stdin carries commands from Racket}
|
||||
@item{child stdout carries the handshake, command results, and events}
|
||||
@item{child stderr carries diagnostic logging}]
|
||||
|
||||
All GUI state lives in the helper process. The embedding side holds no direct
|
||||
references to Qt objects. Communication is explicit and happens through shared
|
||||
memory.
|
||||
Messages on stdin and stdout are compact UTF-8 JSON objects, one object per
|
||||
line. Stdout must not contain ordinary logging because the Racket protocol
|
||||
reader treats every value there as a protocol record.
|
||||
|
||||
@section{Shared Memory and Queues}
|
||||
@section{Protocol Compatibility}
|
||||
|
||||
A shared memory region is created during initialization. Inside that region,
|
||||
three FIFO queues are established: a command queue, a result queue, and an event
|
||||
queue.
|
||||
The command identifiers in @tt{rkt_protocol.h} and the JSON objects used as
|
||||
command payloads are unchanged. The process transport adds an outer message
|
||||
object with a type and, for commands and results, a request identifier.
|
||||
|
||||
Each message consists of a numeric code and a payload, typically JSON:
|
||||
A command has this form:
|
||||
|
||||
@centerline{@tt{(code, payload)}}
|
||||
@verbatim|{{"type":"command","id":17,"command":6,"data":{"wv":1,"url":"https://example.test"}}}|
|
||||
|
||||
The queues have distinct roles. The @italic{command queue} carries requests from the embedding process to the Qt process, for example creating a window, loading a URL, or executing JavaScript. The @italic{result queue} carries direct replies to those commands. A synchronous call on the embedding side blocks until a corresponding result is available. The @italic{event queue} carries asynchronous notifications generated by the Qt side, such as page load completion, navigation requests, window movement, or events
|
||||
originating from JavaScript.
|
||||
A corresponding result has this form:
|
||||
|
||||
@verbatim|{{"type":"result","id":17,"result":0,"data":null}}|
|
||||
|
||||
The request identifier prevents concurrent calls from consuming each other's
|
||||
results. It is not visible in the public API.
|
||||
|
||||
@section{Command Execution}
|
||||
|
||||
A function call on the embedding side is translated into a command and written to
|
||||
the command queue.
|
||||
A Racket caller registers a private response channel, writes a command while
|
||||
holding the transport write lock, and waits on that channel.
|
||||
|
||||
From there the flow is fixed: (1) the command is read by a worker thread in the
|
||||
helper process, (2) it is reposted onto the Qt GUI thread, (3) the GUI thread
|
||||
executes the operation, and (4) the result is written back to the result queue.
|
||||
The Qt process has a command-reader thread. That thread parses stdin records and
|
||||
calls the existing @tt{Rktwebview_qt} methods. Those methods post commands to the
|
||||
Qt GUI thread and wait for completion. The command-reader thread then writes the
|
||||
result record to stdout.
|
||||
|
||||
The worker thread never manipulates Qt objects. All GUI work happens on the GUI
|
||||
thread.
|
||||
|
||||
From the caller’s perspective, a synchronous call returns only after the GUI
|
||||
thread has completed the action.
|
||||
The Racket stdout reader is the only thread that reads protocol output. It uses
|
||||
the request identifier to forward each result to the correct waiting caller.
|
||||
|
||||
@section{Event Delivery}
|
||||
|
||||
Many relevant events are not tied to a specific command. Page loading, navigation
|
||||
attempts, window movement, and JavaScript-originated events are delivered through
|
||||
the event queue.
|
||||
Qt events remain the original JSON strings. The Qt callback converts them only
|
||||
into this transport record:
|
||||
|
||||
Events are retrieved explicitly by polling.
|
||||
@verbatim|{{"type":"event","wv":1,"data":"{...original event JSON...}"}}|
|
||||
|
||||
Each event contains a name, an identifier, and optional fields depending on its
|
||||
type. Events are delivered in FIFO order.
|
||||
The stdout reader captures the callback associated with the handle and places
|
||||
the callback plus event string on an asynchronous Racket channel. A separate
|
||||
event thread invokes user callbacks. Therefore a slow user callback cannot block
|
||||
protocol parsing or prevent command results from being received.
|
||||
|
||||
@section{Contexts and Webviews}
|
||||
Capturing the callback while the event record is read is important for window
|
||||
closure. Qt writes the @tt{"closed"} event before the result of the close command.
|
||||
The handle can be removed immediately after the result reaches its caller, but
|
||||
the already captured callback still receives the event.
|
||||
|
||||
The backend uses a two-level model consisting of contexts and webviews.
|
||||
@section{Contexts and Handles}
|
||||
|
||||
A context represents a browser environment and corresponds to a
|
||||
@tt{QWebEngineProfile}. It defines how pages run, including injected scripts and optional trust configuration using explicitly trusted self-signed certificates.
|
||||
A context corresponds to a @tt{QWebEngineProfile} and is identified by an
|
||||
integer. A webview or tray icon is also identified by an integer handle. Only
|
||||
these numbers cross the process boundary; Qt object pointers never do.
|
||||
|
||||
Each context is identified by an integer handle.
|
||||
The existing Racket @tt{rkt-wv} structure is retained so the higher layers do
|
||||
not need to change.
|
||||
|
||||
Within a context, one or more webviews can be created. A webview represents a
|
||||
window containing a browser view. Webviews are also identified by integer
|
||||
handles.
|
||||
@section{Lifecycle and Failure Handling}
|
||||
|
||||
A webview always belongs to exactly one context. When creating a webview, the
|
||||
context handle must be provided.
|
||||
The backend writes a protocol-version handshake after initialization. Racket
|
||||
waits for this handshake before sending application commands.
|
||||
|
||||
Webviews may optionally have a parent webview. If a parent is specified, the
|
||||
resulting window is created as a modal child of that parent; otherwise it is
|
||||
created as a top-level window.
|
||||
For normal shutdown, Racket closes open handles, sends the existing quit
|
||||
command, waits for its result, closes the child's stdin, and waits for the child
|
||||
process and reader threads.
|
||||
|
||||
From the Racket side, this means that a context must be created first. That
|
||||
context handle is then used to create webviews, which are subsequently addressed
|
||||
through their own handles.
|
||||
If Racket terminates unexpectedly, the operating system closes the stdin pipe.
|
||||
The command-reader thread sees EOF, closes all Qt windows, and quits the Qt event
|
||||
loop. This replaces the former shared-memory alive and alive-ack queues.
|
||||
|
||||
All Qt objects remain internal to the helper process; only these integer handles
|
||||
cross the process boundary.
|
||||
If stdout closes unexpectedly, Racket marks all handles invalid and delivers an
|
||||
exception to every command still waiting for a result.
|
||||
|
||||
@section{JavaScript Bridge}
|
||||
@section{Logging}
|
||||
|
||||
Each context installs a small JavaScript bridge into every page, allowing
|
||||
JavaScript code to send structured data to the host via:
|
||||
The Qt backend writes its existing log messages to stderr. Racket forwards that
|
||||
stream to its current error port. The @racket[rkt-webview-info] result reports
|
||||
@tt{"stderr"} as the backend log destination and retains zero-valued
|
||||
shared-memory metrics for compatibility with callers that display those fields.
|
||||
|
||||
@centerline{@tt{window.rkt_send_event(obj)}}
|
||||
@section{Development Override}
|
||||
|
||||
The objects are collected and forwarded to the event queue.
|
||||
|
||||
@section{Navigation and Window Behavior}
|
||||
|
||||
User actions are not always executed immediately; navigation initiated by the
|
||||
user may result in a @tt{"navigation-request"} event instead of being followed
|
||||
automatically, and closing a window may result in a @tt{"can-close?"} event. The
|
||||
Racket side is expected to decide how to handle these situations.
|
||||
|
||||
@section{Design Considerations}
|
||||
|
||||
@bold{Qt WebEngine lifecycle.}
|
||||
Qt WebEngine cannot be safely reinitialized within a single process.
|
||||
|
||||
In practice, once a @tt{QApplication} using WebEngine has been started and shut
|
||||
down, the WebEngine runtime cannot be started again. This is a known and
|
||||
documented limitation (see for example QTBUG-70519, QTBUG-87460,
|
||||
QTBUG-145033). The underlying cause is that WebEngine starts internal threads
|
||||
and resources that are not fully released, even after application shutdown.
|
||||
|
||||
Attempts to reinitialize WebEngine in the same process result in undefined
|
||||
behavior, including crashes, hangs, or inconsistent state.
|
||||
|
||||
In the DrRacket environment, where components may be restarted under a
|
||||
custodian, this makes an in-process design fundamentally unsuitable. Even when
|
||||
libraries are loaded and unloaded using @tt{#:custodian}, the WebEngine runtime
|
||||
cannot be reset to a clean state.
|
||||
|
||||
By moving Qt and WebEngine into a separate process, this limitation is avoided
|
||||
entirely: each start of the backend creates a fresh runtime, and terminating the
|
||||
helper process guarantees that all associated threads and resources are released
|
||||
by the operating system.
|
||||
|
||||
@bold{Event loop and threading.}
|
||||
Qt requires that GUI operations are performed on the Qt GUI thread.
|
||||
|
||||
Instead of attempting to integrate Qt’s event loop with Racket, the design
|
||||
isolates Qt completely and runs it in its own process.
|
||||
|
||||
Within that process, a worker thread receives commands and forwards them to the
|
||||
GUI thread using Qt’s event mechanism (via @tt{postEvent}). The Racket side never
|
||||
interacts with Qt objects directly.
|
||||
|
||||
@bold{Failure isolation.}
|
||||
Qt WebEngine is a large subsystem with its own internal processes (including the
|
||||
Chromium-based @tt{QtWebEngineProcess}) and is generally stable in practice.
|
||||
|
||||
Running the Qt side in a separate process provides isolation: if the helper
|
||||
process terminates, the embedding Racket process remains unaffected and can
|
||||
decide how to recover.
|
||||
|
||||
@bold{Shared memory communication.}
|
||||
The communication pattern consists of commands, results, and events, mapped onto
|
||||
shared memory FIFO queues, keeping the model simple and explicit.
|
||||
|
||||
@bold{JSON encoding.}
|
||||
All payloads are encoded as JSON, providing a natural bridge between JavaScript,
|
||||
Qt/C++, and Racket: JavaScript produces JSON natively, Qt maps it to variant
|
||||
types, and Racket can decode it easily.
|
||||
|
||||
For control commands, payload sizes are small and infrequent, so serialization
|
||||
cost is negligible compared to GUI-thread execution and WebEngine processing; for
|
||||
dynamic data such as JavaScript results and custom events, JSON is the
|
||||
appropriate representation. A binary protocol would reduce overhead but increase
|
||||
complexity and reduce inspectability.
|
||||
|
||||
@section{Shared Memory Architecture}
|
||||
|
||||
Communication between the Racket process and @tt{rktwebview_prg} is implemented
|
||||
using a shared memory region. This region serves three purposes at once: it
|
||||
stores shared data structures, it provides a simple allocator, and it hosts the
|
||||
FIFO queues used for message passing.
|
||||
|
||||
At the start of the shared memory block, a small administration area is stored,
|
||||
including a pointer to the current end of allocated memory, a list of active
|
||||
allocations, a free list, and a fixed slot table. The slot table acts as a
|
||||
directory of shared objects; queues are created once, stored in slots, and can be
|
||||
retrieved by both processes using only the slot number.
|
||||
|
||||
Memory allocation inside the shared block is intentionally simple. Each
|
||||
allocation is preceded by a small header containing size information and links
|
||||
for a double-linked list. Allocation first attempts to reuse a block from the
|
||||
free list; if no suitable block is available, memory is taken from the unused
|
||||
tail of the region. Freed blocks are returned to the free list and may later be
|
||||
reused. Blocks are not compacted or coalesced. This is not a general-purpose
|
||||
heap; it is a small, predictable allocator for queue items and payload strings.
|
||||
|
||||
Shared objects are referenced primarily through offsets (via @tt{ShmPlace})
|
||||
rather than raw pointers. This makes the layout independent of the virtual
|
||||
address at which the shared memory is mapped in each process.
|
||||
|
||||
Queues are built directly on top of this allocator. Each queue consists of a
|
||||
small header containing the first item, the last item, and a count, followed by a
|
||||
linked list of queue items. Each item stores a numeric command or event code, a
|
||||
pointer to its payload in shared memory, and links to neighboring items.
|
||||
|
||||
Synchronization is split into two layers. A shared lock protects allocator and
|
||||
queue metadata, while each queue has its own semaphore indicating whether items
|
||||
are available. One mechanism protects the structure; the other tells you whether
|
||||
there is anything worth reading.
|
||||
|
||||
On POSIX systems such as Linux, shared memory is implemented using
|
||||
@tt{shm_open}, @tt{ftruncate}, and @tt{mmap}, with synchronization via named
|
||||
POSIX semaphores created using @tt{sem_open}. The owner process initializes these
|
||||
objects and removes them again using @tt{shm_unlink} and @tt{sem_unlink}.
|
||||
|
||||
On Windows, the same model is implemented using @tt{CreateFileMappingA} and
|
||||
@tt{MapViewOfFile} for shared memory, and @tt{CreateSemaphoreA} or
|
||||
@tt{OpenSemaphoreA} for synchronization. The design is identical, but the kernel
|
||||
objects follow the Windows lifetime model and are released when the last handle
|
||||
is closed.
|
||||
|
||||
The shared memory region has a fixed size (currently 10MB) and is not resized at
|
||||
runtime. Although the use of @tt{ShmPlace} offsets would in principle allow
|
||||
relocation, resizing would require coordinated remapping in both processes while
|
||||
all activity is paused. The current design therefore treats the region as
|
||||
fixed-size and relies on reuse of freed blocks.
|
||||
|
||||
This implies that the communication channel is bounded. Payloads such as
|
||||
@tt{set_html} or large JavaScript results must fit within the available free
|
||||
space in the shared memory block. In practice, the usable limit is somewhat below
|
||||
the nominal 10MB due to allocator overhead, queue administration, and concurrent
|
||||
messages.
|
||||
|
||||
This is a message transport, not an infinite sack of HTML.
|
||||
Setting @tt{RKT_WEBVIEW_PRG} to a backend executable bypasses the installed
|
||||
release directory. This makes it possible to test a newly compiled backend
|
||||
without repackaging or changing the downloader first.
|
||||
|
||||
Reference in New Issue
Block a user