documentation and small bug fixes
This commit is contained in:
@@ -1,149 +1,196 @@
|
||||
#lang scribble/manual
|
||||
|
||||
@(require (for-label racket/base
|
||||
(file "../main.rkt")))
|
||||
"../main.rkt"))
|
||||
|
||||
@title{audio-library-manager}
|
||||
@author{Hans Dijkema}
|
||||
|
||||
The @racketmodname[audio-library-manager] package contains command-line tools for
|
||||
maintaining audio library trees.
|
||||
@defmodule[audio-library-manager]
|
||||
|
||||
The @racketmodname[audio-library-manager] package contains a simple batch manager
|
||||
for maintaining a FLAC source library and an Opus mirror tree.
|
||||
|
||||
The current public entry point is @racket[audio-manager]. The manager scans a
|
||||
source tree, compares the scan with a saved state hash, enriches file information
|
||||
through a small ordered pipeline, performs the necessary FLAC and Opus actions,
|
||||
writes the updated state, logs progress, and sends a mail report.
|
||||
|
||||
The design is deliberately synchronous. It does not use places, channels,
|
||||
threads or a worker protocol. Conversion is performed by ordinary function calls
|
||||
to @racketmodname[racket-audio].
|
||||
|
||||
@section{Administration files}
|
||||
|
||||
The managers use these files below the source directory:
|
||||
The manager stores these files in the source directory:
|
||||
|
||||
@itemlist[
|
||||
@item{@filepath{.music-info.db}: keystore database with file state.}
|
||||
@item{@filepath{.flac-48khz-manager.ini}: configuration.}
|
||||
@item{@filepath{.flac-48khz-manager.log}: log file.}]
|
||||
@item{@filepath{.audio-manager.ini}: configuration read through
|
||||
@racketmodname[simple-ini].}
|
||||
@item{@filepath{.audio-manager.db}: serialized state hash, keyed by normalized
|
||||
relative paths.}
|
||||
@item{@filepath{.audio-manager.log}: log file written through
|
||||
@racketmodname[simple-log].}]
|
||||
|
||||
The FLAC-to-Opus manager uses a separate key prefix in @filepath{.music-info.db},
|
||||
so it can share the same database with the 48 kHz manager.
|
||||
State keys are relative to the configured source tree. Absolute mount points are
|
||||
not stored, and path separators are normalized to @tt{/}. This makes the state
|
||||
usable across machines where the same library may be mounted at different base
|
||||
paths.
|
||||
|
||||
@section{FLAC 48 kHz manager}
|
||||
@section{Usage}
|
||||
|
||||
The tool @filepath{flac-48khz-manager.rkt} keeps a FLAC tree at a maximum sample
|
||||
rate of 48 kHz. Files with a higher sample rate are converted in place by a
|
||||
direct call to @racketmodname[racket-audio/audio-encoder]. The batch conversion
|
||||
path is deliberately synchronous; it does not use places, channels or a worker
|
||||
protocol.
|
||||
From a Racket program or REPL:
|
||||
|
||||
Run the manager as:
|
||||
@racketblock[
|
||||
(require audio-library-manager)
|
||||
|
||||
@verbatim{racket flac-48khz-manager.rkt <base-directory>}
|
||||
(audio-manager "/mnt/music" "/mnt/music-opus")
|
||||
]
|
||||
|
||||
When a FLAC file has no embedded picture and its directory contains
|
||||
@filepath{cover.jpg}, @filepath{folder.jpg}, @filepath{cover.png} or
|
||||
@filepath{folder.png}, the manager embeds that image as front-cover picture
|
||||
before fingerprinting and conversion.
|
||||
The optional log level defaults to @racket['debug]:
|
||||
|
||||
The manager only processes @filepath{.flac} files; @filepath{.mp3} files and
|
||||
other sidecar files are ignored by this command.
|
||||
|
||||
@section{FLAC to Opus mirror manager}
|
||||
|
||||
The tool @filepath{flac2opus-manager.rkt} mirrors a source directory to a target
|
||||
directory. FLAC files are converted to Ogg Opus files with extension
|
||||
@filepath{.opus}. Other regular files are copied unchanged, preserving their
|
||||
relative path and modification time. This includes sidecar files such as
|
||||
@filepath{booklet.pdf}, @filepath{cover.jpg}, cue sheets and text files.
|
||||
|
||||
Run the manager as:
|
||||
|
||||
@verbatim{racket flac2opus-manager.rkt <source-directory> <target-directory>}
|
||||
|
||||
The default Opus bitrate is 224 kbps. A different bitrate can be selected with:
|
||||
|
||||
@verbatim{racket flac2opus-manager.rkt --kbps 192 <source-directory> <target-directory>}
|
||||
|
||||
Metadata is copied through @racketmodname[racket-audio/taglib] and
|
||||
@racketmodname[racket-audio/audio-encoder]. TagLib properties and embedded
|
||||
pictures are transferred to the Opus file. The manager also writes a
|
||||
@tt{FLAC2OPUS} comment indicating that the file was converted by the manager.
|
||||
The conversion function is a normal synchronous function call.
|
||||
|
||||
When a source file disappears, the corresponding target file is removed on the
|
||||
next run. The manager does not mirror its own root-level administration files.
|
||||
Paths are handled as Racket paths instead of by splitting on @litchar{/}, so
|
||||
Windows absolute paths and UNC paths are left to the platform path
|
||||
implementation.
|
||||
@racketblock[
|
||||
(audio-manager "/mnt/music" "/mnt/music-opus" #:log-level 'info)
|
||||
]
|
||||
|
||||
@section{Configuration}
|
||||
|
||||
The configuration file is created automatically when it does not exist. The managers read it directly as sections and keys through @racketmodname[simple-ini]; there is no separate configuration struct layered on top of the ini model. The main settings are:
|
||||
The manager expects @filepath{.audio-manager.ini} in the source tree. A minimal
|
||||
configuration looks like this:
|
||||
|
||||
@verbatim{
|
||||
[manager]
|
||||
max-sample-rate=48000
|
||||
hash-algorithm="sha256"
|
||||
change-detection="flac-taglib"
|
||||
dry-run=#f
|
||||
display-log=#t
|
||||
log-file=".flac-48khz-manager.log"
|
||||
compression-level=5
|
||||
@verbatim|{
|
||||
[flac]
|
||||
max-khz=48000
|
||||
|
||||
[opus-manager]
|
||||
log-file=".flac2opus-manager.log"
|
||||
[opus]
|
||||
kbps=224
|
||||
|
||||
[mail]
|
||||
enabled=#f
|
||||
send-on-success=#f
|
||||
send-on-error=#t
|
||||
host=""
|
||||
from=Audio Manager <audio@example.org>
|
||||
to=Hans <hans@example.org>
|
||||
server=smtp.example.org
|
||||
port=25
|
||||
tls=#f
|
||||
username=""
|
||||
password=""
|
||||
from=""
|
||||
to=""
|
||||
cc=""
|
||||
bcc=""
|
||||
subject-prefix="[flac-48khz-manager]"
|
||||
}
|
||||
user=
|
||||
passwd=
|
||||
}|
|
||||
|
||||
When mail is enabled, the report is sent as HTML. The message contains the
|
||||
summary counters and the error table; it does not dump the full log by default.
|
||||
The mail configuration is required in the current implementation. If it is
|
||||
missing or incomplete, the manager fails at startup. This is intentional for a
|
||||
batch process where reporting should not silently disappear.
|
||||
|
||||
@section{Change detection}
|
||||
@section{Processing model}
|
||||
|
||||
The default @tt{change-detection} mode is @tt{flac-taglib}. Unchanged files are
|
||||
first skipped by comparing @tt{size} and @tt{mtime} from the keystore state. New
|
||||
or visibly changed files get a semantic fingerprint made from FLAC STREAMINFO
|
||||
and TagLib metadata. The FLAC part includes the STREAMINFO audio MD5 signature;
|
||||
the TagLib part includes properties and an embedded-picture content hash.
|
||||
The file walker produces entries with an @racket[info] hash. The manager then
|
||||
runs an ordered list of steps. Each step receives three values:
|
||||
|
||||
For non-FLAC files mirrored by @filepath{flac2opus-manager.rkt}, the default
|
||||
signature is @tt{mtime} plus @tt{size}. A full-file hash remains available by
|
||||
setting @tt{change-detection="hash"}.
|
||||
@racketblock[
|
||||
base-path
|
||||
path
|
||||
info
|
||||
]
|
||||
|
||||
@section{Library API}
|
||||
and returns the same three values, possibly with @racket[info] enriched. The
|
||||
important states in @racket[info] are:
|
||||
|
||||
@defproc[(manage-flac-tree
|
||||
[base-directory path-string?]
|
||||
[#:inspect-flac-proc inspect-flac-proc procedure? inspect-flac-sample-rate]
|
||||
[#:fingerprint-proc fingerprint-proc procedure? flac-taglib-fingerprint]
|
||||
[#:convert-proc convert-proc procedure? convert-flac-to-target-in-place])
|
||||
list?]{
|
||||
Scans @racket[base-directory], updates the keystore state, converts FLAC files
|
||||
above the configured maximum sample rate, sends the optional HTML mail report,
|
||||
and returns a summary association list.
|
||||
@itemlist[
|
||||
@item{@racket['new]: the relative path is not present in the saved state.}
|
||||
@item{@racket['changed]: the path exists, but type, size or modification time
|
||||
changed.}
|
||||
@item{@racket['unchanged]: the path exists and its basic filesystem state is
|
||||
unchanged.}
|
||||
@item{@racket['deleted]: the path existed in the saved state but no longer
|
||||
exists in the source tree.}]
|
||||
|
||||
The keyword arguments are intended for tests and dry integration work. In normal
|
||||
use, the default inspector, fingerprint function and converter are used.}
|
||||
For @racket['unchanged] entries, the previous enriched @racket[info] hash is
|
||||
reused. For @racket['changed] entries, the previous @racket[info] hash is reused
|
||||
but marked as changed and updated with the current filesystem data. Content-aware
|
||||
steps should therefore use @racket['changed] as the signal to re-inspect data
|
||||
such as sample rate, bit depth and sniffed audio format.
|
||||
|
||||
@defproc[(manage-flac2opus-tree
|
||||
[source-directory path-string?]
|
||||
[target-directory path-string?]
|
||||
[#:kbps kbps exact-positive-integer? 224]
|
||||
[#:convert-proc convert-proc procedure? convert-flac-to-opus])
|
||||
list?]{
|
||||
Mirrors @racket[source-directory] to @racket[target-directory]. FLAC files are
|
||||
converted to Opus at @racket[kbps] kbps; all other regular files are copied.
|
||||
The state is stored in the source directory's @filepath{.music-info.db} using a
|
||||
separate @tt{flac2opus} prefix. Keystore keys and stored target paths use
|
||||
normalized paths relative to the configured source or target base directory;
|
||||
absolute mount points are not stored and separators are always @tt{/}.}
|
||||
Conceptually, the pipeline has three groups:
|
||||
|
||||
@defproc[(summary->lines [summary list?]) (listof string?)]{
|
||||
Formats the summary association list as display lines.}
|
||||
@itemlist[
|
||||
@item{Enrichment: extension normalization, audio sniffing, FLAC-with-ID3
|
||||
detection, and FLAC sample-rate/bit-depth inspection.}
|
||||
@item{Actions: FLAC downsampling, embedding sidecar cover art, FLAC-to-Opus
|
||||
conversion, and copying non-FLAC sidecar files.}
|
||||
@item{Cleanup and logging: removal of target files for deleted source entries
|
||||
and progress logging.}]
|
||||
|
||||
@section{FLAC handling}
|
||||
|
||||
FLAC source files are presumed by extension and then confirmed by audio sniffing.
|
||||
A file with extension @filepath{.flac} but an ID3 prefix is reported as a
|
||||
FLAC-with-ID3 case. Such a file is not copied to the Opus tree as an ordinary
|
||||
sidecar file.
|
||||
|
||||
For valid FLAC files, the manager reads sample rate and bit depth through the
|
||||
FLAC decoder. Files above the configured @tt{[flac] max-khz} value are converted
|
||||
in place by @racketmodname[racket-audio/audio-encoder]. The conversion is a
|
||||
normal synchronous function call.
|
||||
|
||||
If a FLAC file has no embedded picture, the manager looks in the same directory
|
||||
for @filepath{cover.jpg}, @filepath{cover.jpeg}, @filepath{folder.jpg},
|
||||
@filepath{folder.jpeg}, @filepath{cover.png}, or @filepath{folder.png}. When one
|
||||
is found, it is embedded as front-cover artwork.
|
||||
|
||||
@section{Opus mirror handling}
|
||||
|
||||
For every processable FLAC source file, the manager writes an Opus target file
|
||||
with the same relative path and extension @filepath{.opus}:
|
||||
|
||||
@verbatim|{
|
||||
source: album/track.flac
|
||||
target: album/track.opus
|
||||
}|
|
||||
|
||||
The default bitrate is 224 kbps and can be changed with @tt{[opus] kbps}.
|
||||
|
||||
Non-FLAC files are copied unchanged to the target tree when they are new,
|
||||
changed, or missing from the target. Hidden files are ignored. This is intended
|
||||
for sidecar files such as cover images, PDFs, cue sheets and logs.
|
||||
|
||||
When a source entry disappears, the corresponding target entry is removed. For a
|
||||
deleted FLAC source, the target extension is changed from @filepath{.flac} to
|
||||
@filepath{.opus}. Renames are deliberately represented as a deleted old path and
|
||||
a new path; no rename detection is attempted.
|
||||
|
||||
@section{Logging and report}
|
||||
|
||||
Logging goes to @filepath{.audio-manager.log} in the source tree and is also
|
||||
shown on the console according to the selected log level. At the end of a
|
||||
completed run the manager writes the serialized state to @filepath{.audio-manager.db}
|
||||
and sends a report by mail.
|
||||
|
||||
The report contains sections for FLAC-with-ID3 cases, FLAC actions, Opus actions,
|
||||
file counters, failed conversions and failed copies.
|
||||
|
||||
@section{API}
|
||||
|
||||
@defproc[(audio-manager
|
||||
[music-path path-string?]
|
||||
[opus-path path-string?]
|
||||
[#:log-level log-level symbol? 'debug])
|
||||
symbol?]{
|
||||
Runs the complete batch process. @racket[music-path] is the source music tree.
|
||||
@racket[opus-path] is the target Opus mirror tree. The function returns
|
||||
@racket['done] after a completed run.}
|
||||
|
||||
@section{Module responsibilities}
|
||||
|
||||
The package keeps the responsibilities separated:
|
||||
|
||||
@itemlist[
|
||||
@item{@filepath{audio-manager.rkt}: orchestration, counters, report text and
|
||||
the ordered processing pipeline.}
|
||||
@item{@filepath{private/file-walker.rkt}: filesystem scan and state transition
|
||||
model.}
|
||||
@item{@filepath{private/flac-handling.rkt}: FLAC metadata inspection and FLAC
|
||||
downsampling.}
|
||||
@item{@filepath{private/opus-handling.rkt}: FLAC-to-Opus conversion.}
|
||||
@item{@filepath{private/mail.rkt}: SMTP report sending.}
|
||||
@item{@filepath{private/log.rkt}: logger definition.}
|
||||
@item{@filepath{private/util.rkt}: small path, date and cover-art helpers.}]
|
||||
|
||||
The manager does not implement audio codecs itself. Decoding, encoding,
|
||||
metadata, cover handling and resampling are delegated to @racketmodname[racket-audio].
|
||||
|
||||
Reference in New Issue
Block a user