node/browser based transclusion engine

resource resolvers

given a resource address, return a reliable and easy to consume resource; currently, text is the primary interest.

resolvers may contain resource-provider-specific syntax, such as using globs for local files, or text search queries for full-text-backed resource providers

csv/tsv file

jsonl file

pdf file

html file

zotero

passes either to pdf or html resolvers

calibre

passes to its epub resolver

org mode file

local git repo

follow github syntax, e.g.: https://github.com/hubotio/hubot/blob/ed25584f5ac2520a6c28547ffd0961c7abd7ea49/README.md

git:path-to-repo.git/blob/$COMMIT_HASH/$filename

content resolvers

given content resolved from a resource address, resolve to a specific portion of the content

see src/xcl/content_interop.cljs for resolvers; some examples:

• whole-file
• line-range
• org-heading
• org section

example usage

for API see test.cljs

let xcl = require("./build/transclusion.js")

// corresponds to corpus.cljs
const FileCache = {
    "xcl-test-3-a.org": "content of A!\n\naye aye aye",
    "xcl-test-3-b.org": "* I am B and I include A\n\n** {{{transclude(xcl:xcl-test-3-a.org)}}}",
    "xcl-test-3-c.org": "* I am C and I include B\n\n*{{{transclude(xcl:xcl-test-3-b.org)}}}",
}

function list_files() {
    return Object.keys(FileCache)
}
function load_content(filename) {
    return FileCache[filename]
}

console.log("=== INPUT ===")
console.log(load_content("xcl-test-3-c.org"))
console.log("=== OUTPUT ===")
console.log(xcl.render(
    list_files,
    load_content,
    load_content("xcl-test-3-c.org")))

there is no special escape syntax for the directive; to output a result that contains the directive verbatim, you can pass an escaped directive, e.g. "\{\{\{...\}\}\}", along with a postprocessor function that removes the escape char.

test environment setup

browser testing + cider usage

yarn global add shadow-cljs
yarn add react create-react-class react-dom
shadow-cljs watch browser-sample

open the browser to http://localhost:8020/ to load the tests

once the browser is loaded, you can also connect from emacs

node library testing

shadow-cljs watch node-test

this will watch + recompile the node-test profile, which builds a file that runs nodejs examples. In another terminal, run

node build/test.js

which should auto-rerun when the shadow-cljs watch process recompiles the tests

node REPL setup + cider

first get these 2 processes running

terminal 1

shadow-cljs watch node-test

terminal 2 (node example)

TODO fix the process exiter in test, or create a separate node file for repl use

# you will have to comment out the process exiter;
# this command should block and provide the repl process
node build/test.js

cider connect

M-x cider-connect-cljs

socket / web server + reload with watchexec

in one terminal

shadow-cljs watch node-content-server

in another terminal

watchexec --restart --no-ignore --watch ./build node build/node-content-server.js

default server response test

curl 'http://localhost:23120/'  -H 'Accept: application/json' -H 'Content-Type: application/json' --data-raw '{"jsonrpc":"2.0","id":0,"method":"help","params":[]}'

building for node.js

shadow-cljs release lib-sample

test files sources

alice.epub

http://www.gutenberg.org/ebooks/11

tracemonkey.pdf

https://github.com/mozilla/pdf.js/blob/master/test/pdfs/tracemonkey.pdf

syntax comparison and support for {{ ... }} for inline transclusion

provider	example	description	reference
mediawiki	{{Welcome}}	template (default) namespace	https://www.mediawiki.org/wiki/Transclusion
	{{:VisualEditor}}	main article namespace
	{{OtherNamespace:article}}	arbitrary namespace
	{{/ja}}	subpage
org-transclusion	#+transclude: [[file:path/to/file.org]] :argument-symbol	org link syntax	https://github.com/nobiot/org-transclusion
	[[file:~/org/file.org][My Org Notes]]	whole file
	[[file:~/org/file.org::*heading]]	heading
	[[file:~/org/file.org::#custom-id]]	arbitrary target by property	https://orgmode.org/manual/Internal-Links.html
	[[file:~/org/file.org::named-resource]]	tables, code blocks, etc
	[[id:uuid]]	another method for arbitrary target
obsidian	![[article-or-block-reference]]
multimarkdown	{{some-other-file.txt}}

for inline transclusion, since org-mode already uses double brackets, that leaves us with {{...}} as the best-candidate shortest syntax

implement xcl git directive using string pattern matching

so e.g.

git:path-to-repo.git/blob/$COMMIT_HASH/$pattern

or

git:path-to-repo.git/$pattern?

where $pattern requests a regexp/glob through full text of repo, (assuming unique?)