project_name: Spine, Doc Reform description: [ "documents, structuring, processing, publishing", search, object numbering, static content generator, sisu markup ] author: name: Ralph Amissah email: ralph.amissah@gmail.com copyright: "(C) 2015 - 2024 Ralph Amissah, All Rights Reserved." license: "AGPL 3 or later" homepage: [ "https://www.sisudoc.org", "https://www.doc-reform.org" ] ## Installation, Compilation Development of sisudoc-spine started in 2015 on a Debian linux box. Development since 2020 has been on a NixOS linux box, my laptop. If you are fortunate enough to be using the same the build instructions should be presented on entering the sisudoc-spine directory. It should be little problem building on other linuxes with the right dependencies. At one time, debconf-18 I was persuaded to try meson, and for a couple of years maintained a meson build, that dropped out of use before or on my making the switch to nixos in 2020. ❯❯ D compiler and build manager SiSU spine is written in the programming language D for which there are 3 compilers: dmd, ldc, gdc - https://wiki.dlang.org/Compilers D projects tend to use dub as project manager - https://code.dlang.org/packages/dub - https://github.com/dlang/dub/blob/master/source/dub/commandline.d The default build tools used are dub with ldc2 (dub is also tested) ## make a directory and clone the sisudoc-spine project mkdir ~/git.sisudoc cd ~/git.sisudoc git clone git://git.sisudoc.org/software/sisudoc-spine && \ git clone git://git.sisudoc.org/software/sisudoc-spine-search-cgi && \ git clone git://git.sisudoc.org/markup/sisudoc-spine-samples such a relative directory layout will be assumed in the examples that provided all work in this installation of and use of sisudoc-spine will take place in the directory: sisudoc-spine ## build sisudoc-spine NOTE all actions to build sisudoc-spine are taken within the directory sisudoc-spine cd sisudoc-spine ## directly with dub ### ldc2 # on nix (get dependencies by setting your development environment): nix develop ".#dsh-nixpkgs-ldc-dub" --print-build-logs -c zsh dub run --compiler=ldmd2 --config=ldmd2 --combined --skip-registry=all dub --compiler=ldmd2 --config=ldmd2 dub run --compiler=ldc2 --config=ldc2 --combined --skip-registry=all dub --compiler=ldc2 --config=ldc2 ### dmd # on nix (get dependencies by setting your development environment): nix develop ".#dsh-nixpkgs-dmd-dub" --print-build-logs -c zsh dub run --compiler=dmd --config=dmd --combined --skip-registry=all dub --compiler=dmd --config=dmd ## with make ### ldc2 make ldc ### dmd make dmd ## with nix on linux / nixos ### ldc2 nix build ".#spine-nixpkgs-ldc" --print-build-logs ### dmd nix build ".#spine-nixpkgs-dmd" --print-build-logs ## the Meson build system was used briefly On recommendation at debconf-18 meson was used briefly. It has neither been tested nor used since the move to nix. - https://mesonbuild.com/ meson ninja -C build meson setup --wipe build && ninja -v -C build make meson dub --force --compiler=ldc2 && sudo cp -v cgi-bin/spine-search /usr/lib/cgi-bin/. # Document processing examples These examples assume the file layout suggested in cloning the git.sisudoc.org repository, i.e. that the directories sisudoc-spine and sisudoc-spine-samples are next to each other on a directory tree. Assuming this to be the case, you may wish to set the following exports with adjustments accoring to your specific needs for these examples. # ❯❯ set spine binary location: export SpineBIN=./result/bin/spine # ❯❯ nix builds spine binary: #export SpineBIN=./result/bin/spine # ❯❯ dub builds spine binary (name depends on build, check): #export SpineBIN=./bin/spine #export SpineBIN=./bin/spine-ldc #export SpineBIN=./bin/spine-dmd # ❯❯ location of source files: export SpineDOC=../sisudoc-spine-samples # ❯❯ location of source files pod: export SpinePOD=${SpineDOC}/markup/pod # ❯❯ sisudoc-spine output processing path: export SpineOUT=./OUTPUT_TEST_sisudocSpine # ❯❯ sisudoc-spine output processing path (web server e.g.): #export SpineOUT=/srv/www/spine export SpineSearchActionLocal='http://localhost/spine_search' export SpineSearchActionRemote='https://sisudoc.org/spine_search' # ❯❯ path configured for cgi search form: export SpineCGIform='spine_search' # ❯❯ search form db name: export SpineSQLdb='spine.search.db' # ❯❯ configuration cgi search form path: #export SpineCGIbin=/var/www/cgi/cgi-bin # ❯❯ configuration db path: #export SpineDBpath=/var/www/sqlite ### html with links to search form ${SpineBIN} -v --epub --html --html-link-curate --curate --output=${SpineOUT} ${SpinePOD}/* ${SpineBIN} -v --source --pod --latex --latex-init --epub --html --html-link-pdf --html-link-curate --html-link-markup --curate --output=${SpineOUT} ${SpinePOD}/* ${SpineBIN} -v --source --pod --latex --latex-init --epub --html --html-link-search --html-link-pdf --html-link-curate --html-link-markup --cgi-sqlite-search-filename="${SpineCGIform}" --cgi-url-action="${SpineSearchActionLocal}" --curate --sqlite-update --sqlite-db-filename="${SpineSQLdb}" --output=${SpineOUT} ${SpinePOD}/* spine -v --html \ --html-link-search \ --output=`echo ~webDocRoot` \ ${SpinePOD}/* ## curate if you have a document collection with documents that have metadata headers a summary of the collection can be made using the curate command spine -v --curate --output=`echo ~webDocRoot` ~spineMarkupSamples/pod/* spine -v --curate ~spineMarkupSamples/pod/* spine -v --html --html-link-search --html-link-curate --curate --output=`echo ~webDocRoot` ~spineMarkupSamples/pod/* spine -v --html --html-link-search --html-link-curate --curate ~spineMarkupSamples/pod/* ## sqlite if configuartion has been set specify just - the desired output and - the markup document/pod(s) to process spine -v --html ~spineMarkupSamples/markup/pod/sisu-manual if configuration has not been set or to overide the set configuration specify - the output path as well as - the desired output and - the markup document/pod(s) to process note: ~webDocRoot should be the path to web doc root, provide a suitable output path. spine -v --html --html-link-search --html-link-curate --output=`echo ~webDocRoot` ~spineMarkupSamples/pod/* spine -v --html --html-link-search --html-link-curate --epub --output=`echo ~webDocRoot` ~spineMarkupSamples/pod/* spine -v --html --epub --latex --odt --curate --output=`echo ~webDocRoot` ~spineMarkupSamples/pod/* ### create db if there is no sqlite db you first need to create one, to do so - the name of the db and - the root path for document output must be specified: spine -v \ --sqlite-db-create --sqlite-db-filename="spine.search.db" \ --output=/var/www/html \ ~spineMarkupSamples/pod/* spine -v --sqlite-db-create --sqlite-db-filename="spine.search.db" --output=`echo ~webDocRoot` if you have a configration file providing this information that is to be used for a document collection you can point to the document collection: spine -v --sqlite-db-create ~spineMarkupSamples/pod ### populate db must specify: - the name of the db and - the root path for document output spine -v --sqlite-update \ --sqlite-db-filename="spine.search.db" \ --output=/var/www/html \ ~spineMarkupSamples/pod/* spine -v --sqlite-update --sqlite-db-filename="spine.search.db" --output=`echo ~webDocRoot` ~spineMarkupSamples/pod/* if you have a configration file providing this information that is to be used for a document collection you can point to the document collection: spine -v --sqlite-update ~spineMarkupSamples/pod/* ### generate a cgi search form in d spine -v --cgi-search-form-codegen \ --output=/var/www/html \ ~spineMarkupSamples/pod spine -v --cgi-search-form-codegen --config=~spineMarkupSamples/pod spine -v --cgi-search-form-codegen --config=~spineMarkupSamples/pod/.dr/config_local_site spine --cgi-search-form-codegen --output=`echo ~webDocRoot` ~spineMarkupSamples/pod spine --cgi-search-form-codegen --cgi-sqlite-search-filename="spine_search" --output=`echo ~webDocRoot` spine -v --cgi-search-form-codegen \ --sqlite-db-filename="spine.search.db" \ --cgi-sqlite-search-filename="spine-search" \ --output=/var/www/html \ ~spineMarkupSamples/pod #### compile the cgi search form cd /var/www/html/cgi # /var/www/html (default document root) cd ~webDocRoot/cgi the directory ~webDocRoot/cgi/src should contain two files - spine_search.d (or whatever you named it) - cgi.d (by Adam Rupee) dub --force --compiler=ldc2 && sudo cp -v cgi-bin/spine-search /usr/lib/cgi-bin/. should compile spine-search in ~webDocRoot/cgi/cgi-bin and copy it to the cgi-bin directory spine -v --sqlite-db-create --sqlite-db-filename="spine.search.db" --cgi-sqlite-search-filename="spine-search" --output=`echo ~webDocRoot` spine -v --sqlite-db-create ~spineMarkupSamples/pod spine -v --html --html-link-search --cgi-sqlite-search-filename="spine-search" --output=`echo ~webDocRoot` ~spineMarkupSamples/pod/* spine -v --html --html-link-search --output=`echo ~webDocRoot` ~spineMarkupSamples/pod/* spine -v --html --html-link-search --cgi-sqlite-search-filename="spine-search" --html-link-curate --curate --output=`echo ~webDocRoot` ~spineMarkupSamples/pod/* ${SpineBIN} -v --source --pod --latex --latex-init --epub --html --html-link-search --html-link-pdf --html-link-curate --html-link-markup --cgi-sqlite-search-filename="${SpineCGIform}" --cgi-url-action="${SpineSearchActionLocal}" --curate --sqlite-update --sqlite-db-filename="${SpineSQLdb}" --output=${SpineOUT} ${SpinePOD}/* ### create db & search form spine -v \ --sqlite-db-create --sqlite-db-filename="spine.search.db" \ --cgi-search-form-codegen --cgi-sqlite-search-filename="spine-search" \ --output=/var/www/html \ ~spineMarkupSamples/pod/* # Commands for a list of commands from the program type: spine -h at the time of writing this provides the following output: --abstraction document abstraction --allow-downloads allow downloads (includes cgi.d from github) --assert set optional assertions on --cgi-bin-root path to cgi-bin directory --cgi-url-root url to cgi-bin (to find cgi-bin) --cgi-url-action url to post to cgi-bin search form --cgi-search-title if generating a cgi search form the title to use for it --cgi-sqlite-search-filename =[filename] default is spine-search --concordance file for document --curate extract info on authors & topics from document header metadata --curate-authors extract info on authors from document header metadata --curate-topics extract info on topics from document header metadata --dark alternative dark theme --digest hash digest for each object --epub process epub output --generated-by generated by headers (software version & time) --hide-ocn object cite numbers --html process html output --html-link-curate place links back to curate in segmented html --html-link-markup provide html link to markup source, shared optionally --html-link-pdf provide html link to pdf a4 & letter output --html-link-pdf-a4 provide html link to pdf a4 output --html-link-pdf-letter provide html link to pdf letter size output --html-link-search html embedded search submission --html-seg process html output --html-scroll process html output --lang =[lang code e.g. =en or =en,es] --latex latex output (for pdfs) --latex-color-links mono or color links for pdfs --latex-init initialise latex shared files (see latex-header-sty) --latex-header-sty latex document header sty files --light default light theme --manifest process manifest output --ocn-off object cite numbers --odf open document format text (--odt) --odt open document format text --output =/path/to/output/dir specify where to place output --parallel parallelisation --parallel-subprocesses nested parallelisation --pdf latex output for pdfs --pdf-color-links mono or color links for pdfs --pdf-init initialise latex shared files (see latex-header-sty) --pod spine (doc reform) pod source content bundled -q --quiet output to terminal --section-backmatter document backmatter (default) --section-biblio document biblio (default) --section-blurb document blurb (default) --section-body document body (default) --section-bookindex document bookindex (default) --section-endnotes document endnotes (default) --section-glossary document glossary (default) --section-toc table of contents (default) --serial serial processing --skip-output skip output --show-config show config --show-curate show curate --show-curate-authors show curate authors --show-curate-topics show curate topics --show-epub show epub --show-html show html --show-latex show latex --show-make show make --show-manifest show manifest --show-metadata show metadata --show-pod show pod --show-sqlite show sqlite --show-summary show summary --source document markup source --set-digest default hash digest type (e.g. sha256) --set-papersize default papersize (latex pdf eg. a4 or a5 or b4 or letter) --set-textwrap default textwrap (e.g. 80 (characters) --sqlite-discrete process discrete sqlite output --sqlite-db-create create db, create tables --sqlite-db-drop drop tables & db --sqlite-db-filename sqlite db to create, populate & make available for search --sqlite-db-path sqlite db path --sqlite-db-recreate create db, create tables --sqlite-delete sqlite output --sqlite-insert sqlite output --sqlite-update sqlite output --www-http http or https --www-host web server host (domain) name --www-host-doc-root web host host (domain) name with path to doc root --www-url-doc-root e.g. http://localhost --text text output --theme-dark alternative dark theme --theme-light default light theme --txt text output -v --verbose output to terminal --very-verbose output to terminal --workon (reserved for some matters under development & testing) --xhtml xhtml output --config =/path/to/config/file/including/filename --debug debug --debug-curate debug curate --debug-curate-authors debug curate authors --debug-curate-topics debug curate topics --debug-epub debug epub --debug-harvest debug harvest --debug-html debug html --debug-latex debug latex --debug-manifest debug manifest --debug-metadata debug metadata --debug-pod debug pod --debug-sqlite debug sqlite --debug-stages debug stages -h --help This help information.