#+TITLE: sdp (project) discussion
#+AUTHOR: Ralph Amissah
#+EMAIL: ralph.amissah@gmail.com
#+STARTUP: indent
#+LANGUAGE: en
#+OPTIONS: H:3 num:nil toc:t \n:nil @:t ::t |:t ^:nil _:nil -:t f:t *:t <:t
#+OPTIONS: TeX:t LaTeX:t skip:nil d:nil todo:t pri:nil tags:not-in-toc
#+OPTIONS: author:nil email:nil creator:nil timestamp:nil
#+PROPERTY: header-args :padline no :exports none :noweb yes
#+EXPORT_SELECT_TAGS: export
#+EXPORT_EXCLUDE_TAGS: noexport
#+PRIORITIES: A F E
#+FILETAGS: :sdp:rel:makefile:
#+TAGS: assert(a) class(c) debug(d) mixin(m) sdp(s) tangle(T) template(t) WEB(W) noexport(n)

* sdp                                                                 :sdp:
** debug                                                            :debug:
  objects
  header
  heading
  poem verse ocn

** tasks                                                            :tasks:
*** sisu_loop.d
**** TODO [#A] header extraction (make instructions & metadata)
***** metadata
title & author heading
***** make
****** header make: search and replace
****** auto-identify structure from make instructions
****** auto-number headings, with starting level as instructed in header make
**** markup
***** para markers: extract (hash store) & clean (remove from text)
place in hash
  heading level,                 numeric
  indent info,                   numeric: first, rest
  bullet,                        bool
issue representing complex structures, consider using bits of json!
string h = "{ \"heading_level\": \"A\" }";
string para_attrib = "{ \"indent_first\": 0, \"indent_rest\": 0, \"bullet\": false }";
# string s = "{
#   \"indent_first\": 0,
#   \"indent_rest\": 0,
#   \"bullet\": false,
#   \"name\": "",
#   \"tags\": "",
#   \"digest\": "",
# }";
string para_attrib = "{
  \"indent_first\": 0,
  \"indent_rest\": 0,
  \"bullet\": false,
}";
string obj_ids = "{
  \"name\": \"\",
  \"tags\": \"\",
  \"digest\": \"\",
}";
string obj_lv = "{
  \"markup\": \"\",         // [A-D1-4] lv
  \"num_markup\": 0,      // [0-7]    ln
  \"num_collapsed\": 0,   // [0-7]    lc
}";
string obj_citation = "{
  \"ocn\": 0,             //
  \"on\": true,           //
}";

***** structure as json?
odv
osp
node
parent

you could keep ocn info (bool & number?) this way, consider

also clean
  block markers
  book index!
***** inline markup of text for subsequent processing
regex substitution
search and replace inline ascii markup with less ambiguous utf-8 markers
****** footnotes count/number
**** TODO [#B] endnotes/ footnotes for doc objects other than paragraphs & headings various forms of grouped text
**** TODO [#C] ocn (on & off)
**** TODO [#B] headings
***** heading numbering?

***** segment naming & id tags

**** TODO [#B] backmatter
***** book index
clean from object, store under separate key
consider json
****** TODO [#B] book index sort
    sort algorithm currently sorts A-Za-z need Aa-Zz
    also main terms references need to follow main term ... placed under _a which is not always sorted first
***** bibliography
identify and store under separate hash
consider using json to structure
***** glossary
identify and store under separate hash?
**** DONE [#B] composite documents
import files

<< path_to/file.sst

*** cleanup

*** ranges & node structure info: structure information (levels, ranges & relationships)
**** heading pointers
***** headings_array          heading pointer
***** data_abstration_array   heading pointer
**** ocn
**** heading
***** markup level            [A-D1-4]
***** collapsed level         [0-7]
**** parent
***** heading markup level    [A-D1-4]
***** heading ocn
**** range, heading & children (till next heading of same level or higher (close level mark))
***** array range             (x..y)
includes sub headings and non-heading objects till next heading
debate whether to use relative or absolute values (i.e. array points)
***** ocn range               (x..y)

NodeStructureMetadata see node_jstr
abstract_doc: obj_att\|node_jstr\|node
emitter: ObjAttrib
heading_pointer
*** misc
**** temporarily turn off ocn

--~# ---# --+#

~# & -#

**** parent & children
heading parent & children
paragraph parent

**** dir (read using dlang lib)
**** how to deal with complex data structures?
try hashes with json

**** read files
***** config
***** make
** compile                                                        :compile:
  [[http://www.dprogramming.com/tutorial.php][tutorial]]
  [[http://www.dprogramming.com/tutorial.php#newusers][tutorial new users]]
*** rdmd
rdmd --build-only --chatty -d-debug sdp.d
rdmd -d-debug sisu_ref0.d ../markup/test0.sst

VER='sdp2' && rdmd --build-only --chatty lib/${VER}/sdp.d

*** dmd                                                                :dmd:
dmd -de -w -unittest -debug sdp.d

VER='sdp2'
dmd -debug -of./bin/${VER} lib/${VER}/sdp.d
VER='sdp2' && dmd -debug=checkdoc -of./bin/${VER} lib/${VER}/sdp.d

VER='2' && dmd -debug=checkdoc -debug=summary -of./bin/sdp${VER} lib/sdp${VER}/sdp.d

*** ldc2                                                                :ldc:
ldc2 -d-debug=checkdoc -d-debug=summary -of=./bin/sdp lib/sdp/sdp.d

ldc2 -de -w -unittest -d-debug sdp.d

VER='2' && ldc2 -d-debug=checkdoc -d-debug=summary -of=./bin/sdp${VER} ./lib/sdp${VER}/sdp.d

VER='2' && ldc2 -unittest -d-debug=summary -of=./bin/sdp${VER} lib/sdp${VER}/sdp.d
VER='2' && ldc2 -d-debug=checkdoc -d-debug=summary -of=./bin/sdp${VER} lib/sdp${VER}/sdp.d
VER='2' && ldc2 -d-debug=checkdoc -d-debug=summary -of=./bin/sdp${VER} lib/sdp${VER}/sdp.d

VER='sdp2' && ldc2 -d-debug=objects -d-debug=summary -of=./bin/${VER} lib/${VER}/sdp.d

VER='sdp0'
VER='sdp1'
VER='sdp2'
VER='sdp3'
VER='sdp'
ldc2 -d-debug -of=./bin/sisu_${VER} lib/${VER}/sdp.d
ldc2 -d-debug -of=./bin/sisu_${VER} lib/${VER}/sdp.d
ldc2 -d-debug=heading -of=./bin/sisu_${VER} lib/${VER}/sdp.d
ldc2 -d-debug=objects -of=./bin/sisu_${VER} lib/${VER}/sdp.d

VER='sdp2' && ldc2 -d-debug=objects -d-debug=summary -of=./bin/sdp lib/${VER}/sdp.d

// VER='2' && ldc2 -unittest -d-debug=insert -d-debug=objects -d-debug=headings -d-debug=summary -d-debug=checkdoc -d-debug=subheader -of=./bin/sdp${VER} lib/sdp${VER}/sdp.d

**** remove later
binbuild="sdp1"; ldc2 -d-debug ./${binbuild}.d && time ./${binbuild} markup/test0.sst
binbuild="sdp1"; gdc -fdebug -o ./${binbuild} ./${binbuild}.d && time ./${binbuild} markup/test0.sst

binbuild="sdp2"
ldc2 -release ./${binbuild}.d && time ./${binbuild} markup/test0.sst

#cd lib
#ldc2 -d-debug -of=../bin/sdp0 sdp/sdp.d

*** gdc                                                                 :gdc:
  [[http://wiki.dlang.org/GDC/Using_GDC][Using GDC]]
copy/symlink deps, and then "*.d **/*.d"

gdc -o ./bin/sdp0 lib/sdp.d
VER='sdp2' &&
gdc -o ./bin/${VER}  lib/${VER}/sdp.d

VER='sdp2' && gdc-5 -o ./bin/${VER} ./lib/${VER}/sdp.d

VER='sdp2' && gdc-5 -o ./bin/${VER} ./lib/${VER}/sdp.d ./lib/${VER}/*.d

# VER='sdp2' &&  gdc -o -d-debug=objects -d-debug=summary -of=./bin/${VER} lib/${VER}/sdp.d

# check flags
# VER='sdp2' && gdc-5 -I. -O3 -fPIC -c -o ./bin/${VER} ./lib/${VER}/sdp.d

**** remove later
binbuild="sdp1"
ldc2 -d-debug ${binbuild}.d && time ${binbuild} markup/test0.sst
gdc -fdebug -o ${binbuild} ${binbuild}.d && time ${binbuild} markup/test0.sst

// gdc -release -o ./${binbuild} ./${binbuild}.d && time ./${binbuild} markup/test0.sst

**** bug

[http://forum.dlang.org/thread/mailman.284.1442659522.22025.digitalmars-d-learn@puremagic.com?page=3]

[http://www.mail-archive.com/digitalmars-d-learn@puremagic.com/msg61467.html]

??
[https://github.com/Dicebot/Arch-PKGBUILDs/blob/master/gdc/folders.diff]
from
[https://www.mail-archive.com/d.gnu@puremagic.com/msg03844.html]

[https://www.mail-archive.com/digitalmars-d-learn@puremagic.com/msg61470.html]

*** run                                                                 :run:

sdp0 markup/test0.sst

~utils/d/bin/sdp0 filename.sst
~utils/d/bin/sdp1 filename.sst

cd markup
sdp0 test0.sst

*** compile notes
**** ldc
import path[0] = /usr/include/d/ldc
import path[1] = /usr/include/d
**** gdc
gdmd -help
import path[0] = /usr/include/d/4.9/x86_64-linux-gnu
import path[1] = /usr/include/d/4.9
**** dmd (non-free) install arch?
**** issues

** notes                                                              :notes:
**** read file
char[][] markup_sourcefile_content = split(cast(char[])read(fn_src), rgx_line_delimiter);
char[][] markup_sourcefile_content = markupSourceLineArray(markupSourceString(fn_src));
*** build

** book index
// [http://forum.dlang.org/post/k8of07$1bgu$1@digitalmars.com]
// [http://forum.dlang.org/post/dfyowpjhdaemhxhepfmk@forum.dlang.org]
  // recast ---
  // ocns ; sub  ; main
  string[][string][string] bookindex;
  // as  ---
  // ocns ; sub  ; main
  string[]string[][string][] bookindex_the;
  // with sorted main & sub

// vim ./lib/sdp2/sisu_*.d
// vim **/sdp2/sisu_*.d

// emacs **/sdp2/sisu_*.d &
// emacs ./lib/sdp2/sisu_*.d &

** bugs                                                                :bugs:
ok
time ~dlang/bin/sdp2 --html --no-assert en/[a-eg-z]*
not ok
time ~dlang/bin/sdp2 --html --no-assert en/free_for_all.peter_wayner.sst en/gpl3.fsf.sst
works if:
  poems removed from gpl3;
  biblio removed from free for all
time ~dlang/bin/sdp2 --html --no-assert en/free_for_all.peter_wayner.sst en/gpl2.fsf.sst en/gpl3.fsf.sst
time ~dlang/bin/sdp2 --html --no-assert en/[fg]*
time ~dlang/bin/sdp2 --html --no-assert en/[a-z]*
leaving out free_for_all seems to make it ok
time ~dlang/bin/sdp2 --html --no-assert en/[a-eg-z]*
leaving out any two bibliography entries within free_for_all appears to fix the problem!

works in dmd not in ldc2

**** Error in `/home/ralph/grotto/repo/git.repo/utils/d/bin/sdp2': corrupted double-linked list: 0x00000008b905b310 ***
in free_for_all bibliography first instance FatBrain ref
gets stuck after:
en/free_for_all.peter_wayner.sst
** desc                                                                :desc:
*** process files
.sst (text) regular
.ssm (master) contains either .sst or .ssi
.ssi (insert) processed within .ssm (or called into a .ssm by another .ssi)
*** header
**** metadata
**** make (@make:)
cover_image
home_button_image
home_button_text
footer
headings
num_top
breaks
substitute
bold
italics
emphasis
texpdf_font
css

*** structure
document structure is determined by headings of different levels
headings must either
(a) be explicitly marked as such, or
(b) given by a regex (in the appropriate make header) that allows the program determine headings within text
types of heading:
**** document separators (A-D)
level A is the title
**** text headings       (1-4)
**** header make heading regex
***** heading levels
****** markup level [A-D1-4]
****** markup level numbers [0-7] or [1-8]
****** collapsed level numbers [0-7] or [1-8]
****** nodes
****** json search segments? chapter equivalent, decide
***** switches, ocn on off (dummy header)
*** object attributes
types of object:
**** headings (document structure objects)
***** level
***** segment name
***** numbering
**** paragraphs
**** blocks
types of block object:
***** group
***** block
***** poem (verse)
***** code
***** table
***** quote
***** TODO alt?
*** paragraph attributes
types of paragraph attribute:
**** indent
***** paragraph (1 - 9)
***** first line level (1 - 9), & subsequent text level (1 - 9)
indent (first, rest), bullet
**** bullets
*** inline text (paragraph) attributes
bold, italics, emphasis, superscript, substcript, strike, add, monospace, footnote (number them)
types of text (within paragraph) attribute:
**** bold
**** italics
**** emphasis
**** underscore
**** strike
**** superscript
**** subscript
** package
*** dub
  against dub:
  [[http://forum.dlang.org/post/hmdyrzbbhbcgqwqznqwz@forum.dlang.org][against dub]]
** dlang general
*** books
   [[http://wiki.dlang.org/Bookshttp://wiki.dlang.org/Books][D books]]
   [[http://ddili.org/ders/d.en/index.html][programming in D]]