kramdown-rfc2629-1.7.14/0000755000004100000410000000000014630024004014542 5ustar www-datawww-datakramdown-rfc2629-1.7.14/data/0000755000004100000410000000000014630024004015453 5ustar www-datawww-datakramdown-rfc2629-1.7.14/data/encoding-fallbacks.txt0000644000004100000410000000234414630024004021725 0ustar www-datawww-data00a0 00a1 ! 00a2 [cents] 00a3 GBP 00a4 [currency units] 00a5 JPY 00a6 | 00a7 S. 00a9 (C) 00aa a 00ab << 00ac [not] 00ae (R) 00af _ 00b0 o 00b1 +/- 00b2 ^2 00b3 ^3 00b4 ' 00b5 [micro] 00b6 P. 00b7 . 00b8 , 00b9 ^1 00ba o 00bb >> 00bc 1/4 00bd 1/2 00be 3/4 00bf ? 00c0 A 00c1 A 00c2 A 00c3 A 00c4 Ae 00c5 Ae 00c6 AE 00c7 C 00c8 E 00c9 E 00ca E 00cb E 00cc I 00cd I 00ce I 00cf I 00d0 [ETH] 00d1 N 00d2 O 00d3 O 00d4 O 00d5 O 00d6 Oe 00d7 x 00d8 Oe 00d9 U 00da U 00db U 00dc Ue 00dd Y 00de [THORN] 00df ss 00e0 a 00e1 a 00e2 a 00e3 a 00e4 ae 00e5 ae 00e6 ae 00e7 c 00e8 e 00e9 e 00ea e 00eb e 00ec i 00ed i 00ee i 00ef i 00f0 [eth] 00f1 n 00f2 o 00f3 o 00f4 o 00f5 o 00f6 oe 00f7 / 00f8 oe 00f9 u 00fa u 00fb u 00fc ue 00fd y 00fe [thorn] 00ff y 0152 OE 0153 oe 0161 s 0178 Y 0192 f 02dc ~ 2002 2003 2009 2013 -- 2014 --- 2018 ' 2019 ' 201a ' 201c " 201d " 201e " 2020 *!* 2021 *!!* 2022 o 2026 ... 2030 [/1000] 2032 ' 2039 < 203a > 2044 / 20ac EUR 2122 [TM] 2190 <-- 2192 --> 2194 <-> 21d0 <== 21d2 ==> 21d4 <=> 2212 - 2217 * 2264 <= 2265 >= 2329 < 232a > 0021 ! 0023 # 0024 $ 0025 % 0028 ( 0029 ) 002a * 002b + 002c , 002d - 002e . 002f / 003a : 003b ; 003d = 003f ? 0040 @ 005b [ 005d ] 005e ^ 005f _ 0060 ` 007b { 007c | 007d } 017d Z 017e z 2010 - kramdown-rfc2629-1.7.14/data/math.json0000644000004100000410000003534214630024004017306 0ustar www-datawww-data{"replacements":[["\\textfractionsolidus","⁄"],["\\leftrightsquigarrow","↭"],["\\textpertenthousand","‱"],["\\blacktriangleright","▸"],["\\blacktriangledown","▾"],["\\blacktriangleleft","◂"],["\\twoheadrightarrow","↠"],["\\leftrightharpoons","⇋"],["\\rightleftharpoons","⇌"],["\\textreferencemark","※"],["\\circlearrowright","↻"],["\\rightrightarrows","⇉"],["\\vartriangleright","⊳"],["\\textordmasculine","º"],["\\textvisiblespace","␣"],["\\twoheadleftarrow","↞"],["\\downharpoonright","⇂"],["\\ntrianglerighteq","⋭"],["\\rightharpoondown","⇁"],["\\textperthousand","‰"],["\\leftrightarrows","⇆"],["\\textmusicalnote","♪"],["\\nleftrightarrow","↮"],["\\rightleftarrows","⇄"],["\\bigtriangledown","▽"],["\\textordfeminine","ª"],["\\ntrianglelefteq","⋬"],["\\rightthreetimes","⋌"],["\\trianglerighteq","⊵"],["\\vartriangleleft","⊲"],["\\rightsquigarrow","⇝"],["\\downharpoonleft","⇃"],["\\curvearrowright","↷"],["\\circlearrowleft","↺"],["\\leftharpoondown","↽"],["\\nLeftrightarrow","⇎"],["\\curvearrowleft","↶"],["\\guilsinglright","›"],["\\leftthreetimes","⋋"],["\\leftrightarrow","↔"],["\\rightharpoonup","⇀"],["\\guillemotright","»"],["\\downdownarrows","⇊"],["\\hookrightarrow","↪"],["\\hspace{0.25em}"," "],["\\dashrightarrow","⇢"],["\\leftleftarrows","⇇"],["\\trianglelefteq","⊴"],["\\ntriangleright","⋫"],["\\doublebarwedge","⌆"],["\\upharpoonright","↾"],["\\rightarrowtail","↣"],["\\looparrowright","↬"],["\\Leftrightarrow","⇔"],["\\sphericalangle","∢"],["\\divideontimes","⋇"],["\\measuredangle","∡"],["\\blacktriangle","▴"],["\\ntriangleleft","⋪"],["\\mathchar\"1356","⁁"],["\\texttrademark","™"],["\\mathchar\"2208","⌖"],["\\triangleright","▹"],["\\leftarrowtail","↢"],["\\guilsinglleft","‹"],["\\upharpoonleft","↿"],["\\mathbb{gamma}","ℽ"],["\\fallingdotseq","≒"],["\\looparrowleft","↫"],["\\textbrokenbar","¦"],["\\hookleftarrow","↩"],["\\smallsetminus","﹨"],["\\dashleftarrow","⇠"],["\\guillemotleft","«"],["\\leftharpoonup","↼"],["\\mathbb{Gamma}","ℾ"],["\\bigtriangleup","△"],["\\textcircledP","℗"],["\\risingdotseq","≓"],["\\triangleleft","◃"],["\\mathsterling","£"],["\\textcurrency","¤"],["\\triangledown","▿"],["\\blacklozenge",""],["\\sfrac{5}{6}","⅚"],["\\preccurlyeq","≼"],["\\Rrightarrow","⇛"],["\\circledcirc","⊚"],["\\nRightarrow","⇏"],["\\sfrac{3}{8}","⅜"],["\\sfrac{1}{3}","⅓"],["\\sfrac{2}{5}","⅖"],["\\vartriangle","▵"],["\\Updownarrow","⇕"],["\\nrightarrow","↛"],["\\sfrac{1}{2}","½"],["\\sfrac{3}{5}","⅗"],["\\succcurlyeq","≽"],["\\sfrac{4}{5}","⅘"],["\\diamondsuit","♦"],["\\hphantom{0}"," "],["\\sfrac{1}{6}","⅙"],["\\curlyeqsucc","⋟"],["\\blacksquare","▪"],["\\hphantom{,}"," "],["\\curlyeqprec","⋞"],["\\sfrac{1}{8}","⅛"],["\\sfrac{7}{8}","⅞"],["\\sfrac{1}{5}","⅕"],["\\sfrac{2}{3}","⅔"],["\\updownarrow","↕"],["\\backepsilon","∍"],["\\circleddash","⊝"],["\\eqslantless","⋜"],["\\sfrac{3}{4}","¾"],["\\sfrac{5}{8}","⅝"],["\\hspace{1pt}"," "],["\\sfrac{1}{4}","¼"],["\\mathbb{Pi}","ℿ"],["\\mathcal{M}","ℳ"],["\\mathcal{o}","ȓ4"],["\\mathcal{O}","ᵊA"],["\\nsupseteqq","⊉"],["\\mathcal{B}","ℬ"],["\\textrecipe","℞"],["\\nsubseteqq","⊈"],["\\subsetneqq","⊊"],["\\mathcal{I}","ℑ"],["\\upuparrows","⇈"],["\\mathcal{e}","ℯ"],["\\mathcal{L}","ℒ"],["\\nleftarrow","↚"],["\\mathcal{H}","ℋ"],["\\mathcal{E}","ℰ"],["\\eqslantgtr","⋝"],["\\curlywedge","⋏"],["\\varepsilon","ε"],["\\supsetneqq","⊋"],["\\rightarrow","→"],["\\mathcal{R}","ℛ"],["\\sqsubseteq","⊑"],["\\mathcal{g}","ℊ"],["\\sqsupseteq","⊒"],["\\complement","∁"],["\\Rightarrow","⇒"],["\\gtreqqless","⋛"],["\\lesseqqgtr","⋚"],["\\circledast","⊛"],["\\nLeftarrow","⇍"],["\\Lleftarrow","⇚"],["\\varnothing","∅"],["\\mathcal{N}","𝒩"],["\\Leftarrow","⇐"],["\\gvertneqq","≩"],["\\mathbb{C}","ℂ"],["\\supsetneq","⊋"],["\\leftarrow","←"],["\\nleqslant","≰"],["\\mathbb{Q}","ℚ"],["\\mathbb{Z}","ℤ"],["\\llbracket","〚"],["\\mathbb{H}","ℍ"],["\\spadesuit","♠"],["\\mathit{o}","ℴ"],["\\mathbb{P}","ℙ"],["\\rrbracket","〛"],["\\supseteqq","⊇"],["\\copyright","©"],["\\textsc{k}","ĸ"],["\\gtreqless","⋛"],["\\mathbb{j}","ⅉ"],["\\pitchfork","⋔"],["\\estimated","℮"],["\\ngeqslant","≱"],["\\mathbb{e}","ⅇ"],["\\therefore","∴"],["\\triangleq","≜"],["\\varpropto","∝"],["\\subsetneq","⊊"],["\\heartsuit","♥"],["\\mathbb{d}","ⅆ"],["\\lvertneqq","≨"],["\\checkmark","✓"],["\\nparallel","∦"],["\\mathbb{R}","ℝ"],["\\lesseqgtr","⋚"],["\\downarrow","↓"],["\\mathbb{D}","ⅅ"],["\\mathbb{i}","ⅈ"],["\\backsimeq","⋍"],["\\mathbb{N}","ℕ"],["\\Downarrow","⇓"],["\\subseteqq","⊆"],["\\setminus","∖"],["\\succnsim","⋩"],["\\doteqdot","≑"],["\\clubsuit","♣"],["\\emptyset","∅"],["\\sqsupset","⊐"],["\\fbox{~~}","▭"],["\\curlyvee","⋎"],["\\varkappa","ϰ"],["\\llcorner","⌞"],["\\varsigma","ς"],["\\approxeq","≊"],["\\backcong","≌"],["\\supseteq","⊇"],["\\circledS","Ⓢ"],["\\circledR","®"],["\\textcent","¢"],["\\urcorner","⌝"],["\\lrcorner","⌟"],["\\boxminus","⊟"],["\\texteuro","€"],["\\vartheta","ϑ"],["\\barwedge","⊼"],["\\ding{86}","✶"],["\\sqsubset","⊏"],["\\subseteq","⊆"],["\\intercal","⊺"],["\\ding{73}","☆"],["\\ulcorner","⌜"],["\\recorder","⌕"],["\\precnsim","⋨"],["\\parallel","∥"],["\\boxtimes","⊠"],["\\ding{55}","✗"],["\\multimap","⊸"],["\\maltese","✠"],["\\nearrow","↗"],["\\swarrow","↙"],["\\lozenge","◊"],["\\sqrt[3]","∛"],["\\succsim","≿"],["\\dotplus","∔"],["\\tilde{}","~"],["\\check{}","ˇ"],["\\lessgtr","≶"],["\\Upsilon","ϒ"],["\\Cdprime","Ъ"],["\\gtrless","≷"],["\\backsim","∽"],["\\nexists","∄"],["\\dotplus","∔"],["\\searrow","↘"],["\\lessdot","⋖"],["\\boxplus","⊞"],["\\upsilon","υ"],["\\epsilon","ε"],["\\diamond","⋄"],["\\bigstar","★"],["\\ddagger","‡"],["\\cdprime","ъ"],["\\Uparrow","⇑"],["\\sqrt[4]","∜"],["\\between","≬"],["\\sqangle","∟"],["\\digamma","Ϝ"],["\\uparrow","↑"],["\\nwarrow","↖"],["\\precsim","≾"],["\\breve{}","˘"],["\\because","∵"],["\\bigcirc","◯"],["\\acute{}","´"],["\\grave{}","`"],["\\check{}","ˇ"],["\\lesssim","≲"],["\\partial","∂"],["\\natural","♮"],["\\supset","⊃"],["\\hstrok","ħ"],["\\Tstrok","Ŧ"],["\\coprod","∐"],["\\models","⊧"],["\\otimes","⊗"],["\\degree","°"],["\\gtrdot","⋗"],["\\preceq","≼"],["\\Lambda","Λ"],["\\lambda","λ"],["\\cprime","ь"],["\\varrho","ϱ"],["\\Bumpeq","≎"],["\\hybull","⁃"],["\\lmidot","ŀ"],["\\nvdash","⊬"],["\\lbrace","{"],["\\bullet","•"],["\\varphi","φ"],["\\bumpeq","≏"],["\\ddot{}","¨"],["\\Lmidot","Ŀ"],["\\Cprime","Ь"],["\\female","♀"],["\\rtimes","⋊"],["\\gtrsim","≳"],["\\mapsto","↦"],["\\daleth","ℸ"],["\\square","■"],["\\nVDash","⊯"],["\\rangle","〉"],["\\tstrok","ŧ"],["\\oslash","⊘"],["\\ltimes","⋉"],["\\lfloor","⌊"],["\\marker","▮"],["\\Subset","⋐"],["\\Vvdash","⊪"],["\\propto","∝"],["\\Hstrok","Ħ"],["\\dlcrop","⌍"],["\\forall","∀"],["\\nVdash","⊮"],["\\Supset","⋑"],["\\langle","〈"],["\\ominus","⊖"],["\\rfloor","⌋"],["\\circeq","≗"],["\\eqcirc","≖"],["\\drcrop","⌌"],["\\veebar","⊻"],["\\ulcrop","⌏"],["\\nvDash","⊭"],["\\urcrop","⌎"],["\\exists","∃"],["\\approx","≈"],["\\dagger","†"],["\\boxdot","⊡"],["\\succeq","≽"],["\\bowtie","⋈"],["\\subset","⊂"],["\\Sigma","Σ"],["\\Omega","Ω"],["\\nabla","∇"],["\\colon",":"],["\\boxHu","╧"],["\\boxHd","╤"],["\\aleph","ℵ"],["\\gnsim","⋧"],["\\boxHU","╩"],["\\boxHD","╦"],["\\equiv","≡"],["\\lneqq","≨"],["\\alpha","α"],["\\amalg","∐"],["\\boxhU","╨"],["\\boxhD","╥"],["\\uplus","⊎"],["\\boxhu","┴"],["\\kappa","κ"],["\\sigma","σ"],["\\boxDL","╗"],["\\Theta","Θ"],["\\Vdash","⊩"],["\\boxDR","╔"],["\\boxDl","╖"],["\\sqcap","⊓"],["\\boxDr","╓"],["\\bar{}","¯"],["\\dashv","⊣"],["\\vDash","⊨"],["\\boxdl","┐"],["\\boxVl","╢"],["\\boxVh","╫"],["\\boxVr","╟"],["\\boxdr","┌"],["\\boxdL","╕"],["\\boxVL","╣"],["\\boxVH","╬"],["\\boxVR","╠"],["\\boxdR","╒"],["\\theta","θ"],["\\lhblk","▄"],["\\uhblk","▀"],["\\ldotp","."],["\\ldots","…"],["\\boxvL","╡"],["\\boxvH","╪"],["\\boxvR","╞"],["\\boxvl","┤"],["\\boxvh","┼"],["\\boxvr","├"],["\\Delta","Δ"],["\\boxUR","╚"],["\\boxUL","╝"],["\\oplus","⊕"],["\\boxUr","╙"],["\\boxUl","╜"],["\\doteq","≐"],["\\happy","㋡"],["\\varpi","ϖ"],["\\smile","☺"],["\\boxul","┘"],["\\simeq","≃"],["\\boxuR","╘"],["\\boxuL","╛"],["\\boxhd","┬"],["\\gimel","ℷ"],["\\Gamma","Γ"],["\\lnsim","⋦"],["\\sqcup","⊔"],["\\omega","ω"],["\\sharp","♯"],["\\times","×"],["\\block","█"],["\\hat{}","^"],["\\wedge","∧"],["\\vdash","⊢"],["\\angle","∠"],["\\infty","∞"],["\\gamma","γ"],["\\asymp","≍"],["\\rceil","⌉"],["\\dot{}","˙"],["\\lceil","⌈"],["\\delta","δ"],["\\gneqq","≩"],["\\frown","⌢"],["\\phone","☎"],["\\vdots","⋮"],["\\boxr","└"],["\\k{i}","į"],["\\`{I}","Ì"],["\\perp","⊥"],["\\\"{o}","ö"],["\\={I}","Ī"],["\\`{a}","à"],["\\v{T}","Ť"],["\\surd","√"],["\\H{O}","Ő"],["\\vert","|"],["\\k{I}","Į"],["\\\"{y}","ÿ"],["\\\"{O}","Ö"],["\\'{Y}","Ý"],["\\u{u}","ў"],["\\u{G}","Ğ"],["\\.{E}","Ė"],["\\.{z}","ż"],["\\v{t}","ť"],["\\prec","≺"],["\\H{o}","ő"],["\\mldr","…"],["\\'{y}","ý"],["\\cong","≅"],["\\.{e}","ė"],["\\'{L}","Ĺ"],["\\star","*"],["\\.{Z}","Ż"],["\\'{e}","é"],["\\geqq","≧"],["\\cdot","⋅"],["\\`{U}","Ù"],["\\'{l}","ĺ"],["\\v{L}","Ľ"],["\\c{s}","ş"],["\\'{s}","ś"],["\\~{A}","Ã"],["\\Vert","‖"],["\\k{e}","ę"],["\\lnot","¬"],["\\'{z}","ź"],["\\leqq","≦"],["\\beta","β"],["\\beth","ℶ"],["\\'{E}","É"],["\\~{n}","ñ"],["\\u{i}","й"],["\\c{S}","Ş"],["\\c{N}","Ņ"],["\\H{u}","ű"],["\\v{n}","ň"],["\\'{S}","Ś"],["\\={U}","Ū"],["\\~{O}","Õ"],["\\'{Z}","Ź"],["\\v{E}","Ě"],["\\'{R}","Ŕ"],["\\H{U}","Ű"],["\\v{N}","Ň"],["\\prod","∏"],["\\v{s}","š"],["\\\"{U}","Ü"],["\\c{n}","ņ"],["\\k{U}","Ų"],["\\c{R}","Ŗ"],["\\'{A}","Á"],["\\~{o}","õ"],["\\v{e}","ě"],["\\v{S}","Š"],["\\u{A}","Ă"],["\\circ","∘"],["\\\"{u}","ü"],["\\flat","♭"],["\\v{z}","ž"],["\\r{U}","Ů"],["\\`{O}","Ò"],["\\={u}","ū"],["\\oint","∮"],["\\c{K}","Ķ"],["\\k{u}","ų"],["\\not<","≮"],["\\not>","≯"],["\\`{o}","ò"],["\\\"{I}","Ï"],["\\v{D}","Ď"],["\\.{G}","Ġ"],["\\r{u}","ů"],["\\not=","≠"],["\\`{u}","ù"],["\\v{c}","č"],["\\c{k}","ķ"],["\\.{g}","ġ"],["\\'{N}","Ń"],["\\odot","⊙"],["\\`{e}","э"],["\\c{T}","Ţ"],["\\v{d}","ď"],["\\\"{e}","ё"],["\\'{I}","Í"],["\\v{R}","Ř"],["\\k{a}","ą"],["\\nldr","‥"],["\\`{A}","À"],["\\'{n}","ń"],["\\~{N}","Ñ"],["\\nmid","∤"],["\\.{C}","Ċ"],["\\zeta","ζ"],["\\~{u}","ũ"],["\\`{E}","Э"],["\\~{a}","ã"],["\\c{t}","ţ"],["\\={o}","ō"],["\\v{r}","ř"],["\\={A}","Ā"],["\\.{c}","ċ"],["\\~{U}","Ũ"],["\\k{A}","Ą"],["\\\"{a}","ä"],["\\u{U}","Ў"],["\\iota","ι"],["\\={O}","Ō"],["\\c{C}","Ç"],["\\gneq","≩"],["\\'{c}","ć"],["\\boxH","═"],["\\hbar","ℏ"],["\\\"{A}","Ä"],["\\boxv","│"],["\\boxh","─"],["\\male","♂"],["\\'{u}","ú"],["\\sqrt","√"],["\\succ","≻"],["\\c{c}","ç"],["\\'{C}","Ć"],["\\v{l}","ľ"],["\\u{a}","ă"],["\\v{Z}","Ž"],["\\'{o}","ó"],["\\c{G}","Ģ"],["\\v{C}","Č"],["\\lneq","≨"],["\\\"{E}","Ё"],["\\={a}","ā"],["\\c{l}","ļ"],["\\'{a}","á"],["\\={E}","Ē"],["\\boxV","║"],["\\u{g}","ğ"],["\\'{O}","Ó"],["\\'{g}","ǵ"],["\\u{I}","Й"],["\\c{L}","Ļ"],["\\k{E}","Ę"],["\\.{I}","İ"],["\\~{I}","Ĩ"],["\\quad"," "],["\\c{r}","ŗ"],["\\'{r}","ŕ"],["\\\"{Y}","Ÿ"],["\\={e}","ē"],["\\'{U}","Ú"],["\\leq","≤"],["\\Cup","⋓"],["\\Psi","Ψ"],["\\neq","≠"],["\\k{}","˛"],["\\={}","‾"],["\\H{}","˝"],["\\cup","∪"],["\\geq","≥"],["\\mho","℧"],["\\Dzh","Џ"],["\\cap","∩"],["\\bot","⊥"],["\\psi","ψ"],["\\chi","χ"],["\\c{}","¸"],["\\Phi","Φ"],["\\ast","*"],["\\ell","ℓ"],["\\top","⊤"],["\\lll","⋘"],["\\tau","τ"],["\\Cap","⋒"],["\\sad","☹"],["\\iff","⇔"],["\\eta","η"],["\\eth","ð"],["\\d{}","̣"],["\\rho","ρ"],["\\dzh","џ"],["\\div","÷"],["\\phi","ϕ"],["\\Rsh","↱"],["\\vee","∨"],["\\b{}","ˍ"],["\\t{}","͡"],["\\int","∫"],["\\sim","∼"],["\\r{}","˚"],["\\Lsh","↰"],["\\yen","¥"],["\\ggg","⋙"],["\\mid","∣"],["\\sum","∑"],["\\Dz","Ѕ"],["\\Re","ℜ"],["\\oe","œ"],["\\DH","Ð"],["\\ll","≪"],["\\ng","ŋ"],["\\'G","Ѓ"],["\\wr","≀"],["\\wp","℘"],["\\=I","І"],["\\:)","☺"],["\\:(","☹"],["\\AE","Æ"],["\\AA","Å"],["\\ss","ß"],["\\dz","ѕ"],["\\ae","æ"],["\\aa","å"],["\\th","þ"],["\\to","→"],["\\Pi","Π"],["\\mp","∓"],["\\Im","ℑ"],["\\pm","±"],["\\pi","π"],["\\\"I","Ї"],["\\'C","Ћ"],["\\in","∈"],["\\'K","Ќ"],["\\'k","ќ"],["\\'c","ћ"],["\\'g","ѓ"],["\\ni","∋"],["\\ne","≠"],["\\TH","Þ"],["\\Xi","Ξ"],["\\nu","ν"],["\\NG","Ŋ"],["\\:G","㋡"],["\\xi","ξ"],["\\OE","Œ"],["\\gg","≫"],["\\DJ","Đ"],["\\=e","є"],["\\=E","Є"],["\\mu","μ"],["\\dj","đ"],["\\:"," "],["\\;"," "],["\\&","&"],["\\$","$"],["\\%","%"],["\\#","#"],["\\,"," "],["\\-","­"],["\\S","§"],["\\P","¶"],["\\O","Ø"],["\\L","Ł"],["\\}","}"],["\\o","ø"],["\\l","ł"],["\\h","ℎ"],["\\i","ℹ"],["-","−"]],"combiningmarks":[["\\doubleunderline","̳"],["\\strikethrough","̵"],["\\underline","̲"],["\\overline","̅"],["\\tilde","̃"],["\\grave","̀"],["\\acute","́"],["\\slash","̸"],["\\breve","̆"],["\\ddot","̈"],["\\dot","̇"],["\\bar","̅"],["\\vec","⃗"],["\\hat","̂"]],"subsuperscripts":[["_x","ₓ"],["_v","ᵥ"],["_u","ᵤ"],["_t","ₜ"],["_s","ₛ"],["_r","ᵣ"],["_p","ₚ"],["_o","ₒ"],["_n","ₙ"],["_m","ₘ"],["_l","ₗ"],["_k","ₖ"],["_j","ⱼ"],["_i","ᵢ"],["_h","ₕ"],["_e","ₑ"],["_a","ₐ"],["^∫","ᶴ"],["_>","˲"],["_=","₌"],["_<","˱"],["_9","₉"],["_8","₈"],["_7","₇"],["_6","₆"],["_5","₅"],["_4","₄"],["_3","₃"],["_2","₂"],["_1","₁"],["_0","₀"],["_-","₋"],["_−","₋"],["_+","₊"],["_)","₎"],["_(","₍"],["_ρ","ᵨ"],["_χ","ᵪ"],["_φ","ᵩ"],["_β","ᵦ"],["_γ","ᵧ"],["^φ","ᵠ"],["^χ","ᵡ"],["^δ","ᵟ"],["^γ","ᵞ"],["^β","ᵝ"],["^8","⁸"],["^9","⁹"],["^<","˂"],["^=","⁼"],["^>","˃"],["^0","⁰"],["^1","¹"],["^2","²"],["^3","³"],["^4","⁴"],["^5","⁵"],["^6","⁶"],["^7","⁷"],["^(","⁽"],["^)","⁾"],["^*","*"],["^+","⁺"],["^-","⁻"],["^−","⁻"],["^P","ᴾ"],["^R","ᴿ"],["^T","ᵀ"],["^U","ᵁ"],["^V","ᄑ"],["^W","ᵂ"],["^H","ᴴ"],["^I","ᴵ"],["^J","ᴶ"],["^K","ᴷ"],["^L","ᴸ"],["^M","ᴹ"],["^N","ᴺ"],["^O","ᴼ"],["^A","ᴬ"],["^B","ᴮ"],["^D","ᴰ"],["^E","ᴱ"],["^G","ᴳ"],["^x","ˣ"],["^y","ʸ"],["^z","ᶻ"],["^p","ᵖ"],["^r","ʳ"],["^s","ˢ"],["^t","ᵗ"],["^u","ᵘ"],["^v","ᵛ"],["^w","ʷ"],["^h","ʰ"],["^i","ⁱ"],["^j","ʲ"],["^k","ᵏ"],["^l","ˡ"],["^m","ᵐ"],["^n","ⁿ"],["^o","ᵒ"],["^a","ᵃ"],["^b","ᵇ"],["^c","ᶜ"],["^d","ᵈ"],["^e","ᵉ"],["^f","ᶠ"],["^g","ᵍ"]]} kramdown-rfc2629-1.7.14/data/kramdown-rfc2629.erb0000644000004100000410000001513614630024004021070 0ustar www-datawww-data"?> <% if $options.v3 %> [ <% else %> <% ps.arr("bibxml") do |tag, sys| -%> SYSTEM "<%= sys %>"> <% end -%> <% ps.arr("entity", false) do |en, ev| -%> "<%=ev%>"> <% end -%> ]> <% ps.rest.fetch("consensus") do # consensus not given -- default intelligently cat = ps.has("category") || ps.has("cat") MUST_CONSENSUS = {"std" => true, "bcp" => true} ps["consensus"] = true if MUST_CONSENSUS[cat] end rfcattrs = ps.attrs("ipr", "docName=docname", "category=cat", "consensus", "submissionType=submissiontype=stream", "xml:lang=lang", "number", "obsoletes", "updates", "seriesNo=seriesno") TRUE_FALSE = {nil => "true", false => "false", true => "true", "yes" => "true", "no" => "false"} YES_NO = {"true" => "yes", "false" => "no"} pis = KramdownRFC::ParameterSet.new({}) ps.arr("pi", false) do |pi, val| pis[pi] = TRUE_FALSE[val] || val end if $options.v3 piattrs = pis.attrs("tocDepth=tocdepth", "tocInclude=toc", "sortRefs=sortrefs", "symRefs=symrefs", "indexInclude=index") if piattrs != "" rfcattrs << " " << piattrs end end pis.rest.each do |pi, val| v = YES_NO[val] || pis.escattr(val) -%> ="<%=v%>"?> <% end -%> > <%= ps.ele("title", ps.attr("abbrev=titleabbrev")) %> <% ps.arr("author") do |au| aups = KramdownRFC::authorps_from_hash(au) -%> <%= KramdownRFC::person_element_from_aups("author", aups) -%> <% aups.warn_if_leftovers -%> <% end -%> /> <%= ps.ele("area") %> <%= ps.ele("workgroup=wg") %> <%= ps.ele("keyword=kw") %> <%= sechash.delete("abstract") %> <% if $options.v3 -%> <% venue = ps[:venue] -%> <% if venue -%> <% venue = KramdownRFC::ParameterSet.new(venue) -%> <% if (dn = ps.av[:docName]) && (dt = dn.rpartition('-')[0]) != "" -%> <% if latest = venue[:latest] -%> The latest revision of this draft can be found at . <% end -%> Status information for this document may be found at . <% end -%> <% mail = venue[:mail] -%> <% homepage = venue[:home] -%> <% gtype = venue[:type] -%> <% if mail || homepage -%> <% end -%> <% if mail mail_local, mail_host = mail.split("@", 2) end if mail_host -%> <% default_links = { "iab.org" => true, "ietf.org" => true, "irtf.org" => true, }[mail_host] mail_subdomain, mail_domain = mail_host.split(".", 2) group = venue[:group] || mail_local # XXX arch = venue[:arch] || default_links && "https://mailarchive.ietf.org/arch/browse/#{mail_local}/" subscribe = venue[:subscribe] || default_links && "https://www.ietf.org/mailman/listinfo/#{mail_local}/" GROUPS = {"ietf" => "Working ", "irtf" => "Research "} gtype ||= "#{GROUPS[mail_subdomain]}Group" -%> Discussion of this document takes place on the <%=group%> <%=gtype%> mailing list ()<% if arch -%>, which is archived at <% end -%>. <% if subscribe -%> Subscribe at . <% end -%> <% end -%> <% if homepage -%> <%=gtype%> information can be found at . <% end -%> <% if mail || homepage -%> <% end -%> <% if repo = venue[:repo] || ((gh = venue[:github]) && "https://github.com/#{gh}") -%> Source for this draft and an issue tracker can be found at . <% end -%> <%= venue.ele("t=text", nil, nil, true) -%> <% venue.warn_if_leftovers -%> <% end -%> <% end -%> <% sechash.keys.each do |k| -%> <% if k =~ /\A(to_be_removed_)?note_(.*)/ -%> <% option = "" text = "" if $1 if $options.v3 option = " removeInRFC=\"true\"" else text = " [This note is to be removed before publishing as an RFC.]\n" end end -%> "<%= option %>> <%= text -%> <%= sechash.delete(k) -%> <% end -%> <% end -%> {:/nomarkdown} {:quote: gi="blockquote"} {:aside: gi="aside"} {:markers: sourcecode-markers="true"} {:unnumbered: numbered="false"} {:vspace: vspace="0"} {:removeinrfc: removeinrfc="true"} {:notoc: toc="exclude"} {:compact: spacing="compact"} {:noabbrev: noabbrev="true"} {::nomarkdown} <%= sechash.delete("middle") %> <% displayref.each do |k, v| -%> <% end -%> <% if sh = sechash.delete("normative") -%> <%= sh %> <% end -%> <% if sh = sechash.delete("informative") -%> <%= sh %> <% end -%> <%= sechash.delete("back") %> <% sh = sechash.delete("contributor") -%> <% consec = ps.has("contributor") -%> <% if sh || consec -%> <% if $options.v3 -%>
Contributors <% else -%>
<% warn "*** To use YAML contributors, use --v3 (kdrfc -3)" if consec -%> <% end -%> <%= sh -%> <% if $options.v3 && consec ps.arr("contributor") do |au| aups = KramdownRFC::authorps_from_hash(au) -%> <%= KramdownRFC::person_element_from_aups("contact", aups) -%> <%= if contrib = aups["contribution"] < <% aups.warn_if_leftovers -%> <% end -%> <% end -%>
<% end -%> <% if $source -%> <% end -%> kramdown-rfc2629-1.7.14/bin/0000755000004100000410000000000014630024004015312 5ustar www-datawww-datakramdown-rfc2629-1.7.14/bin/kramdown-rfc0000755000004100000410000000021714630024004017632 0ustar www-datawww-data#!/usr/bin/env ruby # -*- coding: utf-8 -*- require 'ostruct' $options ||= OpenStruct.new $options.v3 = true require 'kramdown-rfc/command' kramdown-rfc2629-1.7.14/bin/kramdown-rfc-clean-svg-ids0000755000004100000410000000076414630024004022273 0ustar www-datawww-data#!/usr/bin/env ruby # -*- coding: utf-8 -*- require 'rexml/document' require 'kramdown-rfc/rexml-formatters-conservative' require 'kramdown-rfc/svg-id-cleanup' def svg_clean_ids(s) d = REXML::Document.new(s) d.context[:attribute_quote] = :quote # Set double-quote as the attribute value delimiter svg_id_cleanup(d) tr = REXML::Formatters::Conservative.new o = '' tr.write(d, o) o rescue => detail warn "*** Can't clean SVG: #{detail}" d.to_s end puts svg_clean_ids(ARGF.read) kramdown-rfc2629-1.7.14/bin/kramdown-rfc26290000755000004100000410000000011414630024004020151 0ustar www-datawww-data#!/usr/bin/env ruby # -*- coding: utf-8 -*- require 'kramdown-rfc/command' kramdown-rfc2629-1.7.14/bin/kramdown-rfc-cache-subseries-bibxml0000755000004100000410000000710414630024004024152 0ustar www-datawww-data#!/usr/bin/env ruby # prerequisite: # gem install net-http-persistent # # generates referencegroup files for BCP and STD series (like bibxml9) # unfortunately, needs to re-fetch rfc-index.xml # uses pretty slow built-in XML parser, so it will take a while even on regen # # uses ENV["KRAMDOWN_REFCACHEDIR"] for where you want to have your bibxml9 data # require 'rexml/document' require 'fileutils' begin require 'net/http/persistent' rescue LoadError warn "*** please install net-http-persistent:" warn " gem install net-http-persistent" warn "(prefix by sudo only if required)." exit 72 # EX_OSFILE end TARGET_DIR = ENV["KRAMDOWN_REFCACHEDIR"] || ( path = File.expand_path("~/.cache/xml2rfc") warn "*** set environment variable KRAMDOWN_REFCACHEDIR to #{path} to actually use the cache" path ) FileUtils.mkdir_p(TARGET_DIR) FileUtils.chdir(TARGET_DIR) $http = Net::HTTP::Persistent.new name: 'subseries' KRAMDOWN_PERSISTENT_VERBOSE = true def get_and_write_resource_persistently(url, fn, age_verbose=false) t1 = Time.now response = $http.request(URI(url)) if response.code != "200" raise "*** Status code #{response.code} while fetching #{url}" else File.write(fn, response.body) end t2 = Time.now warn "#{url} -> #{fn} (#{"%.3f" % (t2 - t1)} s)" if KRAMDOWN_PERSISTENT_VERBOSE if age_verbose if age = response.get_fields("age") warn "(working from a web cache, index is #{age.first} seconds stale)" end end end CLEAR_RET = "\e[K\r" # XXX all the world is ECMA-48 (ISO 6429), no? def noisy(name) print "#{name}...#{CLEAR_RET}" end def clear_noise print CLEAR_RET end def normalize_name(n) n.sub(/([A-Z])0+/) {$1} end def regress_name(n) n.sub(/([A-Z])(\d+)/) {"#$1#{"%04d" % $2.to_i}"} end def regress_name_dot(n) n.sub(/([A-Z])(\d+)/) {"#$1.#{"%04d" % $2.to_i}"} end def get_ref(rfcname) name = "reference.#{regress_name_dot(rfcname)}.xml" begin file = File.read(name) # no age check rescue Errno::ENOENT get_and_write_resource_persistently("https://www.rfc-editor.org/refs/bibxml/" << name, name) file = File.read(name) end d = REXML::Document.new(file) d.xml_decl.nowrite "\n" << d.to_s.lstrip end def create_bib(series, subname, rfcnames) p [series, subname, rfcnames] subname_norm = normalize_name(subname) refs = %{\n} refs << %{\n} refs << rfcnames.map {|x| get_ref(x)}.join refs << "\n" File.write("reference.#{regress_name_dot(subname)}.xml", refs) end def handle_sub(series, entry) ids = entry.get_elements("doc-id") warn "** ids #{ids} #{entry}" unless ids.size == 1 subname = ids.first.text isalso = entry.get_elements("is-also") if isalso.size == 1 rfcs = isalso.first.get_elements("doc-id") if rfcs.size > 0 rfcnames = rfcs.map {|r| r.text} create_bib(series, subname, rfcnames) end end end RFCINDEX_SOURCE = "https://www.rfc-editor.org/rfc/rfc-index.xml" RFCINDEX_COPY = File.basename(RFCINDEX_SOURCE) get_and_write_resource_persistently(RFCINDEX_SOURCE, RFCINDEX_COPY, true) unless ENV["KRAMDOWN_DONT_REFRESH_RFCINDEX"] doc = REXML::Document.new(File.read(RFCINDEX_COPY)) REXML::XPath.each(doc.root, "/rfc-index/bcp-entry") { |e| handle_sub("BCP", e) } REXML::XPath.each(doc.root, "/rfc-index/std-entry") { |e| handle_sub("STD", e) } kramdown-rfc2629-1.7.14/bin/kramdown-rfc-extract-sourcecode0000755000004100000410000001104714630024004023436 0ustar www-datawww-data#!/usr/bin/env ruby -KU require 'rexml/document' # for SVG and bibxml acrobatics require 'yaml' require 'shellwords' require 'fileutils' require_relative '../lib/kramdown-rfc/rfc8792' require_relative '../lib/kramdown-rfc/rexml-all-text.rb' def clean(s) s.gsub!(/\//, ":") s.gsub!(/\A([-.]+)/) {"_" * $1.size } s.gsub!(/[^-.:\w]+/, "-") s end $seen_slugs = {} def slugify_like_xml2rfc(s, delta = 0) s = s.unicode_normalize(:nfkd).force_encoding(Encoding::ASCII).scrub('').downcase s.gsub!(/[^-\w\s\/]/, '') s = s.strip s.gsub!(/[-\s\/]/, '-') n = 32 - delta nmax = [s.size, 40 - delta].min while $seen_slugs[slugex = s[0...n]] && n < nmax n += 1 end if $seen_slugs[slugex] m = 2 while $seen_slugs[slugex = "#{s[0...n]}-#{m}"] m += 1 if m == 1000 raise ArgumentError, "Can't build distinguishable slug for '#{s}'" end end end $seen_slugs[slugex] = s slugex end target = nil dir = nil unfold = true targets = [:list, :files, :zip, :yaml] require 'optparse' begin op = OptionParser.new do |opts| opts.banner = "Usage: kramdown-rfc-extract-sourcecode [options] document.xml" opts.on("-tFMT", "--to=FMT", targets, "Target format #{targets.map(&:to_s)}") do |v| target = v end opts.on("-dDIR", "--dir=DIR", "Target directory (default: sourcecode)") do |v| dir = v end opts.on("-f", "--[no-]unfold", "RFC8792-unfold (default: yes)") do |v| unfold = v end end op.parse! rescue Exception => e warn e exit 1 end if dir target ||= :files unless [:files, :zip].include? target warn "** Unused argument --dir=#{dir}" end end dir ||= "sourcecode" target ||= :list gensym = "unnamed-000" taken = Hash.new { |h, k| h[k] = Hash.new } warned = Hash.new { |h, k| h[k] = Hash.new } d = REXML::Document.new(ARGF) REXML::XPath.each(d.root, "//sourcecode|//artwork") do |x| if ty = x[:type] is_svg = false REXML::XPath.each(x, "svg") do is_svg = true end ty = clean(ty) if is_svg && ty != "svg" warn "** replacing SVG type '#{ty}' by type 'svg'" ty = "svg" end ext = ty if ty.empty? ty = "=txt" ext = "txt" end name = x[:name] name_is_from_slug = false if !name || name.empty? REXML::XPath.each(x, "ancestor::*/name[position() = 1]") do |nameel| unless sname = nameel['slugifiedName'] alltext = nameel.all_text nameel.add_attribute('slugifiedName', sname = clean("name-" << slugify_like_xml2rfc(alltext, 5))) end name = clean(sname.to_s.sub(/^name-/, '')) name_is_from_slug = true end if !name || name.empty? name = gensym.succ!.dup end name = "#{name}.#{ext}" end name = clean(name) if taken[ty][name] if name_is_from_slug || is_svg # rename old entry as well if it was from slug? nameparts = name.split(".") if is_svg && nameparts[-1] != "svg" nameparts << "svg" end ext1 = nameparts.pop || "txt" suffix = "b" while taken[ty][name = [*nameparts[0...-1], "#{nameparts[-1]}-#{suffix}", ext1].join(".")] suffix.succ! end taken[ty][name] = '' else unless warned[ty][name] warn "Concatenating to #{ty}/#{name}." warned[ty][name] = true end end else taken[ty][name] = '' end extracted = false REXML::XPath.each(x, "svg") do |svg| # According to v3.rnc, there should be only one... taken[ty][name] << svg.to_s extracted = true end unless extracted taken[ty][name] << handle_artwork_sourcecode(x.all_text, unfold) end end end def make_directory_from(dir, taken) if File.exist?(dir) bak = "#{dir}.bak" begin FileUtils.mv(dir, bak) rescue Errno::EEXIST bak.succ! retry end end FileUtils.mkdir_p(dir) taken.each { |dir1, v| FileUtils.mkdir_p("#{dir}/#{dir1}") v.each { |fn, value| IO.write("#{dir}/#{dir1}/#{fn}", value) } } end case target when :yaml puts taken.to_yaml when :list puts Hash[ taken.map {|k, v| [k, v.keys] } ].to_yaml when :files make_directory_from(dir, taken) when :zip make_directory_from(dir, taken) zip = "#{dir}.zip" if File.exist?(zip) bak = "#{zip}.bak" begin FileUtils.mv(zip, bak) rescue Errno::EEXIST # This doesn't actually happen. XXX bak.succ! retry end end cmd = ["zip", "-mr", zip, dir].shelljoin warn cmd system(cmd) end kramdown-rfc2629-1.7.14/bin/de-gfm0000755000004100000410000000163414630024004016403 0ustar www-datawww-data#!/usr/bin/env ruby -Ku Encoding.default_external = "UTF-8" # wake up, smell the coffee require 'kramdown' require 'kramdown-parser-gfm' options = '' while /\A-([4bck]+)\z/ === ARGV[0] ARGV.shift options << $1 end if /k/ === options # kramdown MARKDOWN_BR = "\\\\\n" end if /c/ === options # commonmark MARKDOWN_BR = "\\\n" end if /b/ === options # universal HTML MARKDOWN_BR = "
\n" end MARKDOWN_BR ||= " \n" # original Gruber module Kramdown module Converter # Converts an element tree to the kramdown format. class Kramdown < Base # Argh def convert_br(_el, _opts) MARKDOWN_BR end end end end list_indent = 2 list_indent = 4 if /4/ === options doc = Kramdown::Document.new(ARGF.read, input: 'GFM', gfm_quirks: 'paragraph_end', list_indent: list_indent) puts doc.to_kramdown kramdown-rfc2629-1.7.14/bin/kramdown-rfc-lsr0000755000004100000410000001623314630024004020435 0ustar www-datawww-data#!/usr/bin/env ruby -KU # frozen_string_literal: true # List Section References from a RFCXML document # # (PoC, in urgent need of refactoring) # Requires xml2rfc and tidy commands in path # Use without open-uri-cached is untested require 'rexml/document' require 'yaml' require 'json' require 'shellwords' require 'fileutils' begin require 'open-uri/cached' rescue LoadError warn '*** please "gem install open-uri-cached" to enable caching' require 'open-uri' end require_relative '../lib/kramdown-rfc/rexml-all-text.rb' target = :shortname require 'optparse' begin op = OptionParser.new do |opts| opts.banner = "Usage: kramdown-rfc-lsr [options] xml-source" opts.on("-u", "--url", "Source is URL") do |v| target = :url end opts.on("-s", "--shortname", "Source is shortname (default)") do |v| target = :shortname end opts.on("-f", "--file", "Source is filename") do |v| target = :file end end op.parse! rescue Exception => e warn e exit 1 end $exit_code = 0 if target != :file && ARGV.size != 1 puts op exit 1 end xmlsource = ARGV[0] if target == :shortname xmlsource = case xmlsource when /^(?:rfc)?(\d+)$/i "https://www.rfc-editor.org/rfc/rfc#{$1.to_i.to_s}.xml" when /^(?:draft-|I-D.|)(.*-\d\d)$/ "https://www.ietf.org/archive/id/draft-#$1.xml" # XXX find xml source for most recent version! else warn "*** Can't parse shortname #{xmlsource.inspect}" puts op exit 1 end target = :url end begin xml = case target when :file ARGF.read when :url URI(xmlsource).open.read else fail end rescue Exception => e warn "#{xmlsource.inspect}: #{e}" exit 1 end doc = REXML::Document.new(xml) def series_info_to_URI(si) case si[:name] when "RFC" "https://www.rfc-editor.org/rfc/rfc#{si[:value]}.xml" when "Internet-Draft" "https://www.ietf.org/archive/id/#{si[:value]}.xml" end end def series_info_to_name(si) case si[:name] when "RFC" "RFC#{si[:value]}" when "Internet-Draft" si[:value] end end def section_number_to_pn_candidates(s) if /^[0-9]/ =~ s ["section-#{s}"] elsif /[.]/ =~ s ["section-#{s.downcase}", "section-appendix.#{s.downcase}"] else ["section-appendix.#{s.downcase}"] end end def section_number_to_htmlid(s) if /^[0-9]/ =~ s "section-#{s}" else "appendix-#{s.upcase}" end end references = Hash[REXML::XPath.match(doc.root, "//reference").map {|r| si = REXML::XPath.match(r, "seriesInfo").map {|s| series_info_to_URI(s)}.compact.first sn = REXML::XPath.match(r, "seriesInfo").map {|s| series_info_to_name(s)}.compact.first [r[:anchor], si ? [si, sn] : nil] }] # XXX duplicates? heading1 = "# #{xmlsource}" title = REXML::XPath.first(doc.root, "//title") heading1 << "\n(#{title.all_text})" if title puts heading1 per_reference = Hash.new { |h, k| h[k] = Set[]} REXML::XPath.each(doc.root, "//xref[@section]") do |x| trg = x[:target] if x[:relative] puts "\n## #{x[:target]}#{x[:relative]}: #{x[:section]}" else # p x per_reference[trg] << x[:section] end end def error_out(s) warn "" warn s warn "" $exit_code = 1 end def num_expand(s) s.gsub(/\d+/) {|n| "%09d" % n.to_i} end def want_one(secs, what) case secs.size when 0 error_out "*** cannot match #{what}" "*** DOESN'T EXIST ***" when 1 yield secs.first else error_out "*** multiple matches for #{what}" "*** MULTIPLE MATCHES ***" end end require 'open3' module OpenURI class << self def processed(uri, old, camo, *rest) newuri = uri.to_s.sub(old, camo) # camo name for processed data response = Cache.get(newuri) || ( unprocessed = open_uri(uri, *rest).read fn = [OpenURI::Cache.cache_path, uri.sub(/.*\//, '')].join('/') File.open(fn, 'wb'){|f| f.write unprocessed } new_fn = yield newuri, fn Cache.set(newuri, File.open(new_fn)) ) response end def prepped(uri) processed(uri, /\.xml$/, ".prepped.xml") do |newuri, fn| _prep_out, s = Open3.capture2("xml2rfc", "--prep", fn) fail s.inspect unless s.success? fn.sub(/\.xml$/, ".prepped.xml") # xml2rfc creates new file end end def tidied(uri) processed(uri, /\.html$/, ".tidied.html") do |newuri, fn| _prep_out, s = Open3.capture2("tidy", "-mq", "-asxml", "-f", "/dev/null", fn) fail s.inspect unless s.exited? # can't check success fn # -m makes in-place change end end end end # go through section-referenced documents in sequence per_reference.keys.sort_by {|x| num_expand(x)}.each do |trg| uri, sname = references[trg] add = +'' if sname != trg add << " [#{sname}]" end begin ref = URI(uri).open.read refdoc = REXML::Document.new(ref) if REXML::XPath.match(refdoc.root, "/rfc/front/abstract[@pn]").size == 0 ref = OpenURI.prepped(uri).read refdoc = REXML::Document.new(ref) add << " [+prep]" end add << " (#{REXML::XPath.match(refdoc.root, "//title").first.all_text})" rescue OpenURI::HTTPError => e begin jsonuri = uri.sub(/\.xml$/, ".json") refjson = URI(jsonuri).open.read refdata = JSON.load(refjson) add << " (#{refdata["title"].strip})" rescue OpenURI::HTTPError => e add << " [No XML or JSON]" rescue Exception => e warn "*** error getting #{jsonuri.inspect}: #{e}" end rescue Exception => e warn "*** error getting #{uri.inspect}: #{e}" end puts "\n## #{trg}#{add}" unless refdoc begin htmluri = uri.sub(/\.xml$/, ".html") refhtml = OpenURI.tidied(htmluri).read refhtmldoc = REXML::Document.new(refhtml) rescue Exception => e warn "*** error tidying up HTML for #{htmluri.inspect}: #{e}" end end # go through individual section references in sequence per_reference[trg].to_a.sort_by {|x| num_expand(x)}.each do |s| add = +'' if refdoc # find section name in XML from anchor s secpn = section_number_to_pn_candidates(s) secs = secpn.flat_map{ |c| REXML::XPath.match(refdoc.root, "//section[@pn=$pn]", {}, {"pn" => c})} what = "#{secpn.join(" or ")} in #{trg}" add << " (#{want_one(secs, what) do |sec| sec[:title] || sec.elements["name"].all_text end})" elsif refhtmldoc # find section name in HTML from anchor s secpn = section_number_to_htmlid(s) secs = REXML::XPath.match(refhtmldoc.root, "//xmlns:a[@id=$pn]/ancestor::xmlns:span", {"xmlns" => "http://www.w3.org/1999/xhtml"}, {"pn" => secpn}) what = "#{secpn} in #{trg}" add << " (#{want_one(secs, what) do |sec| sec.text.sub(/^\.\s+/, '') end})" end puts "* #{/^[0-9]/ =~ s ? "Section" : "Appendix"} #{s}#{add}" end end exit $exit_code kramdown-rfc2629-1.7.14/bin/kdrfc0000755000004100000410000000502614630024004016334 0ustar www-datawww-data#!/usr/bin/env ruby -KU require 'kramdown-rfc/kdrfc-processor' require 'optparse' # try to get this from gemspec. KDRFC_VERSION=Gem.loaded_specs["kramdown-rfc2629"].version rescue "unknown-version" kdrfc = KramdownRFC::KDRFC.new kdrfc.options.txt = true # default op = OptionParser.new do |opts| opts.banner = <= 1645567342 # Time.parse("2022-02-22T22:02:22Z").to_i kdrfc.options.v3 = true # new default from the above date end end warn "*** v2 #{kdrfc.options.v2.inspect} v3 #{kdrfc.options.v3.inspect}" if kdrfc.options.verbose case ARGV.size when 1 fn = ARGV[0] begin kdrfc.process(fn) rescue StandardError => e warn e.to_s exit 1 end else puts op exit 1 end kramdown-rfc2629-1.7.14/bin/doilit0000755000004100000410000000233114630024004016523 0ustar www-datawww-data#!/usr/bin/env ruby require 'yaml' require 'kramdown-rfc2629' require 'kramdown-rfc/parameterset' require 'kramdown-rfc/refxml' require 'kramdown-rfc/doi' # doilit -c 10.6028/NIST.SP.800-183 10.1016/j.adhoc.2015.04.007 10.1109/MIC.2012.29 10.1145/2507924.2507954 $verbose = false $fuzzy = false $handle = "a" $xml = false $site = "https://dx.doi.org" litent = {} ARGV.each do |doi| case doi when "-c" begin require 'open-uri/cached' rescue LoadError warn '*** please "gem install open-uri-cached" to enable caching' end next when "-f" $fuzzy = true next when "-v" $verbose = true next when /\A-s=(.*)/ $site = $1 next when /\A-h=(.*)/ $handle = $1 next when /\A-x=(.*)/ $handle = $1 $xml = true next when /\A-/ warn "*** Usage: doilit [-c] [-f] [-v] [-h=handle|-x=xmlhandle] doi..." exit 1 end lit = doi_fetch_and_convert(doi, fuzzy: $fuzzy, verbose: $verbose, site: $site) while litent[$handle] $handle.succ! end litent[$handle] = lit end if $xml litent.each do |k, v| puts KramdownRFC::ref_to_xml(k, v) end else # 1.9 compat: s/lines/each_line.to_a/ puts litent.to_yaml.gsub(/^/, " ").each_line.to_a[1..-1] end kramdown-rfc2629-1.7.14/bin/kramdown-rfc-extract-markdown0000755000004100000410000000047614630024004023131 0ustar www-datawww-data#!/usr/bin/env ruby -KU require 'kramdown-rfc/gzip-clone' require 'base64' EMBEDDED_RE = %r{} embedded = ARGF.read.scan(EMBEDDED_RE) unless embedded.empty? puts Gzip.decompress(Base64.decode64(embedded[0][0])) else warn "*** No embedded markdown source found!" end kramdown-rfc2629-1.7.14/bin/kramdown-rfc-cache-i-d-bibxml0000755000004100000410000000605614630024004022624 0ustar www-datawww-data#!/usr/bin/env ruby # prerequisite: # gem install net-http-persistent # # dumps all bibxml for current, "Active" I-Ds in cache # reasonably efficient after initial call if the output is retained # # requires Ruby 2.4 or above because of "liberal_parsing" option # # uses ENV["KRAMDOWN_REFCACHEDIR"] for where you want to have your bibxml3 data # require 'csv' require 'fileutils' begin require 'net/http/persistent' rescue LoadError warn "*** please install net-http-persistent:" warn " gem install net-http-persistent" warn "(prefix by sudo only if required)." exit 72 # EX_OSFILE end TARGET_DIR = ENV["KRAMDOWN_REFCACHEDIR"] || ( path = File.expand_path("~/.cache/xml2rfc") warn "*** set environment variable KRAMDOWN_REFCACHEDIR to #{path} to actually use the cache" path ) FileUtils.mkdir_p(TARGET_DIR) FileUtils.chdir(TARGET_DIR) $http = Net::HTTP::Persistent.new name: 'allid' KRAMDOWN_PERSISTENT_VERBOSE = true def get_and_write_resource_persistently(url, fn, age_verbose=false) t1 = Time.now response = $http.request(URI(url)) if response.code != "200" raise "*** Status code #{response.code} while fetching #{url}" else File.write(fn, response.body) end t2 = Time.now warn "#{url} -> #{fn} (#{"%.3f" % (t2 - t1)} s)" if KRAMDOWN_PERSISTENT_VERBOSE if age_verbose if age = response.get_fields("age") warn "(working from a web cache, index is #{age.first} seconds stale)" end end end CLEAR_RET = "\e[K\r" # XXX all the world is ECMA-48 (ISO 6429), no? def noisy(name) print "#{name}...#{CLEAR_RET}" end def clear_noise print CLEAR_RET end ALL_ID2_SOURCE = "https://www.ietf.org/id/all_id2.txt" ALL_ID2_COPY = ".all_id2.txt" get_and_write_resource_persistently(ALL_ID2_SOURCE, ALL_ID2_COPY, true) unless ENV["KRAMDOWN_DONT_REFRESH_ALL_ID2"] ix = File.read(ALL_ID2_COPY).lines.grep_v(/^#/).join csv = CSV.new(ix, col_sep: "\t", liberal_parsing: true) drafts = csv.read active = drafts.select { |d| d[2] == "Active" } active_names = active.map { |a| a[0] } puts "#{active_names.size} active drafts" active_names.each do |name| if name =~ /\Adraft-(.*)-(\d\d)\z/ namepart = $1 version = $2 name0 = "reference.I-D.#{namepart}.xml" noisy(name0) if File.exist?(name0) name1 = "reference.I-D.draft-#{namepart}-#{version}.xml" if File.exist?(name1) noisy(name1) FileUtils.touch(name0) # because name1 already exists, we believe name0 is fresh else begin url0 = "https://datatracker.ietf.org/doc/bibxml3/draft-#{namepart}.xml" get_and_write_resource_persistently(url0, name0) # get name0 first url1 = "https://datatracker.ietf.org/doc/bibxml3/draft-#{namepart}-#{version}.xml" get_and_write_resource_persistently(url1, name1) # then name1 to mark this as updated rescue => e warn "*** #{name0}: #{e}" end end else warn "*** Malformed draft name: #{name}" end end clear_noise kramdown-rfc2629-1.7.14/bin/kramdown-rfc-autolink-iref-cleanup0000755000004100000410000000027314630024004024030 0ustar www-datawww-data#!/usr/bin/env ruby # -*- coding: utf-8 -*- require 'rexml/document' require 'kramdown-rfc/autolink-iref-cleanup' d = REXML::Document.new(ARGF.read) autolink_iref_cleanup(d) puts d.to_s kramdown-rfc2629-1.7.14/bin/echars0000755000004100000410000000313714630024004016511 0ustar www-datawww-data#!/usr/bin/env ruby require 'unicode/name' require 'unicode/scripts' require 'unicode/blocks' require 'json' require 'differ' module Differ module Format module Color class << self def as_change(change) # monkey patch as_insert(change) << "\n" << as_delete(change) end end end end end def readable(c) j = c.to_json if j.size == 3 j[1...-1] else j end end def explain(s) ret = '' hist = Hash.new(0) s.each_char do |c| hist[c] += 1 unless c.ord.between?(32, 126) end hist.delete("\n") hist.keys.sort.group_by {|c| Unicode::Blocks.block(c)}.each do |block, l| scripts = Set[] l.each do |c| scripts << Unicode::Scripts.scripts(c) end ret << "*** #{block}" ret << " (#{scripts.join})" if scripts.size == 1 ret << "\n" l.each do |c| ret << "#{readable(c)}: U+#{"%04X %4d" % [c.ord, hist[c]] } #{Unicode::Name.correct(c) || Unicode::Name.label(c) }" ret << " (#{Unicode::Scripts.scripts(c).join(", ")})" if scripts.size != 1 ret << "\n" end end ret end s = ARGF.read es = explain(s) n = s.unicode_normalize en = explain(n) if es == en puts es else puts "*** Warning: some characters are not normalized and are shown in red." puts " ...showing a normalized variant (NFC) in green." puts " Lack of normalization may or may not be a problem." puts " (Characters may appear to be under wrong block heading.)" puts Differ.diff_by_line(en, es).format_as(:color) end kramdown-rfc2629-1.7.14/kramdown-rfc2629.gemspec0000644000004100000410000000356114630024004021031 0ustar www-datawww-dataspec = Gem::Specification.new do |s| s.name = 'kramdown-rfc2629' s.version = '1.7.14' s.summary = "Kramdown extension for generating RFCXML (RFC 799x)." s.description = %{An RFCXML (RFC 799x) generating backend for Thomas Leitner's "kramdown" markdown parser. Mostly useful for RFC writers.} s.add_dependency('kramdown', '~> 2.4.0') s.add_dependency('kramdown-parser-gfm', '~> 1.1') s.add_dependency('certified', '~> 1.0') s.add_dependency('json_pure', '~> 2.0') s.add_dependency('unicode-name', '~> 1.0') s.add_dependency('unicode-blocks', '~> 1.0') s.add_dependency('unicode-scripts', '~> 1.0') s.add_dependency('net-http-persistent', '~> 4.0') s.add_dependency('differ', '~> 0.1') s.add_dependency('base64', '>= 0.1') s.files = Dir['lib/**/*.rb'] + %w(README.md LICENSE kramdown-rfc2629.gemspec bin/kdrfc bin/kramdown-rfc bin/kramdown-rfc2629 bin/doilit bin/echars bin/kramdown-rfc-extract-markdown bin/kramdown-rfc-extract-sourcecode bin/kramdown-rfc-lsr data/kramdown-rfc2629.erb data/encoding-fallbacks.txt data/math.json bin/kramdown-rfc-cache-subseries-bibxml bin/kramdown-rfc-autolink-iref-cleanup bin/de-gfm bin/kramdown-rfc-clean-svg-ids) s.require_path = 'lib' s.executables = ['kramdown-rfc', 'kramdown-rfc2629', 'doilit', 'echars', 'kramdown-rfc-extract-markdown', 'kramdown-rfc-extract-sourcecode', 'kramdown-rfc-lsr', 'kdrfc', 'kramdown-rfc-cache-i-d-bibxml', 'kramdown-rfc-cache-subseries-bibxml', 'kramdown-rfc-autolink-iref-cleanup', 'de-gfm', 'kramdown-rfc-clean-svg-ids'] s.required_ruby_version = '>= 2.3.0' # s.requirements = 'wget' # s.has_rdoc = true s.author = "Carsten Bormann" s.email = "cabo@tzi.org" s.homepage = "http://github.com/cabo/kramdown-rfc" s.license = 'MIT' end kramdown-rfc2629-1.7.14/lib/0000755000004100000410000000000014630024004015310 5ustar www-datawww-datakramdown-rfc2629-1.7.14/lib/kramdown-rfc/0000755000004100000410000000000014630024004017702 5ustar www-datawww-datakramdown-rfc2629-1.7.14/lib/kramdown-rfc/gzip-clone.rb0000644000004100000410000000177114630024004022304 0ustar www-datawww-datarequire 'zlib' require 'stringio' # cloned from module ActiveSupport # A convenient wrapper for the zlib standard library that allows # compression/decompression of strings with gzip. # # gzip = Gzip.compress('compress me!') # # => "\x1F\x8B\b\x00o\x8D\xCDO\x00\x03K\xCE\xCF-(J-.V\xC8MU\x04\x00R>n\x83\f\x00\x00\x00" # # Gzip.decompress(gzip) # # => "compress me!" module Gzip class Stream < StringIO def initialize(*) super set_encoding "BINARY" end def close; rewind; end end # Decompresses a gzipped string. def self.decompress(source) Zlib::GzipReader.new(StringIO.new(source)).read end # Compresses a string using gzip, setting mtime to 0 def self.compress_m0(source, level=Zlib::DEFAULT_COMPRESSION, strategy=Zlib::DEFAULT_STRATEGY) output = Stream.new gz = Zlib::GzipWriter.new(output, level, strategy) gz.mtime = 0 gz.write(source) gz.close output.string end end # end kramdown-rfc2629-1.7.14/lib/kramdown-rfc/parameterset.rb0000644000004100000410000000462414630024004022731 0ustar www-datawww-datamodule KramdownRFC class ParameterSet include Kramdown::Utils::Html attr_reader :f def initialize(y) raise "*** invalid parameter set #{y.inspect}" unless Hash === y @f = y @av = {} end attr :av def [](pn) @f.delete(pn.to_s) end def []=(pn, val) @f[pn.to_s] = val end def default(pn, &block) @f.fetch(pn.to_s, &block) end def default!(pn, value) default(pn) { @f[pn.to_s] = value } end def has(pn) @f[pn.to_s] end def has?(pn) @f.key?(pn.to_s) end def escattr(str) escape_html(str.to_s, :attribute) end def van(pn) # pn is a parameter name, possibly with =aliases names = pn.to_s.split("=") [self[names.reverse.find{|nm| has?(nm)}], names.first] end def attr(pn) val, an = van(pn) @av[an.intern] = val %{#{an}="#{escattr(val)}"} if val # see attrtf below end def attrs(*pns) pns.map{ |pn| attr(pn) if pn }.compact.join(" ") end def attrtf(pn) # can do an overriding false value val, an = van(pn) @av[an.intern] = val %{#{an}="#{escattr(val)}"} unless val.nil? end def attrstf(*pns) pns.map{ |pn| attrtf(pn) if pn }.compact.join(" ") end def ele(pn, attr=nil, defcontent=nil, markdown=false) val, an = van(pn) val ||= defcontent val = [val] if Hash === val Array(val).map do |val1| a = Array(attr).dup if Hash === val1 val1.each do |k, v| if k == ":" val1 = v else k = Kramdown::Element.attrmangle(k) || k a.unshift(%{#{k}="#{escattr(v)}"}) end end end v = val1.to_s.strip contents = if markdown ::Kramdown::Converter::Rfc2629::process_markdown(v) else escape_html(v) end %{<#{[an, *a.map(&:to_s)].join(" ").strip}>#{contents}} end.join(" ") end def arr(an, converthash=true, must_have_one=false, &block) arr = self[an] || [] arr = [arr] if Hash === arr && converthash arr << { } if must_have_one && arr.empty? Array(arr).each(&block) end def rest @f end def warn_if_leftovers if !@f.empty? warn "*** attributes left #{@f.inspect}!" end end end end kramdown-rfc2629-1.7.14/lib/kramdown-rfc/rfc8792.rb0000644000004100000410000000567414630024004021347 0ustar www-datawww-data # Note that this doesn't attempt to handle HT characters def remove_indentation(s) l = s.lines indent = l.grep(/\S/).map {|l| l[/^\s*/].size}.min l.map {|li| li.sub(/^ {0,#{indent}}/, "")}.join end def trim_empty_lines_around(s) # this deletes the trailing newline, which may need to be reconstructed s.sub(/\A(\r?\n)*/, '').sub(/(\r?\n)*\z/, '') end def fix_unterminated_line(s) s.sub(/[^\n]\z/) { "#$&\n" } # XXX end def handle_artwork_sourcecode(s, unfold = true) s = trim_empty_lines_around(s) s = unfold8792(s) if unfold fix_unterminated_line(s) end FOLD_MSG = "NOTE: '\\' line wrapping per RFC 8792".freeze UNFOLD_RE = /\A.*#{FOLD_MSG.sub("\\", "(\\\\\\\\\\\\\\\\?)")}.*\n\r?\n/ def unfold8792(s) if s =~ UNFOLD_RE indicator = $1 s = $' sub = case indicator when "\\" s.gsub!(/\\\n[ \t]*/, '') when "\\\\" s.gsub!(/\\\n[ \t]*\\/, '') else fail "indicator" # Cannot happen end warn "** encountered RFC 8792 header without folded lines" unless sub end s end MIN_FOLD_COLUMNS = FOLD_MSG.size FOLD_COLUMNS = 69 RE_IDENT = /\A[A-Za-z0-9_]\z/ def fold8792_1(s, columns = FOLD_COLUMNS, left = false, dry = false) if s.index("\t") warn "*** HT (\"TAB\") in text to be folded. Giving up." return s end if columns < MIN_FOLD_COLUMNS columns = if columns == 0 FOLD_COLUMNS else warn "*** folding to #{MIN_FOLD_COLUMNS}, not #{columns}" MIN_FOLD_COLUMNS end end lines = s.lines.map(&:chomp) did_fold = false ix = 0 while li = lines[ix] col = columns if li[col].nil? if li[-1] == "\\" lines[ix..ix] = [li << "\\", ""] ix += 1 end ix += 1 else did_fold = true min_indent = left || 0 col -= 1 # space for "\\" while li[col] == " " # can't start new line with " " col -= 1 end if col <= min_indent warn "*** Cannot RFC8792-fold1 to #{columns} cols #{"with indent #{left}" if left} |#{li.inspect}|" else if RE_IDENT === li[col] # Don't split IDs col2 = col while col2 > min_indent && RE_IDENT === li[col2-1] col2 -= 1 end if col2 > min_indent col = col2 end end rest = li[col..-1] indent = left || columns - rest.size if !left && li[-1] == "\\" indent -= 1 # leave space for next round end if indent > 0 rest = " " * indent + rest end lines[ix..ix] = [li[0...col] << "\\", rest] end ix += 1 end end if did_fold msg = FOLD_MSG.dup if !dry && columns >= msg.size + 4 delta = columns - msg.size - 2 # 2 spaces half = delta/2 msg = "#{"=" * half} #{msg} #{"=" * (delta - half)}" end lines[0...0] = [msg, ""] lines.map{|x| x << "\n"}.join else s end end kramdown-rfc2629-1.7.14/lib/kramdown-rfc/autolink-iref-cleanup.rb0000644000004100000410000000154314630024004024430 0ustar www-datawww-datarequire 'rexml/document' def autolink_iref_cleanup(d) d.root.get_elements("//section[@anchor]").each do |sec| anchor = sec['anchor'] irefs = {} sec.get_elements(".//xref[@target='#{anchor}'][@format='none']").each do |xr| ne = xr.previous_element # 9c87e84 iref now before xref if ne && ne.name == "iref" && (item = ne['item']) irefs[item] = ne['subitem'] # XXX one subitem only ne.remove chi = xr.children chi[1..-1].reverse.each do |ch| xr.parent.insert_after(xr, ch) end xr.replace_with(chi[0]) end end irefs.each do |k, v| sec.insert_after(sec.get_elements("name").first, e = REXML::Element.new("iref", sec)) e.attributes["item"] = k e.attributes["subitem"] = v e.attributes["primary"] = 'true' end end end kramdown-rfc2629-1.7.14/lib/kramdown-rfc/rexml-all-text.rb0000644000004100000410000000054214630024004023107 0ustar www-datawww-datarequire 'rexml/document' module REXML # all_text: Get all text from descendants that are Text or CData class Element def all_text @children.map {|c| c.all_text}.join end end class Text # also: ancestor of CData def all_text value end end class Child def all_text '' end end end kramdown-rfc2629-1.7.14/lib/kramdown-rfc/command.rb0000755000004100000410000005110014630024004021645 0ustar www-datawww-data#!/usr/bin/env ruby # -*- coding: utf-8 -*- require 'kramdown-rfc2629' require 'kramdown-rfc/parameterset' require 'kramdown-rfc/refxml' require 'kramdown-rfc/rfc8792' require 'yaml' require 'kramdown-rfc/erb' require 'date' # try to get this from gemspec. KDRFC_VERSION=Gem.loaded_specs["kramdown-rfc2629"].version rescue "unknown-version" Encoding.default_external = "UTF-8" # wake up, smell the coffee def add_quote(s) l = s.lines l.map {|li| "> #{li}"}.join end def process_chunk(s, nested, dedent, fold, quote) process_includes(s) if nested s = remove_indentation(s) if dedent s = fold8792_1(s, *fold) if fold s = add_quote(s) if quote s end def process_includes(input) input.gsub!(/^\{::include((?:-[a-z0-9]+)*)\s+(.*?)\}/) { include_flags = $1 fn = [$2] chunks = false nested = false dedent = false fold = false quote = false include_flags.split("-") do |flag| case flag when "" when "nested" nested = true when "quote" quote = true when "dedent" dedent = true when /\Afold(\d*)(left(\d*))?(dry)?\z/ fold = [$1.to_i, # col 0 for '' ($3.to_i if $2), # left 0 for '', nil if no "left" $4] # dry when "all", "last" fn = fn.flat_map{|n| Dir[n]} fn = [fn.last] if flag == "last" chunks = fn.map{ |f| ret = process_chunk(File.read(f), nested, dedent, fold, quote) nested = false; dedent = false; fold = false; quote = false ret } else warn "** unknown include flag #{flag}" end end chunks = fn.map{|f| File.read(f)} unless chunks # no all/last chunks = chunks.map {|ch| process_chunk(ch, nested, dedent, fold, quote)} chunks.join.chomp } end def boilerplate(key) ret = '' case key.downcase when /\Abcp14(info)?(\+)?(-tagged)?(-bcp)?\z/i # $1 $2 $3 $4 if $1 ret << < *[MUST NOT]: *[REQUIRED]: *[SHALL]: *[SHALL NOT]: *[SHOULD]: *[SHOULD NOT]: *[RECOMMENDED]: *[NOT RECOMMENDED]: *[MAY]: *[OPTIONAL]: TAGGED end if $4 # experimental; idnits complains: # ** The document seems to lack a both a reference to RFC 2119 and the # recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 # keywords -- however, there's a paragraph with a matching beginning. # Boilerplate error? ret.sub!("{{!RFC2119}} {{!RFC8174}}", "{{!BCP14}}") end ret when /\Arfc\s*7942(info)?\z/i if $1 ret << < :normative, "?" => :informative } def yaml_load(input, *args) begin if YAML.respond_to?(:safe_load) begin YAML.safe_load(input, *args) rescue ArgumentError YAML.safe_load(input, permitted_classes: args[0], permitted_symbols: args[1], aliases: args[2]) end else YAML.load(input) end rescue Psych::SyntaxError => e warn "*** YAML syntax error: #{e}" exit 65 # EX_DATAERR end end def process_kramdown_options(coding_override = nil, smart_quotes = nil, typographic_symbols = nil, header_kramdown_options = nil) ascii_target = coding_override && coding_override =~ /ascii/ suppress_typography = ascii_target || $options.v3 entity_output = ascii_target ? :numeric : :as_char; options = {input: 'RFC2629Kramdown', entity_output: entity_output, link_defs: {}} if smart_quotes.nil? && suppress_typography smart_quotes = false end if smart_quotes == false smart_quotes = ["'".ord, "'".ord, '"'.ord, '"'.ord] end case smart_quotes when Array options[:smart_quotes] = smart_quotes when nil, true # nothin else warn "*** Can't deal with smart_quotes value #{smart_quotes.inspect}" end if typographic_symbols.nil? && suppress_typography typographic_symbols = false end if typographic_symbols == false typographic_symbols = Hash[::Kramdown::Parser::Kramdown::TYPOGRAPHIC_SYMS.map { |k, v| if Symbol === v [v.intern, k] end }.compact] end # warn [:TYPOGRAPHIC_SYMBOLS, typographic_symbols].to_yaml case typographic_symbols when Hash options[:typographic_symbols] = typographic_symbols when nil, true # nothin else warn "*** Can't deal with typographic_symbols value #{typographic_symbols.inspect}" end if header_kramdown_options options.merge! header_kramdown_options end $global_markdown_options = options # For nested calls in bibref annotation processing and xref text options end XREF_SECTIONS_RE = ::Kramdown::Parser::RFC2629Kramdown::SECTIONS_RE XSR_PREFIX = "#{XREF_SECTIONS_RE} of " XSR_SUFFIX = ", (#{XREF_SECTIONS_RE})| \\((#{XREF_SECTIONS_RE})\\)" XREF_TXT = ::Kramdown::Parser::RFC2629Kramdown::XREF_TXT XREF_TXT_SUFFIX = " \\(#{XREF_TXT}\\)" def spacify_re(s) s.gsub(' ', '[\u00A0\s]+') end include ::Kramdown::Utils::Html def xml_from_sections(input) unless ENV["KRAMDOWN_NO_SOURCE"] require 'kramdown-rfc/gzip-clone' require 'base64' compressed_input = Gzip.compress_m0(input) $source = Base64.encode64(compressed_input) end sections = input.scan(RE_SECTION) # resulting in an array; each section is [section-label, nomarkdown-flag, section-text] line = 1 # skip "---" sections.each do |section| section << line line += 1 + section[2].lines.count end # warn "#{line-1} lines" # the first section is a YAML with front matter parameters (don't put a label here) # We put back the "---" plus gratuitous blank lines to hack the line number in errors yaml_in = input[/---\s*/] << sections.shift[2] ps = KramdownRFC::ParameterSet.new(yaml_load(yaml_in, [Date], [], true)) if v = ps[:v] warn "*** unsupported RFCXML version #{v}" if v != 3 if $options.v2 warn "*** command line --v2 wins over document's 'v: #{v}'" else $options.v3 = true $options.v = 3 ps.default!(:stand_alone, true) ps.default!(:ipr, "trust200902") ps.default!(:pi, {"toc" => true, "sortrefs" => true, "symrefs" => true}) end end if r = ENV["KRAMDOWN_RFC_DOCREV"] warn "** building document revision -#{r}" unless n = ps.has(:docname) and n.sub!(/-latest\z/, "-#{r}") warn "** -d#{r}: docname #{n.inspect} doesn't have a '-latest' suffix" end end if o = ps[:'autolink-iref-cleanup'] $options.autolink_iref_cleanup = o end if o = ps[:'svg-id-cleanup'] $options.svg_id_cleanup = o end coding_override = ps.has(:coding) smart_quotes = ps[:smart_quotes] || ps[:"smart-quotes"] typographic_symbols = ps[:typographic_symbols] header_kramdown_options = ps[:kramdown_options] kramdown_options = process_kramdown_options(coding_override, smart_quotes, typographic_symbols, header_kramdown_options) # all the other sections are put in a Hash, possibly concatenated from parts there sechash = Hash.new{ |h,k| h[k] = ""} snames = [] # a stack of section names sections.each do |sname, nmdflag, text, line| # warn [:SNAME, sname, nmdflag, text[0..10]].inspect nmdin, nmdout = { "-" => ["", ""], # stay in nomarkdown "" => NMDTAGS, # pop out temporarily }[nmdflag || ""] if sname snames << sname # "--- label" -> push label (now current) else snames.pop # just "---" -> pop label (previous now current) end sechash[snames.last] << "#{nmdin}\n#{text}#{nmdout}" end ref_replacements = { } anchor_to_bibref = { } displayref = {} [:ref, :normative, :informative].each do |sn| if refs = ps.has(sn) warn "*** bad section #{sn}: #{refs.inspect}" unless refs.respond_to? :each refs.each do |k, v| if v.respond_to? :to_str if bibtagsys(v) # enable "foo: RFC4711" as a custom anchor definition anchor_to_bibref[k] = v.to_str end ref_replacements[v.to_str] = k end if Hash === v if aliasname = v.delete("-") ref_replacements[aliasname] = k end if bibref = v.delete("=") anchor_to_bibref[k] = bibref end if dr = v.delete("display") displayref[k.gsub("/", "_")] = dr end end end end end open_refs = ps[:ref] || { } # consumed norm_ref = { } # convenience replacement of {{-coap}} with {{I-D.ietf-core-coap}} # collect normative/informative tagging {{!RFC2119}} {{?RFC4711}} sechash.each do |k, v| next if k == "fluff" v.gsub!(/{{(#{ spacify_re(XSR_PREFIX) })?([\w.\/_\-]+@)?(?:([?!])(-)?|(-))([\w._\-]+)(?:=([\w.\/_\-]+))?(#{ XREF_TXT_SUFFIX })?(#{ spacify_re(XSR_SUFFIX) })?}}/) do |match| xsr_prefix = $1 subref = $2 norminform = $3 replacing = $4 || $5 word = $6 bibref = $7 xrt_suffix = $8 xsr_suffix = $9 if replacing if new = ref_replacements[word] word = new else warn "*** no alias replacement for {{-#{word}}}" word = "-#{word}" end end # now, word is the anchor if bibref if old = anchor_to_bibref[word] if bibref != old warn "*** conflicting definitions for xref #{word}: #{old} != #{bibref}" end else anchor_to_bibref[word] = bibref end end # things can be normative in one place and informative in another -> normative # collect norm/inform above and assign it by priority here if norminform norm_ref[word] ||= norminform == '!' # one normative ref is enough end "{{#{xsr_prefix}#{subref}#{word}#{xrt_suffix}#{xsr_suffix}}}" end end [:normative, :informative].each do |k| ps.rest[k.to_s] ||= { } end norm_ref.each do |k, v| # could check bibtagsys here: needed if open_refs is nil or string target = ps.has(v ? :normative : :informative) warn "*** overwriting #{k}" if target.has_key?(k) target[k] = open_refs[k] # add reference to normative/informative end # note that unused items from ref are considered OK, therefore no check for that here # also should allow norm/inform check of other references # {{?coap}} vs. {{!coap}} vs. {{-coap}} (undecided) # or {{?-coap}} vs. {{!-coap}} vs. {{-coap}} (undecided) # could require all references to be decided by a global flag overlap = [:normative, :informative].map { |s| (ps.has(s) || { }).keys }.reduce(:&) unless overlap.empty? warn "*** #{overlap.join(', ')}: both normative and informative" end stand_alone = ps[:stand_alone] [:normative, :informative].each do |sn| if refs = ps[sn] refs.each do |k, v| href = ::Kramdown::Parser::RFC2629Kramdown.idref_cleanup(k) kramdown_options[:link_defs][k] = ["##{href}", nil] # allow [RFC2119] in addition to {{RFC2119}} bibref = anchor_to_bibref[k] || k bts, url = bibtagsys(bibref, k, stand_alone) ann = v.delete("annotation") || v.delete("ann") if Hash === v if bts && (!v || v == {} || v.respond_to?(:to_str)) if stand_alone a = %{{: anchor="#{k}"}} a[-1...-1] = %{ ann="#{escape_html(ann, :attribute)}"} if ann sechash[sn.to_s] << %{\n#{NMDTAGS[0]}\n![:include:](#{bts})#{a}\n#{NMDTAGS[1]}\n} else warn "*** please use standalone mode for adding annotations to references" if ann bts.gsub!('/', '_') (ps.rest["bibxml"] ||= []) << [bts, url] sechash[sn.to_s] << %{&#{bts};\n} # ??? end else unless v && Hash === v warn "*** don't know how to expand ref #{k}" next end if bts && !v.delete("override") warn "*** warning: explicit settings completely override canned bibxml in reference #{k}" end v["ann"] = ann if ann sechash[sn.to_s] << KramdownRFC::ref_to_xml(href, v) end end end end erbfilename = File.expand_path '../../../data/kramdown-rfc2629.erb', __FILE__ erbfile = File.read(erbfilename, coding: "UTF-8") erb = ERB.trim_new(erbfile, '-') # remove redundant nomarkdown pop outs/pop ins as they confuse kramdown input = erb.result(binding).gsub(%r"{::nomarkdown}\s*{:/nomarkdown}"m, "") ps.warn_if_leftovers sechash.delete("fluff") # fluff is a "commented out" section if !sechash.empty? # any sections unused by the ERb file? warn "*** sections left #{sechash.keys.inspect}!" end [input, kramdown_options, coding_override] end XML_RESOURCE_ORG_PREFIX = Kramdown::Converter::Rfc2629::XML_RESOURCE_ORG_PREFIX # return XML entity name, url, rewrite_anchor flag def bibtagsys(bib, anchor=nil, stand_alone=true) if bib =~ /\Arfc(\d+)/i rfc4d = "%04d" % $1.to_i [bib.upcase, "#{XML_RESOURCE_ORG_PREFIX}/bibxml/reference.RFC.#{rfc4d}.xml"] elsif $options.v3 && bib =~ /\A(bcp|std)(\d+)/i n4d = "%04d" % $2.to_i [bib.upcase, "#{XML_RESOURCE_ORG_PREFIX}/bibxml-rfcsubseries-new/reference.#{$1.upcase}.#{n4d}.xml"] elsif bib =~ /\A([-A-Z0-9]+)\./ && (xro = Kramdown::Converter::Rfc2629::XML_RESOURCE_ORG_MAP[$1]) dir, _ttl, rewrite_anchor = xro bib1 = ::Kramdown::Parser::RFC2629Kramdown.idref_cleanup(bib) if anchor && bib1 != anchor if rewrite_anchor a = %{?anchor=#{anchor}} else if !stand_alone warn "*** selecting a custom anchor '#{anchor}' for '#{bib1}' requires stand_alone mode" warn " the output will need manual editing to correct this" end end end [bib1, "#{XML_RESOURCE_ORG_PREFIX}/#{dir}/reference.#{bib}.xml#{a}"] end end def read_encodings encfilename = File.expand_path '../../../data/encoding-fallbacks.txt', __FILE__ encfile = File.read(encfilename, coding: "UTF-8") Hash[encfile.lines.map{|l| l.chomp!; x, s = l.split(" ", 2) [x.hex.chr(Encoding::UTF_8), s || " "]}] end FALLBACK = read_encodings def expand_tabs(s, tab_stops = 8) s.gsub(/([^\t\n]*)\t/) do $1 + " " * (tab_stops - ($1.size % tab_stops)) end end require 'optparse' require 'ostruct' $options ||= OpenStruct.new op = OptionParser.new do |opts| opts.banner = < file.xml Version: #{KDRFC_VERSION} BANNER opts.on("-V", "--version", "Show version and exit") do |v| puts "kramdown-rfc #{KDRFC_VERSION}" exit end opts.on("-H", "--help", "Show option summary and exit") do |v| puts opts exit end opts.on("-v", "--[no-]verbose", "Run verbosely") do |v| $options.verbose = v end opts.on("-3", "--[no-]v3", "Use RFCXML v3 processing rules") do |v| $options.v3 = v end opts.on("-2", "--[no-]v2", "Use RFCXML v2 processing rules") do |v| $options.v2 = v end end op.parse! if $options.v2 && $options.v3 warn "*** can't have v2 and eat v3 cake" $options.v2 = false end if $options.v3.nil? && !$options.v2 if Time.now.to_i >= 1645567342 # Time.parse("2022-02-22T22:02:22Z").to_i $options.v3 = true # new default from the above date end end warn "*** v2 #{$options.v2.inspect} v3 #{$options.v3.inspect}" if $options.verbose input = ARGF.read input.scrub! do |c| warn "*** replaced invalid UTF-8 byte sequence #{c.inspect} by U+FFFD REPLACEMENT CHARACTER" 0xFFFD.chr(Encoding::UTF_8) end if input[0] == "\uFEFF" warn "*** There is a leading byte order mark. Ignored." input[0..0] = '' end if input[-1] != "\n" # warn "*** added missing newline at end" input << "\n" # fix #26 end process_includes(input) unless ENV["KRAMDOWN_SAFE"] input.gsub!(/^\{::boilerplate\s+(.*?)\}/) { bp = boilerplate($1) delta = bp.lines.count bp + "\n" } if input =~ /[\t]/ warn "*** Input contains HT (\"tab\") characters. Undefined behavior will ensue." input = expand_tabs(input) end if input =~ /\A---/ # this is a sectionized file do_the_tls_dance unless ENV["KRAMDOWN_DONT_VERIFY_HTTPS"] input, options, coding_override = xml_from_sections(input) else options = process_kramdown_options # all default end if input =~ /\A<\?xml/ # if this is a whole XML file, protect it input = "{::nomarkdown}\n#{input}\n{:/nomarkdown}\n" end if $options.v3_used && !$options.v3 warn $options.v3_used $options.v3_used = nil $options.v3 = true end if coding_override input = input.encode(Encoding.find(coding_override), fallback: FALLBACK) end # 1.4.17: because of UTF-8 bibxml files, kramdown always needs to see UTF-8 (!) if input.encoding != Encoding::UTF_8 input = input.encode(Encoding::UTF_8) end # warn "options: #{options.inspect}" doc = Kramdown::Document.new(input, options) $stderr.puts doc.warnings.to_yaml unless doc.warnings.empty? output = doc.to_rfc2629 if $options.v3_used && !$options.v3 warn $options.v3_used $options.v3 = true end # only reparse output document if cleanup actions required if $options.autolink_iref_cleanup || $options.svg_id_cleanup require 'rexml/document' d = REXML::Document.new(output) d.context[:attribute_quote] = :quote # Set double-quote as the attribute value delimiter if $options.autolink_iref_cleanup require 'kramdown-rfc/autolink-iref-cleanup' autolink_iref_cleanup(d) end if $options.svg_id_cleanup require 'kramdown-rfc/svg-id-cleanup' svg_id_cleanup(d) end output = d.to_s end if coding_override output = output.encode(Encoding.find(coding_override), fallback: FALLBACK) end puts output kramdown-rfc2629-1.7.14/lib/kramdown-rfc/svg-id-cleanup.rb0000644000004100000410000000225314630024004023047 0ustar www-datawww-datarequire 'rexml/document' SVG_NAMESPACES = {"svg"=>"http://www.w3.org/2000/svg", "xlink"=>"http://www.w3.org/1999/xlink"} def svg_id_cleanup(d) gensym = "gensym000" REXML::XPath.each(d.root, "//svg:svg", SVG_NAMESPACES) do |x| gensym = gensym.succ # warn "*** SVG" # warn "*** SVG: #{x.to_s.size}" found_as_id = Set[] found_as_href = Set[] REXML::XPath.each(x, ".//*[@id]", SVG_NAMESPACES) do |y| # warn "*** ID: #{y}" name = y.attributes["id"] if found_as_id === name warn "*** duplicate ID #{name}" end found_as_id.add(name) y.attributes["id"] = "#{name}-#{gensym}" end REXML::XPath.each(x, ".//*[@xlink:href]", SVG_NAMESPACES) do |y| # warn "*** HREF: #{y}" name = y.attributes["href"] name1 = name[1..-1] if !found_as_id === name1 warn "*** unknown HREF #{name}" end found_as_href.add(name1) y.attributes["xlink:href"] = "#{name}-#{gensym}" end found_as_id -= found_as_href warn "*** warning: unused ID: #{found_as_id.to_a.join(", ")}" unless found_as_id.empty? end rescue => detail warn "*** Can't clean SVG: #{detail}" end kramdown-rfc2629-1.7.14/lib/kramdown-rfc/doi.rb0000644000004100000410000000643014630024004021005 0ustar www-datawww-datarequire 'open-uri' require 'json' require 'yaml' ACCEPT_CITE_JSON = {"Accept" => "application/citeproc+json"} def doi_fetch_and_convert(doi, fuzzy: false, verbose: false, site: "https://dx.doi.org") doipath = doi.sub(/^([0-9.]+)_/) {"#$1/"} # convert initial _ back to / # warn "** SUB #{doi} #{doipath}" if doi != doipath begin cite = JSON.parse(URI("#{site}/#{doipath}").open(ACCEPT_CITE_JSON).read) puts cite.to_yaml if verbose doi_citeproc_to_lit(cite, fuzzy) rescue OpenURI::HTTPError => e begin site = "https://dl.acm.org" percent_escaped = doipath.gsub("/", "%2F") path = "#{site}/action/exportCiteProcCitation?targetFile=custom-bibtex&format=bibTex&dois=#{percent_escaped}" op = URI(path).open # first get a cookie, ignore result # warn [:META, op.meta].inspect cook = op.meta['set-cookie'].split('; ', 2)[0] cite = JSON.parse(URI(path).open("Cookie" => cook).read) cite = cite["items"].first[doipath] puts cite.to_yaml if verbose doi_citeproc_to_lit(cite, fuzzy) rescue raise e end end end def doi_citeproc_to_lit(cite, fuzzy) lit = {} ser = lit["seriesinfo"] = {} refcontent = [] lit["title"] = cite["title"] if (st = cite["subtitle"]) && Array === st # defensive st.delete('') if st != [] lit["title"] << ": " << st.join("; ") end end if authors = cite["author"] lit["author"] = authors.map do |au| lau = {} if (f = au["family"]) if (g = au["given"]) lau["name"] = "#{g} #{f}" lau["ins"] = "#{g[0]}. #{f}" else lau["name"] = "#{f}" # lau["ins"] = "#{g[0]}. #{f}" end end if (f = au["affiliation"]) && Array === f names = f.map { |affn| if Hash === affn && (n = affn["name"]) && String === n n end }.compact if names.size > 0 lau["org"] = names.join("; ") end end lau end end if iss = cite["issued"] if dp = iss["date-parts"] if Integer === (dp = dp[0])[0] lit["date"] = ["%04d" % dp[0], *dp[1..-1].map {|p| "%02d" % p}].join("-") end end end if !lit.key?("date") && fuzzy && (iss = cite["created"]) if dp = iss["date-parts"] if Integer === (dp = dp[0])[0] lit["date"] = ["%04d" % dp[0], *dp[1..-1].map {|p| "%02d" % p}].join("-") end end end if (ct = cite["container-title"]) && ct != [] info = [] if v = cite["volume"] vi = "vol. #{v}" if (v = cite["journal-issue"]) && (issue = v["issue"]) vi << ", no. #{issue}" end info << vi end if p = cite["page"] info << "pp. #{p}" end rhs = info.join(", ") if info != [] ser[ct] = rhs else spl = ct.split(" ") ser[spl[0..-2].join(" ")] = spl[-1] end end if pub = cite["publisher"] refcontent << pub # info = [] # if t = cite["type"] # info << t # end # rhs = info.join(", ") # if info != [] # ser[pub] = rhs # else # spl = pub.split(" ") # ser[spl[0..-2].join(" ")] = spl[-1] # end end ["DOI", "ISBN"].each do |st| if a = cite[st] ser[st] = a end end if refcontent != [] lit["refcontent"] = refcontent.join(", ") end lit end kramdown-rfc2629-1.7.14/lib/kramdown-rfc/erb.rb0000644000004100000410000000042314630024004020776 0ustar www-datawww-datarequire 'erb' class ERB case version.sub("erb.rb [", "") when /\A2.1/ # works back to 1.9.1 def self.trim_new(s, trim) ERB.new(s, nil, trim) end else def self.trim_new(s, trim) ERB.new(s, trim_mode: trim) end end end kramdown-rfc2629-1.7.14/lib/kramdown-rfc/kdrfc-processor.rb0000644000004100000410000001557414630024004023351 0ustar www-datawww-datarequire 'uri' require 'net/http' require 'net/http/persistent' require 'open3' require 'ostruct' require 'json' module KramdownRFC class KDRFC attr_reader :options def initialize @options = OpenStruct.new end # ))) KDRFC_PREPEND = [ENV["KDRFC_PREPEND"]].compact def v3_flag? [*(@options.v3 ? ["--v3"] : []), *(@options.v2 ? ["--v2"] : [])] end def process_mkd(input, output) warn "* converting locally from markdown #{input} to xml #{output}" if @options.verbose o, s = Open3.capture2(*KDRFC_PREPEND, "kramdown-rfc2629", *v3_flag?, input) if s.success? File.open(output, "w") do |fo| fo.print(o) end warn "* #{output} written" if @options.verbose else raise IOError.new("*** kramdown-rfc failed, status #{s.exitstatus}") end end def run_idnits(*args) if @options.remote run_idnits_remotely(*args) else run_idnits_locally(*args) end end def run_idnits_locally(txt_fn) warn "* running idnits locally in txt #{txt_fn}" if @options.verbose unless system("idnits", txt_fn) warn "*** problem #$? running idnits" if @options.verbose warn "*** problem running idnits -- falling back to remote idnits processing" run_idnits_remotely(txt_fn) end end # curl -s https://author-tools.ietf.org/api/idnits -X POST -F file=@draft-ietf-core-comi.txt -F hidetext=true IDNITS_WEBSERVICE = ENV["KRAMDOWN_IDNITS_WEBSERVICE"] || 'https://author-tools.ietf.org/api/idnits' def run_idnits_remotely(txt_fn) url = URI(IDNITS_WEBSERVICE) req = Net::HTTP::Post.new(url) form = [["file", File.open(txt_fn), {filename: "input.txt", content_type: "text/plain"}], ["hidetext", "true"]] diag = ["url/form: ", url, form].inspect req.set_form(form, 'multipart/form-data') warn "* requesting idnits at #{url}" if @options.verbose t0 = Time.now res = persistent_http.request(url, req) warn "* elapsed time: #{Time.now - t0}" if @options.verbose case res when Net::HTTPBadRequest result = checked_json(res.body) raise IOError.new("*** Remote Error: #{result["error"]}") when Net::HTTPOK case res.content_type when 'text/plain' if res.body == '' raise IOError.new("*** HTTP response is empty with status #{res.code}, not written") end puts res.body else warning = "*** HTTP response has unexpected content_type #{res.content_type} with status #{res.code}, #{diag}" warning << "\n" warning << res.body raise IOError.new(warning) end else raise IOError.new("*** HTTP response: #{res.code}, #{diag}") end end def process_xml(*args) if @options.remote process_xml_remotely(*args) else process_xml_locally(*args) end end def process_xml_locally(input, output, *flags) warn "* converting locally from xml #{input} to txt #{output}" if @options.verbose begin o, s = Open3.capture2(*KDRFC_PREPEND, "xml2rfc", *v3_flag?, *flags, input) puts o if s.success? warn "* #{output} written" if @options.verbose else raise IOError.new("*** xml2rfc failed, status #{s.exitstatus} (possibly try with -r)") end rescue Errno::ENOENT warn "*** falling back to remote xml2rfc processing (web service)" # if @options.verbose process_xml_remotely(input, output, *flags) end end # curl https://author-tools.ietf.org/api/render/text -X POST -F "file=@..." XML2RFC_WEBSERVICE = ENV["KRAMDOWN_XML2RFC_WEBSERVICE"] || 'https://author-tools.ietf.org/api/render/' MODE_AS_FORMAT = { "--text" => "text", "--html" => "html", "--v2v3" => "xml", "--pdf" => "pdf", } def checked_json(t) begin JSON.load(t) rescue => e raise IOError.new("*** JSON result: #{e.detailed_message}, #{diag}") end end def persistent_http $http ||= Net::HTTP::Persistent.new name: 'kramdown-rfc' end def process_xml_remotely(input, output, *flags) format = flags[0] || "--text" warn "* converting remotely from xml #{input} to #{format} #{output}" if @options.verbose maf = MODE_AS_FORMAT[format] unless maf raise ArgumentError.new("*** don't know how to convert remotely from xml #{input} to #{format} #{output}") end url = URI(XML2RFC_WEBSERVICE + maf) req = Net::HTTP::Post.new(url) form = [["file", File.open(input), {filename: "input.xml", content_type: "text/plain"}]] diag = ["url/form: ", url, form].inspect req.set_form(form, 'multipart/form-data') warn "* requesting at #{url}" if @options.verbose t0 = Time.now res = persistent_http.request(url, req) warn "* elapsed time: #{Time.now - t0}" if @options.verbose case res when Net::HTTPBadRequest result = checked_json(res.body) raise IOError.new("*** Remote Error: #{result["error"]}") when Net::HTTPOK case res.content_type when 'application/json' if res.body == '' raise IOError.new("*** HTTP response is empty with status #{res.code}, not written") end # warn "* res.body #{res.body}" if @options.verbose result = checked_json(res.body) if logs = result["logs"] if errors = logs["errors"] errors.each do |err| warn("*** Error: #{err}") end end if warnings = logs["warnings"] warnings.each do |w| warn("** Warning: #{w}") end end end raise IOError.new("*** No useful result from remote") unless result["url"] res = persistent_http.request(URI(result["url"])) warn "* result content type #{res.content_type}" if @options.verbose if res.body == '' raise IOError.new("*** Second HTTP response is empty with status #{res.code}, not written") end File.open(output, "w") do |fo| fo.print(res.body) end warn "* #{output} written" if @options.verbose else warning = "*** HTTP response has unexpected content_type #{res.content_type} with status #{res.code}, #{diag}" warning << "\n" warning << res.body raise IOError.new(warning) end else raise IOError.new("*** HTTP response: #{res.code}, #{diag}") end end def process_the_xml(fn, base) process_xml(fn, "#{base}.prepped.xml", "--preptool") if @options.prep process_xml(fn, "#{base}.v2v3.xml", "--v2v3") if @options.v2v3 process_xml(fn, "#{base}.txt") if @options.txt || @options.idnits process_xml(fn, "#{base}.html", "--html") if @options.html process_xml(fn, "#{base}.pdf", "--pdf") if @options.pdf run_idnits("#{base}.txt") if @options.idnits end def process(fn) case fn when /(.*)\.xml\z/ if @options.xml_only warn "*** You already have XML" else # FIXME: copy/paste process_the_xml(fn, $1) end when /(.*)\.mk?d\z/ xml = "#$1.xml" process_mkd(fn, xml) process_the_xml(xml, $1) unless @options.xml_only when /(.*)\.txt\z/ run_idnits(fn) if @options.idnits else raise ArgumentError.new("Unknown file type: #{fn}") end end # ((( end end kramdown-rfc2629-1.7.14/lib/kramdown-rfc/refxml.rb0000644000004100000410000001562214630024004021532 0ustar www-datawww-datarequire 'kramdown-rfc/erb' module KramdownRFC extend Kramdown::Utils::Html def self.escattr(str) escape_html(str.to_s, :attribute) end AUTHOR_ATTRIBUTES = %w{ initials surname fullname asciiInitials asciiSurname asciiFullname role } def self.ref_to_xml(k, v) vps = KramdownRFC::ParameterSet.new(v) erb = ERB.trim_new <<-REFERB, '-' > <%= vps.ele("title") -%> <% vps.arr("author", true, true) do |au| aups = authorps_from_hash(au) -%> > <%= aups.ele("organization=org", aups.attr("abbrev=orgabbrev"), "") %> <% aups.warn_if_leftovers -%> <% end -%> /> <% vps.arr("seriesinfo", false) do |k, v| -%> <% end -%> <% vps.arr("format", false) do |k, v| -%> <% end -%> <%= vps.ele("annotation=ann", nil, nil, true) -%> <%= vps.ele("refcontent=rc", nil, nil, true) -%> REFERB ret = erb.result(binding) vps.warn_if_leftovers ret end def self.treat_multi_attribute_member(ps, an) value = ps.rest[an] if Hash === value value.each do |k, v| ps.rest[if k == ':' an else Kramdown::Element.attrmangle(k + an) || Kramdown::Element.attrmangle(k) || k end] = v end end end def self.initializify(s) # XXX Jean-Pierre w = '\p{Lu}\p{Lo}' if s =~ /\A[-.#{w}]+[.]/u $& elsif s =~ /\A([#{w}])[^-]*/u ret = "#$1." while (s = $') && s =~ /\A(-[\p{L}])[^-]*/u ret << "#$1." end ret else warn "*** Can't initializify #{s}" s end end def self.looks_like_initial(s) s =~ /\A[\p{Lu}\p{Lo}]([-.][\p{Lu}\p{Lo}]?)*\z/u end def self.initials_from_parts_and_surname(aups, parts, s) ssz = s.size nonsurname = parts[0...-ssz] if (ns = parts[-ssz..-1]) != s warn "*** inconsistent surnames #{ns} and #{s}" end nonsurname.map{|x| initializify(x)}.join(" ") end def self.handle_ins(aups, ins_k, initials_k, surname_k) if ins = aups[ins_k] parts = ins.split('.').map(&:strip) # split on dots first if parts == [] warn "*** an empty '#{ins_k}:' value is not useful, try leaving it out" return end # Coalesce H.-P. i = 1; while i < parts.size if parts[i][0] == "-" parts[i-1..i] = [parts[i-1] + "." + parts[i]] else i += 1 end end # Multiple surnames in ins? parts[-1..-1] = parts[-1].split s = if surname = aups.rest[surname_k] surname.split else parts.reverse.take_while{|x| !looks_like_initial(x)}.reverse end aups.rest[initials_k] = initials_from_parts_and_surname(aups, parts, s) aups.rest[surname_k] = s.join(" ") end end def self.handle_name(aups, fn_k, initials_k, surname_k) if name = aups.rest[fn_k] names = name.split(/ *\| */, 2) # boundary for given/last name if names == [] warn "*** an empty '#{fn_k}:' value is not useful, try leaving it out" return end if names[1] aups.rest[fn_k] = name = names.join(" ") # remove boundary if surname = aups.rest[surname_k] if surname != names[1] warn "*** inconsistent embedded surname #{names[1]} and surname #{surname}" end end aups.rest[surname_k] = names[1] end parts = name.split if parts == [] warn "*** a blank '#{fn_k}:' value is not useful, try leaving it out" return end surname = aups.rest[surname_k] || parts[-1] s = surname.split aups.rest[initials_k] ||= initials_from_parts_and_surname(aups, parts, s) aups.rest[surname_k] = s.join(" ") end end def self.authorps_from_hash(au) aups = KramdownRFC::ParameterSet.new(au) if n = aups[:name] warn "** both name #{n} and fullname #{fn} are set on one author" if fn = aups.rest["fullname"] aups.rest["fullname"] = n usename = true end ["fullname", "ins", "initials", "surname"].each do |an| treat_multi_attribute_member(aups, an) end handle_ins(aups, :ins, "initials", "surname") handle_ins(aups, :asciiIns, "asciiInitials", "asciiSurname") # hack ("heuristic for") initials and surname from name # -- only works for people with exactly one last name and uncomplicated first names # -- add surname for people with more than one last name if usename handle_name(aups, "fullname", "initials", "surname") handle_name(aups, "asciiFullname", "asciiInitials", "asciiSurname") end aups end # The below anticipates the "postalLine" changes. # If a postalLine is used (abbreviated "postal" in YAML), # non-postalLine elements are appended as further postalLines. # This prepares for how "country" is expected to be handled # specially with the next schema update. # So an address is now best keyboarded as: # postal: # - Foo Street # - 28359 Bar # country: Germany PERSON_ERB = <<~ERB <<%= element_name%> <%=aups.attrs(*AUTHOR_ATTRIBUTES)%>> <%= aups.ele("organization=org", aups.attrs("abbrev=orgabbrev", *[$options.v3 && "ascii=orgascii"]), "") %>
<% postal_elements = %w{extaddr pobox street cityarea city region code sortingcode country postal}.select{|gi| aups.has(gi)} if postal_elements != [] -%> <% if pl = postal_elements.delete("postal") -%> <%= aups.ele("postalLine=postal") %> <% postal_elements.each do |gi| -%> <%= aups.ele("postalLine=" << gi) %> <% end -%> <% else -%> <% postal_elements.each do |gi| -%> <%= aups.ele(gi) %> <% end -%> <% end -%> <% end -%> <% %w{phone facsimile email uri}.select{|gi| aups.has(gi)}.each do |gi| -%> <%= aups.ele(gi) %> <% end -%>
> ERB def self.person_element_from_aups(element_name, aups) erb = ERB.trim_new(PERSON_ERB, '-') erb.result(binding) end def self.dateattrs(date) begin case date when /\A\d\d\d\d\z/ %{year="#{date}"} when Integer %{year="#{"%04d" % date}"} when String Date.parse("#{date}-01").strftime(%{year="%Y" month="%B"}) when Date date.strftime(%{year="%Y" month="%B" day="%d"}) when Array # this allows to explicitly give a string %{year="#{date.join(" ")}"} when nil %{year="n.d."} end rescue ArgumentError warn "*** Invalid date: #{date} -- use 2012, 2012-07, or 2012-07-28" end end end kramdown-rfc2629-1.7.14/lib/kramdown-rfc/rexml-formatters-conservative.rb0000644000004100000410000000174214630024004026254 0ustar www-datawww-datarequire 'rexml/document' module REXML module Formatters # The Conservative formatter writes an XML document that parses to an # identical document as the source document. This means that no extra # whitespace nodes are inserted, and whitespace within text nodes is # preserved. Attributes are not sorted. class Conservative < Default def initialize @indentation = 0 @level = 0 @ie_hack = false end protected def write_element( node, output ) output << "<#{node.expanded_name}" node.attributes.each_attribute do |attr| output << " " attr.write( output ) end unless node.attributes.empty? if node.children.empty? output << "/" else output << ">" node.children.each { |child| write( child, output ) } output << "" end end end end kramdown-rfc2629-1.7.14/lib/kramdown-rfc2629.rb0000644000004100000410000017007514630024004020564 0ustar www-datawww-data# -*- coding: utf-8 -*- # #-- # Copyright (C) 2009-2010 Thomas Leitner # Copyright (C) 2010-2014 Carsten Bormann # # This file was derived from a part of the kramdown gem which is licensed under the MIT license. # This derived work is also licensed under the MIT license, see LICENSE. #++ # require 'shellwords' raise "sorry, 1.8 was last decade" unless RUBY_VERSION >= '1.9' gem 'kramdown', '~> 2.4.0' require 'kramdown' my_span_elements = %w{list xref eref iref cref spanx vspace} Kramdown::Parser::Html::Constants::HTML_SPAN_ELEMENTS.concat my_span_elements require 'rexml/parsers/baseparser' require 'open3' # for math require 'json' # for math require 'rexml/document' # for SVG and bibxml acrobatics require 'kramdown-rfc/doi' # for fetching information for a DOI require 'kramdown-rfc/rfc8792' class Object def deep_clone Marshal.load(Marshal.dump(self)) end end module Kramdown Kramdown::Options.define(:ol_start_at_first_marker, Kramdown::Options::Boolean, false, <<~EOF) If this option is `true`, an ordered list ( a.length } regexps = [Regexp.union(*sorted_abbrevs.map {|k| /#{Regexp.escape(k).gsub(/\\\s/, "[\\s\\p{Z}]+").force_encoding(Encoding::UTF_8)}/})] regexps << /(?=(?:\W|^)#{regexps.first}(?!\w))/ # regexp should only match on word boundaries # warn regexps.inspect end super(el, regexps) end def initialize(*doc) super @span_parsers.unshift(:xref) @span_parsers.unshift(:iref) @span_parsers.unshift(:span_pi) @block_parsers.unshift(:block_pi) end XREF_BASE = /#{REXML::XMLTokens::NAME_CHAR}+/ # a token for a reference XREF_TXT = /(?:[^\(]|\([^\)]*\))+/ # parenthesized text XREF_RE = /#{XREF_BASE}(?: \(#{XREF_TXT}\))?/ XREF_RE_M = /\A(#{XREF_BASE})(?: \((#{XREF_TXT})\))?/ # matching version of XREF_RE XREF_SINGLE = /(?:Section|Appendix) #{XREF_RE}/ XREF_MULTI = /(?:Sections|Appendices) (?:#{XREF_RE}, )*#{XREF_RE},? and #{XREF_RE}/ XREF_ANY = /(?:#{XREF_SINGLE}|#{XREF_MULTI})/ SECTIONS_RE = /(?:#{XREF_ANY} and )?#{XREF_ANY}/ def self.idref_cleanup(href) # can't start an IDREF with a number or reserved start if href =~ / / if $options.v3 warn "** space(s) in cross-reference '#{href}' -- are you trying to use section references?" else warn "** space(s) in cross-reference '#{href}' -- note that section references are supported in v3 mode only." end end href.gsub(/\A(?:[0-9]|section-|u-|figure-|table-|iref-)/) { "_#{$&}" } end def rfc_mention(target1) # only works for RFCnnnn target1 =~ /\A([A-Z]*)(.*)\z/ "#$1 #$2 " end def handle_bares(s, attr, format, href, last_join = nil) if s.match(/\A(#{XREF_ANY}) and (#{XREF_ANY})\z/) handle_bares($1, {}, nil, href, " and ") handle_bares($2, {}, nil, href, " of ") return end href = href.split(' ')[0] # Remove any trailing (...) target1, target2 = href.split("@", 2) multi = last_join != nil (sn, s) = s.split(' ', 2) loop do m = s.match(/\A#{XREF_RE_M}(, (?:and )?| and )?/) break if not m if not multi and not m[2] and not m[3] and not target2 # Modify |attr| if there is a single reference. This can only be # used if there is only one section reference and the section part # has no title. attr['section'] = m[1] attr['sectionFormat'] = format attr['text'] = m[2] return end if sn @tree.children << Element.new(:text, "#{sn} ", {}) sn = nil end multi = true s[m[0]] = '' attr1 = { 'target' => target1, 'section' => m[1], 'sectionFormat' => 'bare', 'text' => m[2] } @tree.children << Element.new(:xref, nil, attr1) andof = m[3] || last_join || " of " if andof == " of " && target2 andof += rfc_mention(target1) end @tree.children << Element.new(:text, andof, {}) end end XREF_START = /\{\{(?:(?:\{(.*?\n??.*?)\}(?:\{(.*?\n??.*?)\})?)|(\X*?))((?:\}\})|\})/u # Introduce new {{target}} syntax for empty xrefs, which would # otherwise be an ugly ![!](target) or ![ ](target) # (I'd rather use [[target]], but that somehow clashes with links.) def parse_xref @src.pos += @src.matched_size unless @src[4] == "}}" warn "*** #{@src[0]}: unmatched braces #{@src[4].inspect}" end if contact_name = @src[1] attr = {'fullname' => contact_name.gsub("\n", " ")} if ascii_name = @src[2] attr["asciiFullname"] = ascii_name.gsub("\n", " ") end el = Element.new(:contact, nil, attr) else href = @src[3] attr = {} handled_subref = false if $options.v3 # match Section ... of ...; set section, sectionFormat case href.gsub(/[\u00A0\s]+/, ' ') # may need nbsp and/or newlines when /\A(#{SECTIONS_RE}) of (.*)\z/ href = $2 handle_bares($1, attr, "of", href) handled_subref = true when /\A(.*), (#{SECTIONS_RE})\z/ href = $1 handle_bares($2, attr, "comma", href) handled_subref = true when /\A(.*) \((#{SECTIONS_RE})\)\z/ href = $1 handle_bares($2, attr, "parens", href) handled_subref = true when /#{XREF_RE_M}<(.+)\z/ href = $3 if $2 attr['section'] = $2 attr['relative'] = "#" << $1 else attr['section'] = $1 end attr['sectionFormat'] = 'bare' when /\A<<(.+)\z/ href = $1 attr['format'] = 'title' when /\A<(.+)\z/ href = $1 attr['format'] = 'counter' end end if href.match(/#{XREF_RE_M}\z/) href = $1 attr['text'] = $2 end target1, target2 = href.split("@", 2) # should do this only for sectionref... if target2 href = target2 unless handled_subref @tree.children << Element.new(:text, rfc_mention(target1), {}) end end href = self.class.idref_cleanup(href) attr['target'] = href el = Element.new(:xref, nil, attr) end @tree.children << el end define_parser(:xref, XREF_START, '\{\{') IREF_START = /\(\(\((.*?)\)\)\)/u # Introduce new (((target))) syntax for irefs def parse_iref @src.pos += @src.matched_size href = @src[1] el = Element.new(:iref, nil, {'target' => href}) # XXX @tree.children << el end define_parser(:iref, IREF_START, '\(\(\(') # HTML_INSTRUCTION_RE = /<\?(.*?)\?>/m # still defined! # warn [:OPT_SPACE, OPT_SPACE, HTML_INSTRUCTION_RE].inspect PI_BLOCK_START = /^#{OPT_SPACE}<\?/u def parse_block_pi # warn [:BLOCK].inspect line = @src.current_line_number if (result = @src.scan(HTML_INSTRUCTION_RE)) @tree.children << Element.new(:xml_pi, result, nil, category: :block, location: line) @src.scan(TRAILING_WHITESPACE) true else false end end define_parser(:block_pi, PI_BLOCK_START) PI_SPAN_START = /<\?/u def parse_span_pi # warn [:SPAN].inspect line = @src.current_line_number if (result = @src.scan(HTML_INSTRUCTION_RE)) @tree.children << Element.new(:xml_pi, result, nil, category: :span, location: line) else add_text(@src.getch) end end define_parser(:span_pi, PI_SPAN_START, '<\?') # warn [:HERE, @@parsers.keys].inspect end end class Element # Not fixing studly element names postalLine and seriesInfo yet # occasionally regenerate the studly attribute name list via # script in data/studly.rb STUDLY_ATTR = %w( asciiAbbrev asciiFullname asciiInitials asciiName asciiSurname asciiValue blankLines derivedAnchor derivedContent derivedCounter derivedLink displayFormat docName expiresDate hangIndent hangText indexInclude iprExtract keepWithNext keepWithPrevious originalSrc prepTime quoteTitle quotedFrom removeInRFC sectionFormat seriesNo showOnFrontPage slugifiedName sortRefs submissionType symRefs tocDepth tocInclude ) STUDLY_ATTR_MAP = Hash[STUDLY_ATTR.map {|s| [s.downcase, s]}] TRUTHY = Hash.new {|h, k| k} TRUTHY["false"] = false TRUTHY["no"] = false # explicit or automatic studlification # note that explicit (including trailing "_") opts out of automatic def self.attrmangle(k) if (d = k.gsub(/_(.|$)/) { $1.upcase }) != k or d = STUDLY_ATTR_MAP[k] d end end def rfc2629_fix(opts) if a = attr if anchor = a.delete('id') a['anchor'] = ::Kramdown::Parser::RFC2629Kramdown.idref_cleanup(anchor) end if anchor = a.delete('href') a['target'] = ::Kramdown::Parser::RFC2629Kramdown.idref_cleanup(anchor) end if lang = a.delete('lang-') a['xml:lang'] = lang end if av = a.delete('noabbrev') # pseudo attribute -> opts opts = opts.merge(noabbrev: TRUTHY[av]) # updated copy end attr.keys.each do |k| if d = self.class.attrmangle(k) a[d] = a.delete(k) end end end opts end end module Converter # Converts a Kramdown::Document to HTML. class Rfc2629 < Base # we use these to do XML stuff, too include ::Kramdown::Utils::Html def el_html_attributes(el) html_attributes(el.attr) end def el_html_attributes_with(el, defattr) html_attributes(defattr.merge(el.attr)) end # :stopdoc: KRAMDOWN_PERSISTENT = ENV["KRAMDOWN_PERSISTENT"] KRAMDOWN_PERSISTENT_VERBOSE = /v/ === KRAMDOWN_PERSISTENT if KRAMDOWN_PERSISTENT begin require 'net/http/persistent' $http = Net::HTTP::Persistent.new name: 'kramdown-rfc' rescue Exception => e warn "** Not using persistent HTTP -- #{e}" warn "** To silence this message and get full speed, try:" warn "** gem install net-http-persistent" warn "** If this doesn't work, you can ignore this warning." end end # Defines the amount of indentation used when nesting XML tags. INDENTATION = 2 # Initialize the XML converter with the given Kramdown document +doc+. def initialize(*doc) super @sec_level = 1 @location_delta = 100000 # until reset @location_correction = 0 # pre-scanning corrections @in_dt = 0 @footnote_names_in_use = {} end def correct_location(location) location + @location_delta + @location_correction end def convert(el) opts = el.options[:options] # warn "** tree opts #{opts.inspect}" if nested_ol_types = @options[:nested_ol_types] opts[:nested_ol_types] ||= nested_ol_types # warn "** tree opts out #{opts.inspect}" end indent = -INDENTATION if el.children[-1].type == :raw raw = convert1(el.children.pop, indent, opts) end "#{convert1(el, indent, opts)}#{end_sections(1, indent, el.options[:location])}#{raw}" end def convert1(el, indent, opts = {}) nopts = el.rfc2629_fix(opts) send("convert_#{el.type}", el, indent, nopts) end def inner_a(el, indent, opts) indent += INDENTATION el.children.map do |inner_el| nopts = inner_el.rfc2629_fix(opts) send("convert_#{inner_el.type}", inner_el, indent, nopts) end end def inner(el, indent, opts) inner_a(el, indent, opts).join('') end def convert_blank(el, indent, opts) "\n" end def convert_text(el, indent, opts) escape_html(el.value, :text) end def convert_p(el, indent, opts) if (el.children.size == 1 && el.children[0].type == :img) || opts[:unpacked] inner(el, indent, opts) # Part of the bad reference hack else "#{' '*indent}#{inner(el, indent, opts)}\n" end end def saner_generate_id(value) generate_id(value).gsub(/-+/, '-') end def self.process_markdown1(v) # Uuh. Heavy coupling. doc = ::Kramdown::Document.new(v, $global_markdown_options) $stderr.puts doc.warnings.to_yaml unless doc.warnings.empty? doc.to_rfc2629 end def self.process_markdown(v) process_markdown1(v)[3..-6] # skip ...\n end def self.process_markdown_to_rexml(v) s = process_markdown1(v) REXML::Document.new(s) end SVG_COLORS = Hash.new {|h, k| k} < white def svg_munch_color(c, fill) c = SVG_COLORS[c] case c when /\A#(..)(..)(..)\z/ if hex_to_lin($1)*0.2126 + hex_to_lin($2)*0.7152 + hex_to_lin($3)*0.0722 >= B_W_THRESHOLD 'white' else 'black' end when 'none' 'none' if fill # delete for stroke else c end end SVG_NAMESPACES = {"xmlns"=>"http://www.w3.org/2000/svg", "xlink"=>"http://www.w3.org/1999/xlink"} def svg_clean_kgt(s) d = REXML::Document.new(s) REXML::XPath.each(d.root, "/xmlns:svg", SVG_NAMESPACES) do |x| if (w = x.attributes["width"]) && (h = x.attributes["height"]) x.attributes["viewBox"] = "0 0 %d %d" % [w, h] end if x.attributes["viewBox"] x.attributes["width"] = nil x.attributes["height"] = nil end end REXML::XPath.each(d.root, "//rect|//line|//path") do |x| x.attributes["fill"] = "none" x.attributes["stroke"] = "black" x.attributes["stroke-width"] = "1.5" end d.to_s rescue => detail warn "*** Can't clean SVG: #{detail}" d end def svg_clean(s) # expensive, risky d = REXML::Document.new(s) REXML::XPath.each(d.root, "//*[@shape-rendering]") { |x| x.attributes["shape-rendering"] = nil } #; warn x.inspect } REXML::XPath.each(d.root, "//*[@text-rendering]") { |x| x.attributes["text-rendering"] = nil } #; warn x.inspect } REXML::XPath.each(d.root, "//*[@stroke]") { |x| x.attributes["stroke"] = svg_munch_color(x.attributes["stroke"], false) } REXML::XPath.each(d.root, "//*[@fill]") { |x| x.attributes["fill"] = svg_munch_color(x.attributes["fill"], true) } REXML::XPath.each(d.root, "//*[@id]") { |x| x.attributes["id"] = svg_munch_id(x.attributes["id"]) } ## REXML::XPath.each(d.root, "//rect") { |x| x.attributes["style"] = "fill:none;stroke:black;stroke-width:1" unless x.attributes["style"] } # Fix for mermaid: REXML::XPath.each(d.root, "//polygon") { |x| x.attributes["rx"] = nil; x.attributes["ry"] = nil } d.to_s rescue => detail warn "*** Can't clean SVG: #{detail}" d end def memoize(meth, *args) require 'digest' Dir.mkdir(REFCACHEDIR) unless Dir.exist?(REFCACHEDIR) kdrfc_version = Gem.loaded_specs["kramdown-rfc2629"].version.to_s.gsub('.', '_') rescue "UNKNOWN" fn = "#{REFCACHEDIR}/kdrfc-#{kdrfc_version}-#{meth}-#{Digest::SHA256.hexdigest(Marshal.dump(args))[0...40]}.cache" begin out = Marshal.load(File.binread(fn)) rescue StandardError => e # warn e.inspect out = method(meth).call(*args) File.binwrite(fn, Marshal.dump(out)) end out end def capture_croak(t, err) if err != '' err.lines do |l| warn "*** [#{t}:] #{l.chomp}" end end end def shell_prepare(opt) " " << opt.shellsplit.shelljoin end DEFAULT_AASVG="aasvg --spaces=1" def svg_tool_process(t, svg_opt, txt_opt, result) require 'tempfile' file = Tempfile.new("kramdown-rfc") file.write(result) file.close dont_clean = false dont_check = false svg_opt = shell_prepare(svg_opt) if svg_opt txt_opt = shell_prepare(txt_opt) if txt_opt case t when "protocol", "protocol-goat", "protocol-aasvg" cmdparm = result.lines.map(&:strip).select {|x| x != ''}.join(',') result, err, _s = Open3.capture3("protocol #{Shellwords.escape(cmdparm)}#{txt_opt}", stdin_data: '') if t == "protocol-goat" file.unlink file = Tempfile.new("kramdown-rfc") file.write(result) file.close result1, err, _s = Open3.capture3("goat#{svg_opt} #{file.path}", stdin_data: result); dont_clean = true elsif t == "protocol-aasvg" result1, err, _s = Open3.capture3("#{DEFAULT_AASVG}#{svg_opt}", stdin_data: result); dont_clean = true dont_check = true else result1 = nil end when "goat" result1, err, _s = Open3.capture3("goat#{svg_opt} #{file.path}", stdin_data: result); dont_clean = true when "aasvg" result1, err, _s = Open3.capture3("#{DEFAULT_AASVG}#{svg_opt}", stdin_data: result); dont_clean = true dont_check = true when "ditaa" # XXX: This needs some form of option-setting result1, err, _s = Open3.capture3("ditaa #{file.path} --svg -o -#{svg_opt}", stdin_data: result); when "mscgen" result1, err, _s = Open3.capture3("mscgen -T svg -i #{file.path} -o -#{svg_opt}", stdin_data: result); when "mermaid" result1, err, _s = Open3.capture3("mmdc -i #{file.path}#{svg_opt}", stdin_data: result); # -b transparent outpath = file.path + ".svg" result1 = File.read(outpath) rescue '' # don't die before providing error message File.unlink(outpath) rescue nil # ditto when "plantuml", "plantuml-utxt" plantuml = "@startuml\n#{result}\n@enduml" result1, err, _s = Open3.capture3("plantuml -pipe -tsvg#{svg_opt}", stdin_data: plantuml); result, err1, _s = Open3.capture3("plantuml -pipe -tutxt#{txt_opt}", stdin_data: plantuml) if t == "plantuml-utxt" err << err1.to_s when "railroad", "railroad-utf8" result1, err1, _s = Open3.capture3("kgt -l abnf -e svg#{svg_opt}", stdin_data: result); result1 = svg_clean_kgt(result1); dont_clean = true result, err, _s = Open3.capture3("kgt -l abnf -e rr#{t == "railroad" ? "text" : "utf8"}#{txt_opt}", stdin_data: result); err << err1.to_s when "math", "math-asciitex" result1, err, _s = Open3.capture3("tex2svg --font STIX --speech=false#{svg_opt} #{Shellwords.escape(' ' << result)}"); begin raise Errno::ENOENT if t == "math-asciitex" result, err1, s = Open3.capture3("utftex -m #{txt_opt}", stdin_data: result) if s.exitstatus != 0 warn "** utftex: #{err1.inspect}" raise Errno::ENOENT end rescue Errno::ENOENT warn "** utftex not working, falling back to asciitex" unless t == "math-asciitex" result, err1, _s = Open3.capture3("asciitex -f #{file.path}#{txt_opt}") end err << err1 end capture_croak(t, err) # warn ["text:", result.inspect] # warn ["svg:", result1.inspect] file.unlink if result1 result1 = svg_clean(result1) unless dont_clean unless dont_check file = Tempfile.new("kramdown-rfc") file.write(result1) file.close result1, err, _s = Open3.capture3("svgcheck -qa #{file.path}"); file.unlink # warn ["svgcheck:", result1.inspect] capture_croak("svgcheck", err) end if result1 == '' warn "*** could not create svg for #{result.inspect[0...20]}..." exit 65 # EX_DATAERR end end [result, result1] # text, svg end ARTWORK_TYPES = %w(ascii-art binary-art call-flow hex-dump svg) def convert_codeblock(el, indent, opts) # el.attr['anchor'] ||= saner_generate_id(el.value) -- no longer in 1.0.6 result = el.value gi = el.attr.delete('gi') blockclass = el.attr.delete('class') if blockclass == 'language-tbreak' result = result.lines.map {|line| [line.chomp, 0]} spaceind = 0 result.each_with_index {|pair, index| if pair[0] == '' result[spaceind][1] += 1 pair[0] = nil unless index == spaceind else spaceind = index end } # $stderr.puts(result.inspect) result = result.map {|line, space| "" if line }.compact.join("\n") "#{' '*indent}#{result}\n" else artwork_attr = {} t = nil if blockclass classes = blockclass.split(' ') classes.each do |cl| if md = cl.match(/\Alanguage-(.*)/) t = artwork_attr["type"] = md[1] # XXX overwrite else $stderr.puts "*** Unimplemented codeblock class: #{cl}" end end end # compensate for XML2RFC idiosyncrasy by insisting on a blank line unless el.attr.delete('tight') result[0,0] = "\n" unless result[0,1] == "\n" end el.attr.each do |k, v| if md = k.match(/\A(?:artwork|sourcecode)-(.*)/) el.attr.delete(k) artwork_attr[md[1]] = v end end case t when "aasvg", "ditaa", "goat", "math", "math-asciitex", "mermaid", "mscgen", "plantuml", "plantuml-utxt", "protocol", "protocol-aasvg", "protocol-goat", "railroad", "railroad-utf8" if gi warn "*** Can't set GI #{gi} for composite SVG artset" end result, result1 = memoize(:svg_tool_process, t, artwork_attr.delete("svg-options"), artwork_attr.delete("txt-options"), result) retart = mk_artwork(artwork_attr, "ascii-art", "") if result1 # nest TXT in artset with SVG retsvg = mk_artwork(artwork_attr, "svg", result1.sub(/.*?#{retart}\n" else gi ||= ( if !$options.v3 || !t || ARTWORK_TYPES.include?(t) || artwork_attr["align"] "artwork" else "sourcecode" end ) loc_str = if anchor = el.attr['anchor'] "##{anchor}" elsif lineno = el.options[:location] "#{correct_location(lineno)}" else "UNKNOWN" end preprocs = el.attr.delete("pre") checks = el.attr.delete("check") postprocs = el.attr.delete("post") case t when "cbor" warn "** There is no sourcecode-type “cbor”." warn "** Do you mean “cbor-diag” (diagnostic notation)" warn "** or “cbor-pretty” (annotated hex-dump)?" when "json" checks ||= "json" when /\A(.*)-from-yaml\z/ t = $1 preprocs ||= "yaml2json" end preprocs = (preprocs || '').split("-") checks = (checks || '').split("-") postprocs = (postprocs || '').split("-") result = sourcecode_checkproc(preprocs, checks, postprocs, loc_str, result) "#{' '*indent}<#{gi}#{html_attributes(artwork_attr)}>#{result}#{result =~ /\n\Z/ ? '' : "\n"}\n" end end end def sourcecode_proc(proc, loc_str, result) case proc when "dedent" result = remove_indentation(result) when /\Afold(\d*)(left(\d*))?(dry)?\z/ fold = [$1.to_i, # col 0 for '' ($3.to_i if $2), # left 0 for '', nil if no "left" $4] # dry result = fix_unterminated_line(fold8792_1(trim_empty_lines_around(result), *fold)) # XXX when "yaml2json" begin y = YAML.safe_load(result, aliases: true, filename: loc_str) result = JSON.pretty_generate(y) rescue => e warn "*** #{loc_str}: YAML isn't: #{e.message}\n" end else warn "*** #{loc_str}: unknown proc '#{proc}'" end result end def sourcecode_checkproc(preprocs, checks, postprocs, loc_str, result) preprocs.each do |proc| result = sourcecode_proc(proc, loc_str, result) end if preprocs check_input = result checks.each do |check| case check when "skipheader" check_input = handle_artwork_sourcecode(check_input).sub(/.*?\n\n/m, '') when "json" # check for 8792; undo if needed: begin JSON.load(handle_artwork_sourcecode(check_input)) rescue => e err1 = "*** #{loc_str}: JSON isn't: #{JSON.dump(e.message[0..40])}\n" begin JSON.load("{" << check_input << "}") rescue => e warn err1 << "*** not even with braces added around: #{JSON.dump(e.message[0..40])}" end end else warn "*** #{loc_str}: unknown check '#{check}'" end end if checks postprocs.each do |proc| result = sourcecode_proc(proc, loc_str, result) end if postprocs result end def mk_artwork(artwork_attr, typ, content) " typ))}>#{content}" end def convert_blockquote(el, indent, opts) text = inner(el, indent, opts) if $options.v3 gi = el.attr.delete('gi') if gi && gi != 'ul' "#{' '*indent}<#{gi}#{el_html_attributes(el)}>\n#{text}#{' '*indent}\n" else "#{' '*indent} 'true'})}>
  • \n#{text}#{' '*indent}
    • \n" end else text = "" unless text =~ /\n#{text}#{' '*indent}\n" end end def end_sections(to_level, indent, location) if indent < 0 indent = 0 end if @sec_level >= to_level delta = (@sec_level - to_level) @sec_level = to_level "#{' '*indent}
    \n" * delta else $stderr.puts "** #{correct_location(location)}: Bad section nesting: start heading level at 1 and increment by 1" end end def clean_pcdata(parts) # hack, will become unnecessary with XML2RFCv3 clean = '' irefs = '' # warn "clean_parts: #{parts.inspect}" parts.each do |p| md = p.match(%r{([^<]*)(.*)}) clean << md[1] irefs << md[2] # breaks for spanx... don't emphasize in headings! end [clean, irefs] end def clean_pcdatav3(parts) # hack, will become unnecessary with v3 tables clean = '' parts.each do |p| next if p.empty? d = REXML::Document.new("#{p}") t = REXML::XPath.each(d.root, "//text()").to_a.join if t != p warn "** simplified markup #{p.inspect} into #{t.inspect} in table heading" end clean << t end clean end def convert_header(el, indent, opts) # todo: handle appendix tags el = el.deep_clone options = @doc ? @doc.options : @options # XXX: 0.11 vs. 0.12 if options[:auto_ids] && !el.attr['anchor'] el.attr['anchor'] = saner_generate_id(el.options[:raw_text]) end if $options.v3 if sl = el.attr.delete('slugifiedName') # could do general name- play attrstring = html_attributes({'slugifiedName' => sl}) end # noabbrev: true -- Workaround for https://github.com/ietf-tools/xml2rfc/issues/683 nm = inner(el, indent, opts.merge(noabbrev: true)) if ttl = el.attr['title'] warn "*** Section has two titles: >>#{ttl}<< and >>#{nm}<<" warn "*** Do you maybe have a loose IAL?" end irefs = "#{nm}" # else clean, irefs = clean_pcdata(inner_a(el, indent, opts)) el.attr['title'] = clean end "#{end_sections(el.options[:level], indent, el.options[:location])}#{' '*indent}#{irefs}\n" end def convert_hr(el, indent, opts) # misuse for page break "#{' '*indent}\n" end STYLES = {ul: 'symbols', ol: 'numbers', dl: 'hanging'} def convert_ul(el, indent, opts) opts = opts.merge(vspace: el.attr.delete('vspace')) attrstring = el_html_attributes_with(el, {"style" => STYLES[el.type]}) if opts[:unpacked] "#{' '*indent}\n#{inner(el, indent, opts)}#{' '*indent}\n" else "#{' '*indent}\n#{inner(el, indent, opts)}#{' '*indent}\n" end end def convert_ol(el, indent, opts) if @options[:ol_start_at_first_marker] and (first_list_marker = el.options[:first_list_marker]) el.attr['start'] ||= first_list_marker[/\d+/] end nested_types = opts[:nested_ol_types] || ["1"] # warn "** ol opts #{opts.inspect} types #{nested_types.inspect}" if nested_attr = el.attr.delete('nestedOlTypes') nested_types = ::Kramdown::Options.parse(:nested_ol_types, nested_attr) end if nested_types = nested_types.dup # warn "** nested_types #{nested_types.inspect}" nested_here = nested_types.shift opts = opts.merge(nested_ol_types: nested_types << nested_here) el.attr['type'] ||= nested_here # warn "** actual ol type #{el.attr['type'].inspect}" end convert_ul(el, indent, opts) end def convert_dl(el, indent, opts) if $options.v3 if hangindent = el.attr.delete('hangIndent') el.attr['indent'] ||= hangindent # new attribute name wins end vspace = el.attr.delete('vspace') if vspace && !el.attr['newline'] el.attr['newline'] = 'true' end "#{' '*indent}\n#{inner(el, indent, opts.dup)}#{' '*indent}\n" else convert_ul(el, indent, opts) end end def convert_li(el, indent, opts) res_a = inner_a(el, indent, opts) if el.children.empty? || el.children.first.options[:category] == :span res = res_a.join('') else # merge multiple elements res = res_a.select { |x| x.strip != '' }.map { |x| x.sub(/\A\s*(.*)<\/t>\s*\Z/m) { $1} }.join("#{' '*indent}\n").gsub(%r{()\s*}) { $1 }.gsub(%r{\s*(#{res}#{(res =~ /\n\Z/ ? ' '*indent : '')}\n" end def convert_dd(el, indent, opts) if $options.v3 out = '' if !opts[:haddt] out ="#{' '*indent}
    \n" # you can't make this one up end opts[:haddt] = false out << "#{' '*indent}\n#{inner(el, indent, opts)}#{' '*indent}\n" else output = ' '*indent if @in_dt == 1 @in_dt = 0 else output << "" end res = inner(el, indent+INDENTATION, opts.merge(unpacked: true)) # if el.children.empty? || el.children.first.options[:category] != :block output << res << (res =~ /\n\Z/ ? ' '*indent : '') # else FIXME: The latter case is needed for more complex cases # output << "\n" << res << ' '*indent # end output << "\n" end end def convert_dt(el, indent, opts) # SERIOUSLY BAD HACK: if $options.v3 out = '' if opts[:haddt] out ="#{' '*indent}
    \n" # you can't make this one up end opts[:haddt] = true out << "#{' '*indent}#{inner(el, indent, opts)}\n" else close = "#{' '*indent}    \n" * @in_dt @in_dt = 1 vspace = opts[:vspace] vspaceel = "" if vspace ht = escape_html(inner(el, indent, opts), :attribute) # XXX this may leave gunk "#{close}#{' '*indent}#{vspaceel}\n" end end HTML_TAGS_WITH_BODY=['div', 'script'] def convert_html_element(el, indent, opts) res = inner(el, indent, opts) if el.options[:category] == :span "<#{el.value}#{el_html_attributes(el)}" << (!res.empty? ? ">#{res}" : " />") else output = '' output << ' '*indent if !el.options[:parent_is_raw] output << "<#{el.value}#{el_html_attributes(el)}" if !res.empty? && el.options[:parse_type] != :block output << ">#{res}" elsif !res.empty? output << ">\n#{res}" << ' '*indent << "" elsif HTML_TAGS_WITH_BODY.include?(el.value) output << ">" else output << " />" end output << "\n" if el.options[:outer_element] || !el.options[:parent_is_raw] output end end def convert_xml_comment(el, indent, opts) if el.value =~ /\A<\?line (([-+]?)[0-9]+)\?>\z/ lineno = $1.to_i case $2 when '' # absolute @location_delta = lineno - el.options[:location] when '+', '-' # correction (pre-scanning!) @location_correction += lineno end end if el.options[:category] == :block && !el.options[:parent_is_raw] ' '*indent + el.value + "\n" else el.value end end alias :convert_xml_pi :convert_xml_comment alias :convert_html_doctype :convert_xml_comment ALIGNMENTS = { default: :left, left: :left, right: :right, center: :center} COLS_ALIGN = { "l" => :left, "c" => :center, "r" => :right} def convert_table(el, indent, opts) # This only works for tables with headers alignment = el.options[:alignment].map { |al| ALIGNMENTS[al]} cols = (el.attr.delete("cols") || "").split(' ') "#{' '*indent}\n#{inner(el, indent, opts.merge(table_alignment: alignment, table_cols: cols))}#{' '*indent}\n" end def convert_thead(el, indent, opts) inner(el, indent, opts) end alias :convert_tbody :convert_thead alias :convert_tfoot :convert_thead alias :convert_tr :convert_thead def convert_td(el, indent, opts) if alignment = opts[:table_alignment] alignment = alignment.shift if cols = opts[:table_cols].shift md = cols.match(/(\d*(|em|[%*]))([lrc])/) if md[1].to_i != 0 widthval = md[1] widthval << "em" if md[2].empty? widthopt = "width='#{widthval}' " end alignment = COLS_ALIGN[md[3]] || :left end end if alignment xmlres = inner_a(el, indent, opts) if $options.v3 res = clean_pcdatav3(xmlres) else res, irefs = clean_pcdata(xmlres) warn "*** lost markup #{irefs} in table heading" unless irefs.empty? end "#{' '*indent}#{res.empty? ? " " : res}\n" # XXX need clean_pcdata else res = inner(el, indent, opts) "#{' '*indent}#{res.empty? ? " " : res}\n" end end alias :convert_th :convert_td def convert_comment(el, indent, opts) ## Don't actually output all those comments into the XML: # if el.options[:category] == :block # "#{' '*indent}\n" # else # "" # end end def convert_br(el, indent, opts) if $options.v3 "
    " else "" end end def convert_a(el, indent, opts) gi = el.attr.delete('gi') res = inner(el, indent, opts) target = el.attr['target'] if target[0..1] == "{{" # XXX ignoring all attributes and content s = ::Kramdown::Converter::Rfc2629::process_markdown(target) # if res != '' && s[-2..-1] == '/>' # if s =~ /\A<([-A-Za-z0-9_.]+) / # gi ||= $1 # end # s[-2..-1] = ">#{res}" # end return s end if target[0] == "#" # handle [](#foo) as xref as in RFC 7328 el.attr['target'] = target = target[1..-1] if target.downcase == res.downcase res = '' # get rid of raw anchors leaking through end gi ||= "xref" else gi ||= "eref" end "<#{gi}#{el_html_attributes(el)}>#{res}" end def convert_xref(el, indent, opts) gi = el.attr.delete('gi') text = el.attr.delete('text') target = el.attr['target'] if target[0] == "&" "#{target};" else if target =~ %r{\A\w+:(?://|.*@)} gi ||= "eref" else gi ||= "xref" end if text tail = ">#{Rfc2629::process_markdown(text)}" else tail = "/>" end "<#{gi}#{el_html_attributes(el)}#{tail}" end end def convert_contact(el, indent, opts) "" end REFCACHEDIR = ENV["KRAMDOWN_REFCACHEDIR"] || ".refcache" # warn "*** REFCACHEDIR #{REFCACHEDIR}" KRAMDOWN_OFFLINE = ENV["KRAMDOWN_OFFLINE"] KRAMDOWN_REFCACHE_REFETCH = ENV["KRAMDOWN_REFCACHE_REFETCH"] def get_and_write_resource(url, fn) options = {} if ENV["KRAMDOWN_DONT_VERIFY_HTTPS"] options[:ssl_verify_mode] = OpenSSL::SSL::VERIFY_NONE end # workaround for OpenSSL on Windows... # URI.open(url, **options) do |uf| # not portable to older versions OpenURI.open_uri(url, **options) do |uf| s = uf.read if uf.status[0] != "200" warn "*** Status code #{status} while fetching #{url}" else File.write(fn, s) end end end def get_and_write_resource_persistently(url, fn) t1 = Time.now response = $http.request(URI(url)) if response.code != "200" raise "Status code #{response.code} while fetching #{url}" else File.write(fn, response.body) end t2 = Time.now warn "(#{"%.3f" % (t2 - t1)} s)" if KRAMDOWN_PERSISTENT_VERBOSE end def get_doi(refname) lit = doi_fetch_and_convert(refname, fuzzy: true) anchor = "DOI_#{refname.gsub("/", "_")}" KramdownRFC::ref_to_xml(anchor, lit) end # this is now slightly dangerous as multiple urls could map to the same cachefile def get_and_cache_resource(url, cachefile, tvalid = 7200, tn = Time.now) fn = "#{REFCACHEDIR}/#{cachefile}" Dir.mkdir(REFCACHEDIR) unless Dir.exist?(REFCACHEDIR) f = File.stat(fn) rescue nil unless KRAMDOWN_REFCACHE_REFETCH if !KRAMDOWN_OFFLINE && (!f || tn - f.mtime >= tvalid) if f message = "renewing (stale by #{"%.1f" % ((tn-f.mtime)/86400)} days)" fetch_timeout = 10 # seconds, give up quickly if just renewing else message = "fetching" fetch_timeout = 60 # seconds; long timeout needed for Travis end $stderr.puts "#{fn}: #{message} from #{url}" if Array === url begin case url[0] when :DOI ref = get_doi(url[1]) File.write(fn, ref) end rescue Exception => e warn "*** Error fetching #{url[0]} #{url[1].inspect}: #{e}" end elsif ENV["HAVE_WGET"] `cd #{REFCACHEDIR}; wget -t 3 -T #{fetch_timeout} -Nnv "#{url}"` # ignore errors if offline (hack) begin File.utime nil, nil, fn rescue Errno::ENOENT warn "Can't fetch #{url} -- is wget in path?" end else require 'open-uri' require 'socket' require 'openssl' require 'timeout' begin Timeout::timeout(fetch_timeout) do if $http begin # belt and suspenders get_and_write_resource_persistently(url, fn) rescue Exception => e warn "*** Can't get with persistent HTTP: #{e}" get_and_write_resource(url, fn) end else get_and_write_resource(url, fn) end end rescue OpenURI::HTTPError, Errno::EHOSTUNREACH, Errno::ECONNREFUSED, SocketError, Timeout::Error => e warn "*** #{e} while fetching #{url}" end end end begin File.read(fn) # this blows up if no cache available after fetch attempt rescue Errno::ENOENT => e warn "*** #{e} for #{fn}" end end def self.bcp_std_ref(t, n) warn "*** #{t} anchors not supported in v2 format" unless $options.v3 [name = "reference.#{t}.#{"%04d" % n.to_i}.xml", "#{XML_RESOURCE_ORG_PREFIX}/bibxml-rfcsubseries/#{name}"] # FOR NOW end # [subdirectory name, cache ttl in seconds, does it provide for ?anchor=] XML_RESOURCE_ORG_MAP = { "RFC" => ["bibxml", 86400*7, false, ->(fn, n){ [name = "reference.RFC.#{"%04d" % n.to_i}.xml", "https://bib.ietf.org/public/rfc/bibxml/#{name}"] } # was "https://www.rfc-editor.org/refs/bibxml/#{name}"] } ], "I-D" => ["bibxml3", false, false, ->(fn, n){ [fn, "https://datatracker.ietf.org/doc/bibxml3/draft-#{n.sub(/\Adraft-/, '')}.xml"] } ], "BCP" => ["bibxml-rfcsubseries", 86400*7, false, ->(fn, n){ Rfc2629::bcp_std_ref("BCP", n) } ], "STD" => ["bibxml-rfcsubseries", 86400*7, false, ->(fn, n){ Rfc2629::bcp_std_ref("STD", n) } ], "W3C" => "bibxml4", "3GPP" => "bibxml5", "SDO-3GPP" => "bibxml5", "ANSI" => "bibxml2", "CCITT" => "bibxml2", "FIPS" => "bibxml2", # "IANA" => "bibxml2", overtaken by bibxml8 "IEEE" => "bibxml6", # copied over to bibxml6 2019-02-27 "ISO" => "bibxml2", "ITU" => "bibxml2", "NIST" => "bibxml2", "OASIS" => "bibxml2", "PKCS" => "bibxml2", "DOI" => ["bibxml7", 86400, true, ->(fn, n){ ["computed-#{fn}", [:DOI, n] ] }, true # always_altproc ], # emulate old 24 h cache "IANA" => ["bibxml8", 86400, true], # ditto } # XML_RESOURCE_ORG_HOST = ENV["XML_RESOURCE_ORG_HOST"] || "xml.resource.org" # XML_RESOURCE_ORG_HOST = ENV["XML_RESOURCE_ORG_HOST"] || "xml2rfc.tools.ietf.org" XML_RESOURCE_ORG_HOST = ENV["XML_RESOURCE_ORG_HOST"] || "bib.ietf.org" XML_RESOURCE_ORG_PREFIX = ENV["XML_RESOURCE_ORG_PREFIX"] || "https://#{XML_RESOURCE_ORG_HOST}/public/rfc" KRAMDOWN_USE_TOOLS_SERVER = ENV["KRAMDOWN_USE_TOOLS_SERVER"] KRAMDOWN_REFCACHETTL = (e = ENV["KRAMDOWN_REFCACHETTL"]) ? e.to_i : 3600 KRAMDOWN_NO_TARGETS = ENV['KRAMDOWN_NO_TARGETS'] KRAMDOWN_KEEP_TARGETS = ENV['KRAMDOWN_KEEP_TARGETS'] def convert_img(el, indent, opts) # misuse the tag! if a = el.attr alt = a.delete('alt').strip alt = '' if alt == '!' # work around re-wrap uglyness if src = a.delete('src') a['target'] = src end end if alt == ":include:" # Really bad misuse of tag... ann = el.attr.delete('ann') anchor = el.attr.delete('anchor') || ( # not yet warn "*** missing anchor for '#{src}'" src ) anchor = ::Kramdown::Parser::RFC2629Kramdown.idref_cleanup(anchor) anchor.gsub!('/', '_') # should take out all illegals to_insert = "" src.scan(/(W3C|3GPP|[A-Z-]+)[.]?([A-Za-z_0-9.\(\)\/\+-]+)/) do |t, n| never_altproc = n.sub!(/^[.]/, "") fn = "reference.#{t}.#{n}.xml" sub, ttl, _can_anchor, altproc, always_altproc = XML_RESOURCE_ORG_MAP[t] ttl ||= KRAMDOWN_REFCACHETTL # everything but RFCs might change a lot puts "*** Huh: #{fn}" unless sub if altproc && !never_altproc && (!KRAMDOWN_USE_TOOLS_SERVER || always_altproc) fn, url = altproc.call(fn, n) else url = "#{XML_RESOURCE_ORG_PREFIX}/#{sub}/#{fn}" fn = "alt-#{fn}" if never_altproc || KRAMDOWN_USE_TOOLS_SERVER end # if can_anchor # create anchor server-side for stand_alone: false # url << "?anchor=#{anchor}" # fn[/.xml$/] = "--anchor=#{anchor}.xml" # end to_insert = get_and_cache_resource(url, fn.gsub('/', '_'), ttl) to_insert.scrub! rescue nil # only do this for Ruby >= 2.1 begin d = REXML::Document.new(to_insert) d.xml_decl.nowrite d.delete d.doctype d.context[:attribute_quote] = :quote # Set double-quote as the attribute value delimiter d.root.attributes["anchor"] = anchor if t == "RFC" or t == "I-D" if KRAMDOWN_NO_TARGETS || !KRAMDOWN_KEEP_TARGETS d.root.attributes["target"] = nil REXML::XPath.each(d.root, "/reference/format") { |x| d.root.delete_element(x) } else REXML::XPath.each(d.root, "/reference/format") { |x| x.attributes["target"].sub!(%r{https?://www.ietf.org/internet-drafts/}, %{https://www.ietf.org/archive/id/}) if t == "I-D" } end elsif t == "IANA" d.root.attributes["target"].sub!(%r{\Ahttp://www.iana.org/assignments/}, 'https://www.iana.org/assignments/') end if ann el = ::Kramdown::Converter::Rfc2629::process_markdown_to_rexml(ann).root el.name = "annotation" d.root.add_element(el) end to_insert = d.to_s rescue Exception => e warn "** Can't manipulate reference XML: #{e}" broken = true to_insert = nil end # this may be a bit controversial: Don't break the build if reference is broken if KRAMDOWN_OFFLINE || broken unless to_insert to_insert = " *** BROKEN REFERENCE *** " warn "*** KRAMDOWN_OFFLINE: Inserting broken reference for #{fn}" end else exit 66 unless to_insert # EX_NOINPUT end end to_insert else "#{alt}" end end def convert_codespan(el, indent, opts) attrstring = el_html_attributes_with(el, {"style" => 'verb'}) "#{escape_html(el.value)}" end def convert_footnote(el, indent, opts) # XXX: footnotes into crefs??? # this would be more like xml2rfc v3: # "\n#{' '*indent}\n#{inner(el.value, indent, opts).rstrip}\n#{' '*indent}" content = inner(el.value, indent, opts).strip content = content.sub(/\A(.*)<\/t>\z/m) {$1} name = ::Kramdown::Parser::RFC2629Kramdown.idref_cleanup(el.options[:name]) o_name = name.dup while @footnote_names_in_use[name] do if name =~ /_\d+\z/ name.succ! else name << "_1" end end @footnote_names_in_use[name] = true attrstring = el_html_attributes_with(el, {"anchor" => name}) if $options.v3 if o_name[-1] == "-" # Ignore HTML attributes. Hmm. content else # do not indent span-level so we can stick to previous word. Good? "#{content}" end else content = escape_html(content, :text) # text only... "\n#{' '*indent}#{content}" end end def convert_raw(el, indent, opts) end_sections(1, indent, el.options[:location]) + el.value + (el.options[:category] == :block ? "\n" : '') end EMPH = { em: "emph", strong: "strong"} def convert_em(el, indent, opts) if $options.v3 gi = el.type "<#{gi}#{el_html_attributes(el)}>#{inner(el, indent, opts)}" else attrstring = el_html_attributes_with(el, {"style" => EMPH[el.type]}) span, irefs = clean_pcdata(inner_a(el, indent, opts)) "#{span}#{irefs}" end end alias :convert_strong :convert_em def convert_entity(el, indent, opts) entity_to_str(el.value) end TYPOGRAPHIC_SYMS = { :mdash => [::Kramdown::Utils::Entities.entity('mdash')], :ndash => [::Kramdown::Utils::Entities.entity('ndash')], :hellip => [::Kramdown::Utils::Entities.entity('hellip')], :laquo_space => [::Kramdown::Utils::Entities.entity('laquo'), ::Kramdown::Utils::Entities.entity('nbsp')], :raquo_space => [::Kramdown::Utils::Entities.entity('nbsp'), ::Kramdown::Utils::Entities.entity('raquo')], :laquo => [::Kramdown::Utils::Entities.entity('laquo')], :raquo => [::Kramdown::Utils::Entities.entity('raquo')] } def convert_typographic_sym(el, indent, opts) if (result = @options[:typographic_symbols][el.value]) escape_html(result, :text) else TYPOGRAPHIC_SYMS[el.value].map {|e| entity_to_str(e) }.join('') end end def convert_smart_quote(el, indent, opts) entity_to_str(smart_quote_entity(el)) end MATH_LATEX_FILENAME = File.expand_path '../../data/math.json', __FILE__ MATH_LATEX = JSON.parse(File.read(MATH_LATEX_FILENAME, encoding: Encoding::UTF_8)) MATH_REPLACEMENTS = MATH_LATEX["replacements"] MATH_COMBININGMARKS = MATH_LATEX["combiningmarks"] def munge_latex(s) MATH_REPLACEMENTS.each do |o, n| s.gsub!(o, n) end MATH_COMBININGMARKS.each do |m, n| re = /\\#{m[1..-1]}\{(\X)\}/ s.gsub!(re) { "#$1#{n}" } end s end # XXX: This is missing sup/sub support, which needs to be added def convert_math(el, indent, opts) # XXX: This is wrong el = el.deep_clone if el.options[:category] == :block el.attr['artwork-type'] ||= '' el.attr['artwork-type'] += (el.attr['artwork-type'].empty? ? '' : ' ') + 'math' artwork_attr = {} el.attr.each do |k, v| if md = k.match(/\Aartwork-(.*)/) el.attr.delete(k) artwork_attr[md[1]] = v end end result, err, _s = Open3.capture3("tex2mail -noindent -ragged -by_par -linelength=69", stdin_data: el.value); # warn "*** tex2mail not in path?" unless s.success? -- doesn't have useful status capture_croak("tex2mail", err) "#{' '*indent}\n" else type = 'spanx' if $options.v3 type = 'contact' result = munge_latex(el.value) attrstring = el_html_attributes_with(el, {"fullname" => result.chomp, "asciiFullname" => ''}) else warn "*** no support for inline math in XML2RFCv2" type = 'spanx' attrstring = el_html_attributes_with(el, {"style" => 'verb'}) content = escape_html(el.value, :text) end "<#{type}#{attrstring}>#{content}" end end ITEM_RE = '\s*(?:"([^"]*)"|([^,]*?))\s*' IREF_RE = %r{\A(!\s*)?#{ITEM_RE}(?:,#{ITEM_RE})?\z} def iref_attr(s) md = s.match(IREF_RE) attr = { item: md[2] || md[3], subitem: md[4] || md[5], primary: md[1] && 'true', } "" end def nobr_hack(s) # replace this by actual once that exists # https://github.com/ietf-tools/xml2rfc/blob/main/xml2rfc/utils.py#L42 s.gsub(/([-\s\/])(?!\s)/) { case rep = $1 when /\A\s\z/ "\u00A0" # nbsp when "-" "\u2011" # nbhy -- XXX this might mangle dashes else "#{rep}\u2060" end } end def convert_iref(el, indent, opts) iref_attr(el.attr['target']) end def convert_abbreviation(el, indent, opts) # XXX: This is wrong if opts[:noabbrev] return el.value end value = el.value ix = value.gsub(/[\s\p{Z}]+/, " ") title = @root.options[:abbrev_defs][ix] if title.nil? warn "*** abbrev mismatch: value = #{value.inspect} ix = #{ix.inspect}" else title = nil if title.empty? end if title == "" && $options.v3 return "#{value}" end hacked_value = value nobr = false if title && title =~ /\A(\z|\s)/ nobr = true _nobr, title = title.split(' ', 2) hacked_value = nobr_hack(value) if title.nil? || title.empty? return hacked_value # we have "exhausted" this abbrev -- suppress normal meaning end end if title && title[0] == "#" target, title = title.split(' ', 2) if target == "#" target = value else target = target[1..-1] end else target = nil end if item = title pairs = title.split(Parser::RFC2629Kramdown::IREF_START).each_slice(2).to_a replacement = pairs.map {|x,| s = x.strip; s unless s.empty?}.compact.join(" ") irefs = pairs.map {|_,x| x && [x]}.compact warn "@@@ ABBREV MISMATCH #{irefs}" if title.scan(Parser::RFC2629Kramdown::IREF_START) != irefs if irefs.empty? subitem = value else iref = irefs.map{|a,| iref_attr(a)}.join('') end unless replacement.empty? replacement = nobr_hack(replacement) if nobr # XXX this can break XML replacement = ::Kramdown::Converter::Rfc2629::process_markdown(replacement) hacked_value = replacement end else item = value end iref ||= "" if target "#{iref}#{hacked_value}" else "#{iref}#{hacked_value}" end end def convert_root(el, indent, opts) result = inner(el, indent, opts) end end end end kramdown-rfc2629-1.7.14/LICENSE0000644000004100000410000000206714630024004015554 0ustar www-datawww-dataCopyright (c) 2010-2015 Carsten Bormann Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. kramdown-rfc2629-1.7.14/README.md0000644000004100000410000006000514630024004016022 0ustar www-datawww-data# kramdown-rfc [kramdown][] is a [markdown][] parser by Thomas Leitner, which has a number of backends for generating HTML, Latex, and markdown again. **kramdown-rfc** is an additional backend to that: It allows the generation of [XML2RFC][] XML markup (originally known as [RFC 2629][] compliant markup, with a newer version documented in [RFC 7749][]; version 3 now documented in [RFC 7991][] etc. and [v3][]). Who would care? Anybody who is writing Internet-Drafts and RFCs in the [IETF][] and prefers (or has co-authors who prefer) to do part of their work in markdown. kramdown-rfc is documented on this page, and also on [the wiki][]. [the wiki]: https://github.com/cabo/kramdown-rfc/wiki # Usage Start by installing the kramdown-rfc gem (this automatically installs appropriate versions of referenced gems such as kramdown as well): gem install kramdown-rfc (Add a `sudo` and a space in front of that command if you don't have all the permissions needed.) The guts of kramdown-rfc are in one Ruby file, `lib/kramdown-rfc2629.rb` --- this melds nicely into the extension structure provided by kramdown. `bin/kramdown-rfc` started out as a simple command-line program showing how to use this, but can now do much more (see below). To use kramdown-rfc, you'll need Ruby (at least version 2.3, but preferably a current version), and maybe [XML2RFC][] if you want to see the fruits of your work. kramdown-rfc mydraft.md >mydraft.xml xml2rfc mydraft.xml (The most popular file name extension that IETF people have for markdown is .md -- for those who tend to think about GNU machine descriptions here, any extension such as .mkd will do, too.) A more brief interface for both calling kramdown-rfc and XML2RFC is provided by `kdrfc`: kdrfc mydraft.md `kdrfc` can also use a remote installation of XML2RFC if needed: kdrfc -r mydraft.md # Versions of RFCXML Since RFC 8650, RFCs are using an updated grammar as defined in RFC 7991 to 7998 and further updated informally since, colloquially called "[v3][]". As RFC 2629 is no longer the governing standard, what was called kramdown-rfc2629 is now called kramdown-rfc. The latter command defaults to v3 processing rules; from 2022-02-22T22:02:22 on, the old kramdown-rfc2629 driver program does as well (1.6.1). (-3/--v3 and -2/--v2 select v3 and v2 explicitly; the latter should only be needed if there is a reason to to make a document look like it's 2016.) See also [v3 announcement mail][]. [v3 announcement mail]: https://mailarchive.ietf.org/arch/msg/rfc-markdown/JC__LDDGuUbSFqyaEntF9r4kwKw # Examples For historical interest `stupid.mkd` was an early markdown version of an actual Internet-Draft (for a protocol called [STuPiD][] \[sic!]). This demonstrated some, but not all features of kramdown-rfc. Since markdown/kramdown does not cater for all the structure of an RFC 7991 style document, some of the markup is in XML, and the example switches between XML and markdown using kramdown's `{::nomarkdown}` and `{:/nomarkdown}` (this is ugly, but works well enough). `stupid.xml` and `stupid.txt` show what kramdown-rfc and xml2rfc make out of this. `stupid-s.mkd` is the same document in the new sectionized format supported by kramdown-rfc. The document metadata are in a short piece of YAML at the start, and from there, `abstract`, `middle`, references (`normative` and `informative`) and `back` are sections delimited in the markdown file. See the example for how this works. The sections `normative` and `informative` can be populated right from the metadata, so there is never a need to write XML any more. Much less scary, and no `{:/nomarkdown}` etc. is needed any more. Similarly, `stupid-s.xml` and `stupid-s.txt` show what kramdown-rfc and xml2rfc make out of this. `draft-ietf-core-block-xx.mkd` is a real-world example of a current Internet-Draft done this way. For RFC and Internet-Draft references, it uses document prolog entities instead of caching the references in the XML (i.e., not standalone mode, this is easier to handle when collaborating with XML-only co-authors). See the `bibxml` metadata. # The YAML header Please consult the examples for the structure of the YAML header, this should be mostly obvious. The `stand_alone` attribute controls whether the RFC/I-D references are inserted into the document (yes) or entity-referenced (no), the latter leads to increased build time, but may be more palatable for a final XML conversion. The author entry can be a single hash or a list, as in: author: ins: C. Bormann name: Carsten Bormann org: Universität Bremen TZI abbrev: TZI street: Bibliothekstr. 1 city: Bremen code: D-28359 country: Germany phone: +49-421-218-63921 email: cabo@tzi.org or author: - ins: C. Bormann name: Carsten Bormann org: Universität Bremen TZI email: cabo@tzi.org - ins: Z. Shelby name: Zach Shelby org: Sensinode role: editor street: Kidekuja 2 city: Vuokatti code: 88600 country: Finland phone: "+358407796297" email: zach@sensinode.com - role: editor ins: P. Thubert name: Pascal Thubert org: Cisco Systems abbrev: Cisco street: - Village d'Entreprises Green Side - 400, Avenue de Roumanille - Batiment T3 city: Biot - Sophia Antipolis code: '06410' country: FRANCE phone: "+33 4 97 23 26 34" email: pthubert@cisco.com (the hash keys are the XML GIs from RFC 7749, with a flattened structure. As RFC 7749 requires giving both the full name and surname/initials, we use `ins` as an abbreviation for "initials/surname". Yes, the toolchain is Unicode-capable, even if the final RFC output is still in ASCII.) Note that the YAML header needs to be syntactically valid YAML. Where there is a potential for triggering some further YAML feature, a string should be put in quotes (like the "+358407796297" above, which might otherwise be interpreted as a number, losing the + sign). ## References The references section is built from the references listed in the YAML header and from references made inline to RFCs and I-Ds in the markdown text. Since kramdown-rfc cannot know whether a reference is normative or informative, no entry is generated by default in the references section. By indicating a normative reference as in `{{!RFC2119}}` or an informative one as in `{{?RFC1925}}`, you can completely automate the referencing, without the need to write anything in the header. Alternatively, you can write something like: informative: RFC1925: normative: RFC2119: and then just write `{{RFC2119}}` or `{{RFC1925}}`. (Yes, there is a colon in the YAML, because this is a hash that could provide other information.) Since version 1.1, references imported from the [BibXML][] databases can be supplied with a replacement label (anchor name). E.g., RFC 793 could be referenced as `{{!TCP=RFC0793}}`, further references then just can say `{{TCP}}`; both will get `[TCP]` as the label. In the YAML, the same replacement can be expressed as in the first example: normative: TCP: RFC0793 informative: SST: DOI.10.1145/1282427.1282421 Notes about this feature: * Thank you, Martin Thomson, for supplying an implementation and insisting this be done. * While this feature is now available, you are not forced to use it for everything: readers of documents often benefit from not having to look up references, so continuing to use the draft names and RFC numbers as labels may be the preferable style in many cases. * As a final caveat, renaming anchors does not work in the `stand_alone: no` mode (except for IANA and DOI), as there is no such mechanism in XML entity referencing; exporting to XML while maintaining live references then may require some manual editing to get rid of the custom anchors. If your references are not in the [BibXML][] databases and do not have a DOI (that also happens to have correct data) either, you need to spell it out like in the examples below: informative: RFC1925: WEI: title: "6LoWPAN: the Wireless Embedded Internet" # see the quotes above? Needed because of the embedded colon. author: - ins: Z. Shelby name: Zach Shelby - ins: C. Bormann name: Carsten Bormann date: 2009 seriesinfo: ISBN: 9780470747995 ann: This is a really good reference on 6LoWPAN. ASN.1: title: > Information Technology — ASN.1 encoding rules: Specification of Basic Encoding Rules (BER), Canonical Encoding Rules (CER) and Distinguished Encoding Rules (DER) # YAML's ">" syntax used above is a good way to write longer titles author: org: International Telecommunications Union date: 1994 seriesinfo: ITU-T: Recommendation X.690 REST: target: http://www.ics.uci.edu/~fielding/pubs/dissertation/fielding_dissertation.pdf title: Architectural Styles and the Design of Network-based Software Architectures author: ins: R. Fielding name: Roy Thomas Fielding org: University of California, Irvine date: 2000 seriesinfo: "Ph.D.": "Dissertation, University of California, Irvine" format: PDF: http://www.ics.uci.edu/~fielding/pubs/dissertation/fielding_dissertation.pdf COAP: title: "CoAP: An Application Protocol for Billions of Tiny Internet Nodes" seriesinfo: DOI: 10.1109/MIC.2012.29 date: 2012 author: - ins: C. Bormann name: Carsten Bormann - ins: A. P. Castellani name: Angelo P. Castellani - ins: Z. Shelby name: Zach Shelby IPSO: title: IP for Smart Objects (IPSO) author: - org: date: false seriesinfo: Web: http://ipso-alliance.github.io/pub/ normative: ECMA262: author: org: European Computer Manufacturers Association title: ECMAScript Language Specification 5.1 Edition date: 2011-06 target: http://www.ecma-international.org/publications/files/ecma-st/ECMA-262.pdf seriesinfo: ECMA: Standard ECMA-262 RFC2119: RFC6690: (as in the author list, `ins` is an abbreviation for "initials/surname"; note that the first title had to be put in double quotes as it contains a colon which is special syntax in YAML.) Then you can simply reference `{{ASN.1}}` and `{{ECMA262}}` in the text. (Make sure the reference keys are valid XML names, though.) # Experimental features Most of the [kramdown syntax][kdsyntax] is supported and does something useful; with the exception of the math syntax (math has no special support in XML2RFC), and HTML syntax of course. A number of more esoteric features have recently been added. (The minimum required version for each full feature is indicated.) (1.3.x) Slowly improving support for SVG generating tools for XML2RFCv3 (i.e., with `-3` flag). These tools must be installed and callable from the command line. The basic idea is to mark an input code block with one of the following labels (language types), yielding some plaintext form in the .TXT output and a graphical form in the .HTML output. The plaintext is the input in some cases (e.g., ASCII art, `mscgen`), or some plaintext output generated by the tool (e.g., `plantuml-utxt`). Currently supported labels as of 1.3.9: * [goat][], [ditaa][]: ASCII (plaintext) art to figure conversion * [mscgen][]: Message Sequence Charts * [plantuml][]: widely used multi-purpose diagram generator * plantuml-utxt: Like plantuml, except that a plantuml-generated plaintext form is used * [mermaid][]: Very experimental; the conversion to SVG is prone to generate black-on-black text in this version * math: display math using [tex2svg][] for HTML/PDF and [utftex][] for plaintext [goat]: https://github.com/blampe/goat [ditaa]: https://github.com/stathissideris/ditaa [mscgen]: http://www.mcternan.me.uk/mscgen/ [plantuml]: https://plantuml.com [mermaid]: https://github.com/mermaid-js/mermaid-cli [tex2svg]: https://github.com/mathjax/MathJax-demos-node/blob/master/direct/tex2svg [utftex]: https://github.com/bartp5/libtexprintf Note that this feature does not play well with the CI (continuous integration) support in Martin Thomson's [I-D Template][], as that may not have the tools installed in its docker instance. More details have been collected on the [wiki][svg]. [svg]: https://github.com/cabo/kramdown-rfc/wiki/SVG (1.2.9:) The YAML header now allows specifying [kramdown_options][]. [kramdown_options]: https://kramdown.gettalong.org/options.html This was added specifically to provide easier access to the kramdown `auto_id_prefix` feature, which prefixes by some distinguishing string the anchors that are auto-generated for sections, avoiding conflicts: ```yaml kramdown_options: auto_id_prefix: sec- ``` (1.2.8:) An experimental feature was added to include [BCP 14] boilerplate: ```markdown {::boilerplate bcp14} ``` which saves some typing. Saying "bcp14+" instead of "bcp14" adds some random clarifications at the end of the [standard boilerplate text][] that you may or may not want to have. (Do we need other boilerplate items beyond BCP14?) [BCP 14]: https://www.rfc-editor.org/info/bcp14 [standard boilerplate text]: https://tools.ietf.org/html/rfc8174#page-3 (1.0.35:) An experimental command `doilit` has been added. It can be used to convert DOIs given on the command line into references entries for kramdown-rfc YAML, saving a lot of typing. Note that the DOI database is not of very consistent quality, so you likely have to hand-edit the result before including it into the document (use `-v` to see raw JSON data from the DOI database, made somewhat readable by converting it into YAML). Use `-c` to enable caching (requires `open-uri-cached` gem). Use `-h=handle` in front of a DOI to set a handle different from the default `a`, `b`, etc. Similarly, use `-x=handle` to generate XML2RFCv2 XML instead of kramdown-rfc YAML. (1.0.31:) The kramdown `smart_quotes` feature can be controlled better. By default, it is on (with default kramdown settings), unless `coding: us-ascii` is in effect (1.3.14: or --v3 is given), in which case it is off by default. It also can be explicitly set on (`true`) or off (`false`) in the YAML header, or to a specific value (an array of four kramdown entity names or character numbers). E.g., for a German text (that is not intended to become an Internet-Draft), one might write: ```yaml smart_quotes: [sbquo, lsquo, bdquo, ldquo] pi: topblock: no private: yes ``` (1.0.30:) kramdown-rfc now uses kramdown 1.10, which leads to two notable updates: * Support for empty link texts in the standard markdown reference syntax, as in `[](#RFC7744)`. * Language names in fenced code blocks now support all characters except whitespace, so you can go wild with `asn.1` and `C#`. A heuristic generates missing initials/surname from the `name` entry in author information. This should save a lot of redundant typing. You'll need to continue using the `ins` entry as well if that heuristic fails (e.g., for Spanish names). Also, there is some rather experimental support for markdown display math (blocks between `$$` pairs) if the `tex2mail` tool is available. (1.0.23:) Move up to kramdown 1.6.0. This inherits a number of fixes and one nice feature: Markdown footnote definitions that turn into `cref`s can have their attributes in the footnote definition: ```markdown {:cabo: source="cabo"} (This section to be removed by the RFC editor.)[^1] [^1]: here is my editorial comment: warble warble. {:cabo} Another questionable paragraph.[^2] [^2]: so why not delete it? {: source="observer"} ``` (1.0.23:) As before, IAL attributes on a codeblock go to the figure element. Language attributes on the code block now become the artwork type, and any attribute with a name that starts "artwork-" is moved over to the artwork. So this snippet now does the obvious things: ```markdown ~~~ abnf a = b / %s"foo" / %x0D.0A ~~~ {: artwork-align="center" artwork-name="syntax"} ``` (1.0.22:) Index entries can be created with `(((item)))` or `(((item, subitem)))`; use quotes for weird entries: `(((",", comma)))`. If the index entry is to be marked "primary", prefix an (unquoted) `!` as in `(((!item)))`. In addition, auto-indexing is supported by hijacking the kramdown "abbrev" syntax: *[IANA]: *[MUST]: BCP14 *[CBOR]: (((Object Representation, Concise Binary))) (((CBOR))) The word in square brackets (which must match exactly, case-sensitively) is entered into the index automatically for each place where it occurs. If no title is given, just the word is entered (first example). If one is given, that becomes the main item (the auto-indexed word becomes the subitem, second example). If full control is desired (e.g., for multiple entries per occurrence), just write down the full index entries instead (third example). (1.0.20:) As an alternative referencing syntax for references with text, `{{ref}}` can be expressed as `[text](#ref)`. As a special case, a simple `[ref]` is interpreted as `[](#ref)` (except that the latter syntax is not actually allowed by kramdown). This syntax does not allow for automatic entry of items as normative/informative. (1.0.16:) Markdown footnotes are converted into `cref`s (XML2RFC formal comments; note that these are only visible if the pi "comments" is set to yes). The anchor is taken from the markdown footnote name. The source, if needed, can be supplied by an [IAL][], as in (first example also uses an [ALD][]): ```markdown {:cabo: source="cabo"} (This section to be removed by the RFC editor.)[^1]{:cabo} [^1]: here is my editorial comment Another questionable paragraph.[^2]{: source="observer"} [^2]: so why not delete it ``` Note that XML2RFC v2 doesn't allow structure in crefs. If you put any, you get the escaped verbatim XML... (1.0.11:) Allow overriding "style" attribute (via IAL = [inline attribute list][IAL]) in lists and spans as in: ```markdown {:req: counter="bar" style="format R(%d)"} {: req} * Foo * Bar * Bax Text outside the list, so a new IAL is needed. * Foof * Barf * Barx {: req} ``` (1.0.5:) An IAL attribute "cols" can be added to tables to override the column layout. For example, `cols="* 20 30c r"` sets the width attributes to 20 and 30 for the middle columns and sets the right two columns to center and right alignment, respectively. The alignment from `cols` overrides that from the kramdown table, if present. (1.0.2:) An IAL attribute "vspace" can be added to a definition list to break after the definition term: ```markdown {: vspace="0"} word: : definition anotherword: : another definition ``` (0.x:) Files can be included with the syntax `{::include fn}` (needs to be in column 1 since 1.0.22; can be suppressed for use in servers by setting environment variable KRAMDOWN_SAFE since 1.0.22). A typical example from a recent RFC, where the contents of a figure was machine-generated: ```markdown ~~~~~~~~~~ {::include ../ghc/packets-new/p4.out} ~~~~~~~~~~ {: #example2 title="A longer RPL example"} ``` (0.x:) A page break can be forced by adding a horizontal rule (`----`, note that this creates ugly blank space in some HTML converters). # Risks and Side-Effects The code is not very polished, but now quite stable; it has been successfully used for hundreds of non-trivial Internet-Drafts and RFCs. You probably still need to skim [v3][] if you want to write an Internet-Draft, but you don't really need to understand XML very much. Knowing the basics of YAML helps with the metadata (but you'll understand it from the examples). Occasionally, you do need to reach through to the XML arcana, e.g. by setting attribute values using kramdown's ["IAL" syntax][IAL]. This can for instance be used to obtain unnumbered appendices: ```markdown Acknowledgements ================ {: numbered="false"} John Mattsson was nice enough to point out the need for this being documented. ``` Note that this specific example is covered by a predefined kramdown-rfc ["attribute list definition" (ALD)][ALD]: ```markdown {:unnumbered: numbered="false"} ``` so the conventional way to write this example would be the somewhat simpler: ```markdown Acknowledgements ================ {:unnumbered} John Mattsson was nice enough to point out the need for this being documented. ``` # Upconversion If you have an old RFC and want to convert it to markdown, try just using that RFC, it is 80 % there. It may be possible to automate the remaining 20 % some more, but that hasn't been done. If you have XML, there is an experimental upconverter that does 99 % of the work. Please [contact the author](mailto:cabo@tzi.org?subject=Markdown%20for%20RFCXML) if you want to try it. Actually, if the XML was generated by kramdown-rfc, you can simply extract the input markdown from that XML file (but will of course lose any edits that have been made to the XML file after generation): kramdown-rfc-extract-markdown myfile.xml >myfile.md # Tools Joe Hildebrand has a [grunt][] plugin for kramdown-rfc at: https://github.com/hildjj/grunt-kramdown-rfc2629 . Get started with it at: https://github.com/hildjj/grunt-init-rfc . This provides a self-refreshing web page with the kramdown-rfc/xml2rfc rendition of the draft you are editing. [grunt]: http://gruntjs.com Martin Thomson has an [I-D Template][] for github repositories that enable collaboration on draft development. This supports kramdown-rfc out of the box. Just name your draft like `draft-ietf-unicorn-protocol.md` and follow the installation instructions. [I-D Template]: https://github.com/martinthomson/i-d-template # Related Work Moving from XML to Markdown for RFC writing apparently is a no-brainer, so I'm not the only one who has written code for this. [Miek Gieben][] has done a [similar thing][pandoc2rfc] employing pandoc, now documented in [RFC 7328][]. He uses multiple input files instead of kramdown-rfc's sectionized input format. He keeps the metadata in a separate XML file, similar to the way the previous version of kramdown-rfc stored (and still can store) the metadata in XML in the markdown document. He also uses a slightly different referencing syntax, which is closer to what markdown does elsewhere but more verbose (this syntax is now also supported in kramdown-rfc). (Miek now also has a new thing going on with mostly different syntax, see [mmark][] and its [github repository][mmark-git].) Other human-oriented markup input languages that are being used for authoring RFCXML include: * [asciidoc][], with the [asciidoctor-rfc][] tool, as documented in [draft-ribose-asciirfc][]. * [orgmode][] (please help supply a more specific link here). # License Since kramdown version 1.0, kramdown itself is MIT licensed, which made it possible to license kramdown-rfc under the same license. [kramdown]: https://kramdown.gettalong.org [kdsyntax]: http://kramdown.gettalong.org/syntax.html [IAL]: https://kramdown.gettalong.org/syntax.html#inline-attribute-lists [ALD]: https://kramdown.gettalong.org/syntax.html#attribute-list-definitions [stupid]: http://tools.ietf.org/id/draft-hartke-xmpp-stupid-00 [RFC 2629]: https://www.rfc-editor.org/rfc/rfc2629.html [RFC 7749]: https://www.rfc-editor.org/rfc/rfc7749.html [RFC 7991]: https://www.rfc-editor.org/rfc/rfc7991.html [v3]: https://xml2rfc.tools.ietf.org/xml2rfc-doc.html [markdown]: http://en.wikipedia.org/wiki/Markdown [IETF]: http://www.ietf.org [Miek Gieben]: http://www.miek.nl/ [pandoc2rfc]: https://github.com/miekg/pandoc2rfc/ [XML2RFC]: https://github.com/ietf-tools/xml2rfc [RFC 7328]: http://tools.ietf.org/html/rfc7328 [mmark-git]: https://github.com/miekg/mmark [mmark]: https://mmark.nl [YAML]: http://www.yaml.org/spec/1.2/spec.html [draft-ribose-asciirfc]: https://tools.ietf.org/html/draft-ribose-asciirfc [asciidoctor-rfc]: https://github.com/metanorma/asciidoctor-rfc [asciidoc]: http://www.methods.co.nz/asciidoc/ [orgmode]: http://orgmode.org [BibXML]: https://bib.ietf.org/