kramdown-rfc2629-1.2.7/0000755000004100000410000000000013175754757014513 5ustar www-datawww-datakramdown-rfc2629-1.2.7/data/0000755000004100000410000000000013175754757015424 5ustar www-datawww-datakramdown-rfc2629-1.2.7/data/encoding-fallbacks.txt0000644000004100000410000000233513175754757021676 0ustar www-datawww-data00a1 ! 00a2 [cents] 00a3 GBP 00a4 [currency units] 00a5 JPY 00a6 | 00a7 S. 00a9 (C) 00aa a 00ab << 00ac [not] 00ae (R) 00af _ 00b0 o 00b1 +/- 00b2 ^2 00b3 ^3 00b4 ' 00b5 [micro] 00b6 P. 00b7 . 00b8 , 00b9 ^1 00ba o 00bb >> 00bc 1/4 00bd 1/2 00be 3/4 00bf ? 00c0 A 00c1 A 00c2 A 00c3 A 00c4 Ae 00c5 Ae 00c6 AE 00c7 C 00c8 E 00c9 E 00ca E 00cb E 00cc I 00cd I 00ce I 00cf I 00d0 [ETH] 00d1 N 00d2 O 00d3 O 00d4 O 00d5 O 00d6 Oe 00d7 x 00d8 Oe 00d9 U 00da U 00db U 00dc Ue 00dd Y 00de [THORN] 00df ss 00e0 a 00e1 a 00e2 a 00e3 a 00e4 ae 00e5 ae 00e6 ae 00e7 c 00e8 e 00e9 e 00ea e 00eb e 00ec i 00ed i 00ee i 00ef i 00f0 [eth] 00f1 n 00f2 o 00f3 o 00f4 o 00f5 o 00f6 oe 00f7 / 00f8 oe 00f9 u 00fa u 00fb u 00fc ue 00fd y 00fe [thorn] 00ff y 0152 OE 0153 oe 0161 s 0178 Y 0192 f 02dc ~ 2002 2003 2009 2013 -- 2014 --- 2018 ' 2019 ' 201a ' 201c " 201d " 201e " 2020 *!* 2021 *!!* 2022 o 2026 ... 2030 [/1000] 2032 ' 2039 < 203a > 2044 / 20ac EUR 2122 [TM] 2190 <-- 2192 --> 2194 <-> 21d0 <== 21d2 ==> 21d4 <=> 2212 - 2217 * 2264 <= 2265 >= 2329 < 232a > 0021 ! 0023 # 0024 $ 0025 % 0028 ( 0029 ) 002a * 002b + 002c , 002d - 002e . 002f / 003a : 003b ; 003d = 003f ? 0040 @ 005b [ 005d ] 005e ^ 005f _ 0060 ` 007b { 007c | 007d } 017d Z 017e z 2010 - kramdown-rfc2629-1.2.7/data/kramdown-rfc2629.erb0000644000004100000410000000455613175754757021045 0ustar www-datawww-data"?> SYSTEM "<%= sys %>"> <% end -%> <% ps.arr("entity", false) do |en, ev| -%> "<%=ev%>"> <% end -%> ]> <% ps.arr("pi", false) do |pi, val| -%> ="<%= {true => "yes", false => "no", nil => "yes"}[val] || val %>"?> <% end -%> > <%= ps.ele("title", ps.attr("abbrev=titleabbrev")) %> <% ps.arr("author") do |au| aups = KramdownRFC::authorps_from_hash(au) -%> > <%= aups.ele("organization=org", aups.attr("abbrev=orgabbrev"), "") %>
<% postal = %w{street city region code country}.select{|gi| aups.has(gi)} if postal != [] -%> <% postal.each do |gi| -%> <%= aups.ele(gi) %> <% end -%> <% end -%> <% %w{phone facsimile email uri}.select{|gi| aups.has(gi)}.each do |gi| -%> <%= aups.ele(gi) %> <% end -%>
<% aups.warn_if_leftovers -%> <% end -%> /> <%= ps.ele("area") %> <%= ps.ele("workgroup=wg") %> <%= ps.ele("keyword=kw") %> <%= sechash.delete("abstract") %> <% sechash.keys.each do |k| -%> <% if k =~ /\Anote_(.*)/ -%> "> <%= sechash.delete(k) -%> <% end -%> <% end -%>
<%= sechash.delete("middle") %> <% if sh = sechash.delete("normative") -%> <%= sh %> <% end -%> <% if sh = sechash.delete("informative") -%> <%= sh %> <% end -%> <%= sechash.delete("back") %> <% if $source -%> <% end -%>
kramdown-rfc2629-1.2.7/bin/0000755000004100000410000000000013175754757015263 5ustar www-datawww-datakramdown-rfc2629-1.2.7/bin/kdrfc0000755000004100000410000000573713175754757016316 0ustar www-datawww-data#!/usr/bin/env ruby -KU require 'uri' require 'net/http' require 'open3' def process_mkd(input, output) warn "* converting locally from markdown #{input} to xml #{output}" if $options.verbose o, s = Open3.capture2("kramdown-rfc2629", input) if s.success? File.open(output, "w") do |fo| fo.print(o) end warn "* #{output} written" if $options.verbose else warn "*** kramdown-rfc failed, status #{s.exitstatus}" exit 1 end end def process_xml(*args) if $options.remote process_xml_remotely(*args) else process_xml_locally(*args) end end def process_xml_locally(input, output) warn "* converting locally from xml #{input} to txt #{output}" if $options.verbose begin o, s = Open3.capture2("xml2rfc", input) puts o if s.success? warn "* #{output} written" if $options.verbose else warn "*** xml2rfc failed, status #{s.exitstatus} (possibly try with -r)" exit 1 end rescue Errno::ENOENT warn "*** falling back to remote processing" if $options.verbose process_xml_remotely(input, output) end end def process_xml_remotely(input, output) warn "* converting remotely from xml #{input} to txt #{output}" if $options.verbose url = URI('http://xml2rfc.tools.ietf.org/cgi-bin/xml2rfc.cgi') req = Net::HTTP::Post.new(url) req.set_form([["modeAsFormat", "txt/ascii"], ["type", "binary"], ["input", File.open(input), {filename: "input.xml", content_type: "text/plain"}]], 'multipart/form-data') res = Net::HTTP::start(url.hostname, url.port, :use_ssl => url.scheme == 'https' ) {|http| http.request(req) } case res when Net::HTTPOK case res.content_type when 'application/octet-stream' File.open(output, "w") do |fo| fo.print(res.body) end warn "* #{output} written" if $options.verbose else warn "*** HTTP response has unexpected content_type #{res.content_type} with status #{res.code}" warn res.body exit 1 end else warn "*** HTTP response: #{res.code}" exit 1 end end require 'optparse' require 'ostruct' $options = OpenStruct.new op = OptionParser.new do |opts| opts.banner = "Usage: kdrfc [options] file.md|file.mkd|file.xml" opts.on("-v", "--[no-]verbose", "Run verbosely") do |v| $options.verbose = v end opts.on("-r", "--[no-]remote", "Run xml2rfc remotely even if there is a local one") do |v| $options.remote = v end opts.on("-x", "--[no-]xml", "Convert to xml only") do |v| $options.xml_only = v end end op.parse! case ARGV.size when 1 fn = ARGV[0] case fn when /(.*)\.xml\z/ if $options.xml_only warn "*** You already have XML" else process_xml(fn, "#$1.txt") end when /(.*)\.mk?d\z/ xml = "#$1.xml" process_mkd(fn, xml) process_xml(xml, "#$1.txt") unless $options.xml_only else warn "Unknown file type: #{fn}" exit 1 end else puts op exit 1 end kramdown-rfc2629-1.2.7/bin/kramdown-rfc-extract-markdown0000755000004100000410000000047613175754757023102 0ustar www-datawww-data#!/usr/bin/env ruby -KU require 'kramdown-rfc/gzip-clone' require 'base64' EMBEDDED_RE = %r{} embedded = ARGF.read.scan(EMBEDDED_RE) unless embedded.empty? puts Gzip.decompress(Base64.decode64(embedded[0][0])) else warn "*** No embedded markdown source found!" end kramdown-rfc2629-1.2.7/bin/kramdown-rfc26290000755000004100000410000002337013175754757020133 0ustar www-datawww-data#!/usr/bin/env ruby2.1 # -*- coding: utf-8 -*- require 'kramdown-rfc2629' require 'kramdown-rfc/parameterset' require 'kramdown-rfc/refxml' require 'yaml' require 'erb' require 'date' Encoding.default_external = "UTF-8" # wake up, smell the coffee def do_the_tls_dance begin require 'openssl' File.open(OpenSSL::X509::DEFAULT_CERT_FILE) do end rescue warn "** Configuration problem with OpenSSL certificate store at #{OpenSSL::X509::DEFAULT_CERT_FILE}." warn "** Activating workaround. Occasionally run `certified-update`." require 'certified' end end RE_NL = /(?:\n|\r|\r\n)/ RE_SECTION = /---(?:\s+(\w+)(-?))?\s*#{RE_NL}(.*?#{RE_NL})(?=---(?:\s+\w+-?)?\s*#{RE_NL}|\Z)/m NMDTAGS = ["{:/nomarkdown}\n\n", "\n\n{::nomarkdown}\n"] NORMINFORM = { "!" => :normative, "?" => :informative } def yaml_load(input, *args) if YAML.respond_to?(:safe_load) YAML.safe_load(input, *args) else YAML.load(input) end end def xml_from_sections(input) unless ENV["KRAMDOWN_NO_SOURCE"] require 'kramdown-rfc/gzip-clone' require 'base64' compressed_input = Gzip.compress(input) $source = Base64.encode64(compressed_input) end sections = input.scan(RE_SECTION) # resulting in an array; each section is [section-label, nomarkdown-flag, section-text] # the first section is a YAML with front matter parameters (don't put a label here) # We put back the "---" plus gratuitous blank lines to hack the line number in errors yaml_in = input[/---\s*/] << sections.shift[2] ps = KramdownRFC::ParameterSet.new(yaml_load(yaml_in, [Date], [], true)) coding_override = ps.has(:coding) smart_quotes = ps[:smart_quotes] # all the other sections are put in a Hash, possibly concatenated from parts there sechash = Hash.new{ |h,k| h[k] = ""} snames = [] # a stack of section names sections.each do |sname, nmdflag, text| nmdin, nmdout = { "-" => ["", ""], # stay in nomarkdown "" => NMDTAGS, # pop out temporarily }[nmdflag || ""] if sname snames << sname # "--- label" -> push label (now current) else snames.pop # just "---" -> pop label (previous now current) end sechash[snames.last] << "#{nmdin}#{text}#{nmdout}" end ref_replacements = { } anchor_to_bibref = { } [:ref, :normative, :informative].each do |sn| if refs = ps.has(sn) warn "*** bad section #{sn}: #{refs.inspect}" unless refs.respond_to? :each refs.each do |k, v| if v.respond_to? :to_str if bibtagsys(v) # enable "foo: RFC4711" as a custom anchor definition anchor_to_bibref[k] = v.to_str end ref_replacements[v.to_str] = k end if Hash === v if aliasname = v.delete("-") ref_replacements[aliasname] = k end if bibref = v.delete("=") anchor_to_bibref[k] = bibref end end end end end open_refs = ps[:ref] || { } # consumed norm_ref = { } # convenience replacement of {{-coap}} with {{I-D.ietf-core-coap}} # collect normative/informative tagging {{!RFC2119}} {{?RFC4711}} sechash.each do |k, v| next if k == "fluff" v.gsub!(/{{(?:([?!])(-)?|(-))([\w._\-]+)(?:=([\w.\/_\-]+))?}}/) do |match| norminform = $1 replacing = $2 || $3 word = $4 bibref = $5 if replacing if new = ref_replacements[word] word = new else warn "*** no alias replacement for {{-#{word}}}" word = "-#{word}" end end # now, word is the anchor if bibref if old = anchor_to_bibref[word] if bibref != old warn "*** conflicting definitions for xref #{anchor}: #{old} != #{bibref}" end else anchor_to_bibref[word] = bibref end end # things can be normative in one place and informative in another -> normative # collect norm/inform above and assign it by priority here if norminform norm_ref[word] ||= norminform == '!' # one normative ref is enough end "{{#{word}}}" end end [:normative, :informative].each do |k| ps.rest[k.to_s] ||= { } end norm_ref.each do |k, v| # could check bibtagsys here: needed if open_refs is nil or string target = ps.has(v ? :normative : :informative) warn "*** overwriting #{k}" if target.has_key?(k) target[k] = open_refs[k] # add reference to normative/informative end # note that unused items from ref are considered OK, therefore no check for that here # also should allow norm/inform check of other references # {{?coap}} vs. {{!coap}} vs. {{-coap}} (undecided) # or {{?-coap}} vs. {{!-coap}} vs. {{-coap}} (undecided) # could require all references to be decided by a global flag overlap = [:normative, :informative].map { |s| (ps.has(s) || { }).keys }.reduce(:&) unless overlap.empty? warn "*** #{overlap.join(', ')}: both normative and informative" end stand_alone = ps[:stand_alone] link_defs = {} [:normative, :informative].each do |sn| if refs = ps[sn] refs.each do |k, v| href = k.gsub(/\A[0-9]/) { "_#{$&}" } # can't start an IDREF with a number link_defs[k] = ["##{href}", nil] # allow [RFC2119] in addition to {{RFC2119}} bibref = anchor_to_bibref[k] || k bts, url = bibtagsys(bibref, k, stand_alone) if bts && (!v || v == {} || v.respond_to?(:to_str)) if stand_alone a = %{{: anchor="#{k}"}} sechash[sn.to_s] << %{\n#{NMDTAGS[0]}\n![:include:](#{bts})#{a}\n#{NMDTAGS[1]}\n} else bts.gsub!('/', '_') (ps.rest["bibxml"] ||= []) << [bts, url] sechash[sn.to_s] << %{&#{bts};\n} # ??? end else unless v && Hash === v warn "*** don't know how to expand ref #{k}" next end if bts && !v.delete("override") warn "*** warning: explicit settings completely override canned bibxml in reference #{k}" end options = {input: 'RFC2629Kramdown', entity_output: coding_override, link_defs: link_defs} $global_markdown_options = options # For recursive calls in bibref annotation processing. sechash[sn.to_s] << KramdownRFC::ref_to_xml(k, v) end end end end erbfilename = File.expand_path '../../data/kramdown-rfc2629.erb', __FILE__ erbfile = File.read(erbfilename, coding: "UTF-8") erb = ERB.new(erbfile, nil, '-') # remove redundant nomarkdown pop outs/pop ins as they confuse kramdown input = erb.result(binding).gsub(%r"{::nomarkdown}\s*{:/nomarkdown}"m, "") ps.warn_if_leftovers sechash.delete("fluff") # fluff is a "commented out" section if !sechash.empty? # any sections unused by the ERb file? warn "*** sections left #{sechash.keys.inspect}!" end [input, coding_override, link_defs, smart_quotes] end XML_RESOURCE_ORG_PREFIX = Kramdown::Converter::Rfc2629::XML_RESOURCE_ORG_PREFIX # return XML entity name, url, rewrite_anchor flag def bibtagsys(bib, anchor=nil, stand_alone=true) if bib =~ /\Arfc(\d+)/i rfc4d = "%04d" % $1.to_i [bib.upcase, "#{XML_RESOURCE_ORG_PREFIX}/bibxml/reference.RFC.#{rfc4d}.xml"] elsif bib =~ /\A([-A-Z0-9]+)\./ && (xro = Kramdown::Converter::Rfc2629::XML_RESOURCE_ORG_MAP[$1]) dir, _ttl, rewrite_anchor = xro bib1 = bib.gsub(/\A[0-9]/) { "_#{$&}" } # can't start an ID with a number if anchor && bib1 != anchor if rewrite_anchor a = %{?anchor=#{anchor}} else if !stand_alone warn "*** selecting a custom anchor '#{anchor}' for '#{bib1}' requires stand_alone mode" warn " the output will need manual editing to correct this" end end end [bib1, "#{XML_RESOURCE_ORG_PREFIX}/#{dir}/reference.#{bib}.xml#{a}"] end end def read_encodings encfilename = File.expand_path '../../data/encoding-fallbacks.txt', __FILE__ encfile = File.read(encfilename, coding: "UTF-8") Hash[encfile.lines.map{|l| l.chomp!; x, s = l.split(" ", 2) [x.hex.chr(Encoding::UTF_8), s || " "]}] end FALLBACK = read_encodings def expand_tabs(s, tab_stops = 8) s.gsub(/([^\t\n]*)\t/) do $1 + " " * (tab_stops - ($1.size % tab_stops)) end end coding_override = :as_char input = ARGF.read if input[0] == "\uFEFF" warn "*** There is a leading byte order mark. Ignored." input[0..0] = '' end if input[-1] != "\n" # warn "*** added missing newline at end" input << "\n" # fix #26 end input.gsub!(/^\{::include\s+(.*?)\}/) { File.read($1).chomp } unless ENV["KRAMDOWN_SAFE"] if input =~ /[\t]/ warn "*** Input contains HT (\"tab\") characters. Undefined behavior will ensue." input = expand_tabs(input) end link_defs = {} if input =~ /\A---/ # this is a sectionized file do_the_tls_dance unless ENV["KRAMDOWN_DONT_VERIFY_HTTPS"] input, target_coding, link_defs, smart_quotes = xml_from_sections(input) end if input =~ /\A<\?xml/ # if this is a whole XML file, protect it input = "{::nomarkdown}\n#{input}\n{:/nomarkdown}\n" end options = {input: 'RFC2629Kramdown', entity_output: coding_override, link_defs: link_defs} if target_coding && target_coding =~ /ascii/ && smart_quotes.nil? smart_quotes = false end if smart_quotes == false smart_quotes = ["'".ord, "'".ord, '"'.ord, '"'.ord] end case smart_quotes when Array options[:smart_quotes] = smart_quotes when nil, true # nothin else warn "*** Can't deal with smart_quotes value #{smart_quotes.inspect}" end if target_coding input = input.encode(Encoding.find(target_coding), fallback: FALLBACK) end # warn "options: #{options.inspect}" doc = Kramdown::Document.new(input, options) $stderr.puts doc.warnings.to_yaml unless doc.warnings.empty? puts doc.to_rfc2629 kramdown-rfc2629-1.2.7/bin/doilit0000755000004100000410000000444213175754757016501 0ustar www-datawww-data#!/usr/bin/env ruby require 'open-uri' require 'yaml' require 'json' require 'kramdown-rfc2629' require 'kramdown-rfc/parameterset' require 'kramdown-rfc/refxml' # doilit -c 10.6028/NIST.SP.800-183 10.1016/j.adhoc.2015.04.007 10.1109/MIC.2012.29 10.1145/2507924.2507954 ACCEPT_CITE_JSON = {"Accept" => "application/citeproc+json"} $verbose = false $handle = "a" $xml = false litent = {} ARGV.each do |doi| case doi when "-c" begin require 'open-uri/cached' rescue LoadError warn '*** please "gem install open-uri-cached" to enable caching' end next when "-v" $verbose = true next when /\A-h=(.*)/ $handle = $1 next when /\A-x=(.*)/ $handle = $1 $xml = true next when /\A-/ warn "*** Usage: doilit [-c] [-v] [-h=handle|-x=xmlhandle] doi..." exit 1 end cite = JSON.parse(open("https://dx.doi.org/#{doi}", ACCEPT_CITE_JSON).read) puts cite.to_yaml if $verbose lit = {} ser = lit["seriesinfo"] = {} lit["title"] = cite["title"] if authors = cite["author"] lit["author"] = authors.map do |au| lau = {} if (g = au["given"]) && (f = au["family"]) lau["name"] = "#{g} #{f}" lau["ins"] = "#{g[0]}. #{f}" end lau end end if iss = cite["issued"] if dp = iss["date-parts"] if Integer === (dp = dp[0])[0] lit["date"] = ["%04d" % dp[0], *dp[1..-1].map {|p| "%02d" % p}].join("-") end end end if (ct = cite["container-title"]) && ct != [] info = [] if v = cite["volume"] info << "Vol. #{v}" end if p = cite["page"] info << "pp. #{p}" end rhs = info.join(", ") if info != [] ser[ct] = rhs else spl = ct.split(" ") ser[spl[0..-2].join(" ")] = spl[-1] end elsif pub = cite["publisher"] info = [] if t = cite["type"] info << t end rhs = info.join(", ") if info != [] ser[pub] = rhs else spl = pub.split(" ") ser[spl[0..-2].join(" ")] = spl[-1] end end lit["seriesinfo"]["DOI"] = cite["DOI"] while litent[$handle] $handle.succ! end litent[$handle] = lit end if $xml litent.each do |k, v| puts KramdownRFC::ref_to_xml(k, v) end else # 1.9 compat: s/lines/each_line.to_a/ puts litent.to_yaml.gsub(/^/, " ").each_line.to_a[1..-1] end kramdown-rfc2629-1.2.7/lib/0000755000004100000410000000000013175754757015261 5ustar www-datawww-datakramdown-rfc2629-1.2.7/lib/kramdown-rfc2629.rb0000644000004100000410000005647313175754757020542 0ustar www-datawww-data# -*- coding: utf-8 -*- # #-- # Copyright (C) 2009-2010 Thomas Leitner # Copyright (C) 2010-2014 Carsten Bormann # # This file was derived from a part of the kramdown gem which is licensed under the MIT license. # This derived work is also licensed under the MIT license, see LICENSE. #++ # raise "sorry, 1.8 was last decade" unless RUBY_VERSION >= '1.9' gem 'kramdown', '~> 1.14.0' require 'kramdown' my_span_elements = %w{list figure xref eref iref cref spanx vspace} Kramdown::Parser::Html::Constants::HTML_SPAN_ELEMENTS.concat my_span_elements require 'rexml/parsers/baseparser' require 'open3' # for math class Object def deep_clone Marshal.load(Marshal.dump(self)) end end module Kramdown module Parser class RFC2629Kramdown < Kramdown def initialize(*doc) super @span_parsers.unshift(:xref) @span_parsers.unshift(:iref) end XREF_START = /\{\{(.*?)\}\}/u # Introduce new {{target}} syntax for empty xrefs, which would # otherwise be an ugly ![!](target) or ![ ](target) # (I'd rather use [[target]], but that somehow clashes with links.) def parse_xref @src.pos += @src.matched_size href = @src[1] href = href.gsub(/\A[0-9]/) { "_#{$&}" } # can't start an IDREF with a number el = Element.new(:xref, nil, {'target' => href}) @tree.children << el end define_parser(:xref, XREF_START, '{{') IREF_START = /\(\(\((.*?)\)\)\)/u # Introduce new (((target))) syntax for irefs def parse_iref @src.pos += @src.matched_size href = @src[1] el = Element.new(:iref, nil, {'target' => href}) # XXX @tree.children << el end define_parser(:iref, IREF_START, '\(\(\(') end end class Element def rfc2629_fix if a = attr if anchor = a.delete('id') a['anchor'] = anchor end if anchor = a.delete('href') a['target'] = anchor end end end end module Converter # Converts a Kramdown::Document to HTML. class Rfc2629 < Base # we use these to do XML stuff, too include ::Kramdown::Utils::Html def el_html_attributes(el) html_attributes(el.attr) end def el_html_attributes_with(el, defattr) html_attributes(defattr.merge(el.attr)) end # :stopdoc: # Defines the amount of indentation used when nesting XML tags. INDENTATION = 2 # Initialize the XML converter with the given Kramdown document +doc+. def initialize(*doc) super @sec_level = 1 @in_dt = 0 @footnote_names_in_use = {} end def convert(el, indent = -INDENTATION, opts = {}) if el.children[-1].type == :raw raw = convert1(el.children.pop, indent, opts) end "#{convert1(el, indent, opts)}#{end_sections(1, indent)}#{raw}" end def convert1(el, indent, opts = {}) el.rfc2629_fix send("convert_#{el.type}", el, indent, opts) end def inner_a(el, indent, opts) indent += INDENTATION el.children.map do |inner_el| inner_el.rfc2629_fix send("convert_#{inner_el.type}", inner_el, indent, opts) end end def inner(el, indent, opts) inner_a(el, indent, opts).join('') end def convert_blank(el, indent, opts) "\n" end def convert_text(el, indent, opts) escape_html(el.value, :text) end def convert_p(el, indent, opts) if (el.children.size == 1 && el.children[0].type == :img) || opts[:unpacked] inner(el, indent, opts) # Part of the bad reference hack else "#{' '*indent}#{inner(el, indent, opts)}\n" end end def saner_generate_id(value) generate_id(value).gsub(/-+/, '-') end def convert_codeblock(el, indent, opts) # el.attr['anchor'] ||= saner_generate_id(el.value) -- no longer in 1.0.6 result = el.value blockclass = el.attr.delete('class') if blockclass == 'language-tbreak' result = result.lines.map {|line| [line.chomp, 0]} spaceind = 0 result.each_with_index {|pair, index| if pair[0] == '' result[spaceind][1] += 1 pair[0] = nil unless index == spaceind else spaceind = index end } # $stderr.puts(result.inspect) result = result.map {|line, space| "" if line }.compact.join("\n") "#{' '*indent}#{result}\n" else artwork_attr = {} if blockclass classes = blockclass.split(' ') classes.each do |cl| if md = cl.match(/\Alanguage-(.*)/) artwork_attr["type"] = md[1] # XXX overwrite else $stderr.puts "*** Unimplemented block class: #{cl}" end end end # compensate for XML2RFC idiosyncrasy by insisting on a blank line unless el.attr.delete('tight') result[0,0] = "\n" unless result[0,1] == "\n" end el.attr.each do |k, v| if md = k.match(/\Aartwork-(.*)/) el.attr.delete(k) artwork_attr[md[1]] = v end end "#{' '*indent}\n" end end def convert_blockquote(el, indent, opts) text = inner(el, indent, opts) text = "" unless text =~ /\n#{text}#{' '*indent}\n" end def end_sections(to_level, indent) if indent < 0 indent = 0 end if @sec_level >= to_level delta = (@sec_level - to_level) @sec_level = to_level "#{' '*indent}\n" * delta else $stderr.puts "Incorrect section nesting: Need to start with 1" end end def clean_pcdata(parts) # hack, will become unnecessary with XML2RFCv3 clean = '' irefs = '' # warn "clean_parts: #{parts.inspect}" parts.each do |p| md = p.match(%r{([^<]*)(.*)}) clean << md[1] irefs << md[2] # breaks for spanx... don't emphasize in headings! end [clean, irefs] end def convert_header(el, indent, opts) # todo: handle appendix tags el = el.deep_clone options = @doc ? @doc.options : @options # XXX: 0.11 vs. 0.12 if options[:auto_ids] && !el.attr['anchor'] el.attr['anchor'] = saner_generate_id(el.options[:raw_text]) end clean, irefs = clean_pcdata(inner_a(el, indent, opts)) el.attr['title'] = clean "#{end_sections(el.options[:level], indent)}#{' '*indent}#{irefs}\n" end def convert_hr(el, indent, opts) # misuse for page break "#{' '*indent}\n" end STYLES = {ul: 'symbols', ol: 'numbers', dl: 'hanging'} def convert_ul(el, indent, opts) opts = opts.merge(vspace: el.attr.delete('vspace')) attrstring = el_html_attributes_with(el, {"style" => STYLES[el.type]}) if opts[:unpacked] "#{' '*indent}\n#{inner(el, indent, opts)}#{' '*indent}\n" else "#{' '*indent}\n#{inner(el, indent, opts)}#{' '*indent}\n" end end alias :convert_ol :convert_ul alias :convert_dl :convert_ul def convert_li(el, indent, opts) res_a = inner_a(el, indent, opts) if el.children.empty? || el.children.first.options[:category] == :span res = res_a.join('') else # merge multiple elements res = res_a.select { |x| x.strip != '' }.map { |x| x.sub(/\A\s*(.*)<\/t>\s*\Z/m) { $1} }.join("#{' '*indent}\n").gsub(%r{()\s*}) { $1 }.gsub(%r{\s*(#{res}#{(res =~ /\n\Z/ ? ' '*indent : '')}\n" end def convert_dd(el, indent, opts) output = ' '*indent if @in_dt == 1 @in_dt = 0 else output << "" end res = inner(el, indent+INDENTATION, opts.merge(unpacked: true)) # if el.children.empty? || el.children.first.options[:category] != :block output << res << (res =~ /\n\Z/ ? ' '*indent : '') # else FIXME: The latter case is needed for more complex cases # output << "\n" << res << ' '*indent # end output << "\n" end def convert_dt(el, indent, opts) # SERIOUSLY BAD HACK: close = "#{' '*indent}\n" * @in_dt @in_dt = 1 vspace = opts[:vspace] vspaceel = "" if vspace ht = escape_html(inner(el, indent, opts), :attribute) # XXX this may leave gunk "#{close}#{' '*indent}#{vspaceel}\n" end HTML_TAGS_WITH_BODY=['div', 'script'] def convert_html_element(el, indent, opts) res = inner(el, indent, opts) if el.options[:category] == :span "<#{el.value}#{el_html_attributes(el)}" << (!res.empty? ? ">#{res}" : " />") else output = '' output << ' '*indent if !el.options[:parent_is_raw] output << "<#{el.value}#{el_html_attributes(el)}" if !res.empty? && el.options[:parse_type] != :block output << ">#{res}" elsif !res.empty? output << ">\n#{res}" << ' '*indent << "" elsif HTML_TAGS_WITH_BODY.include?(el.value) output << ">" else output << " />" end output << "\n" if el.options[:outer_element] || !el.options[:parent_is_raw] output end end def convert_xml_comment(el, indent, opts) if el.options[:category] == :block && !el.options[:parent_is_raw] ' '*indent + el.value + "\n" else el.value end end alias :convert_xml_pi :convert_xml_comment alias :convert_html_doctype :convert_xml_comment ALIGNMENTS = { default: :left, left: :left, right: :right, center: :center} COLS_ALIGN = { "l" => :left, "c" => :center, "r" => :right} def convert_table(el, indent, opts) # This only works for tables with headers alignment = el.options[:alignment].map { |al| ALIGNMENTS[al]} cols = (el.attr.delete("cols") || "").split(' ') "#{' '*indent}\n#{inner(el, indent, opts.merge(table_alignment: alignment, table_cols: cols))}#{' '*indent}\n" end def convert_thead(el, indent, opts) inner(el, indent, opts) end alias :convert_tbody :convert_thead alias :convert_tfoot :convert_thead alias :convert_tr :convert_thead def convert_td(el, indent, opts) if alignment = opts[:table_alignment] alignment = alignment.shift if cols = opts[:table_cols].shift md = cols.match(/(\d*(|em|[%*]))([lrc])/) if md[1].to_i != 0 widthval = md[1] widthval << "em" if md[2].empty? widthopt = "width='#{widthval}' " end alignment = COLS_ALIGN[md[3]] || :left end end if alignment res, irefs = clean_pcdata(inner_a(el, indent, opts)) warn "*** lost markup #{irefs} in table heading" unless irefs.empty? "#{' '*indent}#{res.empty? ? " " : res}\n" # XXX need clean_pcdata else res = inner(el, indent, opts) "#{' '*indent}#{res.empty? ? " " : res}\n" end end alias :convert_th :convert_td def convert_comment(el, indent, opts) ## Don't actually output all those comments into the XML: # if el.options[:category] == :block # "#{' '*indent}\n" # else # "" # end end def convert_br(el, indent, opts) "" end def convert_a(el, indent, opts) gi = el.attr.delete('gi') res = inner(el, indent, opts) target = el.attr['target'] if target[0] == "#" # handle [](#foo) as xref as in RFC 7328 el.attr['target'] = target = target[1..-1] if target.downcase == res.downcase res = '' # get rid of raw anchors leaking through end gi ||= "xref" else gi ||= "eref" end "<#{gi}#{el_html_attributes(el)}>#{res}" end def convert_xref(el, indent, opts) gi = el.attr.delete('gi') target = el.attr['target'] if target[0] == "&" "#{target};" else if target =~ %r{\A\w+:(?://|.*@)} gi ||= "eref" else gi ||= "xref" end "<#{gi}#{el_html_attributes(el)}/>" end end REFCACHEDIR = ENV["KRAMDOWN_REFCACHEDIR"] || ".refcache" # warn "*** REFCACHEDIR #{REFCACHEDIR}" KRAMDOWN_OFFLINE = ENV["KRAMDOWN_OFFLINE"] KRAMDOWN_REFCACHE_REFETCH = ENV["KRAMDOWN_REFCACHE_REFETCH"] # this is now slightly dangerous as multiple urls could map to the same cachefile def get_and_cache_resource(url, cachefile, tvalid = 7200, tn = Time.now) fn = "#{REFCACHEDIR}/#{cachefile}" Dir.mkdir(REFCACHEDIR) unless Dir.exists?(REFCACHEDIR) f = File.stat(fn) rescue nil unless KRAMDOWN_REFCACHE_REFETCH if !KRAMDOWN_OFFLINE && (!f || tn - f.ctime >= tvalid) if f message = "renewing (stale by #{"%.1f" % ((tn-f.ctime)/86400)} days)" fetch_timeout = 10 # seconds, give up quickly if just renewing else message = "fetching" fetch_timeout = 60 # seconds; long timeout needed for Travis end $stderr.puts "#{fn}: #{message}" if ENV["HAVE_WGET"] `cd #{REFCACHEDIR}; wget -t 3 -T #{fetch_timeout} -Nnv "#{url}"` # ignore errors if offline (hack) begin File.utime nil, nil, fn rescue Errno::ENOENT warn "Can't fetch #{url} -- is wget in path?" end else require 'open-uri' require 'socket' require 'openssl' require 'timeout' begin Timeout::timeout(fetch_timeout) do options = {} if ENV["KRAMDOWN_DONT_VERIFY_HTTPS"] options[:ssl_verify_mode] = OpenSSL::SSL::VERIFY_NONE end # workaround for OpenSSL on Windows... open(url, options) do |uf| s = uf.read if uf.status[0] != "200" warn "*** Status code #{status} while fetching #{url}" else File.write(fn, s) end end end rescue OpenURI::HTTPError, SocketError, Timeout::Error => e warn "*** #{e} while fetching #{url}" end end end begin File.read(fn) # this blows up if no cache available after fetch attempt rescue Errno::ENOENT => e warn "*** #{e} for ${fn}" end end # [subdirectory name, cache ttl in seconds, does it provide for ?anchor=] XML_RESOURCE_ORG_MAP = { "RFC" => ["bibxml", 86400*7], # these should change rarely "I-D" => "bibxml3", "W3C" => "bibxml4", "3GPP" => "bibxml5", "ANSI" => "bibxml2", "CCITT" => "bibxml2", "FIPS" => "bibxml2", # "IANA" => "bibxml2", overtaken by bibxml8 "IEEE" => "bibxml2", # now, how about bibxml6? "ISO" => "bibxml2", "ITU" => "bibxml2", "NIST" => "bibxml2", "OASIS" => "bibxml2", "PKCS" => "bibxml2", "DOI" => ["bibxml7", 86400, true], # 24 h cache at source anyway "IANA" => ["bibxml8", 86400, true], # ditto } # XML_RESOURCE_ORG_HOST = ENV["XML_RESOURCE_ORG_HOST"] || "xml.resource.org" XML_RESOURCE_ORG_HOST = ENV["XML_RESOURCE_ORG_HOST"] || "xml2rfc.tools.ietf.org" XML_RESOURCE_ORG_PREFIX = ENV["XML_RESOURCE_ORG_PREFIX"] || "https://#{XML_RESOURCE_ORG_HOST}/public/rfc" KRAMDOWN_REFCACHETTL = (e = ENV["KRAMDOWN_REFCACHETTL"]) ? e.to_i : 3600 def convert_img(el, indent, opts) # misuse the tag! if a = el.attr alt = a.delete('alt').strip alt = '' if alt == '!' # work around re-wrap uglyness if src = a.delete('src') a['target'] = src end end if alt == ":include:" # Really bad misuse of tag... anchor = el.attr.delete('anchor') || ( # not yet warn "*** missing anchor for '#{src}'" src ) anchor.sub!(/\A[0-9]/) { "_#{$&}" } # can't start an ID with a number anchor.gsub!('/', '_') # should take out all illegals to_insert = "" src.scan(/(W3C|3GPP|[A-Z-]+)[.]?([A-Za-z_0-9.\/\+-]+)/) do |t, n| fn = "reference.#{t}.#{n}.xml" sub, ttl = XML_RESOURCE_ORG_MAP[t] ttl ||= KRAMDOWN_REFCACHETTL # everything but RFCs might change a lot puts "Huh: ${fn}" unless sub url = "#{XML_RESOURCE_ORG_PREFIX}/#{sub}/#{fn}" to_insert = get_and_cache_resource(url, fn.gsub('/', '_'), ttl) to_insert.scrub! rescue nil # only do this for Ruby >= 2.1 # this may be a bit controversial: Don't break the build if reference is broken if KRAMDOWN_OFFLINE to_insert ||= " *** BROKEN REFERENCE *** " else exit 66 unless to_insert # EX_NOINPUT end end to_insert.sub(/<\?xml version=["']1.0["'] encoding=["']UTF-8["']\?>/, '') .sub(/\banchor=(?:"[^"]+"|'[^']+')/, "anchor=\"#{anchor}\"") else "#{alt}" end end def convert_codespan(el, indent, opts) attrstring = el_html_attributes_with(el, {"style" => 'verb'}) "#{escape_html(el.value)}" end def convert_footnote(el, indent, opts) # XXX: footnotes into crefs??? # this would be more like xml2rfc v3: # "\n#{' '*indent}\n#{inner(el.value, indent, opts).rstrip}\n#{' '*indent}" content = inner(el.value, indent, opts).strip content = escape_html(content.sub(/\A(.*)<\/t>\z/m) {$1}, :text) # text only... name = el.options[:name].sub(/\A[0-9]/) {"_" << $&} while @footnote_names_in_use[name] do if name =~ /:\d+\z/ name.succ! else name << ":1" end end @footnote_names_in_use[name] = true attrstring = el_html_attributes_with(el, {"anchor" => name}) "\n#{' '*indent}#{content}" end def convert_raw(el, indent, opts) end_sections(1, indent) + el.value + (el.options[:category] == :block ? "\n" : '') end EMPH = { em: "emph", strong: "strong"} def convert_em(el, indent, opts) attrstring = el_html_attributes_with(el, {"style" => EMPH[el.type]}) span, irefs = clean_pcdata(inner_a(el, indent, opts)) "#{span}#{irefs}" end alias :convert_strong :convert_em def convert_entity(el, indent, opts) entity_to_str(el.value) end TYPOGRAPHIC_SYMS = { :mdash => [::Kramdown::Utils::Entities.entity('mdash')], :ndash => [::Kramdown::Utils::Entities.entity('ndash')], :hellip => [::Kramdown::Utils::Entities.entity('hellip')], :laquo_space => [::Kramdown::Utils::Entities.entity('laquo'), ::Kramdown::Utils::Entities.entity('nbsp')], :raquo_space => [::Kramdown::Utils::Entities.entity('nbsp'), ::Kramdown::Utils::Entities.entity('raquo')], :laquo => [::Kramdown::Utils::Entities.entity('laquo')], :raquo => [::Kramdown::Utils::Entities.entity('raquo')] } def convert_typographic_sym(el, indent, opts) TYPOGRAPHIC_SYMS[el.value].map {|e| entity_to_str(e)}.join('') end def convert_smart_quote(el, indent, opts) entity_to_str(smart_quote_entity(el)) end def convert_math(el, indent, opts) # XXX: This is wrong el = el.deep_clone if el.options[:category] == :block el.attr['artwork-type'] ||= '' el.attr['artwork-type'] += (el.attr['artwork-type'].empty? ? '' : ' ') + 'math' artwork_attr = {} el.attr.each do |k, v| if md = k.match(/\Aartwork-(.*)/) el.attr.delete(k) artwork_attr[md[1]] = v end end result, _s = Open3.capture2("tex2mail -noindent -ragged -by_par -linelength=69", stdin_data: el.value); # warn "*** tex2mail not in path?" unless s.success? -- doesn't have useful status "#{' '*indent}\n" else warn "*** no support for inline math in XML2RFCv2" type = 'spanx' attrstring = el_html_attributes_with(el, {"style" => 'verb'}) "<#{type}#{attrstring}>#{escape_html(el.value, :text)}" end end ITEM_RE = '\s*(?:"([^"]*)"|([^,]*?))\s*' IREF_RE = %r{\A(!\s*)?#{ITEM_RE}(?:,#{ITEM_RE})?\z} def iref_attr(s) md = s.match(IREF_RE) attr = { item: md[2] || md[3], subitem: md[4] || md[5], primary: md[1] && 'true', } "" end def convert_iref(el, indent, opts) iref_attr(el.attr['target']) end def convert_abbreviation(el, indent, opts) # XXX: This is wrong title = @root.options[:abbrev_defs][el.value] title = nil if title.empty? value = el.value if item = title m = title.scan(Parser::RFC2629Kramdown::IREF_START) if m.empty? subitem = value else iref = m.map{|a,| iref_attr(a)}.join('') end else item = value end iref ||= "" "#{el.value}#{iref}" end def convert_root(el, indent, opts) result = inner(el, indent, opts) end end end end kramdown-rfc2629-1.2.7/lib/kramdown-rfc/0000755000004100000410000000000013175754757017653 5ustar www-datawww-datakramdown-rfc2629-1.2.7/lib/kramdown-rfc/refxml.rb0000644000004100000410000000426413175754757021503 0ustar www-datawww-datamodule KramdownRFC extend Kramdown::Utils::Html def self.escattr(str) escape_html(str.to_s, :attribute) end def self.ref_to_xml(k, v) vps = KramdownRFC::ParameterSet.new(v) erb = ERB.new <<-REFERB, nil, '-' > <%= vps.ele("title") -%> <% vps.arr("author", true, true) do |au| aups = authorps_from_hash(au) -%> > <%= aups.ele("organization=org", aups.attr("abbrev=orgabbrev"), "") %> <% aups.warn_if_leftovers -%> <% end -%> /> <% vps.arr("seriesinfo", false) do |k, v| -%> <% end -%> <% vps.arr("format", false) do |k, v| -%> <% end -%> <%= vps.ele("annotation=ann", nil, nil, true) -%> REFERB ret = erb.result(binding) vps.warn_if_leftovers ret end def self.authorps_from_hash(au) aups = KramdownRFC::ParameterSet.new(au) if ins = aups[:ins] parts = ins.split('.').map(&:strip) aups.rest["initials"] = parts[0..-2].join('.') << '.' aups.rest["surname"] = parts[-1] end # hack ("heuristic for") initials and surname from name # -- only works for people with exactly one last name and uncomplicated first names if n = aups.rest["name"] n = n.split aups.rest["initials"] ||= n[0..-2].map(&:chr).join('.') << '.' aups.rest["surname"] ||= n[-1] end aups end def self.dateattrs(date) begin case date when /\A\d\d\d\d\z/ %{year="#{date}"} when Integer %{year="#{"%04d" % date}"} when String Date.parse("#{date}-01").strftime(%{year="%Y" month="%B"}) when Date date.strftime(%{year="%Y" month="%B" day="%d"}) when Array # this allows to explicitly give a string %{year="#{date.join(" ")}"} when nil %{year="n.d."} end rescue ArgumentError warn "*** Invalid date: #{date} -- use 2012, 2012-07, or 2012-07-28" end end end kramdown-rfc2629-1.2.7/lib/kramdown-rfc/parameterset.rb0000644000004100000410000000323213175754757022674 0ustar www-datawww-datamodule KramdownRFC class ParameterSet include Kramdown::Utils::Html attr_reader :f def initialize(y) raise "*** invalid parameter set #{y.inspect}" unless Hash === y @f = y end def [](pn) @f.delete(pn.to_s) end def has(pn) @f[pn.to_s] end def escattr(str) escape_html(str.to_s, :attribute) end def van(pn) # pn is a parameter name, possibly with an =alias an, pn = pn.to_s.split("=") pn ||= an [self[pn] || self[an], an] end def attr(pn) val, an = van(pn) %{#{an}="#{escattr(val)}"} if val end def attrs(*pns) pns.map{ |pn| attr(pn) }.compact.join(" ") end def ele(pn, attr=nil, defcontent=nil, markdown=false) val, an = van(pn) val ||= defcontent Array(val).map do |val1| v = val1.to_s.strip if markdown # Uuh. Heavy coupling. doc = Kramdown::Document.new(v, $global_markdown_options) $stderr.puts doc.warnings.to_yaml unless doc.warnings.empty? contents = doc.to_rfc2629[3..-6] # skip ...\n else contents = escape_html(v) end %{<#{[an, *Array(attr).map(&:to_s)].join(" ").strip}>#{contents}} end.join(" ") end def arr(an, converthash=true, must_have_one=false, &block) arr = self[an] || [] arr = [arr] if Hash === arr && converthash arr << { } if must_have_one && arr.empty? Array(arr).each(&block) end def rest @f end def warn_if_leftovers if !@f.empty? warn "*** attributes left #{@f.inspect}!" end end end end kramdown-rfc2629-1.2.7/lib/kramdown-rfc/gzip-clone.rb0000644000004100000410000000172013175754757022247 0ustar www-datawww-datarequire 'zlib' require 'stringio' # cloned from module ActiveSupport # A convenient wrapper for the zlib standard library that allows # compression/decompression of strings with gzip. # # gzip = Gzip.compress('compress me!') # # => "\x1F\x8B\b\x00o\x8D\xCDO\x00\x03K\xCE\xCF-(J-.V\xC8MU\x04\x00R>n\x83\f\x00\x00\x00" # # Gzip.decompress(gzip) # # => "compress me!" module Gzip class Stream < StringIO def initialize(*) super set_encoding "BINARY" end def close; rewind; end end # Decompresses a gzipped string. def self.decompress(source) Zlib::GzipReader.new(StringIO.new(source)).read end # Compresses a string using gzip. def self.compress(source, level=Zlib::DEFAULT_COMPRESSION, strategy=Zlib::DEFAULT_STRATEGY) output = Stream.new gz = Zlib::GzipWriter.new(output, level, strategy) gz.write(source) gz.close output.string end end # end kramdown-rfc2629-1.2.7/kramdown-rfc2629.gemspec0000644000004100000410000000172413175754757021001 0ustar www-datawww-dataspec = Gem::Specification.new do |s| s.name = 'kramdown-rfc2629' s.version = '1.2.7' s.summary = "Kramdown extension for generating RFC 7749 XML." s.description = %{An RFC7749 (XML2RFC) generating backend for Thomas Leitner's "kramdown" markdown parser. Mostly useful for RFC writers.} s.add_dependency('kramdown', '~> 1.14.0') s.add_dependency('certified', '~> 1.0') s.files = Dir['lib/**/*.rb'] + %w(README.md LICENSE kramdown-rfc2629.gemspec bin/kdrfc bin/kramdown-rfc2629 bin/doilit bin/kramdown-rfc-extract-markdown data/kramdown-rfc2629.erb data/encoding-fallbacks.txt) s.require_path = 'lib' s.executables = ['kramdown-rfc2629', 'doilit', 'kramdown-rfc-extract-markdown', 'kdrfc'] s.default_executable = 'kramdown-rfc2629' s.required_ruby_version = '>= 1.9.2' # s.requirements = 'wget' # s.has_rdoc = true s.author = "Carsten Bormann" s.email = "cabo@tzi.org" s.homepage = "http://github.com/cabo/kramdown-rfc2629" s.license = 'MIT' end kramdown-rfc2629-1.2.7/LICENSE0000644000004100000410000000206713175754757015525 0ustar www-datawww-dataCopyright (c) 2010-2015 Carsten Bormann Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. kramdown-rfc2629-1.2.7/README.md0000644000004100000410000004503213175754757015776 0ustar www-datawww-data# kramdown-rfc2629 [kramdown][] is a [markdown][] parser by Thomas Leitner, which has a number of backends for generating HTML, Latex, and markdown again. **kramdown-rfc2629** is an additional backend to that: It allows the generation of [XML2RFC][] XML markup (originally known as [RFC 2629][] compliant markup, now documented in [RFC 7749][]). Who would care? Anybody who is writing Internet-Drafts and RFCs in the [IETF][] and prefers (or has co-authors who prefer) to do part of their work in markdown. # Usage Start by installing the kramdown-rfc2629 gem (this automatically installs appropriate versions of referenced gems such as kramdown as well): gem install kramdown-rfc2629 (Add a `sudo` and a space in front of that command if you don't have all the permissions needed.) The guts of kramdown-rfc2629 are in one Ruby file, `lib/kramdown-rfc2629.rb` --- this melds nicely into the extension structure provided by kramdown. `bin/kramdown-rfc2629` started out as a simple command-line program showing how to use this, but can now do much more (see below). To use kramdown-rfc2629, you'll need a Ruby 1.9 or 2.x, and maybe [XML2RFC][] if you want to see the fruits of your work. kramdown-rfc2629 mydraft.mkd >mydraft.xml xml2rfc mydraft.xml (The most popular file name extension that IETF people have for markdown is .md -- for those who tend to think about GNU machine descriptions here, any extension such as .mkd will do, too.) # Examples For historical interest `stupid.mkd` was an early markdown version of an actual Internet-Draft (for a protocol called [STuPiD][] \[sic!]). This demonstrated some, but not all features of kramdown-rfc2629. Since markdown/kramdown does not cater for all the structure of an RFC 7749 style document, some of the markup is in XML, and the example switches between XML and markdown using kramdown's `{::nomarkdown}` and `{:/nomarkdown}` (this is ugly, but works well enough). `stupid.xml` and `stupid.txt` show what kramdown-rfc2629 and xml2rfc make out of this. `stupid-s.mkd` is the same document in the new sectionized format supported by kramdown-rfc2629. The document metadata are in a short piece of YAML at the start, and from there, `abstract`, `middle`, references (`normative` and `informative`) and `back` are sections delimited in the markdown file. See the example for how this works. The sections `normative` and `informative` can be populated right from the metadata, so there is never a need to write XML any more. Much less scary, and no `{:/nomarkdown}` etc. is needed any more. Similarly, `stupid-s.xml` and `stupid-s.txt` show what kramdown-rfc2629 and xml2rfc make out of this. `draft-ietf-core-block-xx.mkd` is a real-world example of a current Internet-Draft done this way. For RFC and Internet-Draft references, it uses document prolog entities instead of caching the references in the XML (i.e., not standalone mode, this is easier to handle when collaborating with XML-only co-authors). See the `bibxml` metadata. # The YAML header Please consult the examples for the structure of the YAML header, this should be mostly obvious. The `stand_alone` attribute controls whether the RFC/I-D references are inserted into the document (yes) or entity-referenced (no), the latter leads to increased build time, but may be more palatable for a final XML conversion. The author entry can be a single hash or a list, as in: author: ins: C. Bormann name: Carsten Bormann org: Universität Bremen TZI abbrev: TZI street: Bibliothekstr. 1 city: Bremen code: D-28359 country: Germany phone: +49-421-218-63921 email: cabo@tzi.org or author: - ins: C. Bormann name: Carsten Bormann org: Universität Bremen TZI email: cabo@tzi.org - ins: Z. Shelby name: Zach Shelby org: Sensinode role: editor street: Kidekuja 2 city: Vuokatti code: 88600 country: Finland phone: "+358407796297" email: zach@sensinode.com - role: editor ins: P. Thubert name: Pascal Thubert org: Cisco Systems abbrev: Cisco street: - Village d'Entreprises Green Side - 400, Avenue de Roumanille - Batiment T3 city: Biot - Sophia Antipolis code: '06410' country: FRANCE phone: "+33 4 97 23 26 34" email: pthubert@cisco.com (the hash keys are the XML GIs from RFC 7749, with a flattened structure. As RFC 7749 requires giving both the full name and surname/initials, we use `ins` as an abbreviation for "initials/surname". Yes, the toolchain is Unicode-capable, even if the final RFC output is still in ASCII.) Note that the YAML header needs to be syntactically valid YAML. Where there is a potential for triggering some further YAML feature, a string should be put in quotes (like the "+358407796297" above, which might otherwise be interpreted as a number, losing the + sign). ## References The references section is built from the references listed in the YAML header and from references made inline to RFCs and I-Ds in the markdown text. Since kramdown-rfc2629 cannot know whether a reference is normative or informative, no entry is generated by default in the references section. By indicating a normative reference as in `{{!RFC2119}}` or an informative one as in `{{?RFC1925}}`, you can completely automate the referencing, without the need to write anything in the header. Alternatively, you can write something like: informative: RFC1925: normative: RFC2119: and then just write `{{RFC2119}}` or `{{RFC1925}}`. (Yes, there is a colon in the YAML, because this is a hash that could provide other information.) Since version 1.1, references imported from the [XML2RFC][] databases can be supplied with a replacement label (anchor name). E.g., RFC 793 could be referenced as `{{!TCP=RFC0793}}`, further references then just can say `{{TCP}}`; both will get `[TCP]` as the label. In the YAML, the same replacement can be expressed as in the first example: normative: TCP: RFC0793 informative: SST: DOI.10.1145/1282427.1282421 Notes about this feature: * Thank you, Martin Thomson, for supplying an implementation and insisting this be done. * While this feature is now available, you are not forced to use it for everything: readers of documents often benefit from not having to look up references, so continuing to use the draft names and RFC numbers as labels may be the preferable style in many cases. * As a final caveat, renaming anchors does not work in the `stand_alone: no` mode (except for IANA and DOI), as there is no such mechanism in XML entity referencing; exporting to XML while maintaining live references then may require some manual editing to get rid of the custom anchors. If your references are not in the [XML2RFC][] databases and do not have a DOI (that also happens to have correct data) either, you need to spell it out like in the examples below: informative: RFC1925: WEI: title: "6LoWPAN: the Wireless Embedded Internet" # see the quotes above? Needed because of the embedded colon. author: - ins: Z. Shelby name: Zach Shelby - ins: C. Bormann name: Carsten Bormann date: 2009 seriesinfo: ISBN: 9780470747995 ann: This is a really good reference on 6LoWPAN. ASN.1: title: > Information Technology — ASN.1 encoding rules: Specification of Basic Encoding Rules (BER), Canonical Encoding Rules (CER) and Distinguished Encoding Rules (DER) # YAML's ">" syntax used above is a good way to write longer titles author: org: International Telecommunications Union date: 1994 seriesinfo: ITU-T: Recommendation X.690 REST: target: http://www.ics.uci.edu/~fielding/pubs/dissertation/fielding_dissertation.pdf title: Architectural Styles and the Design of Network-based Software Architectures author: ins: R. Fielding name: Roy Thomas Fielding org: University of California, Irvine date: 2000 seriesinfo: "Ph.D.": "Dissertation, University of California, Irvine" format: PDF: http://www.ics.uci.edu/~fielding/pubs/dissertation/fielding_dissertation.pdf COAP: title: "CoAP: An Application Protocol for Billions of Tiny Internet Nodes" seriesinfo: DOI: 10.1109/MIC.2012.29 date: 2012 author: - ins: C. Bormann name: Carsten Bormann - ins: A. P. Castellani name: Angelo P. Castellani - ins: Z. Shelby name: Zach Shelby normative: ECMA262: author: org: European Computer Manufacturers Association title: ECMAScript Language Specification 5.1 Edition date: 2011-06 target: http://www.ecma-international.org/publications/files/ecma-st/ECMA-262.pdf seriesinfo: ECMA: Standard ECMA-262 RFC2119: RFC6690: (as in the author list, `ins` is an abbreviation for "initials/surname"; note that the first title had to be put in double quotes as it contains a colon which is special syntax in YAML.) Then you can simply reference `{{ASN.1}}` and `{{ECMA262}}` in the text. (Make sure the reference keys are valid XML names, though.) # Experimental features Most of the [kramdown syntax][kdsyntax] is supported and does something useful; with the exception of the math syntax (math has no special support in XML2RFC), and HTML syntax of course. A number of more esoteric features have recently been added. (The minimum required version for each full feature is indicated.) (1.0.35:) An experimental command `doilit` has been added. It can be used to convert DOIs given on the command line into references entries for kramdown-rfc YAML, saving a lot of typing. Note that the DOI database is not of very consistent quality, so you likely have to hand-edit the result before including it into the document (use `-v` to see raw JSON data from the DOI database, made somewhat readable by converting it into YAML). Use `-c` to enable caching (requires `open-uri-cached` gem). Use `-h=handle` in front of a DOI to set a handle different from the default `a`, `b`, etc. Similarly, use `-x=handle` to generate XML2RFCv2 XML instead of kramdown-rfc YAML. (1.0.31:) The kramdown `smart_quotes` feature can be controlled better. By default, it is on (with default kramdown settings), unless `coding: us-ascii` is in effect, when it is off by default. It also can be explicitly set on (`true`) or off (`false`) in the YAML header, or to a specific value (an array of four kramdown entity names or character numbers). E.g., for a German text (that is not intended to become an Internet-Draft), one might write: ```yaml smart_quotes: [sbquo, lsquo, bdquo, ldquo] pi: topblock: no private: yes ``` (1.0.30:) kramdown-rfc now uses kramdown 1.10, which leads to two notable updates: * Support for empty link texts in the standard markdown reference syntax, as in `[](#RFC7744)`. * Language names in fenced code blocks now support all characters except whitespace, so you can go wild with `asn.1` and `C#`. A heuristic generates missing initials/surname from the `name` entry in author information. This should save a lot of redundant typing. You'll need to continue using the `ins` entry as well if that heuristic fails (e.g., for Spanish names). Also, there is some rather experimental support for markdown display math (blocks between `$$` pairs) if the `tex2mail` tool is available. (1.0.23:) Move up to kramdown 1.6.0. This inherits a number of fixes and one nice feature: Markdown footnote definitions that turn into `cref`s can have their attributes in the footnote definition: ```markdown {:cabo: source="cabo"} (This section to be removed by the RFC editor.)[^1] [^1]: here is my editorial comment: warble warble. {:cabo} Another questionable paragraph.[^2] [^2]: so why not delete it? {: source="observer"} ``` (1.0.23:) As before, IAL attributes on a codeblock go to the figure element. Language attributes on the code block now become the artwork type, and any attribute with a name that starts "artwork-" is moved over to the artwork. So this snippet now does the obvious things: ```markdown ~~~ abnf a = b / %s"foo" / %x0D.0A ~~~ {: artwork-align="center" artwork-name="syntax"} ``` (1.0.22:) Index entries can be created with `(((item)))` or `(((item, subitem)))`; use quotes for weird entries: `(((",", comma)))`. If the index entry is to be marked "primary", prefix an (unquoted) `!` as in `(((!item)))`. In addition, auto-indexing is supported by hijacking the kramdown "abbrev" syntax: *[IANA]: *[MUST]: BCP14 *[CBOR]: (((Object Representation, Concise Binary))) (((CBOR))) The word in square brackets (which must match exactly, case-sensitively) is entered into the index automatically for each place where it occurs. If no title is given, just the word is entered (first example). If one is given, that becomes the main item (the auto-indexed word becomes the subitem, second example). If full control is desired (e.g., for multiple entries per occurrence), just write down the full index entries instead (third example). (1.0.20:) As an alternative referencing syntax for references with text, `{{ref}}` can be expressed as `[text](#ref)`. As a special case, a simple `[ref]` is interpreted as `[](#ref)` (except that the latter syntax is not actually allowed by kramdown). This syntax does not allow for automatic entry of items as normative/informative. (1.0.16:) Markdown footnotes are converted into `cref`s (XML2RFC formal comments; note that these are only visible if the pi "comments" is set to yes). The anchor is taken from the markdown footnote name. The source, if needed, can be supplied by an IAD, as in (first example with ALD): ```markdown {:cabo: source="cabo"} (This section to be removed by the RFC editor.)[^1]{:cabo} [^1]: here is my editorial comment Another questionable paragraph.[^2]{: source="observer"} [^2]: so why not delete it ``` Note that XML2RFC v2 doesn't allow structure in crefs. If you put any, you get the escaped verbatim XML... (1.0.11:) Allow overriding "style" attribute (via IAL = [inline attribute list][kdsyntax-ial]) in lists and spans as in: ```markdown {:req: counter="bar" style="format R(%d)"} {: req} * Foo * Bar * Bax Text outside the list, so a new IAL is needed. * Foof * Barf * Barx {: req} ``` (1.0.5:) An IAL attribute "cols" can be added to tables to override the column layout. For example, `cols="* 20 30c r"` sets the width attributes to 20 and 30 for the middle columns and sets the right two columns to center and right alignment, respectively. The alignment from `cols` overrides that from the kramdown table, if present. (1.0.2:) An IAL attribute "vspace" can be added to a definition list to break after the definition term: ```markdown {: vspace="0"} word: : definition anotherword: : another definition ``` (0.x:) Files can be included with the syntax `{::include fn}` (needs to be in column 1 since 1.0.22; can be suppressed for use in servers by setting environment variable KRAMDOWN_SAFE since 1.0.22). A typical example from a recent RFC, where the contents of a figure was machine-generated: ```markdown ~~~~~~~~~~ {::include ../ghc/packets-new/p4.out} ~~~~~~~~~~ {: #example2 title="A longer RPL example"} ``` (0.x:) A page break can be forced by adding a horizontal rule (`----`, note that this creates ugly blank space in some HTML converters). # Risks and Side-Effects The code is not very polished, but now quite stable; it has been successfully used for a number of non-trivial Internet-Drafts and RFCs. You probably still need to skim [RFC 7749][] if you want to write an Internet-Draft, but you don't really need to understand XML very much. Knowing the basics of YAML helps with the metadata (but you'll understand it from the examples). # Upconversion If you have an old RFC and want to convert it to markdown, try just using that RFC, it is 80 % there. It may be possible to automate the remaining 20 % some more, but that hasn't been done. If you have XML, there is an experimental upconverter that does 99 % of the work. Please contact the author if you want to try it. # Tools Joe Hildebrand has a [grunt][] plugin for kramdown-rfc2629 at: https://github.com/hildjj/grunt-kramdown-rfc2629 . Get started with it at: https://github.com/hildjj/grunt-init-rfc . This provides a self-refreshing web page with the kramdown-rfc2629/xml2rfc rendition of the draft you are editing. [grunt]: http://gruntjs.com Martin Thomson has an [I-D Template][] for github repositories that enable collaboration on draft development. This supports kramdown-rfc2629 out of the box. Just name your draft like `draft-ietf-unicorn-protocol-latest.md` and follow the installation instructions. [I-D Template]: https://github.com/martinthomson/i-d-template # Related Work Moving from XML to Markdown for RFC writing apparently is a no-brainer, so I'm not the only one who has written code for this. [Miek Gieben][] has done a [similar thing][pandoc2rfc] employing pandoc, now documented in [RFC 7328][]. He uses multiple input files instead of kramdown-rfc2629's sectionized input format. He keeps the metadata in a separate XML file, similar to the way the previous version of kramdown-rfc2629 stored (and still can store) the metadata in XML in the markdown document. He also uses a slightly different referencing syntax, which is closer to what markdown does elsewhere but more verbose (this syntax is now also supported in kramdown-rfc2629). (Miek now also has a new thing going on with mostly different syntax, see [mmark][].) # License Since kramdown version 1.0, kramdown itself is MIT licensed, which made it possible to license kramdown-rfc2629 under the same license. [kramdown]: http://kramdown.rubyforge.org/ [kdsyntax]: http://kramdown.gettalong.org/syntax.html [kdsyntax-ial]: http://kramdown.gettalong.org/syntax.html#inline-attribute-lists [stupid]: http://tools.ietf.org/id/draft-hartke-xmpp-stupid-00 [RFC 2629]: http://xml.resource.org/public/rfc/html/rfc2629.html [RFC 7749]: http://tools.ietf.org/html/rfc7749 [markdown]: http://en.wikipedia.org/wiki/Markdown [IETF]: http://www.ietf.org [Miek Gieben]: http://www.miek.nl/ [pandoc2rfc]: https://github.com/miekg/pandoc2rfc/ [XML2RFC]: http://xml.resource.org [RFC 7328]: http://tools.ietf.org/html/rfc7328 [mmark]: https://github.com/miekg/mmark [YAML]: http://www.yaml.org/spec/1.2/spec.html