Create Markdown Table of Content

Purpose

A small utility function to create a table of content section at the beginning of a markdown file.

Let’s assume the following workflow:

  • Usually I use a text editor (preferable with code highlighting for the markdown syntax) for text-writing. Plain markdown fits perfectly for most of the tasks I need to follow up.

    Focusing on the content in an early stage of text creation for me is the most important. When using markdown syntax, text formatting is done almost instantly and so does not tear down the productivity.

  • Later on or in parallel, I also use Markdown2Go for the conversion to html and a to control the layout briefly in the browser.

  • Once finished, the main reference is still the markdown file and depending on the use case other programs come to play:

    • Markdown2Go: to create e.g. a program documentation
    • Hugo static site generator: for web presentation
    • Web-Browser: copy&paste the created html text from the browser window directly into an e-mail body
    • Web-Browser: to create pdf files
    • pandoc: for batch conversion of a bulk of files, etc…

I sometimes missing a “Table of Content” (TOC) creation tool which just adds the required tags on thy fly. So here it is.

For the moment, the function works for my personal use case. The logic behind is fairly simple and as alyways - it is not cranted that every possible case is covered right now.

Usage

The program takes a markdown file as input and writes the TOC + required html tags to it.
In a 2nd run, the TOC information is removed from the file again.

Command line:

createtoc.tcl <path-to-markdown-file\markdown-file.md>

The given file will be modified and a file write action will take place.

When testing the software, please make sure to backup the markdown text beforehand.

Example

The example shows the 3 stages: before, after and fully rendered:

# Lorem ipsum

Lorem ipsum dolor sit amet, consectetur adipisici elit,
sed eiusmod tempor incidunt ut labore et dolore magna aliqua.

## Ut enim ad minim veniam

Ut enim ad minim veniam, quis nostrud exercitation ullamco
laboris nisi ut aliquid ex ea commodi consequat.

### Quis aute iure reprehenderit

Quis aute iure reprehenderit in voluptate velit esse
cillum dolore eu fugiat nulla pariatur.
Excepteur sint obcaecat cupiditat non proident,
sunt in culpa qui officia deserunt mollit anim id est laborum.

#### Lorem ipsum dolor sit amet

Lorem ipsum dolor sit amet, consectetur adipisici elit,
sed eiusmod tempor incidunt ut labore et dolore magna aliqua.
**Table of Content:**
<a name="toc"></a>

-	[Ut enim ad minim veniam](#ut-enim-ad-minim-veniam)
	-	[Quis aute iure reprehenderit](#quis-aute-iure-reprehenderit)
	-	[Lorem ipsum dolor sit amet](#lorem-ipsum-dolor-sit-amet)

# Lorem ipsum

Lorem ipsum dolor sit amet, consectetur adipisici elit,
sed eiusmod tempor incidunt ut labore et dolore magna aliqua.

## Ut enim ad minim veniam
<a name="ut-enim-ad-minim-veniam"></a>

Ut enim ad minim veniam, quis nostrud exercitation ullamco
laboris nisi ut aliquid ex ea commodi consequat.

### Quis aute iure reprehenderit
<a name="quis-aute-iure-reprehenderit"></a>

Quis aute iure reprehenderit in voluptate velit esse
cillum dolore eu fugiat nulla pariatur.
Excepteur sint obcaecat cupiditat non proident,
sunt in culpa qui officia deserunt mollit anim id est laborum.

### Lorem ipsum dolor sit amet
<a name="lorem-ipsum-dolor-sit-amet"></a>

Lorem ipsum dolor sit amet, consectetur adipisici elit,
sed eiusmod tempor incidunt ut labore et dolore magna aliqua.

Table of Content:

Lorem ipsum

Lorem ipsum dolor sit amet, consectetur adipisici elit, sed eiusmod tempor incidunt ut labore et dolore magna aliqua.

Ut enim ad minim veniam

Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquid ex ea commodi consequat.

Quis aute iure reprehenderit

Quis aute iure reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint obcaecat cupiditat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

Lorem ipsum dolor sit amet

Lorem ipsum dolor sit amet, consectetur adipisici elit, sed eiusmod tempor incidunt ut labore et dolore magna aliqua.

Here is the code

The script is written in tcl/tk. Please read through the documentation which available in the source code for more information.

createtoc.tcl


# ------------------------------------------------------------------------
# --- createtoc.tcl
# ------------------------------------------------------------------------
# (c) 2021, Johann Oberdorfer - Engineering Support | CAD | Software
#     johann.oberdorfer [at] gmail.com
#     www.johann-oberdorfer.eu
# ------------------------------------------------------------------------
# This source file is distributed under the BSD license.
#   This program is distributed in the hope that it will be useful,
#   but WITHOUT ANY WARRANTY; without even the implied warranty of
#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
#   See the BSD License for more details.
# ------------------------------------------------------------------------
#
# Purpose:
#
#   Add a table of content (TOC) in pure markdown + html syntax.
#   The TOC does not require any javascript and should work
#   out of the box with any markdown converter.
#
#   Please not that the code is still experimental,
#   give it a try, but make sure to create a backup of the
#   input file beforehand.
#
# Usage:
#
#   The program takes a markdown file as input and writes
#   the TOC + required html tags to it.
#   In a 2nd run, the TOC information is removed from the file again.
#
# Notes:
#   Maybe it could be a good idea to add this functionality
#   to the Markdown2Go program using something like the following:
#      <!-- markdown:toc=3 -->
# ------------------------------------------------------------------------
# Revision history:
# 21-01-09: Hans, V0.1 - Initial release
# ------------------------------------------------------------------------

set dir [file dirname [info script]]

namespace eval markdown_create_toc {
	namespace export *

	variable vars
	
	array set vars {
		TOC_DESCR "**Table of Content:**"
		TOC_TAG   "toc"
		TOC_EOF   "<!-- TOC End -->\n"
		MAX_LEN 30
		NAV_TO_TOP 1
		STYLED 1
	}

	# read the whole file into the content variable
	#
	proc read_file_content {fname content_byref} {
		upvar $content_byref content

		set fp [open $fname "r"]
		set content [read $fp]
		close $fp
	}
	
	proc print__content {content} {
		foreach row [split $content "\n"] {
			puts $row
		}
	}

	proc is_toc_available {content} {
		variable vars

		foreach row [split $content "\n"] {
			if { [string compare $row $vars(TOC_DESCR)] == 0 } {
				return 1
			}
		}
		return 0
	}

	# k.o criteria for remove header
	proc is_header1_available {content} {

		foreach row [split $content "\n"] {
			if { [string range [string trim $row] 0 0] == "#" } {
				return 1
			}
		}
		return 0
	}

	
	# add tab at the beginning of a given string
	proc add_tab {str level} {
	
		# cnt: effect
		#   1: take all header lines into account,
		#   2: start with the 2nd level
		set cnt 2
		
		while {$cnt < $level} {
			set str "\t${str}"
			incr cnt
		}
		return $str
	}

	# re-write the file and
	# add/remove html tag for each header line ...
	#
	proc add_or_remove_html_tags {mode fname content toc_byref} {
		upvar $toc_byref toc
		variable vars
	
		# re-open the file again for writing:
		set fp [open $fname "w"]

		# **Table of Content:**
		set toc [list]
		lappend toc $vars(TOC_DESCR)
		lappend toc "<a name=\"${vars(TOC_TAG)}\"></a>\n"

		set cnt 0
		set inside_codeblock 0
		set content_list [split $content "\n"]

		while { $cnt < [llength $content_list] } {

			set row [lindex $content_list $cnt]

			# -------------------------------------------------
			if {$cnt < [expr {[llength $content_list] -1}]  } {
				puts $fp $row
			} else {
				puts -nonewline $fp $row
			}
			# -------------------------------------------------

			set item [string trim $row]

			# toggle "inside a codeblock" flag
			if { [string range $item 0 2] == "```"} { ;# ```
				if { !$inside_codeblock } {
					set inside_codeblock 1
				} else {
					set inside_codeblock 0
				}
			}

			if { !$inside_codeblock } {

				# check out, if the current row is a markdown header line ?
				set level -1

				if { [string range $item 0 3] == "####" } {
					set level 4
				} elseif { [string range $item 0 2] == "###" } {
					set level 3
				} elseif { [string range $item 0 1] == "##" } {
					set level 2
				}
				
				# elseif { [string range $item 0 0] == "#" } {
				#	set level 1
				# }

				if { $level != -1 } {
			
					set str [string trim [string range $item $level end]]
					set descr [string map {"\"" ""} $str]
					set tag [string tolower [string map {" " "-" "\"" "" ":" ""} $str]]

					# limit the string lenght as there is a limit in outlook when copy paste
					# html into the e-mail body, if this limit is exceeded, links won't work (!)
					
					set tag [string range $tag 0 [expr {$vars(MAX_LEN) -1}]]
					set html "<a name=\"${tag}\"></a>"

					if { $vars(NAV_TO_TOP) } {
						
						if { $vars(STYLED) } {
							# add alternative styling for the top navigation
							#   <a href="#toc" class="button">^</a>

							set html_bttn "<a href=\"#${vars(TOC_TAG)}\" class=\"button\">^</a>"
							set html "$html $html_bttn"
						} else {
							set html "$html \[^\](#${vars(TOC_TAG)})"
						}
					}

					switch -- $mode {
						"add_toc" {
							# check existence of html tag in the next following line,
							# if this line matches the just compiled html tag,
							# there's nothing more to do ...
							if {[lindex $content_list [expr {$cnt +1}]] != $html } {
						
								puts $fp $html
								puts "--> Writing tag: $tag"
							}

							lappend toc [add_tab "-\t\[${descr}\](#${tag})" $level]
						}
						"remove_toc" {
							# check existence of html tag in next following line,
							# if this line matches the just compiled html tag,
							# increment the list counter so that the following line
							# won't be written back to file...

							if {[lindex $content_list [expr {$cnt +1}]] == $html } {
								incr cnt
							}
						}
						default {
							error "unknown argument given to function: $mode"
						}
					}
				}
			}

			incr cnt
		}

		close $fp
	}

	# add toc at the very beginning of the file
	#
	proc add_table_of_content {fname toc} {
		variable vars

		# it is important to read-in the file content
		# one more time before the file is re-written
		# -------------------------------------------
		read_file_content $fname content
		# -------------------------------------------

		# now, re-open the file again for writing:
		set fp [open $fname "w"]

		foreach item $toc {
			puts $fp $item
			# puts "--> $item"
		}

		puts $fp "\n"

		# add alternative styling for the top navigation
		#   <style> .button {
		#	  border: 2px solid LightGrey;
		#	  border-radius:6px; padding: 0px 5px 0px 5px;}
		#   </style>
		#   <a href="#toc" class="button">^</a>

		if { $vars(NAV_TO_TOP) && $vars(STYLED) } {
			puts -nonewline \
			     $fp "<style> .button {border: 2px solid LightGrey;"
			puts $fp "border-radius:6px; padding: 0px 5px 0px 5px;} </style>"
		}

		# future improvement:
		# puts $fp $vars(TOC_EOF)

		puts -nonewline $fp $content
		close $fp
	}

	proc remove_table_of_content {fname} {
		variable vars

		# it is important to read-in the file content
		# one more time before the file is re-written
		# -------------------------------------------
		read_file_content $fname content
		# -------------------------------------------

		if { ![is_header1_available $content] } {
			puts "No header-1 could be detected - not possible to remove table of content!"
			return
		}

		set fp [open $fname "w"]
		set is_toc_text 1

		set content_list [split $content "\n"]
		set cnt 0

		foreach row $content_list {

			# criteria when table of content has ended
			# and the normal markdown text starts ...

			# future improvement: search for vars(TOC_EOF)
			if { $is_toc_text } {
				if { [string range [string trim $row] 0 0] == "#" } {
					set is_toc_text 0
				}
			}
		
			if { ! $is_toc_text } {
				if {$cnt < [expr {[llength $content_list] -1}]  } {
					puts $fp $row
				} else {
					puts -nonewline $fp $row
				}
			}
			incr cnt
		}

		close $fp
	}
}


proc process_file {fname} {

	puts "Processing File: $fname"

	# 1.) read the whole file into the content variable

	read_file_content $fname content
	# print__content $content


	# 2.) check, if table of content is already available ?
	#     and depending on the result add or remove html tag
	#     for each header line ...
	
	if { ! [is_toc_available $content] } {

		puts "TOC will be added!"
		add_or_remove_html_tags "add_toc" $fname $content toc

		# 3.) and finally, write toc at the very beginning of the file
		add_table_of_content $fname $toc

	} else {

		puts "TOC will be removed!"
		add_or_remove_html_tags "remove_toc" $fname $content toc
		remove_table_of_content $fname
	}

}


# ------------------------
# main loop starts here...
# ------------------------

namespace import markdown_create_toc::*

# show console window

catch {
	package require Tk
	wm withdraw .
	console show
	console eval {wm protocol . WM_DELETE_WINDOW {exit 0}}


puts \
"
Welcome to the createtoc.tcl utility v0.1
-----------------------------------------

Purpose:

   Add a table of content (TOC) in pure markdown + html syntax.
   The TOC does not require any javascript and should work
   out of the box with any markdown converter.

   Please not that the code is still experimental,
   give it a try, but make sure to create a backup of the
   input file beforehand.

Usage:

   The program takes a markdown file as input and writes
   the TOC + required html tags to it.
   In a 2nd run, the TOC information is removed from the file again.

   -------------

Copyright:

	(c) 2020, Johann Oberdorfer - Engineering Support | CAD | Software
		johann.oberdorfer \[at\] gmail <dot> com
		www.johann-oberdorfer.eu

	This source file is distributed under the BSD license.
	This program is distributed in the hope that it will be useful,
	but WITHOUT ANY WARRANTY; without even the implied warranty of
	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
	See the BSD License for more details.
"


}


set current_file [lindex $argv 0]

if { ![file exists $current_file] } {
	puts \
"
***Error:
	No argument given to function.
	Please provide a markdown file as argument and try again!
"

} else {
	# test mode:
	# set fname "markdown-test-file.md"
	# set current_file [file join $dir $fname]

	process_file $current_file
}

puts "Done."

CSS goodies

With the following CSS style a dynamic scroll behavior can be achived easily. Just use the “scroll-behavior:” declarative for the html body as following:

<style>
html {
	scroll-padding: 110px;
	scroll-behavior: smooth; /* note: shoas a side effect with scroll to top */
}
</style>

It might happen, that when scrolling to a header line in a html page via e.g. that header line is not shown as it appears to be slightly on top (outside the visible area). This can be adjusted by setting the “scroll-padding:” declaration.

That’s it for the moment.