How to program a recursive directory listing

I came across the idea to create a summary page showing all files
which are stored in a directiory down below Hugo’s /static folder.
Here is a brief description of how such a shortcode could be implemented.

Implementation

The function is implemented as a shortcode in Hugo (I assume here that the reader is familiar with Hugo (a static site generator) and as well with shortcodes (a kind of template declarative which can be repetitively used directly in markdown).

Let’s start with the markdown code needed to call up the shortcode:

{{/* directoryindex_recurse pathURL="/software" */}}
The /* */ is to comment out the shortcode here and needs to be replaced by < > as usual. Doing so will activate the function call and the template engine will evaluate the shortcode so that the output of is rendered as html.

To traverse down a directory tree, a recursive function is required which lists all available file types in a given directory.

In case if the file type is a directory the function calls itself again (and again…). For all remaining files in the current directory a file list is created and rendered as html table.

A function can be defined with the define function and such a function is called as template (I did not check Hugo’s documentation deeply and maybe there are different nomenclature in place).

In fact those two functions allow a kind of procedural programming similar to what we have in higher programming languages. Remember: the template engine is optimized for speed and any kind of processing in the template might slow down the build process a bit.

Arguments can be defined via a dict declaration. This is required as the scope of a declared variable depends on it’s context and is not valid any more inside a define declaration.

Shortcode

The best explanation is the code itself, so here it is.
This shortcode is also used to generate the download section on this site.

 {{/*
	--- shortcodes/directory_index_recurse.html
	
	recursive function which takes the directory name as input
	and traverses down the complete file tree
	to create a list with directory names including the
	file name + size
	note that it is asumed that the given directory (start dir)
	is located down below the /static directory
	
	example call:
	
	< directoryindex_recurse pathURL="/software" >

	(c) 2020, Johann Oberdorfer - Engineering Support | CAD | Software
		www.johann-oberdorfer.eu
	This source file is distributed under the MIT license.
	This program is distributed in the hope that it will be useful,
	but WITHOUT ANY WARRANTY; without even the implied warranty of
	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
	See the MIT License for more details.

*/}}

{{ $pathURL := .Get "pathURL" }}

{{ define "read-dir" }}

	{{ $pathURL := .pathurl }}
	{{ $currentPath := .currentpath }}

	{{ $files := readDir (printf "/static%s" $currentPath) }}

		{{ range $files }}

			{{ $fstat := os.Stat (printf "/static%s/%s" $currentPath .Name ) }}
			{{ $file_or_dir := printf "%s/%s" $currentPath .Name }}
			
			{{ if $fstat.IsDir }}
				<tr>
					<td class="table-active" style="color:DarkGrey;"> {{ .Name }}</td>
					<td class="table-active"></td>
					<td class="table-active"></td>
				</tr> 
				{{/* recursive call */}}
				{{ template "read-dir" dict "pathurl" $pathURL "currentpath" $file_or_dir }}

			{{ else }}

				<tr>
					<td></td>
					<td>
						<a href="{{ $file_or_dir }}" 
						   class="hover-animate-from-left">
							{{ .Name }}
						</a>
					</td>
					<td style="color:DarkGrey;">{{ .Size }}</td>
				</tr>
			{{ end }}
		{{ end }}

{{ end }}

<!-- thead and tbody tags and table id are required for the
	 bs-datatable.js utility script which takes care about search
	 sort and pagination functionality -->

<script>
	$(document).ready(function() {
		$('#datatable2').dataTable( { "ordering": false, "order": [], } );
		$('#datatable2_length select').val('50').change();
	});
</script>

<div class="table-responsive">

	<table id="datatable2"
		   class="table table-bordered nobottommargin"
		   cellspacing="0">
		<thead>
			<th>Name:</th>
			<th>Download-Link:</th>
			<th>Size <small>/ byte</small>:</th>
		</thead>
		<tbody>
			{{ template "read-dir" dict "pathurl" $pathURL "currentpath" $pathURL }}
		</tbody>
	</table>

</div>

Requirements for readDir and os.Stat

It took me a while to work out the requirements for the os.Stat arguments. Here it is required to add /static as a prefix to the string holding the currentPath.

Some more notes:

  • When a web-site is rendered with Hugo all files are copied over to the /public directory. All files means: all files from /static as well.

  • Hugo does not do any processing for static files only a copy operation takes place and only in case a file has changed - all for the sake of performance.

  • This /public directory later on becomes the server’s root directory and that’s the only one accessible for the web server.

So readDir and os.Stat take real directory and files which exists on the file system during the build process, whereas the rendered html pages only need to have URL’s referring files stored on the web server’s root directory - which are files copied over during the build process to the /public directory.

That’s it for the moment.