°25 // CSV2HTML in Nim

I implemented the csv2html idea in nimgithub repo. Nim is a strongly typed, compiles-to-c/c++/js-to-metal language that has advanced programming features, yet looks very friendly (Python-ish syntax).

Here is the core of the csv2html program in nim:

import parsecsv
import argparse
import os
import system


include "row.nimf"
include "table.nimf"
include "html.nimf"

proc csv2html(csvfile: string, delimiter: string, quotechar: string, caption: string, title: string, header: bool, cssurl: string) = 
  var p: CsvParser
  var headerRow = ""
  p.open(csvfile)
  if header:
    p.readHeaderRow()
    headerRow = renderRow(p.headers, rowtype="th")
  var rows = newSeq[string](2)
  while p.readRow():
    rows.add(renderRow(p.row))
  p.close()
  echo renderHtml(renderTable(caption, headerRow, rows), title, cssurl)
  
  
  
when isMainModule:
  var p = newParser("CSV2HTML"):
    arg("csvfile")
    option("-d", "--delimiter", help="Field Delimiter. Default is ,.", default=",")
    option("-q", "--quotechar", help="Quote Character. Default is nothing")
    option("-t", "--title", help="Page Title. Will be printed in h1 tag")
    option("-c", "--caption", help="Table Caption")
    option("-s", "--css", help="Override CSS URL")
    flag("-f", "--header", help="Data has Header. First row will be rendered as `th`")
  if paramCount() < 1:
    echo p.help()
    quit(0)
  var args = p.parse()
  csv2html(args.csvfile, args.delimiter, args.quotechar, args.caption, ar

This looks very familiar to a Python programmer.

It is said that the language you write programs in, influences how you think. While nim is very close to Python syntax wise there are some standard library features of nim, that made me implement a feature differently.

In python, I wanted to keep to using just the standard library, and not pull in a templating language (I like Jinja2), to render the data into an HTML file. So, I ended up mixing the HTML template into the code. Which is fine for a small program like this.

In nim, the Source Code Filters library allows you to easily template things without having to reach for a “proper” templating language.

You might have wondered about lines starting with include above. I’m “including” the source code filter templates. For example, this is the code that renderes the csv data into an html table:

#? stdtmpl(subChar='$', metaChar='#')
#proc renderTable*(caption: string, headerRow: string, rows: seq[string]) : string = 
#  result = ""
<table>
  #if caption != "":
    <caption>${caption}</caption>
  #end if
  ${headerRow}
  #for row in items(rows):
    $row
  #end for
</table>
#end proc

Its clear looking at the template, what’s happening. At the top, we are defining subChar=$, which means the strings with $ prefix will be replaced with the actual value. metaChar=# stands for the character that means something to the nim compiler. Essentially the lines starting with # will be treated as nim code.

The files includedd will be compiled at compile time into corresponding nim code. This allows us to write templating code without having to worry about escaping etc.

The project is defined as a nim project using the nimble package manager definition file:

# csv2html package
version = "0.1.0"
author = "Pradeep Gowda"
description = "render csv files as html files"
license = "BSD3"

# deps
requires "nim >= 1.0.0", "argparse >= 0.10.0"

srcDir = "src"
bin = @["csv2html"]

You can see that I’m defining the argparse third party dependency in the nimble file. Argparse is very much influenced by Python’s stdlib - argparse.

The project can be compiled into a binary file with: nimble build which will produce csv2html binary.

The idea of using source code filters came to me via this Nim forum thread – “AWK-style processing with Nim” mentioned by @deech.

python nim

°24 // csv2html

Today, I had to create a paste a table of values into a JIRA ticket and I had a csv file. There are different ways to “paste” a nicely formatted table into a JIRA WYSIWYG text area, I’m sure.. but none of them fast enough to type: $csv2html source.csv destation.html. So, I wrote one.

You can find it here https://github.com/btbytes.csv2html. The help is self explanatory:

$./csv2html.py -h
usage: csv2html.py [-h] [--delimiter DELIMITER] [--quotechar QUOTECHAR]
                   [--title TITLE] [--caption CAPTION] [--header]
                   csvfile

CSV to HTML converter.

positional arguments:
  csvfile

optional arguments:
  -h, --help            show this help message and exit
  --delimiter DELIMITER
                        Field delimiter. Default is , .
  --quotechar QUOTECHAR
                        Quote Character. Deault is nothing
  --title TITLE         Page title. Will be printed in h1 tag.
  --caption CAPTION     Table caption.
  --header              data has header. First row will be "th".

Ideally, I’d have liked to use the pandoc-csv2table pandoc filter. But, installing this filter using cabal and stack turned out to be disappointing. So instead of figuring out how to compile and install these Haskell packages, I wrote this script.

I can still do some improvements to this script, like:

  1. adding an -o --output flag to write the output to a file instead of stdout.
  2. override the css

But, that’s for another day.

python

°23 // jsonfmt

Sometimes you would like to format a JSON file, and you wish a command called jsonfmt like gofmt would be handy. If you have Python3 installed on your system, you already have such a tool installed, its just called with a different name.

Let’s say you have this file - two.json 1.

{"glossary":{"title":"example glossary","GlossDiv":{"GlossList":{"GlossEntry":{"GlossDef":{"GlossSeeAlso":["GML","XML"],"para":"A meta-markup language, used to create markup languages such as DocBook."},"Abbrev":"ISO 8879:1986","GlossSee":"markup","SortAs":"SGML","GlossTerm":"Standard Generalized Markup Language","Acronym":"SGML","ID":"SGML"}},"title":"S"}}}

Not very readable, is it? You can “pretty-print” it like this:

$ python3 -m json.tool two.json
{
    "glossary": {
        "title": "example glossary",
        "GlossDiv": {
            "GlossList": {
                "GlossEntry": {
                    "GlossDef": {
                        "GlossSeeAlso": [
                            "GML",
                            "XML"
                        ],
                        "para": "A meta-markup language, used to create markup languages such as DocBook."
                    },
                    "Abbrev": "ISO 8879:1986",
                    "GlossSee": "markup",
                    "SortAs": "SGML",
                    "GlossTerm": "Standard Generalized Markup Language",
                    "Acronym": "SGML",
                    "ID": "SGML"
                }
            },
            "title": "S"
        }
    }
}

You can output the formatted JSON to a file like this:

$ python3 -m json.tool two.json three.json

Further, you can sort the keys with the --sort-keys option

$ python3 -m json.tool two.json --sort-keys

{
    "glossary": {
        "GlossDiv": {
            "GlossList": {
                "GlossEntry": {
                    "Abbrev": "ISO 8879:1986",
                    "Acronym": "SGML",
                    "GlossDef": {
                        "GlossSeeAlso": [
                            "GML",
                            "XML"
                        ],
                        "para": "A meta-markup language, used to create markup languages such as DocBook."
                    },
                    "GlossSee": "markup",
                    "GlossTerm": "Standard Generalized Markup Language",
                    "ID": "SGML",
                    "SortAs": "SGML"
                }
            },
            "title": "S"
        },
        "title": "example glossary"
    }
}

CAUTION: The one downside to this tool is, you can’t do an “in-place” formatting. That is, if you try – $ python3 -m json.tool two.json two.json, it will overwrite the original file.

If you want further explore json related tools, look at my json and jq pages.

python json

°22 // Thread Reaper

A simple utitlity to convert Thread Reader pages into simpler HTML pages. You can find the code on github/btbytes/theadreaper

Usage:

./threadreaper URL ["TITLE"] > local_page.html
./threadreaper https://threadreaderapp.com/thread/1202293011150852096.html "On Binbary Numbers" > foone_binary.html

This program also fixes an issue with embedded images that you would have if you were to simply download the Thread Reader HTML to disk.

golang

°21 // Thread reader app browser bookmark

Thread reader is a website that allows you to read twitter threads as a single page, example – this thread on Acropolis of Athens.

When I want to search for a twitter thread on their website or crete a new page of a twitter thread, I have to copy the top tweet of the thread, head over to https://threadreaderapp.com/ and paste the URL in the search box.

This is something that should be automated, so I created a bookmark that you can drag to the bookmarks toolbar. When you see a twitter thread, hit the bookmark, to be taken to the thread’s page on Thread reader.

Drag this bookmark to your browser’s bookmark toolbar » Thread Reader

javascript

Archive

(Reverse chronologic)

Tags

(Chronologic)
computing

Conway’s law 2020-04-29

fonts

Courier is a nice monospace font 2020-04-29

golang

Building Go Programs with Source File Hash baked in 2020-05-05

Notes to self: How to use Go Third Party Packages 2020-05-08

Thread Reaper 2020-05-22

grammar

About Grammarians and Philosophers 2020-05-15

ideas

gistash, RFCs 2020-05-14

interesting

A few interesting things 2020-05-12

Some interesting things 2020-05-13

gistash, RFCs 2020-05-14

Some Interesting Links 2020-05-15

javascript

Thread reader app browser bookmark 2020-05-20

json

jsonfmt 2020-06-05

mathematics

A worked exmaple of - A persian folk method of figuring interest 2020-05-01

nim

CSV2HTML in Nim 2020-06-10

patterns

Some Interesting Links 2020-05-15

philosophy

About Grammarians and Philosophers 2020-05-15

programming

github codespaces 2020-05-06

My code in books 2020-05-07

Getting Out of the Programming Goldilocks Zone 2020-05-14

python

pipx is like brew for python applications 2020-04-29

My Happy Python Workflow 2020-05-06

Python Line Processing Pattern 2020-05-17

Shell Safe API key / Password Generator 2020-05-19

jsonfmt 2020-06-05

csv2html 2020-06-05

CSV2HTML in Nim 2020-06-10

software

Quip 2020-05-02

zettelkasten

About Grammarians and Philosophers 2020-05-15