Console Applications

Overview

A console application (aka “command-line application”) is an application that is intended to run from within a shell, such as bash, zsh, or PowerShell, and uses text and block characters for input and output. This style of application dominated through the early days of computer science, until the 1980s when graphical user interfaces became much more common.

Console applications use text for both input and output: this can be as simple as plan text displayed in a window (e.g. bash), to systems that use text-based graphics and color to enhance their usability (e.g. Midnight Commander). This application style was really driven by the technical constraints of the time. Software was written for text-based terminals, often with very limited resources, and working over slow network connections. Text was faster to process, transmit and display than sophisticated graphics.

Some console applications remain popular, often due to powerful capabilities or features that are difficult to replicate in a graphical environment. Vim, for instance, is a text editor that originated as a console application, and is still used by many developers. Despite various attempts to build a “graphical vim”, the console version is often seen as more desireable due to the benefits of console applications i.e. faster, reduced latency, small memory footprint and ability to easily leverage other command-line tools.

Console applications often favor ‘batch-style processing‘, where you provide arguments to the program, and it executes a single task before exiting. This is very much due to the Unix Philosophy from the 70s, where this interaction style dominated:

1. Make each program do one thing well. To do a new job, build fresh rather than complicate old programs by adding new “features”.

2. Expect the output of every program to become the input to another, as yet unknown, program. Don’t clutter output with extraneous information. Avoid stringently columnar or binary input formats. Don’t insist on interactive input.

3. Design and build software, even operating systems, to be tried early, ideally within weeks. Don’t hesitate to throw away the clumsy parts and rebuild them.

4. Use tools in preference to unskilled help to lighten a programming task, even if you have to detour to build the tools and expect to throw some of them out after you’ve finished using them."

   – Bell Systems Technical Journal (1978)

Although we tend to run graphical operating systems with graphical applications, console applications are still very common. e.g. Windows, macOS, Linux all ship with consoles and powerful tools that are common used, at ledast by a subset of users. e.g. ls, curl, wget, emacs, vim, git and so on.

For expert users in particular, this style has some advantages. Console applications:

• can easily be scripted or automated to run without user intervention.
• can redirect input and output using standard IO streams, to allow interaction with other console applications that support this standard.
• tend to be small and performant, due to their relatively low-overhead (i.e. no graphics, sound, other application overhead).

The disadvantage is the steep learning curve, and lack of feature discoverability.

Design

Architecture

Command-line applications tend towards performing a single action and then exiting. This lends itself to a pipeline architecture, where processing steps are performed on intput, in order, to produce an output. This is also called a pipes and filters design pattern.

Examples of how this pattern might be applied:

• A file rename utility that changes the date on one or more files that are supplied as arguments. The program would (1) process and validate the command-line arguments, (2) apply the rule to produce a set of target output files (likely original:revise filename pairs) and then (3) perform these operations. Any interruption or error would cause the rename to abort.
• An image processing utility that attempts to filter a photo. Filters are applied in sequence on the input image producing a final output image.

Of course, not every command-line program works like this; there are some that are interactive, where the user provides successive input while the program is running. A program like this could use a mode traditional event-driven architecture:

Examples of this might include a text editor like Vi. It can certainly process command-line arguments, but it primarily operates in an interactive mode, where it waits for, and acts upon, user input (where keystrokes are represented as events).

Lifecycle

Command-line applications expect a single-entry point: a method with this familiar signature:

fun main(args:Array<String>) {
    // code here
    System.exit(0)
}

When your program is launched it executes this method. When it reaches the end of the method, the program stops executing. It’s considered “good form” to call System.exit(0) where the 0 is an error code returned to the OS. In this case, 0 means no errors i.e. a normal execution.

Input should be handled through either (a) arguments supplied on the command-line, or (b) values supplied through stdin. Batch-style applications should be able to run to completion without prompting the user for more input.

Output should be directed to stdout. You’re typically limited to textual output, or limited graphics (typically through the use of Unicode characters).

Errors should typically be directed to stderr.

As suggested above, we want to use a standard calling convention. Typically, command-line applications should use this format, or something similar:

$ program_name -option=value parameter

• program_name is the name of your program. It should be meaningful, and reflect what your program does.
• options represent a value that tells your program how to operate on the data. Typically options are prefixed with a dash (”-”) to distinguish them from parameters. If an option also requires a value (e.g. ”-bg=red”) then separate the option and value with a colon (”:”) or an equals sign (”=”).
• parameter represents input, or data that your program would act upon (e.g. the name of a file containing data). If multiple parameters are required, separate them with whitespace (e.g. ”program_name parameter1 parameter2”).

The order of options and parameters should not matter.

Running a program with insufficient arguments should display information on how to successfully execute it.

 $ rename
 Usage: rename [source] [dest]

Features

Command-line applications should have the following standard features:

• Use IO Streams: your application should handle all input through stdin, channel output to stdout and errors to stderr. This ensures that it will work as-expected with other command-line programs.
• Support conventions: The “target” (e.g. filename on which to operate) is usually provided as the primary argument. To disambiguate other input, it is normal to provide additional information using dashes using ¹. For example, it is standard to use --help to display brief help that demonstrates how to use your application.
• Provide user feedback for errors: your program should never print out “successful” messages. Save user feedback for errors. Error messages should be clear and help the user figure out what went wrong, or what they need to do to fix the error.

Typical command-line interaction is shown below:

% exa --help
Usage:
  exa [options] [files...]

META OPTIONS
  -?, --help         show list of command-line options
  -v, --version      show version of exa

DISPLAY OPTIONS
  -1, --oneline      display one entry per line
  -l, --long         display extended file metadata as a table
  -G, --grid         display entries as a grid (default)
  -x, --across       sort the grid across, rather than downwards
  -R, --recurse      recurse into directories
  -T, --tree         recurse into directories as a tree
  -F, --classify     display type indicator by file names
  ...

% exa -T
.
├── 01.syllabus.md
├── 02.introduction.md
├── 03.software_process.md
├── 04.sdlc.md
├── 05.architecture.md
├── 06.development.md
├── 07.testing.md
├── 08.kotlin_primer.md
├── 09.building_desktop.md
├── 10.building_mobile.md
├── 11.building_libraries.md
├── 12.building_services.md
├── 13.multiplatform.md
├── 99.unused.md
├── assets
│  ├── 2_tier_architecture.png
│  ├── 3_tier_architecture.png
│  ├── abstract_factory.png
│  ├── activity_lifecycle.png
...

Processing arguments

The main() method can optionally accept an array of Strings containing the command-line arguments for your program. To process them, you can simply iterate over the array.

This is effectively the same approach that you would take in C/C++.

fun main(args: Array<String>) {
	// args.size returns number of items 
	for (s in args) { 
	  println("Argument: $s")
	}
}

Reading/writing text

The Kotlin Standard Library (“kotlin-stdlib“) includes the standard IO functions for interacting with the console.

• readLine() reads a single value from “stdin“
• println() directs output to “stdout“

code/ucase.kts

// read single value from stdin
val str:String ?= readLine() 
if (str != null) { 
  println(str.toUpperCase())
}

It also includes basic methods for reading from existing files.

import java.io.*
var filename = "transactions.txt" 

// read up to 2GB as a string (incl. CRLF)
val contents = File(filename).readText(Charsets.UTF_8) 
println(contents) 

2020-06-06T14:35:44, 1001, 78.22, CDN
2020-06-06T14:38:18, 1002, 12.10, CDN
2020-06-06T14:42:51, 1003, 44.50, CDN

Example: rename utility

Let’s combine these ideas into a larger example.

Requirements: Write an interactive application that renames one or more files using options that the user provides. Options should support the following operations: add a prefix, add a suffix, or capitalize it.

We need to write a script that does the following:

1. Extracts options and target filenames from the arguments.
2. Checks that we have (a) valid arguments and (b) enough arguments to execute program properly (i.e. at least one filename and one rename option).
3. For each file in the list, use the options to determine the new filename, and then rename the file.

Usage: rename.kts [option list] [filename]

For this example, we need to manipulate a file on the local file system. The Kotlin standard library offers a File class that supports this. (e.g. changing permissions, renaming the file).

Construct a ‘File‘ object by providing a filename, and it returns a reference to that file on disk.

fun main(args: Array<String>) { 
  val files = getFiles(args) 
  val options = getOptions(args) 

  // check minimum required arguments
  if (files.size == 0 || options.size == 0) { 
	  println("Usage: [options] [file]") 
  } else {
  	applyOptions(files, options) 
} 

fun getOptions(args:Array<String>): HashMap<String, String> {
	var options = HashMap<String, String>() 
	for (arg in args) { 
		if (arg.contains(":")) {
			val (key, value) = arg.split(":") 
			options.put(key, value) 
		} 
	} 
	return options 
} 

fun getFiles(args:Array<String>) : List<String> { 
  var files:MutableList<String> = mutableListOf() 
    for (arg in args) { 
	    if (!arg.contains(":")) {
        files.add(arg) 
      }
    } 
  return files 
}

fun applyOptions(files:List<String>,options:HashMap<String, String>) { 
	for (file in files) { 
		var rFile = file
		// capitalize before adding prefix or suffix
    if (options.contains("prefix")) rFile = options.get("prefix") + rFile 
    if (options.contains("suffix")) rFile = rFile + options.get("suffix") 
  
		File(file).renameTo(rFile)
		println(file + " renamed to " + rFile)
	}
} 

Reading/writing binary data

These examples have all talked about reading/writing text data. What if I want to process binary data? Many binary data formats (e.g. JPEG) are defined by a standard, and will have library support for reading and writing them directly.

Kotlin also includes object-streams that support reading and writing binary data, including entire objects. You can, for instance, save an object and it’s state (serializing it) and then later load and restore it into memory (deserializing it).

class Emp(var name: String, var id:Int) : Serializable {} 
var file = FileOutputStream("datafile")
var stream = ObjectOutputStream(file)
var ann = Emp(1001, "Anne Hathaway", "New York") 
stream.writeObject(ann) 

Cursor positioning

ANSI escape codes are specific codes that can be “printed” by the shell but will be interpreted as commands rather than output. Note that these are not standard across consoles, so you are encouraged to test with a specific console that you wish to support.

These escape codes can be used to move the cursor around the console. All of these start with ESC plus a suffix.

For instance, this progress bar example draws a line and updates the starting percentage in-place.

val ESC  = "\u001B";
val STARTLINE = ESC + "[250D";

for (i in 1..100) {
  print(STARTLINE);
  print(i + "% " + BLOCK.repeat(i));
}

// output
100% ▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋

Printing in colour

We can also use escape sequences to change the color that is displayed.

Below, we use ANSI colour codes to display colored output:

val ANSI_BLACK = "\u001B[30m"
val ANSI_YELLOW = "\u001B[33m"
val ANSI_WHITE = "\u001B[37m"

println(ANSI_WHITE + "  ;)(;  ");
println(ANSI_YELLOW + " :----: " + ANSI_BLACK + " Vendor:   " + System.getProperty("java.vendor"));
println(ANSI_YELLOW + "C|" + ANSI_WHITE + "====" + ANSI_YELLOW + "| " + ANSI_BLACK + " JDK Name: " + System.getProperty("java.vm.name"));
println(ANSI_YELLOW + " |    | " + ANSI_BLACK + " Version:  " + System.getProperty("java.version"));
println(ANSI_YELLOW + " `----' ");
println();

Applications

Packaging just refers to putting together your software components in a way that you can distribute to your end-user. As we’ve seen, this could be physical (a retail box including a DVD), or a digital image (ISO image, package file, installer executable). It’s the “bundling” part of building software.

Packaging is also one of those tasks that we tend to hand wave: just compile, link and hand-out the executable right? Unfortunately it’s not that simple. Apart from the executable, your application may also include images, sound clips and other resources, plus external libraries that you don’t necessarily want to statically link into your executable. You need to distribute all of these files, and ensure that they are installed in the correct location, registered with the OS and made usable by your application. You also need to perform steps to ensure that your application respects conventions of the particular environment: installing under Program Files in Windows, putting libraries under Windows\\System32, putting an icon on the desktop and so on.

The way we address the compexities of packaging is to break down the process into multiple steps, which are handled by different tools:

In the next sections, we’ll demonstrate using each of these tools.

1. Compiling classes

Let’s use the HelloWorld sample to demonstrate different methods for preparing, and then packaging our application.

fun main(args: Array<String>) {
	println(“Hello Kotlin!”)
}

To compile from the command-line, we can use the Kotlin compiler, kotlinc. By default, it takes Kotlin source files (.kt) and compiles them into corresponding class files (.class) that can be executed on the JVM.

$ kotlinc Hello.kt 

$ ls 
Hello.kt HelloKt.class

$ kotlin HelloKt
Hello Kotlin!

We could also just do this with IntelliJ IDEA or Android Studio, where Gradle - build - build will generate class files. As we will see at the end of this section, we can often just generate the final installer in a single step without doing each step manually.

2. Creating archives

Class files by themselves are difficult to distribute. The Java platform includes the jar utility, used to create a single archive file from your classes.

A JAR file is a standard mechanism in the Java ecosystem for distributing applications. It’s effectively just a compressed file (just like a ZIP file) which has a specific structure and contents. Most distribution mechanisms expect us to create a JAR file first.

This example compiles Hello.kt, and packages the output in a Hello.jar file.

$ kotlinc Hello.kt -include-runtime -d Hello.jar

$ ls
Hello.jar Hello.kt

The -d option tells the compiler to package all of the required classes into our jar file. The -include-runtime flag tells it to also include the Kotlin runtime classes. These classes are needed for all Kotlin applications, and they’re small, so you should always include them in your distribution (if you fail to include them, you app won’t run unless your user has Kotlin installed).

To run from a jar file, use the java command².

$ java -jar Hello.jar
Hello Kotlin!

Our JAR file from above looks like this if you uncompress it:

$ unzip Hello.jar -d contents
Archive:  Hello.jar
  inflating: contents/META-INF/MANIFEST.MF  
  inflating: contents/HelloKt.class  
  inflating: contents/META-INF/main.kotlin_module  
  inflating: contents/kotlin/collections/ArraysUtilJVM.class 
	...

$ tree -L 2 contents/
.
├── META-INF
│   ├── MANIFEST.MF
│   ├── main.kotlin_module
│   └── versions
└── kotlin
    ├── ArrayIntrinsicsKt.class
    ├── BuilderInference.class
    ├── DeepRecursiveFunction.class
    ├── DeepRecursiveKt.class
    ├── DeepRecursiveScope.class
    ...

The JAR file contains these main features:

• HelloKt.class – a class wrapper generated by the compiler
• META-INF/MANIFEST.MF – a file containing metadata.
• kotlin/ – Kotlin runtime classes not included in the JDK.

The MANIFEST.MF file is autogenerated by the compiler, and included in the JAR file. It tells the runtime which main method to execute. e.g. HelloKt.main().

$ cat contents/META-INF/MANIFEST.MF 
Manifest-Version: 1.0
Created-By: JetBrains Kotlin
Main-Class: HelloKt

In IntelliJIDEA, Gradle - build - jar will create a jar file. As we will see at the end of this section, we can often just generate the final installer in a single step without doing each step manually.

3. Creating scripts

We can distribute JAR files like this to our users, but they’re awkward: users would need to have the Java JDK installed, and typejava -jar filename.jar to actually run our programs. For some scenarios this might be acceptable (e.g. launching a service from an init script), but most of the time we want a more user-friendly installation mechanism.

The simplest thing we can do it create a script to launch our application from a JAR file, with the same effect as executing from the command-line:

$ cat hello
#!/bin/bash 
java -jar hello.jar

$ chmod +x hello

$ ./hello
Hello Kotlin!

For simple applications, especially ones that we use ourselves, this may be sufficient. It has the downside of requiring the user to have the Java JDK installed on their system³. This particular script is also not very robust: it doesn’t set runtime parameters, and doesn’t handle more than a single JAR file.

For a more robust script, we can let Gradle generate one for us. In IntelliJIDEA, Gradle - distribution - distZip will create a zip file that includes a custom runtime script. Unzip it, and it will contain a lib directory with all of the JAR files, and a bin directory with a script that will run the application from the lib directory. Copy both to a new location and you can execute the app from there.

Here’s an example of the Archey project, with a distZip file that was generated by Gradle.

$ tree build/distributions -L 3
build/distributions
├── app
│   ├── bin
│   │   ├── app
│   │   └── app.bat
│   └── lib
│       ├── annotations-13.0.jar
│       ├── app.jar
│       ├── checker-qual-3.8.0.jar
│       ├── error_prone_annotations-2.5.1.jar
│       ├── failureaccess-1.0.1.jar
│       ├── guava-30.1.1-jre.jar
│       ├── j2objc-annotations-1.3.jar
│       ├── jsr305-3.0.2.jar
│       ├── kotlin-stdlib-1.5.31.jar
│       ├── kotlin-stdlib-common-1.5.31.jar
│       ├── kotlin-stdlib-jdk7-1.5.31.jar
│       ├── kotlin-stdlib-jdk8-1.5.31.jar
│       └── listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar
└── app.zip

The bin/app script is quite robust, and will handle many different types of system configurations. It will also handle setting the CLASSPATH for all of the libraries that are included in the lib directory!

Here’s a portion of the app script. You don’t want to have to write this by-hand!

# OS specific support (must be 'true' or 'false').
cygwin=false
msys=false
darwin=false
nonstop=false
case "$( uname )" in                #(
  CYGWIN* )         cygwin=true  ;; #(
  Darwin* )         darwin=true  ;; #(
  MSYS* | MINGW* )  msys=true    ;; #(
  NONSTOP* )        nonstop=true ;;
esac

CLASSPATH=$APP_HOME/lib/app.jar:$APP_HOME/lib/kotlin-stdlib-jdk8-1.5.31.jar:$APP_HOME/lib/guava-30.1.1-jre.jar:$APP_HOME/lib/kotlin-stdlib-jdk7-1.5.31.jar:$APP_HOME/lib/kotlin-stdlib-1.5.31.jar:$APP_HOME/lib/kotlin-stdlib-common-1.5.31.jar:$APP_HOME/lib/failureaccess-1.0.1.jar:$APP_HOME/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:$APP_HOME/lib/jsr305-3.0.2.jar:$APP_HOME/lib/checker-qual-3.8.0.jar:$APP_HOME/lib/error_prone_annotations-2.5.1.jar:$APP_HOME/lib/j2objc-annotations-1.3.jar:$APP_HOME/lib/annotations-13.0.jar


# Determine the Java command to use to start the JVM.
if [ -n "$JAVA_HOME" ] ; then
    if [ -x "$JAVA_HOME/jre/sh/java" ] ; then
        # IBM's JDK on AIX uses strange locations for the executables
        JAVACMD=$JAVA_HOME/jre/sh/java
    else
        JAVACMD=$JAVA_HOME/bin/java
    fi
    if [ ! -x "$JAVACMD" ] ; then
        die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME

The script works well. Here’s archey being executed from the script:

$ build/distributions/app/bin/app 

               ####                   User: jaffe
               ###                    Home: /Users/jaffe
       #######    #######             Name: Bishop.local
     ######################           OS: Mac OS X
    #####################             Version: 11.3
    ####################              CPU: x86_64
    ####################              Cores: 10
    #####################             Free Memory: 514 MB
     ######################           Total Memory: 520 MB
      ####################            Disk Size: 582 GB
        ################              Free Space: 582 GB
         ####     #####               IP Address: 127.0.0.1

Toolkits

In some ways, a terminal-based program is really just an extension of a line printer. It’s designed to output to stdout one line at a time, and doesn’t handle editing in-place very well (which would be required if you wanted to create a terminal-based UI, like Midnight Commander).

There are specific libraries that have been developed to provide more sophisticated capabilities. Here’s a couple of tile-based toolkits that allow you to build up a full UI in the console:

• Mosaic is a Kotlin toolkit for building tiled applications e.g. Rogue-like games.

• Zirzon is a tile-engine and Text GUI library.

Here’s an example of a GUI written with Zircon:

Thse toolkits really serve a rare circumstance, where you want graphical capabilities but you’re running in a non-graphical environment. Typically if you’re running a modern OS, you have more sophisticated graphical capabilities available.

Here’s a console toolkit that provides further capabilities.

• Turtle