Percent complete R function

R function to print percent complete.

If you use
method = message
you may want to use
suffix = ” % complete”

pctComplete <- function(
  progress = 0, last.progress = 0,
  total = 100, increment = 10,
  suffix = " ", method = cat, ...) 
  if (increment < 1)
    increment <- 1
  if (total < 0)
    total <- 100
  pct <- round(progress * 100 / total)
  if ( pct >= increment 
               + last.progress
               - last.progress%% increment )
    method(paste("", pct, suffix, sep = ""),


aggregate free memory and swap

free -b reveals the total, used, free, shared, cache, and available memory.

Here’s the awk code to extract the total of the free physical memory and the free swap.

First, observe the free memory and swap, in bytes

# free -b | \
awk 'BEGIN{s=0}; NR<2{print $3}; NR>1{print $4}'

Now, total the free memory and swap

# free -b | \
awk 'BEGIN{s=0};NR>1{n=$4;s=s+n;};END{print s}'

The sum is slightly different because the second command was run at a different time.

Why I chose MDI during R installation

When installing R on a machine that is shared among several users, some of which are accustomed to the Mac OS, I had to choose between the Multiple Document Interface (MDI) and the Single Document Interface (SDI). The MDI article on Wikipedia pointed out that the MDI interface relates all documents to a common menu bar, which is what Mac users are accustomed to; Windows users probably have encountered both, since Office applications have used at times (depending on the generation) SDI and MDI. Therefore, to make the experience less alien for Mac users on a Windows machine, I installed R configured to use MDI.

Ubuntu temperature monitor


I am trying to figure out how to monitor temperature on my Ubuntu system, and I found this thread:
So I started installing the packages recommended in response #5, but response #4 may be good, too.

$ sudo apt-get install lm-sensors hddtemp
$ sudo sensors-detect
$ sudo /etc/init.d/kmod start
[ ok ] Starting kmod (via systemctl): kmod.service.
$ sensors
Adapter: Virtual device
temp1:        +54.0°C  (crit = +120.0°C)

Adapter: ISA adapter
Physical id 0:  +53.0°C  (high = +100.0°C, crit = +100.0°C)
Core 0:         +52.0°C  (high = +100.0°C, crit = +100.0°C)
Core 1:         +51.0°C  (high = +100.0°C, crit = +100.0°C)

sed script for key:value to csv

I had files with one key-value pair per line, e.g.:

Acquired Name: foo
Acquired Date: 1066-10-14T10:00:00Z

I wanted to extract all of the fields to one CSV line per file.

This is the sed script that I used. As it happens, this would also work if all of the input files were concatenated (because I used the ‘d’ command rather than ‘q’).

/Acquired Name:/ { 
  s/.*Acquired Name: \(.*\)./"\1"/ ; h ; }
/Acquired Date:/ {
  s/.*Acquired Date: \(.*\)./"\1"/ ; x ; G ; s/\n/,/M ; h ; }
/Acquired Time:/ {
  s/.*Acquired Time: \(.*\)./"\1"/ ; x ; G ;
  s/\n/,/M ; h ; }
/Job Code:/ {
  s/.*Job Code: \(.*\)./"\1"/ ; x ; G ;
  s/\n/,/M ; h ; }
/Instrument:/ {
  s/.*Instrument: \(.*\)./"\1"/ ; x ; G ;
  s/\n/,/M ; h ; }
/Bottle Number:/ {
  s/.*Bottle Number: \(.*\)./"\1"/ ; x ; G ;
  s/\n/,/M ; h ; }
/Inlet Method:/ {
  s/.*Inlet Method: \(.*\)./"\1"/ ; x ; G ;
  s/\n/,/M ; h ; }
/MS Method:/ {
  s/.*MS Method: \(.*\)./"\1"/ ; x ; G ;
  s/\n/,/M ; h ; }
/Tune Method:/ {
  s/.*Tune Method: \(.*\)./"\1"/ ; x ; G ;
  s/\n/,/M ; p ; d ; }

The manual for sed
explains the commands used:

  • d – Delete the pattern space; immediately start next cycle with next input line.
  • h – Copy the pattern space to the hold space
  • G – Append the hold space to the pattern space
  • p – Print the pattern space
  • s/pattern/replacement/ – Substitute pattern with replacement
  • s/pattern/replacement/M – Do multi-line substitution; see this
  • x – Exchange the pattern space and the hold space




colordiff is a wrapper for diff and produces the same output as diff but with coloured syntax highlighting at the command line to improve readability.

The output is similar to how a diff-generated patch might appear in Vim or Emacs with the appropriate syntax highlighting options enabled.

The colour schemes can be read from a central configuration file or from a local user ~/.colordiffrc file.

To enjoy on Ubuntu:

sudo apt install colordiff

Check these if you cannot ssh

In addition to checking that /etc/ssh/sshd_config is correct, you will want to check that the permissions on the user’s home directory and ~/.ssh are correct. I have gathered that:

  • The home directory must only be owned by AND only be writable by the login user.
  • The .ssh directory must only be accessible by the user.
  • The authorized_keys file must be only writable by the user.
[foo@foo-nas ~/.ssh]$ ls -la -D ""
total 14
drwx------  2 foo  foogrp    3  .
drwxr-x---  5 foo  foogrp   15  ..
-rw-r--r--  1 foo  foogrp  395  authorized_keys

ZFS expansion drive on Ubuntu

What ZFS has to offer for a laptop

ZFS is best used in a RAID configuration:

  • mirror (at least 2 disks)
  • raidz1 (at least 3 disks)
  • raidz2 (at least 4 disks)
  • raidz3 (5 or more disks)

A laptop does not typically offer the advantage of being able to house so many disks. Nevertheless, ZFS offers other advantages that make it sensible to use on a laptop:

  • scrubbing and on-line file system repair
  • snapshots
  • transparent compression
  • transparent encryption

I have two drives in my laptop:

  • a 256 GB SSD OS drive
  • a 512 GB 7200 RPM expansion drive

It might make sense at some point to replace the expansion drive with a higher capacity drive. For now, I installed ZFS onto the expansion drive, inspired by:

Ideally, I would have a mirror between my OS drive and a partition on the expansion drive; however, not only is my expansion drive fairly limited, but also this is my first attempt at using ZFS beyond FreeNAS, which handles ZFS at arm’s length.  So, I decided to start with something simple: ZFS on an unmirrored disk.

Preliminary step

Although the kernel in Ubuntu 16.04 LTS supports ZFS, I needed to install the ZFS utils:
  apt-get install zfsutils-linux
to use the zpool and zfs commands.

What I should have done, perhaps

If I am reading the man page correctly for the zpool command (which says that the device name path is relative to /dev), I could have deduced the invariant id for my disk
using the command:
  ls -l /dev/disk/by-id
and then creating and mounting the “zpool” called “zpdrone” using the command

zpool create zpdrone disk/by-id/\

What I actually did

Unfortunately for me, I hadn’t quite understood the man page, so I went with the first thing that worked:
  zpool create zpdrone sda

I next created a ZFS file system (“drone”) and enabled compression:
  zfs create zpdrone/drone
  zfs set compression=on zpdrone/drone

I proceeded to add some files to my file system (which was mounted at /zpdrone/drone), rebooted, and verified that I could see the files.

What went wrong, and how to fix it

Shortly thereafter, I booted with an external drive attached.
I found that the external drive was assigned to sda, so my expansion drive was assigned to sdb, and consequently my zpool didn’t get mounted; this gave me the opportunity to learn how to untangle the mess that I had caused.  As it turns out, this was not hard to do.

The first step was to “export” the pool (roughly equivalent to sync’ing and unmounting it):
  zpool export zpdrone

The second step was to “import” the pool using the disk ID and pool ID.  I obtained the pool ID (269414010796639802) from the output of
  zpool import
and then told zpool to import it by searching under
for a zpool with the zpool ID

zpool import -d /dev/disk/by-id \

Once again, my file system was mounted at

Subsequently, I could boot with an attached external hard drive, and, as expected,
  zpool status
showed the disk ID for the pool

Programming R efficiently

I have begun a search for guidance on programming R efficiently.
I hope that I can revise this once I have posted it …

Here are the first URLs that I have found on the subject:


Off-topic but perhaps useful

Pyinotify – Linux filesystem watcher

from the Pyinotify wiki

Pyinotify is a Python module for monitoring filesystems changes. Pyinotify relies on a Linux Kernel feature (merged in kernel 2.6.13) called inotify. inotify is an event-driven notifier, its notifications are exported from kernel space to user space through three system calls. pyinotify binds these system calls and provides an implementation on top of them offering a generic and abstract way to manipulate those functionalities.

See the tutorial