Author Archives: Hai

About Hai

I am a software engineer who spend most of my time coding test automation, mostly in Python. I love the Mac and Linux.

Add Unique Directory to the PATH Environment Variable

The Problem

My .bashrc is full of lines like this one:

export PATH=$PATH:$HOME/bin

There are a couple of problems with this approach: First, the directory $HOME/bin might not exist (I share my .bashrc among my different computers so chances are not all of them have the same directory structure). Second, the directory might already be in the path.

What I need is a function which “adds the directory in question to the PATH if and only if a) the directory exists and b) if it is not already in the path. That brings me to write the bash function I called append_if_exists.

Continue reading

Quickly Set up Python Virtual Environment

I often need to quickly create a Python virtual environment to try out ideas. Normally, I create a temporary directory, create a virtual environment, then install the required packages. While these steps do not take long to finish, it helps to streamline the steps.

Continue reading

My BSD Experience

I have been using Linux for a long time. Before that, I used Unix and BSD at school. In my heart BSD is something I want to explore and turn into my daily driver.

That is why in 2011, I tried out FreeBSD, OpenBSD, NetBSD, then PC-BSD. Sadly, I kept running into problems which prevent me to use BSD for my daily tasks. Did I mention that I am a software engineer? That means the task of setting up and tweaking an operating system should not be too hard for me. However, for BSD, I gave up after a couple of days.

Continue reading

Synchronize Directories in Bash Shell

The Problem

I often work in a Linux environment with multiple windows open in a tmux session. One of the pattern I see often is the need to navigate to the same directory for multiple windows. For example, in window 1, I navigated to a directory deep within my project:

$ cd ~/long/path/to/my/directory

Then, on window 2, I want to navigate to the same directory. This often involves either copy the command from window 1 and paste into window 2, or retype the same command again. There must be a better way.

Continue reading

Customize Jupyter QtConsole Font

Introduction

I am using the Jupyter QtConsole all the time to try out Python ideas. One thing I noticed that on my Windows 10 system, the font looks beautiful whereas on my macOS, it looks quite ugly so I am setting out to fix it.

About Jupyter QtConsole

Jupyter QtConsole comes with Anaconda installation.

Before Customization

By default, the console uses the Monaco font, which I don’t like:
before

Customize it

From the terminal, I issued the following command:

jupyter qtconsole --generate-config

After that, a configuration file was created in ~/.jupyter/jupyter_qtconsole_config.py.bak. Next, I edited it using my favorite editor and made the following changes:

c.JupyterConsoleApp.confirm_exit = False
c.JupyterQtConsoleApp.display_banner = False
c.ConsoleWidget.console_height = 60
c.ConsoleWidget.console_width = 120
c.ConsoleWidget.font_family = 'Inconsolata'

The visual change comes from the last line. At this point, my console looks like this:

before

Overall, I like the font Inconsolata much better than Monaco.

A Simple Directory Walker with Filter

The Problem

In my previous post, I presented a simple directory walker which solved some of my annoyances. That directory walker is not not perfect. There are times when I want to filter out the files:

for path_name in dirwalker('/path/to/dir'):
   if some_condition(path_name):
        pass  # Do something 

The Use Cases

In this case, I want to process the files only if some condition is true. I would be nice if we can tell dirwalker to return only the files that match our condition:

from dirwalker import dirwalker, include, exclude

# Only process *.xml files
for path_name in dirwalker('.', include('*.xml')):
   print path_name

# Process all but *.obj, *.bak
for path_name in dirwalker('.', exclude('*.obj', '*.bak')):
   print path_name

# Create my own predicate: process only empty files
import os
def is_empty(path_name):
   stat = os.stat(path_name)
   return stat.st_size == 0
for path_name is dirwalker('.', is_empty):
   print path_name

The Solution

The implementation of the new dirwalker is:

from fnmatch import fnmatch
import os

def exclude(*patterns):
   """A predicate which excludes any file that matches a pattern """
   def predicate(filename):
       return not any(fnmatch(filename, pattern) for pattern in patterns)
   return predicate

def include(*patterns):
   """ A predicate which includes only files that match a list of patterns """
   def predicate(filename):
       return any(fnmatch(filename, pattern) for pattern in patterns)
   return predicate

def dirwalker(root, predicate=None):
   """ Recursively walk a directory and yield the path names """
   for dirpath, dirnames, filenames in os.walk(root):
       for filename in filenames:
           fullpath = os.path.join(dirpath, filename)
           if predicate is None or predicate(filename):
               yield fullpath

Discussion

The new dirwalker takes in an additional parameter: a predicate which returns True for those files we want to process and False otherwise. To maintain backward compatibility, the predicate is default to None which means dirwalker will yield every file it found.

I also created two predicates creators, include and exclude, which create appropriate predicates. As you can see in the usage, it is easy to create a custom predicate if the built-in ones do not work for your purposes. Here are a few suggestions for predicates:

  • Files that are read-only
  • Files that are larger than a certain threshold
  • Files that have been modified within a time frame
  • Files that are symbolic links
  • Black lists and white lists

Conclusion

The dirwalker is now more powerful, thanks to the added functionality. At the same time, it is still simple to use.