Miller: Command Line CSV File Processing

Often times the lowest common denominator for integrating two systems is a CSV file. Usually I would resort to scripts for processing these CSV files. However, for certain simple operations scripting could be overkill. A battle tested command line would be much more appropriate than writing a possibly bug ridden one off script.

Miller is a command line tool for CSV file processing. It can replace Python scripts for common data transformation tasks such as:

  • Change file delimiters
  • Converting between CSV variants
  • Adding/Removing column
  • Calculating Statistics
  • Sorting
  • Remove records based on some criteria

Python in the Enterprise Part 2: Static Type Checking

Due to the size of the code bases involved and often the lack of time to implement unit test, static typing is a must-have when developing enterprise systems.

Thankfully, Python now has support for optional static type checking in the form of the mypy static type checker, and the addition of type annotations to the Python language.

In this article, I’m going to cover the features of mypy and Python type annotations that I use the most.

Long Running Business Logic in Plain Old Code Part 2

In the previous post, we went through the problem domain that we would be implementing - a loyalty system. We also looked at implementing various Temporal components for our problem domain - workers, workflows and activities.

In this post we’ll look at how we can use signal methods and query methods to implement the “earning points” portion of the loyalty system.

Long Running Business Logic in Plain Old Code Part 1

Usually, when programmers are tasked with programming a long running piece of business logic (e.g. subscriptions, gamification, marketing campaigns, any customer journey) they will reach for familiar tools such as cron, message queues and manual state management with their db of choice.

In this series of posts I want to show an alternative approach - implementing long running business logic as a single long running function. Traditionally, we cannot implement anything long running as a single function because processes and machines fail and the function’s state is in volatile memory. However, what if there was a tool that allowed us to code in such a way that we could treat memory as persistent and reliable (Smalltalk programmers might be familiar with this concept)