Mark's Blog – Page 3 – The wonderful world of software development

September 30, 2020

Flushing your DNS cache on Ubuntu 20.04

Often enough I run into this little problem: I make a DNS change and want to see the results. When I check for the changes though it looks as if nothing has changed, and sometimes this is caused by my system’s DNS cache.

Clearing the cache on Ubuntu 20.04 can be done with the following command: sudo systemd-resolve --flush-caches, you can verify that the caches have been emptied by following that up with the command systemd-resolve --statistics which then gives an output like shown below:

DNSSEC supported by current servers: yes

Transactions               
Current Transactions: 0    
  Total Transactions: 21870
                           
Cache                      
  Current Cache Size: 0    
          Cache Hits: 6722 
        Cache Misses: 15158
                           
DNSSEC Verdicts            
              Secure: 7783 
            Insecure: 21703
               Bogus: 0    
       Indeterminate: 0

The text and numbers will vary per setup, the “Current Cache Size” should however display a value of 0 to indicate your cache has been cleard.

If you are running dnsmasq next to the standard systemd-resolve like I do, restarting dnsmasq (with sudo systemctl restart dnsmasq) will ensure its caches are cleared to (otherwise you may still end up getting cached results).

And that’s all that I have for this very short post. If you have feedback on this article or have questions based on its contents then feel free to reach out to me on Twitter or through e-mail.

September 27, 2020

Obtaining your external IP address from the command-line

Sometimes you need to know what your external IP address is, for example when you need to make use of a protected service where they need to (temporarily) add your IP address to their firewall’s allow list to allow you to connect to said service.

If you have a broadband connection at home it is possible that you have a fixed external IP address and that you already know what it is. If you are however visiting elsewhere you likely do not know what it is, a quick solution in such cases is to open your browser and use a site such as https://www.whatismyip.com/ to see what your external IP address is. But what if you need to know this on a Unix-like server that only offers a command-line interface (or if you simply prefer to use your shell)?

September 24, 2020

Using dnsmasq to run a local TLD

If you work on a lot of different projects you likely have a lot of different entries in your hosts file to point fake domains to localhost or one or more Vagrant boxes. This can be a bit of a pain to manage. If you’re running Linux, BSD or Mac OS then this article offers an option that might ease this problem a bit: dnsmasq.

In this article I will describe how to setup dnsmasq to run your own local (fake) top level domain for which all domains will route to localhost (or another IP address of choice, to point to a Vagrant box for example). While writing this article I used Ubuntu 20.04, if you are running another operating system that can run dnsmasq you may have to do things slightly differently… in that case: be ready to consult some manuals.

September 20, 2020September 24, 2020

Using sed (the non-interactive command line text editor)

If you go to the sed webpage it tells you that it is a non-interactive command line text editor. Unlike interactive editors you do not spend time in sed, instead you invoke sed with a set of command line parameters and it directly outputs or writes its result.

I have often used it in Vagrant provisioning scripts to tweak configuration files, you can however also use it to extract information from text files (such as log files) and convert that into a different format (such as CSV). In this article I will demonstrate how you can use sed to do some of these things to give you some insight into it’s utility.

September 13, 2020September 24, 2020

Consuming a REST API with the Ruby standard library

Making use of REST APIs is a commonplace task for many a software developer. When working with Ruby there are quite a few RubyGems that you can use to aid you in this endeavour. Faraday, httpclient and HTTParty are examples just to name a few. If you however do not want to add dependencies to your project you can also make us of Ruby’s standard library, which is what I will be going through in this article.

August 26, 2020

Vim tip: Visual block editing

Vim is my favourite editor, it has been for many years. Over 10 years ago, in May 2010, I wrote a blog post about visual block editing in Vim. Much to my surprise that article is still read by people every month to this very day. Since its still relevant I have updated it and placed it here on my current blog.

This article assumes that you are familiar with the basics of Vim. If you are not yet familiar with vim then the vimtutor is a good place to start, you run it by executing “vimtutor” instead of “vim”. With that said, on to visual block editing!

August 23, 2020

Using SQL as an every day tool

Every now and then I find myself in the situation where I need to analyze data coming varying sources, be it in the form of files (spreadsheets, CSV, etc.) or data spread across different servers and/or database platforms. The way that I often see people handle this is by loading the data into a spreadsheet (using Excel, LibreOffice or an alternative) and then start filtering and fidgeting away with spreadsheet formulas and functions, creating temporary columns and/or copying and pasting things around until they get their desired result. When you only have to examine a single table of data this is not much of an issue, it may in fact be the best option to start with: its low-barrier and using column filters is pretty intuitive in most spreadsheet software that I’ve used.

When your searching and filtering needs become more complicated and/or involve multiple tables of data that you want to combine in one way or another things tend to get a bit hairy (at least for me) and I personally prefer using SQL instead. In this article I will give an in-depth explanation on how you can do that using a contrived example, if you are experienced with SQL, command line tools and the shell you can probably skim large parts of this article, if most of this is new to you it hopefully contains enough detail for you to follow along.

August 5, 2020August 17, 2020

Tmux and SSH agent forwarding

When you normally connect to a machine using SSH and start a tmux session SSH agent forwarding (if you have that setup) will work normally. When you however detach your session and re-attach it later you will find that it no longer does, this is because the environment variables in the shell inside the tmux session refers to the same SSH_AUTH_SOCK that it did when you originally connected.

Recent enough versions of tmux have something that can fix this for you, if you execute the command tmux show-environment -s you will see a bunch of shell commands to set a series of environment variables. If you execute those it will update the SSH information in the shell and agent forwarding will work once again, a shortcut to doing this in a single line would be eval "$(tmux show-environment -s)".

If you have your own dotfiles on the destination machine you can make this happen automatically. For bash users there’s the PROMP_COMMAND environment variable (see https://www.johntobin.ie/blog/updating_environment_variables_from_tmux/), for zsh users (like myself) you can do this by specifying a precmd hook.

If you add the following to your .zshrc (or a file you source inside it) you can set this up:

function update_environment_from_tmux() {
  if [ -n "${TMUX}" ]; then
    eval "$(tmux show-environment -s)"
  fi
}

add-zsh-hook precmd update_environment_from_tmux

The above will ensure that the update_environment_from_tmux function is executed before each command that you execute, if you re-attach to a tmux session that is using zsh with the above loaded and execute an SSH related command (git pull, scp, sftp, ssh, etc.) it will first automatically update the SSH environment variables and then execute the command making it work seamlessly.

The update_environment_from_tmux function only evals the tmux show-environment -s command if you are inside a tmux session (which will set the TMUX environment variable).

August 2, 2020August 2, 2020

Hello world!

Here we are, there and back again. After the longest of times I have brought back a WordPress blog for myself to write on about software development. Now time will have to tell if I have anything interesting to write here…