Julia Evans


SQL queries don't start with SELECT

Okay, obviously many SQL queries do start with SELECT (and actually this post is only about SELECT queries, not INSERTs or anything).

But! Yesterday I was working on an explanation of window functions, and I found myself googling “can you filter based on the result of a window function”. As in – can you filter the result of a window function in a WHERE or HAVING or something?

Eventually I concluded “window functions must run after WHERE and GROUP BY happen, so you can’t do it”. But this led me to a bigger question – what order do SQL queries actually run in?.

This was something that I felt like I knew intuitively (“I’ve written at least 10,000 SQL queries, some of them were really complicated! I must know this!“) but I struggled to actually articulate what the order was.

SQL queries happen in this order

I looked up the order, and here it is! (SELECT isn’t the first thing, it’s like the 5th thing!) (here it is in a tweet).

(I really want to find a more accurate way of phrasing this than “sql queries happen/run in this order” but I haven’t figured it out yet)

In a non-image format, the order is:

questions this diagram helps you answer

This diagram is about the semantics of SQL queries – it lets you reason through what a given query will return and answers questions like:

Database engines don’t actually literally run queries in this order because they implement a bunch of optimizations to make queries run faster – we’ll get to that a little later in the post.


confounding factor: column aliases

Someone on Twitter pointed out that many SQL implementations let you use the syntax:

SELECT CONCAT(first_name, ' ', last_name) AS full_name, count(*)
FROM table
GROUP BY full_name

This query makes it look like GROUP BY happens after SELECT even though GROUP BY is first, because the GROUP BY references an alias from the SELECT. But it’s not actually necessary for the GROUP BY to run after the SELECT for this to work – the database engine can just rewrite the query as

SELECT CONCAT(first_name, ' ', last_name) AS full_name, count(*)
FROM table
GROUP BY CONCAT(first_name, ' ', last_name)

and run the GROUP BY first.

Your database engine also definitely does a bunch of checks to make sure that what you put in SELECT and GROUP BY makes sense together before it even starts to run the query, so it has to look at the query as a whole anyway before it starts to come up with an execution plan.

queries aren’t actually run in this order (optimizations!)

Database engines in practice don’t actually run queries by joining, and then filtering, and then grouping, because they implement a bunch of optimizations reorder things to make the query run faster as long as reordering things won’t change the results of the query.

One simple example of a reason why need to run queries in a different order to make them fast is that in this query:

owners LEFT JOIN cats ON owners.id = cats.owner
WHERE cats.name = 'mr darcy'

it would be silly to do the whole left join and match up all the rows in the 2 tables if you just need to look up the 3 cats named ‘mr darcy’ – it’s way faster to do some filtering first for cats named ‘mr darcy’. And in this case filtering first doesn’t change the results of the query!

There are lots of other optimizations that database engines implement in practice that might make them run queries in a different order but there’s no room for that and honestly it’s not something I’m an expert on.

LINQ starts queries with FROM

LINQ (a querying syntax in C# and VB.NET) uses the order FROM ... WHERE ... SELECT. Here’s an example of a LINQ query:

var teenAgerStudent = from s in studentList
                      where s.Age > 12 && s.Age < 20
                      select s;

pandas (my favourite data wrangling tool) also basically works like this, though you don’t need to use this exact order – I’ll often write pandas code like this:

df = thing1.join(thing2)      # like a JOIN
df = df[df.created_at > 1000] # like a WHERE
df = df.groupby('something', num_yes = ('yes', 'sum')) # like a GROUP BY
df = df[df.num_yes > 2]       # like a HAVING, filtering on the result of a GROUP BY
df = df[['num_yes', 'something1', 'something']] # pick the columns I want to display, like a SELECT
df.sort_values('sometthing', ascending=True)[:30] # ORDER BY and LIMIT

This isn’t because pandas is imposing any specific rule on how you have to write your code, though. It’s just that it often makes sense to write code in the order JOIN / WHERE / GROUP BY / HAVING. (I’ll often put a WHERE first to improve performance though, and I think most database engines will also do a WHERE first in practice)

dplyr in R also lets you use a different syntax for querying SQL databases like Postgres, MySQL and SQLite, which is also in a more logical order.

I was really surprised that I didn’t know this

I’m writing a blog post about this because when I found out the order I was SO SURPRISED that I’d never seen it written down that way before – it explains basically everything that I knew intuitively about why some queries are allowed and others aren’t. So I wanted to write it down in the hopes that it will help other people also understand how to write SQL queries.


Zine revenue for 2019

I occasionally get questions like “Can you share what you’ve learned about running a business?” The most surprising thing I’ve learned is that it’s possible to make money by teaching people computer things on the internet, so I want to make that a little more concrete by sharing the revenue from the zine business so far in 2019. Here’s a graph of revenue by month (the last month is September 2019):

This adds up to $87,858 USD for 2019 so far, which (depending on what I release in the rest of this year) is on track to be similar to revenue for 2018 ($101,558).

Until quite recently I’d been writing zines in my spare time, and now I’m taking a year to focus on it.

how $30,000 for September breaks down

The most obvious thing in that monthly revenue graph above is that 2 months (September and March) have way more revenue than all the others. This is because I released new zines (Bite Size Networking and HTTP: Learn your browser’s language) in those months.

Here’s how the $30,000 for September breaks down:

This September was the month with the most sales ever, which is mostly because of individual humans who find the zines useful (thank you!!).


The main expenses are paying illustrators and an accountant, a mailing list, and various books I buy to learn how to do things better. They probably come out to about 10% of revenue or so, and then there are taxes after that.

giving away free copies has been great

With the HTTP zine, like many of my previous zines, I’ve been giving away one free copy for every copy that people buy, so that people can get it even if $12 is hard for them to afford. (if you can’t afford $12, here’s the link, there are about 70 available as I’m writing this). I’m pretty happy with this setup – we’ve given away 1358 copies so far. (I think of this as kind of a “sales” statistic too)

I think I want to automate the system to give away free copies a bit more soon (like by automatically updating the number of free zines available using the Gumroad API instead of periodically doing it manually).

hopefully this is a useful data point!

Writing about money on the internet is weird, so this will probably be the first and last zine revenue post, but I’m writing it down in the hopes that it’s a useful data point for others. I thought for a long time that you could only really make money from writing on the internet with ads or sponsorships, but it’s not true!

The goal of this isn’t to say “you should run a business” or anything, just that this is a thing that’s possible in the world and that many developers do really value good educational materials and are happy to pay for them (if you’re one of those people, thank you!)


Notes on building SQL exercises

In the last couple of weeks I’ve been working on some interactive SQL exercises to help people get better at writing SQL queries. This is a pretty new thing for me so I thought I’d write a few notes about my process so far!

why SQL is exciting: distributed SQL engines

To me the reason why SQL is exciting is that a lot of companies are storing their data in distributed SQL databases (Google BigQuery, Amazon Redshift, Spark SQL, Presto, etc) that let you run a complicated query across a billion rows pretty quickly! They’re fast partly because they’re designed to run your query across possibly tens or hundreds of computers.

At my last job I wrote thousands of SQL queries to do data analysis while I was working on the machine learning team, mostly ad hoc queries to answer questions I had about our data. I learned a lot of fun tricks to make them faster / easier to write and I’ve never really talked about it!

So I think SQL is a really nice way to go from “I have this sort of complicated question about billions of rows of data” to “ok, that’s the answer, great, I can move on”.

why write exercises: knowledge != skills

This is the first time I’m really trying in earnest to write exercises to teach something, instead of just explanations of the thing. The reason I’m doing this is that I read Design for how people learn by Julie Dirksen and she makes the point that knowledge is different from skills.

She defines a “skill” as “something you have to practice”. And SQL is definitely something that you have to practice if you want to learn it! So I thought – SQL is a relatively simple skill (as programming/programming-adjacent skills go!), maybe I can make something interactive and relatively simple to help people improve their SQL skills!

It’s also, well, a challenge, and I like trying things I haven’t tried before.

how I’m doing it: start with a challenge

I started out doing these SQL exercises in kind of the obvious way: start out with easy exercises, and then make them harder and harder over time to introduce new concepts. But when I watched people trying it out, I noticed a problem – a lot of people already know some SQL, and sometimes they would go through all the exercises without learning anything at all! That’s no fun!

So I came up with a different structure for each section of the SQL exercises:

  1. Start with a “challenge” that tests the skill the section is trying to teach.
  2. If the challenge is too hard, move on to a bunch of easier exercises that teach you the skills you need to solve the challenge.

Since showing is easier than explaining: here’s a draft of a page teaching GROUP BY. Here’s a screenshot of what the initial “challenge” for basic group by looks like:

I think that challenge in particular isn’t very good yet (I have a lot of work to do!) but that’s the idea.

how I’m getting feedback: anonymously track responses

Early on I also realized that I needed to get feedback about which challenges people were finding hard / easy. Every time someone runs a query, I track

I’ve already learned a lot from this, for example:

So basically (in addition to making more exercises) I think I need to spend more time cataloguing where/how people are getting stuck in practice and helping make sure fewer people get stuck.

the tech stack

To build this, I’m using:

I also bought the Refactoring UI book to try to improve my web design skills a tiny bit. I think it’s helped a little so far.

Vue components let me really easily add new challenges/exercises to a page like this:

   title='Count the number of different cat owners'
   You can use <code>COUNT(DISTINCT column)</code> to count distinct values of a column. (you can also do <code>SUM(DISTINCT column)</code> or <code>AVG(DISTINCT column)</code> but I'm not sure why that would be useful.
   answer= "
   SELECT count(distinct(owner)) AS num_owners
   from cats

the goal: make something that’s worth $100 or so

What I’m working towards is making exercises & challenges that would help someone with beginner/intermediate SQL skills improve their SQL fluency enough that it’d easily be worth $100 to them. We’ll see if I can get there! I don’t know whether I’ll price it at $100, but that’s my goal for how useful it should be.

The person I have in mind is sort of (as usual) myself 6 years ago, when I’d heard of SQL and could write a basic query but if you gave me a table of VERY INTERESTING DATA I couldn’t really effectively use SQL to answer the questions I had about it.


Taking a year to explain computer things

I’ve been working on explaining computer things I’m learning on this blog for 6 years. I wrote one of my first posts, what does a shell even do? on Sept 30, 2013. Since then, I’ve written 11 zines, 370,000 words on this blog, and given 20 or so talks. So it seems like I like explaining things a lot.

tl;dr: I’m going to work on explaining computer things for a year

Here’s the exciting news: I left my job a month ago and my plan is to spend the next year working on explaining computer things!

As for why I’m doing this – I was talking through some reasons with my friend Mat last night and he said “well, sometimes there are things you just feel compelled to do”. I think that’s all there is to it :)

what does “explain computer things” mean?

I’m planning to:

  1. write some more zines (maybe I can write 10 zines in a year? we’ll see! I want to tackle both general-interest and slightly more niche topics, we’ll see what happens).
  2. work on some more interactive ways to learn things. I learn things best by trying things out and breaking them, so I want to see if I can facilitate that a little bit for other people. I started a project around this in May which has been on the backburner for a bit but which I’m excited about. Hopefully I’ll release it soon and then you can try it out and tell me what you think!

I say “a year” because I think I have at least a year’s worth of ideas and I can’t predict how I’ll feel after doing this for a year.

how: run a business

I started a corporation almost exactly a year ago, and I’m planning to keep running my explaining-things efforts as a business. This business has been making more than I made in my first programming job (that is, definitely enough money to live on!), which has been really surprising and great (thank you!).

some parameters of the business:

It’s been pretty interesting to learn more about running a small business and so far I like it more than I thought I would. (except for taxes, which I like exactly as much as I thought I would)

that’s all!

I’m excited to keep making explanations of computer things and to have more time to do it. This blog might change a bit away from “here’s what I’m learning at work these days” and towards “here are attempts at explaining things that I mostly already know”. It’ll be different! We’ll see how it goes!


New zine: HTTP: Learn your browser's language!

Hello! I’ve released a new zine! It’s called “HTTP: Learn your browsers language!”

You can get it for $12 at https://gum.co/http-zine. If you buy it, you’ll get a PDF that you can either read on your computer or print out.

Here’s the cover and table of contents:

why http?

I got the idea for this zine from talking to Marco Rogers – he mentioned that he thought that new web developers / mobile developers would really benefit from understanding the fundamentals of HTTP better, I thought “OOH I LOVE TALKING ABOUT HTTP”, wrote a few pages about HTTP, saw they were helping people, and decided to write a whole zine about HTTP.

HTTP is important to understand because it runs the entire web – if you understand how HTTP requests and responses work, then it makes it WAY EASIER to debug why your web application isn’t working properly. Caching, cookies, and a lot of web security are implemented using HTTP headers, so if you don’t understand HTTP headers those things seem kind of like impenetrable magic. But actually the HTTP protocol is fundamentally pretty simple – there are a lot of complicated details but the basics are pretty easy to understand.

So the goal of this zine is to teach you the basics so you can easily look up and understand the details when you need them.

what it looks like printed out

All of my zines are best printed out (though you get a PDF you can read on your computer too!), so here are a couple of pictures of what it looks like when printed. I always ask my illustrator to make both a black and white version and a colour version of the cover so that it looks great when printed on a black and white printer.

(if you click on that “same origin policy” image, you can make it bigger)

The zine comes with 4 print PDFs in addition to a PDF you can just read on your computer/phone:

zines for your team

You can also buy this zine for your team members at work to help them learn HTTP!

I’ve been trying to get the pricing right for this for a while – I used to do it based on size of company, but that didn’t seem quite right because sometimes people would want to buy the zine for a small team at a big company. So I’ve switched to pricing based on the number of copies you want to distribute at your company.

Here’s the link: zines for your team!.

the tweets

When I started writing zines, I would just sit down, write down the things I thought were important, and be done with it.

In the last year and a half or so I’ve taken a different approach – instead of writing everything and then releasing it, instead I write a page at a time, post the page to Twitter, and then improve it and decide what page to write next based on the questions/comments I get on Twitter. If someone replies to the tweet and asks a question that shows that what I wrote is unclear, I can improve it! (I love getting replies on twitter asking clarifiying questions!).

Here are all the initial drafts of the pages I wrote and posted on twitter, in chronological order. Some of the pages didn’t make it into the zine at all, and I needed to do a lot of editing at the end to figure out the right order and make them all work coherently together in a zine instead of being a bunch of independent tweets.

Writing zines one tweet at a time has been really fun. I think it improves the quality a lot, because I get a ton of feedback along the way that I can use to make the zine better. There are also some experimental 45 second tiny videos in that list, which are definitely not part of the zine, but which were fun to make and which I might expand on in the future.


One tiny easter egg in the zine: I have a lot of examples of HTTP requests, and I wasn’t sure for a long time what domain I should use for the examples. I used example.com a bunch, and google.com and twitter.com sometimes, but none of those felt quite right.

A couple of days before publishing the zine I finally had an epiphany – my example on the cover was requesting a picture of a cat, so I registered https://examplecat.com which just has a single picture of a cat. It also has an ASCII cat if you’re browsing in your terminal.

$ curl https://examplecat.com/cat.txt  -i
HTTP/2 200 
accept-ranges: bytes
cache-control: public, max-age=0, must-revalidate
content-length: 33
content-type: text/plain; charset=UTF-8
date: Thu, 12 Sep 2019 16:48:16 GMT
etag: "ac5affa59f554a1440043537ae973790-ssl"
strict-transport-security: max-age=31536000
age: 5
server: Netlify
x-nf-request-id: c5060abc-0399-4b44-94bf-c481e22c2b50-1772748

\    /\
 )  ( ')
(  /  )

more zines at wizardzines.com

If you’re interested in the idea of programming zines and haven’t seen my zines before, I have a bunch more at https://wizardzines.com. There are 6 free zines there:

next zine: not sure yet!

Some things I’m considering for the next zine:


How to put an HTML page on the internet

One thing I love about the internet is that it’s SO EASY to put static HTML websites on the internet. Someone asked me today how to do it, so I thought I’d write down how really quickly!

just an HTML page

All of my sites are just static HTML and CSS. My web design skills are relatively minimal (https://wizardzines.com is the most complicated site I’ve developed on my own), so keeping all my internet sites relatively simple means that I have some hope of being able to make changes / fix things without spending a billion hours on it.

So we’re going to take as minimal of an approach as possible in this blog post – just one HTML page.

the HTML page

The website we’re going to put on the internet is just one file, called index.html. You can find it at https://github.com/jvns/website-example, which is a Github repository with exactly one file in it.

The HTML file has some CSS in it to make it look a little less boring, which is partly copied from https://example.com.

how to put the HTML page on the internet

Here are the steps:

  1. sign up for a Neocities account
  2. copy the index.html into the index.html in your neocities site
  3. done

The index.html page above is on the internet at julia-example-website.neocities.com, if you view source you’ll see that it’s the same HTML as in the github repo.

I think this is probably the simplest way to put an HTML page on the internet (and it’s a throwback to Geocities, which is how I made my first website in 2003) :). I also like that Neocities (like glitch, which I also love) is about experimentation and learning and having fun..

other options

This is definitely not the only easy way – Github pages and Gitlab pages and Netlify will all automatically publish a site when you push to a Git repository, and they’re all very easy to use (just connect them to your github repository and you’re done). I personally use the Git repository approach because not having things in Git makes me nervous – I like to know what changes to my website I’m actually pushing. But I think if you just want to put an HTML site on the internet for the first time and play around with HTML/CSS, Neocities is a really nice way to do it.

If you want to actually use your website for a Real Thing and not just to play around you probably want to buy a domain and link it to your website so that you can change hosting providers in the future, but that is a bit less simple.

this is a good possible jumping off point for learning HTML

If you are a person who is comfortable editing files in a Git repository but wants to practice HTML/CSS, I think this is a fun way to put a website on the internet and play around! I really like the simplicity of it – there’s literally just one file, so there’s no fancy extra magic to get in the way of understanding what’s going on.

There are also a bunch of ways to complicate/extend this, like this blog is actually generated with Hugo which generates a bunch of HTML files which then go on the internet, but it’s always nice to start with the basics.


How to write zines with simple tools

People often ask me what tools I use to write my zines (the answer is here). Answering this question as written has always felt slightly off to me, though, and I couldn’t figure out why for a long time.

I finally realized last week that instead of “what tools do you use to write zines?” some people may have actually wanted to know “how can I do this myself?”! And “buy a $500 iPad” is not a terribly useful answer to that question – it’s not how I got started, iPads are kind of a weird fancy way to write zines, and most people don’t have them.

So this blog post is about more traditional (and easier to get started with) ways to write zines.

We’re going to start out by talking about the mechanics of how to write the zine, and then talk about how to assemble it into a booklet.

Way 1: Write it on paper

This is how I made my first zine (spying on your programs with strace) which you can see here: https://jvns.ca/strace-zine-unfolded.pdf.

Here’s an example of a page I drew on paper this morning pretty quickly. It looks kind of bad because I scanned it with my phone, but if you use a real scanner (like I did with the strace PDF above), the scanned version comes out better.

Way 2: Use a Google doc

The next option is to use a Google doc (or whatever other word processor you prefer). Here’s the Google doc I wrote for the below image, and here’s what it looks like:

They key thing about this Google doc approach is to apply some “less is more”. It’s intended to be printed as part of a booklet on half a sheet of letter paper, which means everything needs to be twice as big for it to look good.

Way 3: Use an iPad

This is what I do (use the Notability app on iPad). I’m not going to talk about this method much because this post is about using more readily available tools.

Way 4: Use a single sheet of paper

This is a subset of “Write it on paper” – the Wikibooks page on zine making has a great guide that shows how to write out a tiny zine on 1 piece of paper and then fold it up to make a little booklet. Here are the pictures of the steps from the Wikibooks page:

Sumana Harihareswara’s Playing with python zine is a nice example of a zine that’s intended to be folded up in that way.

Way 5: Adobe Illustrator

I’ve never used Adobe Illustrator so I’m not going to pretend that I know anything about it or put together an example using it, but I hear it’s a way people do book layout.

booklets: the photocopier method

So you’ve written a bunch of pages and want to assemble them into a booklet. One way to do this (and what I did for my first zine about strace!) is the photocopier method. There’s a great guide by Julia Gfrörer in this tweet, which I’m going to reproduce here:

That explanation is excellent and I don’t have anything to add. I did it that way and it worked great.

If you want to buy a print copy of that how-to-make-zines zine from Thruban Press, you can get it here on Etsy.

booklets: the computer method

If you’ve made your zine in Google Docs or in another computery way, you probably want a more computery way of assembling the pages into a booklet.

what I use: pdflatex

I do this using the pdfpages LaTeX extension. This sounds complicated but it’s not really, you don’t need to learn latex or anything. You just need to have pdflatex on your system, which is a sudo apt install texlive-base away on Ubuntu. The steps are:

  1. Get a PDF with the pages from your zine (pages need to be a multiple of 4)
  2. Get the latex file from this gist
  3. Replace /home/bork/http-zine.pdf with the path to your PDF and 1-28 with 1-however many pages are in your zine.
  4. run pdflatex formatted-zine.tex
  5. Tweak the parameters until it looks the way you want. The documentation for the pdfpages package is here

I like using this relatively complicated method because there are always small tweaks I want to make like “oh, the right margin is too big, crop it a little bit” and the pdfpages package has tons of options that let me make those tweaks.

other methods

  1. On Linux you can use the pdfjam bash script, which is just a wrapper around the pdfpages latex package. This is what I used to do but today I find it simpler to use the pdfpages latex package directly.
  2. There’s a program called Booklet Creator for Mac and Windows that @mrfb uses. It looks pretty simple to use.
  3. If you convert your PDF to a ps file (with pdf2ps for instance), psnup can do this. I tried cat file.ps | psbook | psnup -2 > booklet.ps and it worked, though the resulting PDFs are a little slow to load in my PDF viewer for some reason.
  4. there are probably a ton more ways to do this, if you know more let me know

making zines is easy and low tech

That’s all! I mostly wanted to explain that zines are an easy low tech thing to do and if you think making them sounds fun, you definitely 100% do not need to use any fancy expensive tools to do it, you can literally use some sheets of paper, a Sharpie, a pen, and spend $3 at your local print shop to use the photocopier.


summary of the resources I linked to:


git exercises: navigate a repository

I think the curl exercises the other day went well, so today I woke up and wanted to try writing some Git exercises. Git is a big thing to learn, probably too big to learn in a few hours, so my first idea for how to break it down was by starting by navigating a repository.

I was originally going to use a toy test repository, but then I thought – why not a real repository? That’s way more fun! So we’re going to navigate the repository for the Ruby programming language. You don’t need to know any C to do this exercise, it’s just about getting comfortable with looking at how files in a repository change over time.

clone the repository

To get started, clone the repository:

git clone https://github.com/ruby/ruby

The big different thing about this repository (as compared to most of the repositories you’ll work with in real life) is that it doesn’t have branches, but it DOES have lots of tags, which are similar to branches in that they’re both just pointers to a commit. So we’ll do exercises with tags instead of branches. The way you change tags and branches are very different, but the way you look at tags and branches is exactly the same.

a git SHA always refers to the same code

The most important thing to keep in mind while doing these exercises is that a git SHA like 9e3d9a2a009d2a0281802a84e1c5cc1c887edc71 always refers to the same code, as explained in this page. This page is from a zine I wrote with Katie Sylor-Miller called Oh shit, git!. (She also has a great site called https://ohshitgit.com/ that inspired the zine).

We’ll be using git SHAs really heavily in the exercises to get you used to working with them and to help understand how they correspond to tags and branches.

git subcommands we’ll be using

All of these exercises only use 5 git subcommands:

git checkout
git log (--oneline, --author, and -S will be useful)
git diff (--stat will be useful)
git show
git status


  1. Check out matz’s commit of Ruby from 1998. The commit ID is 3db12e8b236ac8f88db8eb4690d10e4a3b8dbcd4. Find out how many lines of code Ruby was at that time.
  2. Check out the current master branch
  3. Look at the history for the file hash.c. What was the last commit ID that changed that file?
  4. Get a diff of how hash.c has changed in the last 20ish years: compare that file on the master branch to the file at commit 3db12e8b236ac8f88db8eb4690d10e4a3b8dbcd4.
  5. Find a recent commit that changed hash.c and look at the diff for that commit
  6. This repository has a bunch of tags for every Ruby release. Get a list of all the tags.
  7. Find out how many files changed between tag v1_8_6_187 and tag v1_8_6_188
  8. Find a commit (any commit) from 2015 and check it out, look at the files very briefly, then go back to the master branch.
  9. Find out what commit the tag v1_8_6_187 corresponds to.
  10. List the directory .git/refs/tags. Run cat .git/refs/tags/v1_8_6_187 to see the contents of one of those files.
  11. Find out what commit ID HEAD corresponds to right now.
  12. Find out how many commits have been made to the test/ directory
  13. Get a diff of lib/telnet.rb between the commits 65a5162550f58047974793cdc8067a970b2435c0 and 9e3d9a2a009d2a0281802a84e1c5cc1c887edc71. How many lines of that file were changed?
  14. How many commits were made between Ruby 2.5.1 and 2.5.2 (tags v2_5_1 and v2_5_3) (this one is a tiny bit tricky, there’s more than one step)
  15. How many commits were authored by matz (Ruby’s creator)?
  16. What’s the most recent commit that included the word tkutil?
  17. Check out the commit e51dca2596db9567bd4d698b18b4d300575d3881 and create a new branch that points at that commit.
  18. Run git reflog to see all the navigating of the repository you’ve done so far


curl exercises

Recently I’ve been interested in how people learn things. I was reading Kathy Sierra’s great book Badass: Making Users Awesome. It talks about the idea of deliberate practice.

The idea is that you find a small micro-skill that can be learned in maybe 3 sessions of 45 minutes, and focus on learning that micro-skill. So, as an exercise, I was trying to think of a computer skill that I thought could be learned in 3 45-minute sessions.

I thought that making HTTP requests with curl might be a skill like that, so here are some curl exercises as an experiment!

what’s curl?

curl is a command line tool for making HTTP requests. I like it because it’s an easy way to test that servers or APIs are doing what I think, but it’s a little confusing at first!

Here’s a drawing explaining curl’s most important command line arguments (which is page 6 of my Bite Size Networking zine). You can click to make it bigger.

fluency is valuable

With any command line tool, I think having fluency is really helpful. It’s really nice to be able to just type in the thing you need. For example recently I was testing out the Gumroad API and I was able to just type in:

curl https://api.gumroad.com/v2/sales \
                         -d "access_token=<SECRET>" \
                         -X GET  -d "before=2016-09-03"

and get things working from the command line.

21 curl exercises

These exercises are about understanding how to make different kinds of HTTP requests with curl. They’re a little repetitive on purpose. They exercise basically everything I do with curl.

To keep it simple, we’re going to make a lot of our requests to the same website: https://httpbin.org. httpbin is a service that accepts HTTP requests and then tells you what request you made.

  1. Request https://httpbin.org
  2. Request https://httpbin.org/anything. httpbin.org/anything will look at the request you made, parse it, and echo back to you what you requested. curl’s default is to make a GET request.
  3. Make a POST request to https://httpbin.org/anything
  4. Make a GET request to https://httpbin.org/anything, but this time add some query parameters (set value=panda).
  5. Request google’s robots.txt file (www.google.com/robots.txt)
  6. Make a GET request to https://httpbin.org/anything and set the header User-Agent: elephant.
  7. Make a DELETE request to https://httpbin.org/anything
  8. Request https://httpbin.org/anything and also get the response headers
  9. Make a POST request to https://httpbin.com/anything with the JSON body {"value": "panda"}
  10. Make the same POST request as the previous exercise, but set the Content-Type header to application/json (because POST requests need to have a content type that matches their body). Look at the json field in the response to see the difference from the previous one.
  11. Make a GET request to https://httpbin.org/anything and set the header Accept-Encoding: gzip (what happens? why?)
  12. Put a bunch of a JSON in a file and then make a POST request to https://httpbin.org/anything with the JSON in that file as the body
  13. Make a request to https://httpbin.org/image and set the header ‘Accept: image/png’. Save the output to a PNG file and open the file in an image viewer. Try the same thing with with different Accept: headers.
  14. Make a PUT request to https://httpbin.org/anything
  15. Request https://httpbin.org/image/jpeg, save it to a file, and open that file in your image editor.
  16. Request https://www.twitter.com. You’ll get an empty response. Get curl to show you the response headers too, and try to figure out why the response was empty.
  17. Make any request to https://httpbin.org/anything and just set some nonsense headers (like panda: elephant)
  18. Request https://httpbin.org/status/404 and https://httpbin.org/status/200. Request them again and get curl to show the response headers.
  19. Request https://httpbin.org/anything and set a username and password (with -u username:password)
  20. Download the Twitter homepage (https://twitter.com) in Spanish by setting the Accept-Language: es-ES header.
  21. Make a request to the Stripe API with curl. (see https://stripe.com/docs/development for how, they give you a test API key). Try making exactly the same request to https://httpbin.org/anything.


Get your work recognized: write a brag document

There’s this idea that, if you do great work at your job, people will (or should!) automatically recognize that work and reward you for it with promotions / increased pay. In practice, it’s often more complicated than that – some kinds of important work are more visible/memorable than others. It’s frustrating to have done something really important and later realize that you didn’t get rewarded for it just because the people making the decision didn’t understand or remember what you did. So I want to talk about a tactic that I and lots of people I work with have used!

This blog post isn’t just about being promoted or getting raises though. The ideas here have actually been more useful to me to help me reflect on themes in my work, what’s important to me, what I’m learning, and what I’d like to be doing differently. But they’ve definitely helped with promotions!

You can also skip to the brag document template at the end.

you don’t remember everything you did

One thing I’m always struck by when it comes to performance review time is a feeling of “wait, what did I do in the last 6 months?“. This is a kind of demoralizing feeling and it’s usually not based in reality, more in “I forgot what cool stuff I actually did”.

I invariably end up having to spend a bunch of time looking through my pull requests, tickets, launch emails, design documents, and more. I always end up finding small (and sometimes not-so-small) things that I completely forgot I did, like:

your manager doesn’t remember everything you did

And if you don’t remember everything important you did, your manager (no matter how great they are!) probably doesn’t either. And they need to explain to other people why you should be promoted or given an evaluation like “exceeds expectations” (“X’s work is so awesome!!!!” doesn’t fly).

So if your manager is going to effectively advocate for you, they need help.

here’s the tactic: write a document listing your accomplishments

The tactic is pretty simple! Instead of trying to remember everything you did with your brain, maintain a “brag document” that lists everything so you can refer to it when you get to performance review season! This is a pretty common tactic – when I started doing this I mentioned it to more experienced people and they were like “oh yeah, I’ve been doing that for a long time, it really helps”.

Where I work we call this a “brag document” but I’ve heard other names for the same concept like “hype document” or “list of stuff I did” :).

There’s a basic template for a brag document at the end of this post.

share your brag document with your manager

When I first wrote a brag document I was kind of nervous about sharing it with my manager. It felt weird to be like “hey, uh, look at all the awesome stuff I did this year, I wrote a long document listing everything”. But my manager was really thankful for it – I think his perspective was “this makes my job way easier, now I can look at the document when writing your perf review instead of trying to remember what happened”.

Giving them a document that explains your accomplishments will really help your manager advocate for you in discussions about your performance and come to any meetings they need to have prepared.

Brag documents also really help with manager transitions – if you get a new manager 3 months before an important performance review that you want to do well on, giving them a brag document outlining your most important work & its impact will help them understand what you’ve been doing even though they may not have been aware of any of your work before.

share it with your peer reviewers

Similarly, if your company does peer feedback as part of the promotion/perf process – share your brag document with your peer reviewers!! Every time someone shares their doc with me I find it SO HELPFUL with writing their review for much the same reasons it’s helpful to share it with your manager – it reminds me of all the amazing things they did, and when they list their goals in their brag document it also helps me see what areas they might be most interested in feedback on.

On some teams at work it’s a team norm to share a brag document with peer reviewers to make it easier for them.

explain the big picture

In addition to just listing accomplishments, in your brag document you can write the narrative explaining the big picture of your work. Have you been really focused on security? On building your product skills & having really good relationships with your users? On building a strong culture of code review on the team?

In my brag document, I like to do this by making a section for areas that I’ve been focused on (like “security”) and listing all the work I’ve done in that area there. This is especially good if you’re working on something fuzzy like “building a stronger culture of code review” where all the individual actions you do towards that might be relatively small and there isn’t a big shiny ship.

use your brag document to notice patterns

In the past I’ve found the brag document useful not just to hype my accomplishments, but also to reflect on the work I’ve done. Some questions it’s helped me with:

you can write it all at once or update it every 2 weeks

Many people have told me that it works best for them if they take a few minutes to update their brag document every 2 weeks ago. For me it actually works better to do a single marathon session every 6 months or every year where I look through everything I did and reflect on it all at once. Try out different approaches and see what works for you!

don’t forget to include the fuzzy work

A lot of us work on fuzzy projects that can feel hard to quantify, like:

A lot of people will leave this kind of work out because they don’t know how to explain why it’s important. But I think this kind of work is especially important to put into your brag document because it’s the most likely to fall under the radar! One way to approach this is to, for each goal:

  1. explain your goal for the work (why do you think it’s important to refactor X piece of code?)
  2. list some things you’ve done towards that goal
  3. list any effects you’ve seen of the work, even if they’re a little indirect

If you tell your coworkers this kind of work is important to you and tell them what you’ve been doing, maybe they can also give you ideas about how to do it more effectively or make the effects of that work more obvious!

encourage each other to celebrate accomplishments

One nice side effect of having a shared idea that it’s normal/good to maintain a brag document at work is that I sometimes see people encouraging each other to record & celebrate their accomplishments (“hey, you should put that in your brag doc, that was really good!”). It can be hard to see the value of your work sometimes, especially when you’re working on something hard, and an outside perspective from a friend or colleague can really help you see why what you’re doing is important.

Brag documents are good when you use them on your own to advocate for yourself, but I think they’re better as a collaborative effort to recognize where people are excelling.

Next, I want to talk about a couple of structures that we’ve used to help people recognize their accomplishments.

the brag workshop: help people list their accomplishments

The way this “brag document” practice started in the first place is that my coworker Karla and I wanted to help other women in engineering advocate for themselves more in the performance review process. The idea is that some people undersell their accomplishments more than they should, so we wanted to encourage those people to “brag” a little bit and write down what they did that was important.

We did this by running a “brag workshop” just before performance review season. The format of the workshop is like this:

Part 1: write the document: 1-2 hours. Everybody sits down with their laptop, starts looking through their pull requests, tickets they resolved, design docs, etc, and puts together a list of important things they did in the last 6 months.

Part 2: pair up and make the impact of your work clearer: 1 hour. The goal of this part is to pair up, review each other’s documents, and identify places where people haven’t bragged “enough” – maybe they worked on an extremely critical project to the company but didn’t highlight how important it was, maybe they improved test performance but didn’t say that they made the tests 3 times faster and that it improved everyone’s developer experience. It’s easy to accidentally write “I shipped $feature” and miss the follow up (“… which caused $thing to happen”). Another person reading through your document can help you catch the places where you need to clarify the impact.

biweekly brag document writing session

Another approach to helping people remember their accomplishments: my friend Dave gets some friends together every couple of weeks or so for everyone to update their brag documents. It’s a nice way for people to talk about work that they’re happy about & celebrate it a little bit, and updating your brag document as you go can be easier than trying to remember everything you did all at once at the end of the year.

These don’t have to be people in the same company or even in the same city – that group meets over video chat and has people from many different companies doing this together from Portland, Toronto, New York, and Montreal.

In general, especially if you’re someone who really cares about your work, I think it’s really positive to share your goals & accomplishments (and the things that haven’t gone so well too!) with your friends and coworkers. It makes it feel less like you’re working alone and more like everyone is supporting each other in helping them accomplish what they want.


Thanks to Karla Burnett who I worked with on spreading this idea at work, to Dave Vasilevsky for running brag doc writing sessions, to Will Larson who encouraged me to start one of these in the first place, to my manager Jay Shirley for always being encouraging & showing me that this is a useful way to work with a manager, and to Allie, Dan, Laura, Julian, Kamal, Stanley, and Vaibhav for reading a draft of this.

I’d also recommend the blog post Hype Yourself! You’re Worth It! by Aashni Shah which talks about a similar approach.

Appendix: brag document template

Here’s a template for a brag document! Usually I make one brag document per year. (“Julia’s 2017 brag document”). I think it’s okay to make it quite long / comprehensive – 5-10 pages or more for a year of work doesn’t seem like too much to me, especially if you’re including some graphs/charts / screenshots to show the effects of what you did.

One thing I want to emphasize, for people who don’t like to brag, is – you don’t have to try to make your work sound better than it is. Just make it sound exactly as good as it is! For example “was the primary contributor to X new feature that’s now used by 60% of our customers and has gotten Y positive feedback”.

Goals for this year:

Goals for next year


For each one, go through:

Remember: don’t forget to explain what the results of you work actually were! It’s often important to go back a few months later and fill in what actually happened after you launched the project.

Collaboration & mentorship

Examples of things in this category:

Design & documentation

List design docs & documentation that you worked on

Company building

This is a category we have at work – it basically means “things you did to help the company overall, not just your project / team”. Some things that go in here:

What you learned

My friend Julian suggested this section and I think it’s a great idea – try listing important things you learned or skills you’ve acquired recently! Some examples of skills you might be learning or improving:

It’s really easy to lose track of what skills you’re learning, and usually when I reflect on this I realize I learned a lot more than I thought and also notice things that I’m not learning that I wish I was.

Outside of work

It’s also often useful to track accomplishments outside of work, like:

I think this can be a nice way to highlight how you’re thinking about your career outside of strictly what you’re doing at work.

This can also include other non-career-related things you’re proud of, if that feels good to you! Some people like to keep a combined personal + work brag document.

General prompts

If you’re feeling stuck for things to mention, try:


What does debugging a program look like?

I was debugging with a friend who’s a relatively new programmer yesterday, and showed them a few debugging tips. Then I was thinking about how to teach debugging this morning, and mentioned on Twitter that I’d never seen a really good guide to debugging your code. (there are a ton of really great replies by Anne Ogborn to that tweet if you are interested in debugging tips)

As usual, I got a lot of helpful answers and now I have a few ideas about how to teach debugging skills / describe the process of debugging.

a couple of debugging resources

I was hoping for more links to debugging books/guides, but here are the 2 recommendations I got:

“Debugging” by David Agans: Several people recommended the book Debugging, which looks like a nice and fairly short book that explains a debugging strategy. I haven’t read it yet (though I ordered it to see if I should be recommending it) and the rules laid out in the book (“understand the system”, “make it fail”, “quit thinking and look”, “divide and conquer”, “change one thing at a time”, “keep an audit trail”, “check the plug”, “get a fresh view”, and “if you didn’t fix it, it ain’t fixed”) seem extremely resaonable :). He also has a charming debugging poster.

“How to debug” by John Regehr: How to Debug is a very good blog post based on Regehr’s experience teaching a university embedded systems course. Lots of good advice. He also has a blog post reviewing 4 books about debugging, including Agans’ book.

reproduce your bug (but how do you do that?)

The rest of this post is going to be an attempt to aggregate different ideas about debugging people tweeted at me.

Somewhat obviously, everybody agrees that being able to consistently reproduce a bug is important if you want to figure out what’s going on. I have an intuitive sense for how to do this but I’m not sure how to explain how to go from “I saw this bug twice” to “I can consistently reproduce this bug on demand on my laptop”, and I wonder whether the techniques you use to do this depend on the domain (backend web dev, frontend, mobile, games, C++ programs, embedded etc).

reproduce your bug quickly

Everybody also agrees that it’s extremely useful be able to reproduce the bug quickly (if it takes you 3 minutes to check if every change helped, iterating is VERY SLOW).

A few suggested approaches:

accept that it’s probably your code’s fault

Sometimes I see a problem and I’m like “oh, library X has a bug”, “oh, it’s DNS”, “oh, SOME OTHER THING THAT IS NOT MY CODE is broken”. And sometimes it’s not my code! But in general between an established library and my code that I wrote last month, usually it’s my code that I wrote last month that’s the problem :).

start doing experiments

@act_gardner gave a nice, short explanation of what you have to do after you reproduce your bug

I try to encourage people to first fully understand the bug - What’s happening? What do you expect to happen? When does it happen? When does it not happen? Then apply their mental model of the system to guess at what could be breaking and come up with experiments.

Experiments could be changing or removing code, making API calls from a REPL, trying new inputs, poking at memory values with a debugger or print statements.

I think the loop here may be:

change one thing at a time

Everybody definitely agrees that it is important to change one thing a time when doing an experiment to verify an assumption.

check your assumptions

A lot of debugging is realizing that something you were sure was true (“wait this request is going to the new server, right, not the old one???“) is actually… not true. I made an attempt to list some common incorrect assumptions. Here are some examples:

weird methods to get information

There are a lot of normal ways to do experiments to check your assumptions / guesses about what the code is doing (print out variable values, use a debugger, etc). Sometimes, though, you’re in a more difficult environment where you can’t print things out and don’t have access to a debugger (or it’s inconvenient to do those things, maybe because there are too many events). Some ways to cope:

The point here is that information is the most important thing and you need to do whatever’s necessary to get information.

write your code so it’s easier to debug

Another point a few people brought up is that you can improve your program to make it easier to debug. tef has a nice post about this: Write code that’s easy to delete, and easy to debug too. here. I thought this was very true:

Debuggable code isn’t necessarily clean, and code that’s littered with checks or error handling rarely makes for pleasant reading.

I think one interpretation of “easy to debug” is “every single time there’s an error, the program reports to you exactly what happened in an easy to understand way”. Whenever my program has a problem and says sometihng “error: failure to connect to SOME_IP port 443: connection timeout” I’m like THANK YOU THAT IS THE KIND OF THING I WANTED TO KNOW and I can check if I need to fix a firewall thing or if I got the wrong IP for some reason or what.

One simple example of this recently: I was making a request to a server I wrote and the reponse I got was “upstream connect error or disconnect/reset before headers”. This is an nginx error which basically in this case boiled down to “your program crashed before it sent anything in response to the request”. Figuring out the cause of the crash was pretty easy, but having better error handling (returning an error instead of crashing) would have saved me a little time because instead of having to go check the cause of the crash, I could have just read the error message and figured out what was going on right away.

error messages are better than silently failing

To get closer to the dream of “every single time there’s an error, the program reports to you exactly what happened in an easy to understand way” you also need to be disciplined about immediately returning an error message instead of silently writing incorrect data / passing a nonsense value to another function which will do WHO KNOWS WHAT with it and cause you a gigantic headache. This means adding code like this:

    raise "oh no THING happened"

This isn’t easy to get right (it’s not always obvious where you should be raising errors!“) but it really helps a lot.

failure: print out a stack of errors, not just one error.

Related to returning helpful errors that make it easy to debug: Rust has a really incredible error handling library called failure which basicaly lets you return a chain of errors instead of just one error, so you can print out a stack of errors like:

"error starting server process" caused by
"error initializing logging backend" caused by
"connection failure: timeout connecting to port 1234".

This is SO MUCH MORE useful than just connection failure: timeout connecting to port 1234 by itself because it tells you the significance of (it’s something to do with the logging backend!). And I think it’s also more useful than connection failure: timeout connecting to port 1234 with a stack trace, because it summarizes at a high level the parts that went wrong instead of making you read all the lines in the stack trace (some of which might not be relevant!).

tools like this in other languages:

If you know how to do this in other languages I’d be interested to hear!

understand what the error messages mean

One sub debugging skill that I take for granted a lot of the time is understanding what error messages mean! I came across this nice graphic explaining common Python errors and what they mean, which breaks down things like NameError, IOError, etc.

I think a reason interpreting error messages is hard is that understanding a new error message might mean learning a new concept – NameError can mean “Your code uses a variable outside the scope where it’s defined”, but to really understand that you need to understand what variable scope is! I ran into this a lot when learning Rust – the Rust compiler would be like “you have a weird lifetime error” and I’d like be “ugh ok Rust I get it I will go actually learn about how lifetimes work now!“.

And a lot of the time error messages are caused by a problem very different from the text of the message, like how “upstream connect error or disconnect/reset before headers” might mean “julia, your server crashed!“. The skill of understanding what error messages mean is often not transferable when you switch to a new area (if I started writing a lot of React or something tomorrow, I would probably have no idea what any of the error messages meant!). So this definitely isn’t just an issue for beginner programmers.

that’s all for now!

I feel like the big thing I’m missing when talking about debugging skills is a stronger understanding of where people get stuck with debugging – it’s easy to say “well, you need to reproduce the problem, then make a more minimal reproduction, then start coming up with guesses and verifying them, and improve your mental model of the system, and then figure it out, then fix the problem and hopefully write a test to make it not come back”, but – where are people actually getting stuck in practice? What are the hardest parts? I have some sense of what the hardest parts usually are for me but I’m still not sure what the hardest parts usually are for someone newer to debugging their code.


Why are monoidal categories interesting?

Hello! Someone on Twitter asked a question about tensor categories recently and I remembered “oh, I know something about that!! These are a cool thing!“. Monoidal categories are also called “tensor categories” and I think that term feels a little more concrete: one of the biggest examples of a tensor category is the category of vector spaces with the tensor product as the way you combine vectors / functions. “Monoidal” means “has an associative binary operation with an identity”, and with vector spaces the tensor product is the “associative binary operation” it’s referring to. So I’m going to mostly use “tensor categories” in this post instead.

So here’s a quick stab at explaining why tensor categories are cool. I’m going to make a lot of oversimplifications which I figure is better than trying to explain category theory from the ground up. I’m not a category theorist (though I spent 2 years in grad school doing a bunch of category theory) and I will almost certainly say wrong things about category theory.

In this post I’m going to try to talk about Seven Sketches in Compositionality: An Invitation to Applied Category Theory using mostly plain English.

tensor categories aren’t monads

If you have been around functional programming for a bit, you might see the word “monoid” and “categories” and wonder “oh, is julia writing about monads, like in Haskell”? I am not!!

There is a sentence “monads are a monoid in the category of endofunctors” which includes both the word “monoid” and “category” but that is not what I am talking about at all. We’re not going to talk about types or Haskell or monads or anything.

tensor categories are about proving (or defining) things with pictures

Here’s what I think is a really nice example from this [“seven sketches in compositionality”]((https://arxiv.org/pdf/1803.05316.pdf) PDF (on page 47):

The idea here is that you have 3 inequalities

  1. t <= v + w
  2. w + u <= x + z
  3. v + x <= y,

and you want to prove that t + u <= y + z.

You can do this algebraically pretty easily.

But in this diagram they’ve done something really different! They’ve sort of drawn the inequalities as boxes with lines coming out of them for each variable, and then you can see that you end up with a t and a u on the left and a y and a z on the right, and so maybe that means that t + u <= y + z.

The first time I saw something like this in a math class I felt like – what? what is happening? you can’t just draw PICTURES to prove things?!! And of course you can’t just draw pictures to prove things.

What’s actually happening in pictures like this is that when you put 2 things next to each other in the picture (like t and u), that actually represents the “tensor product” of t and u. In this case the “tensor product” is defined to be addition. And the tensor product (addition in this case) has some special properties –

  1. it’s associative
  2. if a <= b and c <= d then a + c <= b + d

so saying that this picture proves that t + u <= y + z actually means that you can read a proof off the diagram in a straightforward way:

      t    + u 
<= (v + w) + u 
=  v + (w + u) 
<= v + (x + z) 
=  (v + x) + z 
<=   y     + z

So all the things that “look like they would work” according to the picture actually do work in practice because our tensor product thing is associative and because addition works nicely with the <= relationship. The book explains all this in a lot more detail.

draw vector spaces with “string diagrams”

Proving this simple inequality is kind of boring though! We want to do something more interesting, so let’s talk about vector spaces! Here’s a diagram that includes some vector spaces (U1, U2, V1, V2) and some functions (f,g) between them.

Again, here what it means to have U1 stacked on top of U2 is that we’re taking a tensor product of U1 and U2. And the tensor product is associative, so there’s no ambiguity if we stack 3 or 4 vector spaces together!

This is all explained in a lot more detail in this nice blog post called introduction to string diagrams (which I took that picture from).

define the trace of a matrix with a picture

So far this is pretty boring! But in a follow up blog post, they talk about something more outrageous: you can (using vector space duality) take the lines in one of these diagrams and move them backwards and make loops. So that lets us define the trace of a function f : V -> V like this:

This is a really outrageous thing! We’ve said, hey, we have a function and we want to get a number in return right? Okay, let’s just… draw a circle around it so that there are no lines left coming out of it, and then that will be a number! That seems a lot more natural and prettier than the usual way of defining the trace of a matrix (“sum up the numbers on the diagonal”)!

When I first saw this I thought it was super cool that just drawing a circle is actually a legitimate way of defining a mathematical concept!

how are tensor category diagrams different from regular category theory diagrams?

If you see “tensor categories let you prove things with pictures” you might think “well, the whole point of category theory is to prove things with pictures, so what?“. I think there are a few things that are different in tensor category diagrams:

  1. with string diagrams, the lines are objects and the boxes are functions which is the opposite of how usual category theory diagrams are
  2. putting things next to each other in the diagram has a specific meaning (“take the tensor product of those 2 things”) where as in usual category theory diagrams it doesn’t. being able to combine things in this way is powerful!
  3. half circles have a specific meaning (“take the dual”)
  4. you can use specific elements of a (eg vector space) in a diagram which usually you wouldn’t do in a category theory diagram (the objects would be the whole vector space, not one element of that vector space)

what does this have to do with programming?

Even though this is usually a programming blog I don’t know whether this particular thing really has anything to do with programming, I just remembered I thought it was cool. I wrote my master’s thesis (which i will link to even though it’s not very readable) on topological quantum computing which involves a bunch of monoidal categories.

Some of the diagrams in this post are sort of why I got interested in that area in the first place – I thought it was really cool that you could formally define / prove things with pictures. And useful things, like the trace of a matrix!

Someone pointed me to a couple of twitter threads (coincidentally from this week!!) that relate tensor categories & diagrammatic methods to programming:

  1. this thread from @KenScambler (“My best kept secret* is that string & wiring diagrams–plucked straight out of applied category theory–are fabulous for software and system design.)
  2. this other thread by him of 31 interesting related things to this topic


New zine: Bite Size Networking!

Last week I released a new zine: Bite Size Networking! It’s the third zine in the “bite size” series:

  1. Bite Size Linux
  2. Bite Size Command Line
  3. Bite Size Networking

You can get it for $10 at https://wizardzines.com/zines/bite-size-networking/! (or $150/$250/$600 for the corporate rate).

Here’s the cover and table of contents!

A few people have asked for a 3-pack with all 3 “bite size” zines which is coming soon!

why this zine?

In last few years I’ve been doing a lot of networking at work, and along the way I’ve gone from “uh, what even is tcpdump” to “yes I can just type in sudo tcpdump -c 200 -n port 443 -i lo” without even thinking twice about it. As usual this zine is the resource I wish I had 4 years ago. There are so many things it took me a long time to figure out how to do like:

and as often happens with computers none of them are really that hard!! But the man pages for the tols you need to do these things are Very Long and as usual don’t differentiate between “everybody always uses this option and you 10000% need to know it” and “you will never use this option it does not matter”. So I spent a long time staring sadly at the tcpdump man page.

the pitch for this zine is:

It’s Thursday afternoon and your users are reporting SSL errors in production and you don’t know why. Or a HTTP header isn’t being set correctly and it’s breaking the site. Or you just got a notification that your site’s SSL certificate is expiring in 2 days. Or you need to update DNS to point to a new server. Or a server suddenly isn’t able to connect to a service. And networking maybe isn’t your full time job, but you still need to get the problem fixed.

Kamal (my partner) proofreads all my zines and we hit an exciting milestone with this one: this is the first zine where he was like “wow, I really did not know a lot of the stuff in this zine”. This is of course because I’ve spent a lot more time than him debugging weird networking things, and when you practice something you get better at it :)

a couple of example pages

Here are a couple of example pages, to give you an idea of what’s in the zine:

next thing to get better at: getting feedback!

One thing I’ve realized that while I get a ton of help from people while writing these zines (I read probably a thousand tweets from people suggesting ideas for things to include in the zine), I don’t get as much feedback from people about the final product as I’d like!

I often hear positive things (“I love them!”, “thank you so much!”, “this helped me in my job!”) but I’d really love to hear more about which bits specifically helped the most and what didn’t make as much sense or what you would have liked to see more of. So I’ll probably be asking a few questions about that to people who buy this zine!

selling zines is going well

When I made the switch about a year ago from “every zine I release is free” to “the old zines are free but all the new ones are not free” it felt scary! It’s been startlingly totally fine and a very positive thing. Sales have been really good, people take the work more seriously, I can spend more time on them, and I think the quality has gone up.

And I’ve been doing occasional giveaways for people who can’t afford a $10 zine, which feels like a nice way to handle “some people legitimately can’t afford $10 and I would like to get them information too”.

what’s next?

I’m not sure yet! A few options:

We’ll see what I feel most inspired by :)


Organizing this blog into categories

Today I organized the front page of this blog (jvns.ca) into CATEGORIES! Now it is actually possible to make some sense of what is on here!! There are 28 categories (computer networking! learning! “how things work”! career stuff! many more!) I am so excited about this.

How it works: Every post is in only 1 category. Obviously the categories aren’t “perfect” (there is a “how things work” category and a “kubernetes” category and a “networking” category, and so for a “how container networking works in kubernetes” I need to just pick one) but I think it’s really nice and I’m hoping that it’ll make the blog easier for folks to navigate.

If you’re interested in more of the story of how I’m thinking about this: I’ve been a little dissatisfied for a long time with how this blog is organized. Here’s where I started, in 2013, with a pretty classic blog layout (this is Octopress, which was a Jekyll Wordpress-lookalike theme that was cool back then and which served me very well for a long time):

problem with “show the 5 most recent posts”: you don’t know what the person’s writing is about!

This is a super common way to organize a blog: on the homepage of your blog, you display maybe the 5 most recent posts, and then maybe have a “previous” link.

The thing I find tricky about this (as a blog reader) is that

  1. it’s hard to hunt through their back catalog to find cool things they’ve written
  2. it’s SO HARD to get an overall sense for the body of a person’s work by reading 1 blog post at a time

next attempt: show every post in chronological order

My next attempt at blog organization was to show every post on the homepage in chronological order. This was inspired by Dan Luu’s blog, which takes a super minimal approach. I switched to this (according to the internet archive) sometime in early 2016. Here’s what it looked like (with some CSS issues :))

The reason I like this “show every post in chronological order” approach more is that when I discover a new blog, I like to obsessively binge read through the whole thing to see all the cool stuff the person has written. Rachel by the bay also organizes her writing this way, and when I found her blog I was like OMG WOW THIS IS AMAZING I MUST READ ALL OF THIS NOW and being able to look through all the entries quickly and start reading ones that caught my eye was SO FUN.

Will Larson’s blog also has a “list of all posts” page which I find useful because it’s a good blog, and sometimes I want to refer back to something he wrote months ago and can’t remember what it was called, and being able to scan through all the titles makes it easier to do that.

I was pretty happy with this and that’s how it’s been for the last 3 years.

problem: a chronological list of 390 posts still kind of sucks

As of today, I have 390 posts here (360,000 words! that’s, like, 4 300-page books! eep!). This is objectively a lot of writing and I would like people new to the blog to be able to navigate it and actually have some idea what’s going on.

And this blog is not actually just a totally disorganized group of words! I have a lot of specific interests: I’ve written probably 30 posts about computer networking, 15ish on ML/statistics, 20ish career posts, etc. And when I write a new Kubernetes post or whatever, it’s usually at least sort of related to some ongoing train of thought I have about Kubernetes. And it’s totally obvious to me what other posts that post is related to, but obviously to a new person it’s not at all clear what the trains of thought are in this blog.

solution for now: assign every post 1 (just 1) category

My new plan is to assign every post a single category. I got this idea from Itamar Turner-Trauring’s site.

Here are the initial categories:

I guess you can tell this is a systems-y blog because there are 8 different systems-y categories (kubernetes, infrastructure, linux debugging tools, rust, debugging stories, performance, and linux systems stuff, how a computer thing works) :).

But it was nice to see that I also have this huge career / work category! And that category is pretty meaningful to me, it includes a lot of things that I struggled with and were hard for me to learn. And I get to put all my machine learning posts together, which is an area I worked in for 3 years and am still super interested in and every so often learn a new thing about!

How I assign the categories: a big text file

I came up with a scheme for assigning the categories that I thought was really fun! I knew immediately that coming up with categories in advance would be impossible (how was I supposed to know that “fan posts about awesome work by other people” was a substantial category?)

So instead, I took kind of a Marie Kondo approach: I wrote a script to just dump all the titles of every blog post into a text file, and then I just used vim to organize them roughly into similar sections. Seeing everything in one place (a la marie kondo) really helped me see the patterns and figure out what some categories were.

Here’s the final result of that text file. I think having a lightweight way of organizing the posts all in one file made a huge difference and that it would have been impossible for me to seen the patterns otherwise.

How I implemented it: a hugo taxonomy

Once I had that big text file, I wrote a janky python script to assign the categories in that text file to the actual posts.

I use Hugo for this blog, and so I also needed to tell Hugo about the categories. This blog already technically has tags (though they’re woefully underused, I didn’t want to delete them). I use Hugo, and it turns out that in Hugo you can define arbitrary taxonomies. So I defined a new taxonomy for these sections (right now it’s called, unimaginitively, juliasections).

The details of how I did this are pretty boring but here’s the hugo template that makes it display on the homepage. I used this Hugo documentation page on taxonomies a lot.

organizing my site is cool! reverse chronology maybe isn’t the best possible thing!

Amy Hoy has this interesting article called how the blog broke the web about how the rise of blog software made people adopt a site format that maybe didn’t serve what they were writing the best.

I don’t personally feel that mad about the blog / reverse chronology organization: I like blogging! I think it was nice for the first 6 years or whatever to be able to just write things that I think are cool without thinking about where they “fit”. It’s worked really well for me.

But today, 360,000 words in, I think it makes sense to add a little more structure :).

what it looks like now!

Here’s what the new front page organization looks like! These are the blogging / learning / rust sections! I think it’s cool how you can see the evolution of some of my thinking (I sure have written a lot of posts about asking questions :)).

I ❤ the personal website

This is also part of why I love having a personal website that I can organize any way I want: for both of my main sites (jvns.ca and now wizardzines.com) I have total control over how they appear! And I can evolve them over time at my own pace if I decide something a little different will work better for me. I’ve gone from a jekyll blog to octopress to a custom-designed octopress blog to Hugo and made a ton of little changes over time. It’s so nice.

I think it’s fun that these 3 screenshots are each 3 years apart – what I wanted in 2013 is not the same as 2016 is not the same as 2019! This is okay!

And I really love seeing how other people choose to organize their personal sites! Please keep making cool different personal sites.


!!Con 2019: submit a talk!

As some of you might know, for the last 5 years I’ve been one of the organizers for a conferences called !!Con. This year it’s going to be held on May 11-12 in NYC.

The submission deadline is Sunday, March 3 and you can submit a talk here.

(we also expanded to the west coast this year: !!Con West is next week!! I’m not on the !!Con West team since I live on the east coast but they’re doing amazing work, I have a ticket, and I’m so excited for there to be more !!Con in the world)

!!Con is about the joy, excitement, and surprise of computing

Computers are AMAZING. You can make programs that seem like magic, computer science has all kind of fun and surprising tidbits, there are all kinds of ways to make really cool art with computers, the systems that we use every day (like DNS!) are often super fascinating, and sometimes our computers do REALLY STRANGE THINGS and it’s very fun to figure out why.

!!Con is about getting together for 2 days to share what we all love about computing. The only rule of !!Con talks is that the talk has to have an exclamation mark in the title :)

We originally considered calling !!Con ExclamationMarkCon but that was too unwieldy so we went with !!Con :).

!!Con is inclusive

The other big thing about !!Con is that we think computing should include everyone. To make !!Con a space where everyone can participate, we

past !!Con talks

I think maybe the easiest way to explain !!Con if you haven’t been is through the talk titles! Here are a few arbitrarily chosen talks from past !!Cons:

If you want to see more (or get an idea of what !!Con talk descriptions usually look like), here’s every past year of the conference:

this year you can also submit a play / song / performance!

One difference from previous !!Cons is that if you want submit a non-talk-talk to !!Con this year (like a play!), you can! I’m very excited to see what people come up with. For more of that see Expanding the !!Con aesthetic.

all talks are reviewed anonymously

One big choice that we’ve made is to review all talks anonymously. This means that we’ll review your talk the same way whether you’ve never given a talk before or if you’re an internationally recognized public speaker. I love this because many of our best talks are from first time speakers or people who I’d never heard of before, and I think anonymous review makes it easier to find great people who aren’t well known.

writing a good outline is important

We can’t rely on someone’s reputation to determine if they’ll give a good talk, but we do need a way to see that people have a plan for how to present their material in an engaging way. So we ask everyone to give a somewhat detailed outline explaining how they’ll spend their 10 minutes. Some people do it minute-by-minute and some people just say “I’ll explain X, then Y, then Z, then W”.

Lindsey Kuper wrote some good advice about writing a clear !!Con outline here which has some examples of really good outlines which you can see here.

We’re looking for sponsors

!!Con is pay-what-you-can (if you can’t afford a $300 conference ticket, we’re the conference for you!). Because of that, we rely on our incredible sponsors (companies who want to build an inclusive future for tech with us!) to help make up the difference so that we can pay our speakers for their amazing work, pay for speaker travel, have open captioning, and everything else that makes !!Con the amazing conference it is.

If you love !!Con, a huge way you can help support the conference is to ask your company to sponsor us! Here’s our sponsorship page and you can email me at julia@jvns.ca if you’re interested.

hope to see you there ❤

I’ve met so many fantastic people through !!Con, and it brings me a lot of joy every year. The thing that makes !!Con great is all the amazing people who come to share what they’re excited about every year, and I hope you’ll be one of them.


Networking tool comics!

Hello! I haven’t been blogging too much recently because I’m working on a new zine project: Linux networking tools!

I’m pretty excited about this one – I LOVE computer networking (it’s what I spent a big chunk of the last few years at work doing), but getting started with all the tools was originally a little tricky! For example – what if you have the IP address of a server and you want to make a https connection to it and check that it has a valid certificate? But you haven’t changed DNS to resolve to that server yet (because you don’t know if it works!) so you need to use the IP address? If you do curl, curl will tell you that the certificate isn’t valid (because it’s not valid for So you need to know to do curl https://jvns.ca --resolve jvns.ca:443:

I know how to use curl --resolve because my coworker told me how. And I learned that to find out when a cert expires you can do openssl x509 -in YOURCERT.pem -text -noout the same way. So the goal with this zine is basically to be “your very helpful coworker who gives you tips about how to use networking tools” in case you don’t have that person.

And as we know, a lot of these tools have VERY LONG man pages and you only usually need to know like 5 command line options to do 90% of what you want to do. For example I only ever do maybe 4 things with openssl even though the openssl man pages together have more than 60,000 words.

There are a few things I’m also adding (like ethtool and nmap and tc) which I don’t personally use super often but I think are super useful to people with different jobs than me. And I’m a big fan of mixing more advanced things (like tc) with basic things (like ssh) because then even if you’re learning the basic things for the first time, you can learn that the advanced thing exists!

Here’s some work in progress:

It’s been super fun to draw these: I didn’t know about ssh-copy-id or ~. before I made that ssh comic and I really wish I’d known about them earlier!

As usual I’ll announce the zine when it comes out here, or you can sign up for announcements at https://wizardzines.com/mailing-list/.


A few early marketing thoughts

At some point last month I said I might write more about business, so here are some very early marketing thoughts for my zine business (https://wizardzines.com!). The question I’m trying to make some progress on in this post is: “how to do marketing in a way that feels good?”

what’s the point of marketing?

Okay! What’s marketing? What’s the point? I think the ideal way marketing works is:

  1. you somehow tell a person about a thing
  2. you explain somehow why the thing will be useful to them / why it is good
  3. they buy it and they like the thing because it’s what they expected

(or, when you explain it they see that they don’t want it and don’t buy it which is good too!!)

So basically as far as I can tell good marketing is just explaining what the thing is and why it is good in a clear way.

what internet marketing techniques do people use?

I’ve been thinking a bit about internet marketing techniques I see people using on me recently. Here are a few examples of internet marketing techniques I’ve seen:

  1. word of mouth (“have you seen this cool new thing?!”)
  2. twitter / instagram marketing (build a twitter/instagram account)
  3. email marketing (“build a mailing list with a bajillion people on it and sell to them”)
  4. email marketing (“tell your existing users about features that they already have that they might want to use”)
  5. social proof marketing (“jane from georgia bought a sweater”), eg fomo.com
  6. cart notifications (“you left this sweater in your cart??! did you mean to buy it? maybe you should buy it!“)
  7. content marketing (which is fine but whenever people refer to my writing as ‘content’ I get grumpy :))

you need some way to tell people about your stuff

Something that is definitely true about marketing is that you need some way to tell new people about the thing you are doing. So for me when I’m thinking about running a business it’s less about “should i do marketing” and more like “well obviously i have to do marketing, how do i do it in a way that i feel good about?”

what’s up with email marketing?

I feel like every single piece of internet marketing advice I read says “you need a mailing list”. This is advice that I haven’t really taken to heart – technically I have 2 mailing lists:

  1. the RSS feed for this blog, which sends out new blog posts to a mailing list for folks who don’t use RSS (which 3000 of you get)
  2. https://wizardzines.com’s list, for comics / new zine announcements (780 people subscribe to that! thank you!)

but definitely neither of them is a Machine For Making Sales and I’ve put in almost no efforts in that direction yet.

here are a few things I’ve noticed about marketing mailing lists:

what works for me: twitter

Right now for my zines business I’d guess maybe 70% of my sales come from Twitter. The main thing I do is tweet pages from zines I’m working on (for example: yesterday’s comic about ss). The comics are usually good and fun so invariably they get tons of retweets, which means that I end up with lots of followers, which means that when I later put up the zine for sale lots of people will buy it.

And of course people don’t have to buy the zines, I post most of what ends up in my zines on twitter for free, so it feels like a nice way to do it. Everybody wins, I think.

(side note: when I started getting tons of new followers from my comics I was actually super worried that it would make my experience of Twitter way worse. That hasn’t happened! the new followers all seem totally reasonable and I still get a lot of really interesting twitter replies which is wonderful ❤)

I don’t try to hack/optimize this really: I just post comics when I make them and I try to make them good.

a small Twitter innovation: putting my website on the comics

Here’s one small marketing change that I made that I think makes sense!

In the past, I didn’t put anything about how to buy my comics on the comics I posted on Twitter, just my Twitter username. Like this:

After a while, I realized people were asking me all the time “hey, can I buy a book/collection? where do these come from? how do I get more?“! I think a marketing secret is “people actually want to buy things that are good, it is useful to tell people where they can buy things that are good”.

So just recently I’ve started adding my website and a note about my current project on the comics I post on Twitter. It doesn’t say much: just “❤ these comics? buy a collection! wizardzines.com” and “page 11 of my upcoming bite size networking zine”. Here’s what it looks like:

I feel like this strikes a pretty good balance between “julia you need to tell people what you’re doing otherwise how are they supposed to buy things from you” and “omg too many sales pitches everywhere”? I’ve only started doing this recently so we’ll see how it goes.

should I work on a mailing list?

It seems like the same thing that works on twitter would work by email if I wanted to put in the time (email people comics! when a zine comes out, email them about the zine and they can buy it if they want!).

One thing I LOVE about Twitter though is that people always reply to the comics I post with their own tips and tricks that they love and I often learn something new. I feel like email would be nowhere near as fun :)

But I still think this is a pretty good idea: keeping up with twitter can be time consuming and I bet a lot of people would like to get occasional email with programming drawings. (would you?)

One thing I’m not sure about is – a lot of marketing mailing lists seem to use somewhat aggressive techniques to get new emails (a lot of popups on a website, or adding everyone who signs up to their service / buys a thing to a marketing list) and while I’m basically fine with that (unsubscribing is easy!), I’m not sure that it’s what I’d want to do, and maybe less aggressive techniques will work just as well? We’ll see.

should I track conversion rates?

A piece of marketing advice I assume people give a lot is “be data driven, figure out what things convert the best, etc”. I don’t do this almost at all – gumroad used to tell me that most of my sales came from Twitter which was good to know, but right now I have basically no idea how it works.

Doing a bunch of work to track conversion rates feels bad to me: it seems like it would be really easy to go down a dumb rabbit hole of “oh, let’s try to increase conversion by 5%” instead of just focusing on making really good and cool things.

My guess is that what will work best for me for a while is to have some data that tells me in broad strokes how the business works (like “about 70% of sales come from twitter”) and just leave it at that.

should I do advertising?

I had a conversation with Kamal about this post that went:

So, ads! I don’t know anything about advertising except that you can advertise on Facebook or Twitter or Google. Some non-ethical questions I have about advertising:

This seems nontrivial to learn about and I don’t think I’m going to try soon.

other marketing things

a few other things I’ve thought about:

Maybe marketing is all about “make your things discoverable without being annoying”? :)

that’s all!

Hopefully some of this was interesting! Obviously the most important thing in all of this is to make cool things that are useful to people, but I think cool useful writing does not actually sell itself!

If you have thoughts about what kinds of marketing have worked well for you / you’ve felt good about I would love to hear them!


Some nonparametric statistics math

I’m trying to understand nonparametric statistics a little more formally. This post may not be that intelligible because I’m still pretty confused about nonparametric statistics, there is a lot of math, and I make no attempt to explain any of the math notation. I’m working towards being able to explain this stuff in a much more accessible way but first I would like to understand some of the math!

There’s some MathJax in this post so the math may or may not render in an RSS reader.

Some questions I’m interested in:

since these notes are from reading a math book and math books are extremely dense this is basically going to be “I read 7 pages of this math book and here are some points I’m confused about”

what’s nonparametric statistics?

Today I’m looking at “all of nonparametric statistics” by Larry Wasserman. He defines nonparametric inference as:

a set of modern statistical methods that aim to keep the number of underlying assumptions as weak as possible

Basically my interpretation of this is that – instead of assuming that your data comes from a specific family of distributions (like the normal distribution) and then trying to estimate the paramters of that distribution, you don’t make many assumptions about the distribution (“this is just some data!!“). Not having to make assumptions is nice!

There aren’t no assumptions though – he says

we assume that the distribution $F$ lies in some set $\mathfrak{F}$ called a statistical model. For example, when estimating a density $f$, we might assume that $$ f \in \mathfrak{F} = \left\{ g : \int(g^{\prime\prime}(x))^2dx \leq c^2 \right\}$$ which is the set of densities that are not “too wiggly”.

I have not too much intuition for the condition $\int(g^{\prime\prime}(x))^2dx \leq c^2$. I calculated that integral for the normal distribution on wolfram alpha and got 4, which is a good start. (4 is not infinity!)

some questions I still have about this definition:

we still have to assume independence

One assumption we won’t get away from is that the samples in the data we’re dealing with are independent. Often data in the real world actually isn’t really independent, but I think the what people do a lot of the time is to make a good effort at something approaching independence and then close your eyes and pretend it is?

estimating the density function

Okay! Here’s a useful section! Let’s say that I have 100,000 data points from a distribution. I can draw a histogram like this of those data points:

If I have 100,000 data points, it’s pretty likely that that histogram is pretty close to the actual distribution. But this is math, so we should be able to make that statement precise, right?

For example suppose that 5% of the points in my sample are more than 100. Is the probability that a point is greater than 100 actually 0.05? The book gives a nice formula for this:

$$ \mathbb{P}(|\widehat{P}_n(A) - P(A)| > \epsilon ) \leq 2e^{-2n\epsilon^2} $$

(by “Hoeffding’s inequality” which I’ve never heard of before). Fun aside about that inequality: here’s a nice jupyter notebook by henry wallace using it to identify the most common Boggle words.

here, in our example:

So, what’s the probability that the real probability is between 0.04 and 0.06? $\epsilon = 0.01$, so it’s $2e^{-2 \times 100,000 \times (0.01)^2} = 4e^{-9} $ ish (according to wolfram alpha)

here is a table of how sure we can be:

so basically, in this case, using this formula: 100,000 data points is AMAZING, 10,000 data points is pretty good, and 1,000 is much less useful. If we have 1000 data points and we see that 5% of them are more than 100, we DEFINITELY CANNOT CONCLUDE that 4% to 6% of points are more than 100. But (using the same formula) we can use $\epsilon = 0.04$ and conclude that with 92% probability 1% to 9% of points are more than 100. So we can still learn some stuff from 1000 data points!

This intuitively feels pretty reasonable to me – like it makes sense to me that if you have NO IDEA what your distribution that with 100,000 points you’d be able to make quite strong inferences, and that with 1000 you can do a lot less!

more data points are exponentially better?

One thing that I think is really cool about this estimating the density function formula is that how sure you can be of your inferences scales exponentially with the size of your dataset (this is the $e^{-n\epsilon^2}$). And also exponentially with the square of how sure you want to be (so wanting to be sure within 0.01 is VERY DIFFERENT than within 0.04). So 100,000 data points isn’t 10x better than 10,000 data points, it’s actually like 10000000000000x better.

Is that true in other places? If so that seems like a super useful intuition! I still feel pretty uncertain about this, but having some basic intuition about “how much more useful is 10,000 data points than 1,000 data points?“) feels like a really good thing.

some math about the bootstrap

The next chapter is about the bootstrap! Basically the way the bootstrap works is:

  1. you want to estimate some statistic (like the median) of your distribution
  2. the bootstrap lets you get an estimate and also the variance of that estimate
  3. you do this by repeatedly sampling with replacement from your data and then calculating the statistic you want (like the median) on your samples

I’m not going to go too much into how to implement the bootstrap method because it’s explained in a lot of place on the internet. Let’s talk about the math!

I think in order to say anything meaningful about bootstrap estimates I need to learn a new term: a consistent estimator.

What’s a consistent estimator?

Wikipedia says:

In statistics, a consistent estimator or asymptotically consistent estimator is an estimator — a rule for computing estimates of a parameter $\theta_0$ — having the property that as the number of data points used increases indefinitely, the resulting sequence of estimates converges in probability to $\theta_0$.

This includes some terms where I forget what they mean (what’s “converges in probability” again?). But this seems like a very good thing! If I’m estimating some parameter (like the median), I would DEFINITELY LIKE IT TO BE TRUE that if I do it with an infinite amount of data then my estimate works. An estimator that is not consistent does not sound very useful!

why/when are bootstrap estimators consistent?

spoiler: I have no idea. The book says the following:

Consistency of the boostrap can now be expressed as follows.

3.19 Theorem. Suppose that $\mathbb{E}(X_1^2) < \infty$. Let $T_n = g(\overline{X}_n)$ where $g$ is continuously differentiable at $\mu = \mathbb{E}(X_1)$ and that $g\prime(\mu) \neq 0$. Then,

$$ \sup_u | \mathbb{P}_{\widehat{F}_n} \left( \sqrt{n} (T( \widehat{F}_n*) - T( \widehat{F}_n) \leq u \right) - \mathbb{P}_{\widehat{F}} \left( \sqrt{n} (T( \widehat{F}_n) - T( \widehat{F}) \leq u \right) | \rightarrow^\text{a.s.} 0 $$

3.21 Theorem. Suppose that $T(F)$ is Hadamard differentiable with respect to $d(F,G)= sup_x|F(x)-G(x)|$ and that $0 < \int L^2_F(x) dF(x) < \infty$. Then,

$$ \sup_u | \mathbb{P}_{\widehat{F}_n} \left( \sqrt{n} (T( \widehat{F}_n*) - T( \widehat{F}_n) \leq u \right) - \mathbb{P}_{\widehat{F}} \left( \sqrt{n} (T( \widehat{F}_n) - T( \widehat{F}) \leq u \right) | \rightarrow^\text{P} 0 $$

things I understand about these theorems:

I don’t really understand how they’re related to consistency, and in particular the $\sup_u$ thing is weird, like if you’re looking at $\mathbb{P}(something < u)$, wouldn’t you want to minimize $u$ and not maximize it? Maybe it’s a typo and it should be $\inf_u$?

it concludes:

there is a tendency to treat the bootstrap as a panacea for all problems. But the bootstrap requires regularity conditions to yield valid answers. It should not be applied blindly.

this book does not seem to explain why the bootstrap is consistent

In the appendix (3.7) it gives a sketch of a proof for showing that estimating the median using the bootstrap is consistent. I don’t think this book actually gives a proof anywhere that bootstrap estimates in general are consistent, which was pretty surprising to me. It gives a bunch of references to papers. Though I guess bootstrap confidence intervals are the most important thing?

that’s all for now

This is all extremely stream of consciousness and I only spent 2 hours trying to work through this, but some things I think I learned in the last couple hours are:

  1. maybe having more data is exponentially better? (is this true??)
  2. “consistency” of an estimator is a thing, not all estimators are consistent
  3. understanding when/why nonparametric bootstrap estimators are consistent in general might be very hard (the proof that the bootstrap median estimator is consistent already seems very complicated!)
  4. boostrap confidence intervals are not the same thing as bootstrap estimators. Maybe I’ll learn the difference next!


2018: Year in review

I wrote these in 2015 and 2016 and 2017 and it’s always interesting to look back at them, so here’s a summary of what went on in my side projects in 2018.

ruby profiler!

At the beginning of this year I wrote rbspy (docs: https://rbspy.github.io/). It inspired a Python version called py-spy and a PHP profiler called phpspy, both of which are excellent. I think py-spy in particular is probably better than rbspy which makes me really happy.

Writing a program that does something innovative (top for your Ruby program’s functions!) and inspiring other people to make amazing new tools is something I’m really proud of.

started a side business!

A very surprising thing that happened in 2018 is that I started a business! This is the website: https://wizardzines.com/, and I sell programming zines.

It’s been astonishingly successful (it definitely made me enough money that I could have lived on just the revenue from the business this year), and I’m really grateful to everyone’s who’s supported that work. I hope the zines have helped you. I always thought that it was impossible to make anywhere near as much money teaching people useful things as I can as a software developer, and now I think that’s not true. I don’t think that I’d want to make that switch (I like working as a programmer!), but now I actually think that if I was serious about it and was interested in working on my business skills, I could probably make it work.

I don’t really know what’s next, but I plan to write at least one zine next year. I learned a few things about business this year, mainly from:

I used to think that sales / marketing had to be gross, but reading some of these business books made me think that it’s actually possible to run a business by being honest & just building good things.


this is mostly about side projects, but a few things about work:

When I go back to work I’m going to switch to working on something COMPLETELY DIFFERENT (writing code that sends messages to banks!) for 3 months. It’s a lot closer to the company’s core business, and I think it’ll be neat to learn more about how financial infastracture works.

I struggled a bit with understanding/defining my job this year. I wrote What’s a senior engineer’s job? about that, but I have not yet reached enlightenment.


I gave 4 talks in 2018:

In past years I’ve mostly given talks which can mostly be summarized “here are some cool tools” and “here is how to learn hard things”. This year I changed focus to giving talks about the actual work I do – there were two talks about building a Ruby profiler, and one about what I do at work (I spend a lot of time on infrastructure migrations!)

I’m not sure whether if I’ll give any talks in 2019. I travelled more than I wanted to in 2018, and to stay sane I ended up having to cancel on a talk I was planning to give with relatively short notice which wasn’t good.


I also experimented a bit with a new format: the podcast! These were basically all really fun! They don’t take that long (about 2 hours total?).

what I learned about doing podcasts:


I co-organized !!Con for the 4th time – I ran sponsorships. It’s always such a delight and the speakers are so great.

!!Con is expanding to the west coast in 2019 – I’m not directly involved with that but it’s going to be amazing.

blog posts!

I apparently wrote 54 blog posts in 2018. A couple of my favourites are What’s a senior engineer’s job? , How to teach yourself hard things, and batch editing files with ed.

There were basically 4 themes in blogging for 2018:

a tiny inclusion project: a guide to performance reviews

Last year in addition to my actual job, I did a couple of projects at work towards helping make sure the performance/promotion process works well for folks – i collaborated with the amazing karla on the idea of a “brag document”, and redid our engineering levels.

This year, in the same vein, I wrote a document called the “Unofficial guide to the performance reviews”. A lot of folks said it helped them but probably it’s too early to celebrate. I think explaining to folks how the performance review process actually works and how to approach it is really valuable and I might try to publish a more general version here at some point.

I like that I work at a place where it’s possible/encouraged to do projects like this. I spend a relatively small amount of time on them (maybe I spent 15 hours on this one?) but it feels good to be able to make tiny steps towards building a better workplace from time to time. It’s really hard to judge the results though!


some things that worked in 2018:


New talk: High Reliability Infrastructure Migrations

On Tuesday I gave a talk at KubeCon called High Reliability Infrastructure Migrations. The abstract was:

For companies with high availability requirements (99.99% uptime or higher), running new software in production comes with a lot of risks. But it’s possible to make significant infrastructure changes while maintaining the availability your customers expect! I’ll give you a toolbox for derisking migrations and making infrastructure changes with confidence, with examples from our Kubernetes & Envoy experience at Stripe.



Here are the slides:

since everyone always asks, I drew them in the Notability app on an iPad. I do this because it’s faster than trying to use regular slides software and I can make better slides.

a few notes

Here are a few links & notes about things I mentioned in the talk

skycfg: write functions, not YAML

I talked about how my team is working on non-YAML interfaces for configuring Kubernetes. The demo is at skycfg.fun, and it’s on GitHub here. It’s based on Starlark, a configuration language that’s a subset of Python.

My coworker John has promised that he’ll write a blog post about it at some point, and I’m hoping that’s coming soon :)

no haunted forests

I mentioned a deploy system rewrite we did. John has a great blog post about when rewrites are a good idea and how he approached that rewrite called no haunted forests.

ignore most kubernetes ecosystem software

One small point that I made in the talk was that on my team we ignore almost all software in the Kubernetes ecosystem so that we can focus on a few core pieces (Kubernetes & Envoy, plus some small things like kiam). I wanted to mention this because I think often in Kubernetes land it can seem like everyone is using Cool New Things (helm! istio! knative! eep!). I’m sure those projects are great but I find it much simpler to stay focused on the basics and I wanted people to know that it’s okay to do that if that’s what works for your company.

I think the reality is that actually a lot of folks are still trying to work out how to use this new software in a reliable and secure way.

other talks

I haven’t watched other Kubecon talks yet, but here are 2 links:

I heard good things about this keynote from melanie cebula about kubernetes at airbnb, and I’m excited to see this talk about kubernetes security. The slides from that security talk look useful

Also I’m very excited to see Kelsey Hightower’s keynote as always, but that recording isn’t up yet. If you have other Kubecon talks to recommend I’d love to know what they are.

my first work talk I’m happy with

I usually give talks about debugging tools, or side projects, or how I approach my job at a high level – not on the actual work that I do at my job. What I talked about in this talk is basically what I’ve been learning how to do at work for the last ~2 years. Figuring out how to make big infrastructure changes safely took me a long time (and I’m not done!), and so I hope this talk helps other folks do the same thing.


Page created: Sun, Oct 20, 2019 - 09:05 PM GMT