Benki → All Posts

Guess what one of the top disk latency inducers is on my (functionally mostly idle) server.

# zfsslower
Tracing ZFS operations slower than 10 ms
TIME     COMM           PID     T BYTES   OFF_KB   LAT(ms) FILENAME
09:14:10 async_49       2675004 S 0       0          18.29 journal.jif

The mysteriously named async_49 represents Mnesia as used by RabbitMQ as part of… Zulip.

Have I mentioned that a Zulip instance hosting 3 users is a waste of resources? Oh, I have, haven’t I?

How do I fix CGit’s display of a repository’s time of last update?

If you copied Git repositories into CGit at one point, you may have done so without keeping their mtimes intact. In this case, CGit will display an incorrect time of last update for the affected repositories, as it does not determine it based on the most recent Git commit but rather the time the default branch was last touched on the local file system.

By default, CGit uses the mtime of refs/heads/master (assuming that master is your default branch) to determine the time of last update, so this is how you can fix the time to be the same as the commit date of the last commit:

touch -c refs/heads/master -t $(date +"%Y%m%d%H%M.%S" --date=@$(git show -s --format=%ct HEAD))

You can now subscribe to this web site via a weekly email newsletter. The content is the same as in the public news feed. Be warned: The code is beta quality. The very first issue of the newsletter is also going to contain all posts ever made up to this point.

On Reddit there is currently a discussion going on about the newly written Haskell committee guidelines for respectful communication. There’s nothing new about it—one side saying it’s long overdue, another saying it’s the usual overly draconian, one-sided snowflake nonsense, with little more nuanced commentary in between.

Now, I haven’t read the document being discussed nearly carefully enough and from the cursory look I’ve given it, it actually looks like one of the better ones that I can find little to disagree with in, so I’ll not be commenting on it specifically. But I do have an opinion on codes of conduct like it in general and I’ll make use of its wording to illustrate it (in part because it feels like such an okay code of conduct to me, all things considered).

So let’s look at codes of conduct in more generality.

In my opinion, while their goals are clearly noble and worthy of support, the concrete regulations stipulated in such documents often go way overboard and, if implemented, can be harmful and unfair to the parties involved.

To illustrate why, take the following statement the document linked above makes:

In our communication, we consistently honour and affirm the … good intentions of others.

Which sounds very reasonable. But then contrast it with this part:

Our response should usually be to apologise … Even if we feel we have been misinterpreted or unfairly accused, the chances are good there was something we could have communicated better …

There is an apparent contradiction here depending on how you read the word “should.” Under the assumption that person B accuses person A of insulting them, should we:

1. assume that person A had only good intentions and therefore there is no need to apologize, or
2. assume that person A is obliged to apologize because if in doubt, person B was probably right that person A said something wrong?

I think the fundamental problem lies in the three different forms that the communication in question takes and the nontrivial mappings between them:

                               ┌────┐  ┏━━━━━━━━━━━━━━┓   ┌────┐                   
╔═══════════════════╗          │g[B]│  ┃              ┃   │f[A]│                   
║realm of expression║         ┌┴────┴──┃ what is said ┃◀─┴────┴┐                  
╚═══════════════════╝         │        ┃              ┃         │                  
                              │        ┗━━━━━━━━━━━━━━┛         │                  
                              ▼                                │                  
                       ┏━━━━━━━━━━━━┳───────────┐       ┏━━━━━━━━━━━━┳───────────┐
╔═══════════════════╗  ┃  what is   │ Person B  │       ┃  what is   │ Person A  │
║ realm of meaning  ║  ┃ understood ┣───────────┘       ┃  intended  ┣───────────┘
╚═══════════════════╝  ┗━━━━━━━━━━━━┛                   ┗━━━━━━━━━━━━┛            

If what was said causes person B to be offended, do you automatically assume that what was intended was an offense? Or do you assume that if person A maintains that they did not mean to offend, do you completely discount what was understood? Is either extreme reasonable? Clearly not. Hence the contradictory phrasing in the document.

What you need to realize to untangle this mess is that both f and g are dependent on both the person executing them and the situation that they are in at the moment they do so (which I’m going to simplify notationally by assuming that the situation is a part of the person). With this realization we can now rephrase the problem:

How much responsibility do we put on person A to anticipate g[B] for any given person B?

Clearly we cannot expect person A to anticipate g[B] for all possible B in a discussion on a public mailing list, since the space {g[B] | B ∈ Audience} is, for all intents and purposes, infeasible to compute when Audience is sufficiently large. On the other hand it is also obvious that if what is said is in the context of a face-to-face meeting between two individuals who know each other well, the challenge is much simpler and we can expect person A to be more considerate of what they can reasonably expect person B to understand based on what they express.

(What people will actually do in practice when they do not know the audience well is to substitute B := A, which may seem overly simplistic, but is really as good an approximation as any when Audience = World.)

In practice, what this means is that as an outside observer C, the wise thing to do is probably to assume both that:

1. when evaluation person B’s conduct, the only correct way to interpret what was said is in the most malevolent way reasonably interpretable, and
2. when evaluating person A’s conduct, the only correct way to interpret what was said is in the most benevolent way reasonably interpretable.

Which results in the contradiction mentioned previously. As far as I know, there is no way to resolve it, so all we can do is accept it.

Zulip is ridiculous.

Here are the biggest RAM eaters in my Kubernetes cluster, all of which are pretty much completely idle right now:

NAME            CPU(cores)   MEMORY(bytes)
gerrit          5m           315Mi
keycloak        3m           440Mi
mulkcms2        2m           325Mi
zulip           31m          2698Mi

Remember, Keycloak is built on top of JBoss.

Let that sink in for a bit.

What’s going on there? Well, Zulip consists of 20 microservices. It’s the modern way.

How to switch from Docker to Containerd on a kubeadm-managed Kubernetes node:

1. Install Containerd.

2. Edit /etc/default/kubelet and add the following line:

   KUBELET_EXTRA_ARGS=--container-runtime=remote --runtime-request-timeout=15m --container-runtime-endpoint=unix:///run/containerd/containerd.sock

   If KUBELET_EXTRA_ARGS exists already, add the additional parameters to it instead.

3. Restart kubelet.

4. Uninstall Docker.

5. To make kubeadm aware of the change (so that kubeadm upgrade apply works): Let ${NODE_NAME} be the name of your node. Run:

   kubeadm edit nodes/${NODE_NAME}

   Look for the kubeadm.alpha.kubernetes.io/cri-socket annotation. Change its value to /run/containerd/containerd.sock.

Now everything should be set up for Containerd and you can do such fun things as running Kubernetes pods in Kata containers.

Presumably, these instructions will work for a migration to CRI-O and other container runtimes as well, but I have not tried.