510 stories

Monday, June 27, 2022 - aphoristic noodling


Monday, June 27, 2022

aphoristic noodling

I read this post by Baldur Bjarnason, listing "Everything I’ve learned about web development in the almost twenty-five years I’ve been practising", and this followup, which says:

Some of the aphorisms ended up not-so-pithy, but it was overall a fun little experiment that I recommend: note down everything relevant about the craft that you can think of over the space of a week.

I thought about this, and then I thought: Ok, what exactly is my craft? I do computer shit. So I started a list about that, challenging myself to be descriptive about things and not veer too far into pure advice.

A year or so passed, and I noticed this post was still sitting in my "work in progress" directory. I tried picking it back up and noticed how much overlap it would have with other posts like these:

This style of writing is basically catnip to people like me, whether it's of much use to anyone else or not. This post ultimately felt like a dead end, because instead of a blog post, it really wants to be some long document where I collect all sorts of aphorisms, pithy quotes, eponymous laws, and so forth about technical work and maybe just work generally. Maybe I'll start that document one of these days.

Anyway, that very partial and uneven list:

  1. Caching is hard to think about and breaks often.
  2. Cleverness in code is generally a sign of danger.
  3. Business ruins everything.
  4. Some forms of interoperability are a trap.
  5. Bad ideas aren't limited to bad people.
  6. Good people aren't limited to good ideas.
  7. An aesthetic is not an ethic.
  8. The customer is usually wrong.
  9. If it's written in:
    • C: It'll work, but I should remember there's a buffer overflow or something.
    • PHP: It'll probably work, but there's an SQL injection vulnerability somewhere and the cool kids will be shitty about it being PHP.
    • Python: 50/50 whether it'll just barf stack traces into my terminal for non-obvious reasons.
    • Ruby: Decent chance I'll wind up reading the source code and cursing at clever Ruby programmers.
    • Haskell: It works, but I'm not smart enough to understand it.
    • Rust: Probably works, if they finished writing it. I'm not smart enough to understand the code.
    • Go: Total crapshoot, but either way I bet the CLI has a bunch of infuriatingly nested subcommands.
    • JavaScript: Life is too short to deal with whatever package management and runtime I'm supposed to use for this now.
    • Java: If I have to find out it's Java, I'm probably in trouble.
  10. Lightweight markup languages are fundamentally in tension with the range of structures that their users will inevitably want to express.
  11. Design, marketing, and management are all real undertakings, but they are also aggressively self-reproducing ideological systems and political projects.
  12. Environments within which small tools can be combined to operate on simple abstractions are powerful. An environment might be what you think of as an operating system, a programming language, a database, or an application. All else being equal, the ones that can bridge to other environments are more powerful.
  13. There are few abstractions in computing more stable than filesystems, standard IO, text files, and the shell. Boring relational databases aren't too far behind, but the barriers to entry and data transfer are higher.
  14. Technology is at least as fashion-oriented as the sartorial choices of highschoolers, actors, and musicians. Changes are driven as much by a desire for difference from the perceived status quo as anything else.
  15. Technical politics are also organizational, labor, and identity politics. The currents of power they involve are illegible without taking those factors into account.
  16. There's no guarantee that your technical preferences will match up with the ideas, people, or power structures you find agreeable in other domains. (Or vice versa.)

tags: topics/technical, topics/work

p1k3 / 2022 / 6 / 27

Read the whole story
38 days ago
39 days ago
Ojai, CA, US
Share this story

Exit Interviews Are a Trap

1 Comment

Exit Interviews Are a Trap

I was leaving a job after a few years followed by several really bad months. My last day was Friday, and on Tuesday afternoon I got the calendar invite from my boss: “Exit Interview, Thursday 1pm”.

I did something that, in most cases, is a very bad idea: I went to that interview and told my boss exactly why I was leaving, and what I thought he could do to change things. I want to use this story to talk about exit interviews: why they are, generally speaking, a trap; how to handle them if you get asked to go to one; and, finally, why I chose to break my own rule.

Exit interviews are a trap

Say you’re leaving a job, and you have some criticism. Maybe your boss sucked. Maybe the working conditions were bad. Maybe your teammates were hard to work with. Maybe the food in the cafeteria is better on the other side of Sand Hill Road. Whatever: the details of the criticisms are unimportant; the question is, should you air them out at the exit interview?

Well, it’s time for some game theory1:

If you keep your mouth shut, nothing changes. Whatever departing impression your colleagues and bosses have of you, whatever references they may (or may not) give, stay unchanged. The company, too, stays unchanged.

What if you speak up? Total honesty, say what you think.

Let’s consider the upside: maybe the company changes! Probably not, though: whatever the criticism is, it’s highly unlikely it’s something new. People typically don’t quit over one-time problems; they quit over patterns. That obnoxious coworker, everyone knows he’s terrible, they’ve just chosen to avoid the problem rather than addressing it. Or they’ve tried and failed. It’s highly unlikely that your feedback will be the thing that finally moves the needle. Not impossible, it does happen, but it’s rare. And even if your feedback does finally start some change – it’s too late for you. You’re already out the door. The time to fix the problems was months ago, before you got fed up and started your job search.

So, the best case scenario is a very unlikely chance of a positive outcome, and even if you get that positive outcome, it’s of no direct value to you.

Now let’s consider the downside. What if what you say rubs someone the wrong way? Telling your boss exactly why he was a bad manager might not go over well. Maybe that coworker you want to complain about is actually the CEO’s nephew. Maybe HR knows all about the food and is sick and tired of hearing about it. There are all sorts of ways that your former employers might retaliate. They might decide not to give a reference they would have before the exit interview. They might say bad things about you at industry events. It might be more subtle: they might just carry a negative impression of you in the back of their minds, and when you both end up working together again in the future they’ll hesitate to trust you. Or it might be blatantly unethical or even illegal. They could tell you they’re fine to be a reference, but when someone calls they could say bad things. They could refuse to confirm your employment to a background investigator. They could call your new company and tell them you were fired for fraud.

These are all real stories: every single one of those hypotheticals is actually something I’ve seen happen. So, the worst-case scenario is… real bad! Potential job-ending or even career-limiting retaliation.

This is especially true for people early in their careers. If you burn a reference, and this is your first job, it can be very hard to get a second. If you get badmouthed around the industry, and you don’t have an established professional network to refute the allegations, people will believe the lies.

This is why I say exit interviews are a trap. There’s no upside, and a lot of downside. The best case scenario (positive change) is highly unlikely, and even if it happens, of no direct value to you. And the worst case scenario is retaliation that could haunt you for years. If we could calculate an “expected average outcome”, like an expected value of a bet, it’d be deeply negative. Nobody who knows the odds would take this bet.

So how should I handle an exit interview request?

Despite this, exit interviews are exceedingly normal. If you leave a job, you can expect to be asked to sit for one by your boss, a skip-level boss, HR, or maybe all three. What then?

Here’s what I recommend:

  • Avoid the meeting if possible. Don’t go if you think you can get away with it. You can certainly refuse to go (you already quit; what are they going to do about it?2) but sometimes outright refusal will feel too hostile. You can also run out the clock: keep having last-minute conflicts and reschedule until your last day, then discover you have too much to do that last day to make time, so sorry.
  • If you must go – if you get cornered, if you feel declining or avoiding will come across poorly and risk burning a bridge – be totally bland. Why are you leaving? You found a great opportunity elsewhere and had to take it. Do you have any feedback? Nothing comes to mind. What could they have done to keep you? Oh, it’s not about leaving them, it’s about joining the new place.

    If asked for feedback about someone specific, here’s a phrase you can use: “I have nothing bad to say about them”. This has the advantage of being true: even if this is someone you can’t stand, you have nothing bad to say about them. You’re signaling that even if you have things you think, you’re not going to say them. You need to practice saying this in a breezy, friendly, upbeat tone: don’t play tricks; don’t wink.

  • Lie if you must. Usually I’d say dishonesty at work is unethical, but there are situations where it’s not; this is one of them. (I’ll have more to say about honesty at work in my next post; stay tuned). I don’t recommend outright lies: saying you love your manager when they’re the reason you’re leaving helps them continue to hide bad behavior. I recommend instead something simple and bland like saying “nothing comes to mind” or “hm I haven’t really thought about it, not sure.”

  • Let it be awkward. If HR has a 20 question script and you’re answering every question with “nothing comes to mind”, it’s going to be weird. That’s not your fault: it’s weird because exit interviews themselves are weird. Be prepared for the awkwardness, and stick to the plan.

Why I broke my own rules

Now that I’ve explained why exit interviews are a trap, why did I decide to ignore my own rules?

There are several reasons why people might want to air their grievances at an exit interview:

  • They haven’t really thought about it strategically: they just sort of go in without a plan and answer questions like they would at any other meeting.
  • Catharsis: they’ve been wanting to say the thing, and now that they’re leaving they finally feel like they can. Or, they’ve been saying it all along, but now they think they’ll finally be heard.
  • They believe their feedback will lead to change. They want to improve things for their colleagues who are staying, for whoever replaces them. They genuinely want the company to do better! Their compassion leads them to overestimate the likelihood of this change, and underestimate the potential cost of speaking up.

My motivation was that last one. I was leaving a team behind, and I was worried about how things would go for them after I left. I genuinely liked most of my coworkers, even the ones I thought that were making the wrong moves, and wanted better things for them. And, yeah I did overestimate the likelihood that my parting feedback would make a difference.

However, even with all of that, I would have stuck to my “say nothing” guns if not for another factor: I didn’t think I had any downside risk. Let me explain:

  • I’m well-established in my career. I have over 20 years of experience, which means that any one former employer has limited power over my career. I don’t need the reference; I have plenty. I don’t need to emphasize any particular job on my resume; I have others.

  • I have a very strong professional network. People know me, know the kind of person and co-worker I am. If someone were to badmouth me, it almost certainly wouldn’t work: lies would be fairly obvious. Any attempt to sabotage me would likely backfire.

  • I’m financially stable, and had my next job already lined up. I wasn’t going to have a gap in employment, and even if I did, I have the savings to withstand it. The worst-case scenarios aren’t very bad.

  • Finally, and most importantly: I knew my boss and co-workers very well, everyone in a position to react to my feedback, and had a lot of trust they’d take feedback professionally. We had a workplace high in psychological safety. I’d given critical feedback before, many times, and had never experienced defensiveness, retaliation, or any other sort of negative outcome. Further, I knew they’d listen to me, and consider my feedback carefully. Even when we’d disagreed, I’d always felt my feedback was heard and considered.

So by my calculation, there was no real downside to speaking up. The sort of scary retaliation scenarios were deeply unlikely, and even if they happened, I have the professional standing such that they’re not a threat. This makes the average expected outcome of speaking up slightly positive: there’s a small chance I’ll improve things for the folks still working there, and no real downside risk.

So what happened?

As you know from reading my blog, I’m very convincing. My soon-to-be-former boss listened to everything I had to say, and immediately made the changes I was suggesting, and the company is doing very well now.

Er, no. Not at all.

My boss thanked me for my feedback, and nothing changed. Several other folks quit over the next few months. Last I checked, the organization was still very much headed in the direction that I disagreed with.

Like most people, I was wrong about the likelihood of an exit interview truth bomb leading to change. I’m glad to say I my trust was correct, though: my former boss and co-workers continue to say nice things about me.

I think I made the right choice, but it’s still not the decision I recommend. Exit interviews are a trap. Unless you know you’re immune, don’t walk into the trap.

  1. Sorry. ↩︎

  2. Occasionally, unethical HR people will threaten to without a final paycheck or other benefits if you don’t participate in the exit interview. This is illegal (in the US, at least). ↩︎

Read the whole story
125 days ago
I think he's mostly correct.
Ojai, CA, US
Share this story

wednesday, march 16, 2022

1 Comment

wednesday, march 16, 2022

what's the distance
between a nervous habit
and a ritual tradition?

maybe just time and the collection plate
or how much group dynamics and trappings of
the numinous you can gin up

but i notice how
a lot of us have lost all touch with the latter
while accumulating a distinct excess
of the former

p1k3 / 2022 / 3 / 16

Read the whole story
135 days ago
This hits home with some "got dang why can't we find community anymore?" discussions my wife and I have been having.
Ojai, CA, US
Share this story

It’s (Still) Not Rocket Science


In 2014 the creator of the Rust programming language wrote a blog post entitled, “The Not Rocket Science Rule Of Software Engineering.” In the post, Graydon Hoare describes an arcane CI system he cobbled together circa 2001.

the team cooked up a system of cron jobs, rsync, multiple CVS repositories and a small postgres database tracking test results.

– Graydon Hoare, “not rocket science” (the story of monotone and bors)

The system sounds hopelessly antiquated by today’s standards – nobody is using CVS!

At the end of last year, GitHub introduced the Merge Queue, a mirror of GitLab’s (premium-tier only) Merge Trains feature. This means as of 2022, most of the major software forges have almost caught up with a duck tape CI system from 2001.

It’s not rocket science

The concepts developed building this piecemeal system turned out to be vital in the early days of Rust. So vital, the author felt compelled to write a blog post coining a rule to help identify this concept.

The rule itself is straightforward:

The Not Rocket Science Rule Of Software Engineering:

automatically maintain a repository of code that always passes all the tests

Lurking under that placid description is a simple concept, but it’s challenging to execute.

Why is this so hard

Most CI systems only guarantee that the tests pass in your branch. But that’s different than saying your tests will pass when your branch merges.

One reason for this mismatch may be integration friction: your branch diverged from the main branch over time.

It’s unsafe to merge changes in parallel in a conventional CI system. Semantic conflicts between patches can break the build post-merge. Imagine renaming a method in one branch while merging a patch that depends on that method in another.

The queue

Adopting a rebase-and-merge policy is a possible solution — where you rebase and merge all patches one at a time.

But processing a queue in serial is slow and won’t scale up as you add more developers.

Speculative future state

By enqueuing patches to rebase-and-merge in order, you’re creating a deterministic code state. We know what the codebase will look like after each patch is rebased and merged, so we can do that work up-front.

We can test the second patch in parallel with the first as if the first patch has already merged.

Main@𝑥² = HEAD + 𝑥¹
Main@𝑥³ = HEAD + 𝑥¹ + 𝑥²

We can run the tests for all patches in the queue in parallel. Waiting to merge each patch until all the patches in front of it pass their tests.

If a patch fails, we remove it and re-run the tests for all the patches behind it in the queue—it’s not rocket science!

This is how the Merge Queue and the Merge Train work today.

Multiply by 𝑛

One of my former colleagues was fond of saying:

The only numbers that matter in computer science are 0, 1, and 𝑛.

The “not rocket science” problem gets more complicated when a production build is composed of 𝑛 repos.

At my work, we’ve yet to embrace the monorepo trend 😬. We (nominally) have our reasons outside this post’s scope. The result is a release to production composed of roughly 200 different repositories – some of which depend on one another in an ever-spidering graph of maddening dependencies.

What we do

Sometime in 2012, the Openstack Infrastructure team started work on Zuul. Zuul maintains what it calls “pipelines.” Dependent pipelines can ensure that patches across multiple repos get tested and merged together as if they were part of a single Merge Train or Merge Queue.

Zuul treats each pipeline like a queue and maintains a shadow version control system that allows testing speculative future states across repositories in parallel.

Today it’s a decade after development began on Zuul — more than two decades after Graydon Hoare’s duck tape CI system. And the most prominent and best-funded software forges in the history of humanity are pretty gosh darn close to catching up.

Read the whole story
171 days ago
Boulder, CO
171 days ago
Ojai, CA, US
Share this story

Why Not Just Nationalize Tyson Foods?

1 Comment

Yesterday, the Joe Biden administration announced that they were going to provide hundreds of millions of dollars to mom and pop meat processors via grants, loan guarantees, and reduced inspection fees. The idea behind this splash of money is to promote competition in the meatpacking industry and ultimately achieve a redistribution of money away from meatpackers and to farmers in the form of higher prices and consumers in the form of lower prices.

Four large meat-packing companies control 85 percent of the beef market. In poultry, the top four processing firms control 54 percent of the market. And in pork, the top four processing firms control about 70 percent of the market. The meatpackers and processors buy from farmers and sell to retailers like grocery stores, making them a key bottleneck in the food supply chain.

When dominant middlemen control so much of the supply chain, they can increase their own profits at the expense of both farmers—who make less—and consumers—who pay more.

The explanation of the problem makes a lot of sense. On its journey to the consumer, meat starts with a farmer, travels through a meatpacker, and then finds its way to a retail store. If the meatpacking part of the value chain is clamped down by a few players that are colluding with one another, then they can grab an outsized share of the meat revenue.

But the offered solution to this problem — some grants and loans to smaller processors — seems both inadequate to solving the problem and also needlessly indirect. Frankly, this is true of most of the ideas that come out of the competition policy community.

If there are four companies that control the meatpacking market, and the government thinks that they are colluding in a way that generates super-profits at the expense of adjacent market players, then the government should simply buy one of the four companies and run it properly. You don’t need years-long efforts to maybe increase competition through subsidizing new firms. Simply nationalizing one of the big players would allow you to increase competition and bust up collusion in a few months.

Consider Tyson Foods for example. They are one of the big four meatpackers. They also happen to be publicly traded. Right now, the market cap of Tyson Foods is a mere $32 billion. Their profit last year was a bit over $3 billion.

The federal government can currently borrow money for 10 years at an interest rate of 1.63 percent. Thus, if it borrowed $32 billion to buy Tyson Foods, it would only need to generate $522 million per year from the company in order to break even on the interest expense for this borrowing. In all likelihood, even after reforming the new state-owned Tyson so that it competed properly and stopped squeezing consumers and farmers so hard, this type of leveraged buyout would end up being a profitable investment for the federal government.

Of course, I know why this kind of thing is never pursued or even proposed. It reeks a bit too much of the dreaded socialism. But if you put aside these kinds of ideological hangups for a second, I think you would really struggle to find a simpler, faster, or more effective way to tackle these kinds of problems. This kind of selective nationalization would not only allow the government to directly solve the problem in the targeted sector, but could very likely influence behavior in other sectors that do not want to become the next acquisition target.

Read the whole story
214 days ago
Why not?
Ojai, CA, US
Share this story

Volunteer Responsibility Amnesty Day

1 Comment

Volunteer Responsibility Amnesty Day

Tomorrow is Volunteer Responsibility Amnesty Day , a day to reflect on your responsibilities as a volunteer and, if any of them are too burdensome, set them down. My friend Sumana started this after observing that too many open source maintainers felt “trapped” and unable to step back from commitments they no longer were able to meet.

It’s a brilliant idea, and one I need this year. A few years ago, the former maintainer of dj-database-url needed to step down. He asked me if I’d pick the codebase up, and I said “yes”. I didn’t have a strong interest in being a maintainer, but it’s a very widely used package – almost a million downloads a month – and I felt it needed a good home. Unfortunately, after an initial flurry of inspiration, I found I didn’t have the time or energy. The repo has stagnated for the last couple of years.

It’s hard to admit you’ve failed. For me, it’s even harder to admit failure when it was a volunteer responsibility in the first place. Although I knew I needed to hand this off, I procrastinated doing so out of shame and guilt.

This is why having a set day (two per year - on each solstice) for stepping back feels so important to me. There’s a bit of a deadline – “do this by December 21st” – and a sense of strength in numbers.

If you’re volunteering in a way that’s no longer feeling good, why not take tomorrow to do an inventory and consider setting some things down? Or think about it, and set a reminder for next solstice (June 21, 2022)?

If anyone’s interested in the specifics of how I’m passing off maintainership – an interesting topic given the serious issues around software supply chain security – see this issue for how I’ve decided to implement the handoff .

Read the whole story
229 days ago
It's ok, sometimes it is time.
Ojai, CA, US
Share this story
Next Page of Stories