506 stories

Why Not Just Nationalize Tyson Foods?

1 Comment

Yesterday, the Joe Biden administration announced that they were going to provide hundreds of millions of dollars to mom and pop meat processors via grants, loan guarantees, and reduced inspection fees. The idea behind this splash of money is to promote competition in the meatpacking industry and ultimately achieve a redistribution of money away from meatpackers and to farmers in the form of higher prices and consumers in the form of lower prices.

Four large meat-packing companies control 85 percent of the beef market. In poultry, the top four processing firms control 54 percent of the market. And in pork, the top four processing firms control about 70 percent of the market. The meatpackers and processors buy from farmers and sell to retailers like grocery stores, making them a key bottleneck in the food supply chain.

When dominant middlemen control so much of the supply chain, they can increase their own profits at the expense of both farmers—who make less—and consumers—who pay more.

The explanation of the problem makes a lot of sense. On its journey to the consumer, meat starts with a farmer, travels through a meatpacker, and then finds its way to a retail store. If the meatpacking part of the value chain is clamped down by a few players that are colluding with one another, then they can grab an outsized share of the meat revenue.

But the offered solution to this problem — some grants and loans to smaller processors — seems both inadequate to solving the problem and also needlessly indirect. Frankly, this is true of most of the ideas that come out of the competition policy community.

If there are four companies that control the meatpacking market, and the government thinks that they are colluding in a way that generates super-profits at the expense of adjacent market players, then the government should simply buy one of the four companies and run it properly. You don’t need years-long efforts to maybe increase competition through subsidizing new firms. Simply nationalizing one of the big players would allow you to increase competition and bust up collusion in a few months.

Consider Tyson Foods for example. They are one of the big four meatpackers. They also happen to be publicly traded. Right now, the market cap of Tyson Foods is a mere $32 billion. Their profit last year was a bit over $3 billion.

The federal government can currently borrow money for 10 years at an interest rate of 1.63 percent. Thus, if it borrowed $32 billion to buy Tyson Foods, it would only need to generate $522 million per year from the company in order to break even on the interest expense for this borrowing. In all likelihood, even after reforming the new state-owned Tyson so that it competed properly and stopped squeezing consumers and farmers so hard, this type of leveraged buyout would end up being a profitable investment for the federal government.

Of course, I know why this kind of thing is never pursued or even proposed. It reeks a bit too much of the dreaded socialism. But if you put aside these kinds of ideological hangups for a second, I think you would really struggle to find a simpler, faster, or more effective way to tackle these kinds of problems. This kind of selective nationalization would not only allow the government to directly solve the problem in the targeted sector, but could very likely influence behavior in other sectors that do not want to become the next acquisition target.

Read the whole story
17 days ago
Why not?
Ojai, CA, US
Share this story

Volunteer Responsibility Amnesty Day

1 Comment

Volunteer Responsibility Amnesty Day

Tomorrow is Volunteer Responsibility Amnesty Day , a day to reflect on your responsibilities as a volunteer and, if any of them are too burdensome, set them down. My friend Sumana started this after observing that too many open source maintainers felt “trapped” and unable to step back from commitments they no longer were able to meet.

It’s a brilliant idea, and one I need this year. A few years ago, the former maintainer of dj-database-url needed to step down. He asked me if I’d pick the codebase up, and I said “yes”. I didn’t have a strong interest in being a maintainer, but it’s a very widely used package – almost a million downloads a month – and I felt it needed a good home. Unfortunately, after an initial flurry of inspiration, I found I didn’t have the time or energy. The repo has stagnated for the last couple of years.

It’s hard to admit you’ve failed. For me, it’s even harder to admit failure when it was a volunteer responsibility in the first place. Although I knew I needed to hand this off, I procrastinated doing so out of shame and guilt.

This is why having a set day (two per year - on each solstice) for stepping back feels so important to me. There’s a bit of a deadline – “do this by December 21st” – and a sense of strength in numbers.

If you’re volunteering in a way that’s no longer feeling good, why not take tomorrow to do an inventory and consider setting some things down? Or think about it, and set a reminder for next solstice (June 21, 2022)?

If anyone’s interested in the specifics of how I’m passing off maintainership – an interesting topic given the serious issues around software supply chain security – see this issue for how I’ve decided to implement the handoff .

Read the whole story
32 days ago
It's ok, sometimes it is time.
Ojai, CA, US
Share this story

Health Insurance & Retirement Plans For Open Source Maintainers

1 Comment and 2 Shares
This is huge.

The Open Collective Foundation has just announced: "OCF now offers employment options to initiative workers—with health insurance!" As that page says: "Initiatives fiscally hosted by OCF can have employees, with access to benefits like health insurance. Costs related to employment are paid from the initiative's budget, with OCF as the employer." (Employees must be based in the US.) And through the OCF's benefits provider, employees can also opt into 401(k) retirement savings plans.

This is such a huge step forward for open source sustainability, in particular for projects with key contributors in the United States. Let's talk about why!


  1. Open source and fiscal hosts
  2. The United States, employment, retirement, and health care
  3. What you can now do via OpenCollective
  4. What this unlocks

Open source and fiscal hosts

A "fiscal host" is a nonprofit organization that helps out charitable endeavors by giving them certain kinds of legal and financial infrastructure and services. Here's why they exist:

If you start a mutual aid food pantry in your neighborhood, or a meditation meetup that turns into a real community, or an open source software project, eventually you'll likely need to find ways to take in and spend money without having everything go through one person's personal bank account/PayPal/Venmo. And sometimes you need a trademark to protect people against imitators, or you'd like for the domain name and the fridges and the storage unit to actually be held by the group and not just the founder.

In the United States, this means creating or getting help from a "legal entity" -- a corporation or some other organization that is registered with the government. And if you want to ask for donations or apply for grant funding, people often expect or require that your organization is a registered charity, often referred to as a "501(c)3", which means that donors can deduct their donations from the yearly taxes they pay.

It is hard and annoying to set up a 501(c)3 organization! You probably need to pay a lawyer and accountant to do bits of startup paperwork, appoint a board of trustees and have regular meetings, and so on. Sometimes this burden is way more than volunteers want to take on -- and if you mess up the recordkeeping and fall behind in tax filings, it's a real headache to catch up.

So some nonprofit organizations offer "fiscal host" (also known as "fiscal sponsor") services. Just like it's a big pain to set up your own datacenter and so a lot of people instead rent server time from Amazon Web Services or Heroku, a lot of small projects get a membership with a fiscal host to get access to legal and financial infrastructure. In this analogy it's like getting a dorm room instead of building a stately manor. The fiscal host covers its own costs by taking a percentage of donations given to member projects.

In the arts, a popular fiscal host is Fractured Atlas. In open source software, you'll see NumFOCUS, Software Freedom Conservancy, and others.

Open Collective is particularly interesting here because its fiscal host service is fairly turnkey -- the application process is pretty streamlined -- and because a fiscal host within it, Open Source Collective, serves as fiscal host to nearly three thousand open source software projects. I would be surprised if there's another fiscal host out there that supports more.

The United States, employment, retirement, and health care

Your open source software project, once you're set up as a member project at a fiscal host, can now receive and spend funds. Great! So you can register domains, buy AWS credits and laptops and plane tickets, pay contractors...

Right, yes, you can compensate people for their labor, but in the US, the way you compensate them gets complicated. Because it's fairly easy to hire someone as a contractor ("freelancer"), but hard to hire them as an employee. And to talk about the difference I need to talk about how weird the United States is. In short: being hired as a "full-time" employee (usually at least 30 hours of work per week) usually gets a knowledge worker (such as a programmer) a lot of concrete benefits that would be unavailable, inconvenient, or more expensive if they were hired as a contractor, in particular concerning health care and saving for retirement. If you've been in the US workforce for several years you can probably skip this.

The United States, compared to approximately all other countries that have its level of wealth and infrastructure and so on, is completely strange and deficient in how we deal with healthcare and retirement-type care for senior citizens. A lot of this stuff is tied to employment here.

First: retirement. (I'll cover it first because healthcare will take longer.) How are people in the US supposed to support themselves after they stop working? Through a patchwork combination of stuff.

  • You can save and invest "normally" in bank accounts, real estate,* securities, and so on.
  • Some people get pensions (the employer keeps paying them after they retire) but far less than half the workforce can count on this, for various reasons.
  • Since 1935, we've had the Social Security program. Starting in one's 60s, almost every US worker is eligible for Social Security payments, and you get more if you earned more during your working lifetime. Some people can also get Supplemental Security Income. Many politicians scare voters by saying that Social Security is in crisis and that you will not be able to depend on it actually paying you any money by the time you retire.
  • Since the 1980s, under Internal Revenue Code Section 401(k), there's a special kind of account called a "401(k)" where a person can make contributions to be saved/invested towards retirement. An employer can sweeten the deal by "matching" your contributions up to some amount, such as $5,000 per year. A 401(k) must be employer-sponsored -- that is, you can't do it just by yourself -- and you usually only get access to 401(k) benefits if you are a full-time employee. But there are alternatives called Individual Retirement Accounts which a person can create independently. It's a bit complicated but, when changing jobs, one can often "roll over" a 401(k) from one employer to another so that you have one big growing account rather than a bunch of little ones. The money in a 401(k) or IRA account gets invested in a securities portfolio; the accountholder gets to make some choices about what to invest in. You and your employer contribute the money "pre-tax" (it's deducted from your taxable income, so you pay lower income taxes) and you can't withdraw money, till you retire, without paying tax on that withdrawal -- but there's often a one-time tax exemption where you can take money out to use when buying a home.

That last item, sponsorship for 401(k)-type account and possibly some employer matching for contributions, is basically what a knowledge worker in the US now expects as a part of an employment benefit package. (And I don't love it! I don't love being handed a bunch of poker chips and directed to the casino that is Wall Street and told: go invest your retirement savings! You're in charge!** But that's the current state of play.) If no organization is your employer, then you have to do a bunch of workarounds to get a similar means of saving for retirement, and you miss out on the possibility of employer-matched contributions.

And then there's healthcare. How are US residents supposed to pay for doctor visits, medicines, and so on?

This gets super complicated as you can tell by just skimming the table of contents for the English Wikipedia entry on "Health insurance in the United States". But to painfully summarize: instead of paying out-of-pocket for medical stuff, most people have a health insurance policy, and their health insurer decrees what is approved and what's not, what bills the individual has to pay, etc. And insurance companies negotiate down the rates for what they pay for stuff, compared to the "standard"/"out of pocket" rate, so uninsured people -- generally least able to afford healthcare! -- actually get the highest bills! The main ways people get health insurance in the US:

It's way too complicated! Even people eligible for government-subsidized insurance often don't know how to get it! "More Than 6 in 10 of the Remaining 27.4 Million Uninsured People in the U.S. are Eligible for Subsidized ACA Marketplace Coverage, Medicaid or the Children’s Health Insurance Program"! Costs have gone way up because for-profit insurers came into the industry and started raising premiums! We spend way more per person on health care than in other comparable countries and the quality and speed of care we get is less! And even insured people end up with huge medical bills -- medical bills are the number one cause of people in the US going bankrupt, which means selling or liquidating all your assets to pay your creditors!

And I haven't even gotten into the huge pain of choosing or changing health insurers and policies! Any given doctor, hospital, procedure, or medication may be covered by some health insurance policies but not others, and it can be tedious or even impossible to find out ahead of time whether a particular insurer will cover something! If you switch insurers, you'll sometimes have to find a new general practitioner or specialist! If your GP or a specialist stops taking your insurance then you have to scramble to find a new one! (Yes, this includes mental health practitioners!) This is particularly awful in rural areas with few doctors, or places where the only health facility around is affiliated with a religion that prohibits care that you need!

There's nearly a century of politics I haven't gotten into here -- the main thing to understand is that middle-class people in the United States are, reasonably, pretty scared of being really poor during our final years, or of being ill and really poor due to huge medical bills (which is way more likely if you don't have health insurance). And the main way we protect ourselves against those outcomes is by getting employed someplace that will give us employer-sponsored health insurance coverage and a 401(k) account.

If you make your wages as a contractor instead of as an employee, then it's harder and more tedious and more error-prone and more expensive to arrange for health insurance coverage and retirement savings. And you're less protected against changes in health insurance costs and thus against the headache of switching insurers. This basically is also true if you run a tiny business and are self-employed. And the precarity is particularly scary if you're disabled, or if your spouse or child has expensive health needs.

And so: if an organization wants to hire someone, to compensate them for labor, some people will only do it as an employee, not as a contractor.

But it's tedious and expensive to get set up to employ someone and give them those benefits, and to fund and administer those benefits on an ongoing basis! In contrast, there's very little paperwork needed to pay someone as a contractor. And that brings us back to open source projects....

What you can now do via OpenCollective

Thus: In the United States, the need for reliable health care and health insurance causes a tremendous number of open source contributors to have to take full-time jobs with employers. Sometimes these employers hired them to work on their open source projects, but more often, they're working either 0% or a very small percent of the time on open source, and they're working most of the time on proprietary software. So they squeeze in open source maintenance work during vacations, nights, and weekends.

A big focus in open source sustainability right now is finding ways to pay the maintainers. Instead of maintainers scrambling for nights-and-weekends spare time to maintain software, we should get them wages that would enable them to spend their core labor hours on open source maintenance. And though some companies and academic institutions are interested in employing particular maintainers full-time, it's probably more resilient if projects can take in relatively smaller donation streams from many sources, and combine them to hire maintainers.

But all the fiscal hosts and similar services I'm aware of that serve open source projects -- until now -- only let you pay contractors, not employees. They did not, until now, help member projects get employee-level benefits for individual laborers.

Until now.

Now, an open source project fiscally hosted by OpenCollective "can have employees, with access to benefits like health insurance. Costs related to employment are paid from the initiative's budget, with OCF as the employer." Employees must be based in the US. They're using Justworks, a company that helps small businesses provide employment benefits. In particular: 401(k) retirement plans and health insurance coverage.

So your open source project can gather donations via the OpenCollective platform, then use them to hire a US-based employee -- who reports to the project as a whole, not just one company, yet gets the benefits and at least some of the stability of a traditional employee.

Open source maintainers in the US now have substantially greater freedom to leave their jobs, go independent, and still protect their health and their future.

What this unlocks

Look at what's already happening with people who don't have to worry as much about health insurance. Check out Freexian, which is an effort where Debian developers club together and get sponsorship money, so they can each spend a certain number of hours each month consulting on really important parts of Debian software and infrastructure. A lot of those people who can take advantage of that are in Europe, or are in other places where health care isn't in question. So they can choose contracting work (or switch back and forth between full time employment and consulting, or combine flexible contracting with a stable part-time job) a lot more easily.

So now this possibility opens up more to US-based open source maintainers. We can better crowdfund and recruit US-based programmers and other workers to work on under-produced under-supported infrastructure, like Debian, or autoconf, or various glue libraries.

All the stuff we've been trying to do with grants, Tidelift, GitHub Sponsors, and similar initiatives: they're more likely to succeed, because more people -- both existing maintainers and apprentices willing to learn -- will be available to hire. If you run a program like Django Fellows, where you pay contractors to support the project through community management and code review, you can now expand your candidate pool and recruit US workers who want to work as employees.

And! we can better crowdfund and support innovative research, possibly in directions that big companies don't love. Indeed, we can better invest in FLOSS software that has no commercial competitor, or whose commercial competitors are much worse, because for-profit companies would be far warier of liability or other legal issues surrounding the project, such as youtube-dl.

More generally: any given open source software project that has a substantial user base now has a better chance at being able to hire one of its US contributors to provide ongoing maintenance and support. And so more projects will be able to sustain themselves with user support, instead of burning out unpaid volunteers and stagnating to a crawl and then a halt.

Some of this I'm basically copying and pasting from the "what if we had universal healthcare" section of my talk "What Would Open Source Look Like If It Were Healthy?" Because this is, potentially, a huge step for the health of open source.

I do consulting to help open source software projects get unstuck. Sometimes I advise them on which fiscal host or funding platform might suit their needs. The advice to get set up on OpenCollective has just gotten more attractive, and I hope other startups and nonprofits in the space pay attention. Adding this benefit to more fiscal hosting or funding services would be a tangible and significant way to improve open source contributors' freedom.

* There are a bunch of strange financial and tax advantages to buying a home, so one way people save for retirement is by buying a home, so they'll own it free and clear after retiring and won't need to pay housing expenses. When we say "buy" we usually mean "pay an initial payment called a 'down payment' to the seller, take out a loan called a 'mortgage,' move into the home, and gradually pay off the mortgage over 30 years." Yes, 30 specifically. Employers who want their benefits packages to help with this aspect of retirement planning might offer -- as Electronic Frontier Foundation does -- interest-free second mortgage loans for up to a portion or percentage of a home's price.

** Daniel Davies talks about pension "reform" in case you want some more thoughts.

Read the whole story
58 days ago
This is a good thing.
Ojai, CA, US
57 days ago
Boulder, CO
Share this story

What I've Learned About Writing as a New Engineering Manager

1 Comment

Managers at every level are prisoners of the notion that a simple style reflects a simple mind. Actually a simple style is the result of hard work and hard thinking; a muddled style reflects a muddled thinker or a person too arrogant, or too dumb, or too lazy to organize his thoughts. 

– William Zinsser, On Writing Well

While squinting hard at an email, I asked my boss:

"What does it mean that we're going to operationalize the plan?"
"It means we're going to do it," he said. 😐

Software engineering managers’ most precious resource is time. But you wouldn’t know it from reading our emails–bloated screeds of business buzzwords that we expect engineers to decipher.

Here are some tips on how to write an email so people will read it.

1️⃣ Have a point

Make sure you have something to say before you write.

Corporate-speak will write your email for you unless you remain vigilant1. Jargon lulls the writer into the false belief that they’ve said something precise while your reader is left wondering whether you’ve said anything at all.

It’s easier to say you want to touch base than to think about exactly why you want to have a meeting. But, really, why? If you can’t articulate a tangible goal, you haven’t thought about it enough to ask for a meeting.

Be direct and start your draft with the purpose of the email; ask what you want to ask or say what you want to say before writing anything else. This way: you can’t overwrite and forget to include your point. You can always wrap your plain language in the requisite business shibboleths later.

2️⃣ Keep it short

Write as if you were dying. At the same time, assume you write for an audience consisting solely of terminal patients.

– Annie Dillard, The Writing Life

Everything you write is too long.

People reading your emails aren’t fans of your writing—they’re trying to get through their email.

When you’ve finished writing your email: make it 10% shorter. Your writing will be more effective.

3️⃣ Make it easy

All visually displayed text involves typography

– Matthew Butterick, Butterick’s Practical Typography

It’s not enough for the action in your email to be easy; it’s got to look easy.

Appropriate typography and thoughtful information architecture make your email easier to read.

Researchers at the University of Michigan gave student test subjects two identical sets of instructions: one in a hard-to-read font and one in an easy-to-read font.

Despite the steps being identical, the student’s predictions of task difficulty were different. They thought the more intimidating text was a more demanding task. The author’s conclusion is the title of their study: “If it’s Hard to Read, It’s Hard to Do.”

Break up long text with headings. Keep your paragraphs short. Use bullet points and short sentences to make your text look less intimidating and easier to read.

A squint test of encyclopedia article vs. a popular article.

Squint test: Compare the shape of the first three paragraphs of a popular article about Barack Obama with the first three paragraphs of Barack Obama's Wikipedia entry.

Professional writers know to make it easy for their readers—the first paragraph on the left is a single sentence.

4️⃣ Make it ✨pretty✨

I know. Emojis 🙄.

In her book Because Internet, author and linguist Gretchen McCulloh posits that people embraced emojis because of their utility. Emojis are emblems of natural gesticulation — they add body language to our text.

Emojis can help people process the shape of your text at a glance. I use emojis to lead the eye through a text. Emojis are precognitive signposts you can use to reinforce the meaning of your writing.

Emojis can’t substitute for substance, but they can lower barriers for your readers.

I ❤️ the judicious use of emojis.

Further Reading


  • Hemingway Editor – In-browser editor that points out problems like overuse of adverbs and passive voice.
  • Grammarly – I currently pay $60 quarterly for this. I don’t use the browser extension since that seems likely to send every plain text field I fill in my browser to their servers. I don’t trust this service with my data, but I do like this service.

  1. Orwell said in “Politics and the English Language”: jargon will “construct your sentences for you—even think your thoughts for you, to a certain extent.”↩︎

Read the whole story
75 days ago
Heh, he mentions me :)
Ojai, CA, US
Share this story

5 Great Phabricator features that inspired GitLab

1 Comment

Innovation often happens because competition sparks new ideas. We unpack how Phabricator inspired GitLab to add new features.

Turning back time a bit, what exactly is Phabricator? Built on the concept of web-based applications, Phabricator enables developers to collaborate with code reviews, repository browser, change monitoring, bug tracking and wiki. On May 29, 2021, Phacility, the maintainer and sponsor of Phabricator announced end-of-life and stopped maintaining Phabricator.

GitLab co-founder and CEO, Sid Sijbrandij gives credit to Phabricator on HackerNews:

Phabricator was an inspiration to me when starting GitLab. It is shutting down now. Many of its features were years ahead of its time and there was a lot of humor in the product. As a tribute to it shall we add Clowcoptarize as a way to merge? This would be an opt in option introduced in GitLab 14.0.

It got me curious: What are these inspirations Sid is referring to? Let's dive into GitLab's history together and see what we can learn.

Tip: Features in the GitLab documentation often have a Version History box. You can use the issue URLs to dive deeper into feature proposals, discussions, etc.

Review workflows

A typical engineering workflow is as follows: The engineering manager assigns a new issue as a task to a developer. The developer works in their preferred IDE – local in VS Code or in the Gitpod cloud environment. Changes happen in a new feature branch in Git, which gets pushed to the remote Git server for collaboration.

The Git branch is not ready yet and stays hidden in a potentially long list of branches. To keep better track of their feature branches, developers could copy-paste the branch name or URL into the related issue - which I did 10 years ago. The concept of a "diff linked to a task for review" in Phabricator, likewise a "Git branch with commits linked to Merge Requests" in GitLab was not invented yet.

Phabricator inspired GitLab to create a default workflow for reviews. The Phabricator workflow makes the review more dominant and squashes all changes into a single commit after the review is approved. There are upsides and downsides to automatically squashing commits. Squashing the commits could mean losing information from review history and create more discussion. Depending on the application architecture, the frequency of changes, and debugging requirements, this can be a good thing or a bad thing. GitLab allows you to choose to squash commits before merging a MR and/or specifying the default project settings around squashing commits.

Phabricator treated a MR (or what they call "diff tasks") as the single source of truth for tracking changes and the review history. We felt this was a great idea, and replicated the process of a "diff task" in Phabricator in GitLab MRs. One of the major upsides to GitLab's version is that collaboration and discussion that happened in issues and epics is still available even after the change is merged.

Draft MR (or "diff tasks")

Many times when a MR is created in GitLab, the branch requires additional work before it is ready to be merged. Phabricator introduced a formal "Draft" / "Not Yet Ready for Review" state in 2013 for "diff tasks", which helped keep track of work in this state. GitLab added WIP MRs in 2016, which we then renamed to draft merge requests in 2020. While WIP may make sense to some people, acronyms can exclude newcomers. We found Draft is more recognizable. To avoid confusion, GitLab deprecated WIP and moved forward with draft merge requests.

Keep history in MRs for future debugging

The commit history in GitLab is enriched with links to the MR and the corresponding Git review history. In case of a production emergency, having everything documented allows for faster research and debugging.

GitLab stores the MR short URL with <namespace>/<project>!1234 in the merge commit message. Check the history of a demo project for the Kubernetes agent to see how the merge commit is rendered.

GitLab history with MR commit links GitLab commit history includes link to the MR.

This raw information is stored in the Git repository, whereas the MR itself stays in GitLab's database backend. You can verify this by cloning a repository and inspecting the history with this command:

$ git log

git log MR metadata MR metadata included in output from git log command.

Code coverage in MRs

Code coverage reports provide insight into how many lines of the source code are covered with unit tests. Reaching 100% test coverage is a developer myth - visualizing a decrease or increase can help monitor a trend in code quality. Phabricator implemented support for various languages with unit test engines and parsing the output, for example in Golang.

With many different languages and report output formats, integrating code coverage reports into GitLab MRs was challenging. GitLab launched the first iteration of code coverage reports in 2016, which generated the reports with CI/CD jobs and used GitLab pages to publish the HTML reports.

In this first iteration, the test coverage is parsed with a regular expression from the CI/CD job output, specified in the project settings or with the coverage keyword inside the CI/CD job configuration. We can see this in the job view inside the MR widget and as a coverage badge for the project. See the test coverage history by navigating into Analytics > Repository.

Test coverage as project badge in GitLab The test coverage badge in a GitLab project.

JUnit XML test reports were introduced as common format specification and added as an MR widget in 2018. The test reports runs in the background, using CI/CD artifacts to upload the XML reports from the runner to the server, where the MR/pipeline view visualizes the coverage reports in a tab.

The generic JUnit integration also helped with customization requests to unit tests, updated CLI commands, or changed coverage report outputs to parse. GitLab provides CI/CD template examples

The missing piece for GitLab was having inline code coverage remarks inside MR diffs. It took about five years for Sid's initial proposal for inline code coverage remarks to be implemented. In 2020, inline code coverage remarks were released in GitLab 13.5.

Test Coverage with Rust in GitLab How inline code coverage works in GitLab.

Check out this MR to practice verifying the test coverage. Make sure to select the inline diff view.

Automated workflows and integrated CI

Phabricator provides Herald as an automated task runner and rule engine to listen for changes. Herald can also be used to ensure protected branches and approval rules to enforce a strong permission model in development workflows. There are more examples in this HackerNews post from 2016 and somehow, I feel like an explorer seeing many great GitLab features in similar ways. 🦊

GitLab CI/CD pipeline schedules remind me of the task runner, similarly to webhooks and the REST API being instrumented from CI/CD jobs. The pipeline schedules are also a great way to periodically regenerate caches and rebuild container images for cloud native deployments.

Harbormaster is Phabricator's integration for CI. It's not built from multiple tools in the DevOps stack, but is instead fully integrated in the product.

The first version of GitLab CI was created in November 2012. In 2015, a GitLab team member came up with the idea of combining SCM with CI and the all-in-one DevOps platform was born. Built-in CI/CD inspired for more features and fostered a better way to innovate together. The new pipeline editor is just one example of a streamlined way to configure CI/CD pipelines in GitLab.

Let's throwback to 2017 and watch as we demonstrate how to take an idea to production in GitLab, using GKE:

Work boards for issue management

Work needs to be organized. Phabricator led the way with a board which allowed users to filter tasks and provide a more detailed view into planning and project management.

Phabricator work boards Inside Phabricator work boards.

GitLab users will recognize the similar look between Phabricator's work boards and GitLab issue boards. In GitLab 14.1, we built on existing epic tracking and labeling to create Epic boards to keep teams organized and measure progress.

In Phabricator, users can drag and drop between columns, which automatically changes the work status for a particular task. This feature inspired the boards in GitLab to automatically change the labels in a defined workflow by dragging and dropping between columns. Users can go a level deeper with scoped labels to switch between workflow states:

  • workflow::design
  • workflow::planning breakdown
  • workflow::ready for development
  • workflow::in dev
  • workflow::verification

The GitLab engineering handbook documents the different workflows.

Epic boards in GitLab Take a look at the Epic boards in GitLab.

Put it all together

In Phabricator, a diff task (in GitLab they're MRs) in the "review" state is linked to another task specifying the requirements. The UX needs to be clear so the relationship between the diffs can be accessed and understood. Unless necessary, the user shouldn't have to navigate manually. The context of the change review defines possible links to labels, states, dependent issues, diff tasks (MRs), and more.

GitLab links related issues. If an issue is mentioned in a MR, or vice versa, GitLab automatically links them. The user also has the option to have the issue close automatically once a change is merged. Read a blog post from 2016 to learn more about how issues and MRs can relate to each other in GitLab.

Linked issues and MRs in GitLab Linked issues and related MRs in GitLab.

UX work is challenging, and we continue to iterate to improve workflows in GitLab. For example, in GitLab 13.8, we reduced the number of clicks it takes to download a CI/CD job artifact from the MR.

Did we miss a feature Phabricator inspired?

While writing this blog post, my research revealed more gems. For example, I found a proposal to add visual graphs for issue dependencies in the HN thread.

Which features from Phabricator are missing in GitLab? Let us know in the comments, create a new feature proposal or start your contribution journey in a new MR right away!

Cover image by Johannes Plenio on Unsplash

Read the whole story
161 days ago
A nice gesture. To be fair, there is a community supported fork called phorge (of course).
Ojai, CA, US
Share this story

Nobody gives a hoot about groupthink


People want what’s bad for them; managers want what’s bad for the organisation.

Many in software development like to pretend that management is an elite kind of human being. The kind who does their job every day with a laser focus on rational organisational improvement. People who come in every day and work with an inhuman disregard for their career and standing. After all, the root cause of all software flaws lies in the code and coders, right?

Unfortunately, the buck always stops at management. And, most of the time, a person can only care so much about the organisation as a whole.

In the immortal words of Deming:

Nobody gives a hoot about profits.

A recurring problem specific to software development is that many of the popular features that people keep asking for have a detrimental effect on the quality of their actual work.

Namely: managers and stakeholders frequently prioritise fashion and ‘being current’ over objective improvements in software quality or user experience. Because those are the priorities of those who buy software.

Two relatively common ‘fashions’ today are real-time collaboration and shared data repositories of one kind or another.

Both increase productivity in the naive sense. We work more; everybody is more active; the group feels more cohesive.

The downside is that they also both tend to reduce the quality of the work and increase busywork.

Real-time collaboration

One of the oldest observed phenomena in psychology is groupthink and other forms of group influence on behaviour and decision-making.

A common implementation style for real-time collaboration is the Google Docs model:

  • The list of those present in the document is visible to all collaborators.
  • Everybody’s current activity is visible to all.
  • Everybody’s notes are visible to all, with attribution
  • Each person’s contribution is generally identifiable.

The consequences of this design should be obvious. The group’s opinion will converge on that of the highest authority present.

As soon as an authority of any kind makes their opinion known, the group will shift in that direction. Even the most rational will tweak their responses after that. After all, who wants to risk going up against an authority? Interns will hesitate to comment. All objections will be a little bit more qualified or toned down.

Generally speaking, if you are writing a document and want to get the most out of a group’s feedback, each contributor should be able to form their opinions independently and give their responses without fear of social or community repercussions.

It’s one of the basic precepts of the book The Wisdom of Crowds, but it applies to any context where you want to leverage the expertise of a heterogeneous group.

This style of collaboration also increases busywork: everything is moving and changing everywhere all the time. Comments fly by. You change what you’re writing half a dozen times in response to what appears. You write a series of notes, noticing later that six other people said the same thing on another page and that somebody added three pages to that same effect towards the end.

Real-time is exciting because it’s busy, much busier than any other form of online collaboration.

But stakeholders never ask for collaboration features based on the dynamic aggregation of independent contributions. Cause that isn’t the fashion. Google’s crap is the fashion. So, we get real-time collaboration and busywork everywhere.

Shared data repositories

Another Google-inspired user experience atrocity that has become commonplace is the shared drive.

Did you know that once-upon-a-time Information Architecture was considered a specialised field of study? Did you know that organising information is considered such a complicated endeavour that there is a massive field dedicated to it?

Organising information so that it’s easy for a group of people to find the documents they need is very hard.

Not that you’d know it from how most companies work. Almost everybody builds their internal library of documents as an improvised layer of garbage and weeds. Like a junkyard, converted to a garden, that overgrew with weeds, and was converted back to a junkyard. If you’re extremely unlucky, one of your employees will be obsessive-compulsive enough to restructure everything. They’ll clean the place up and reorganise it in a manner that makes sense to them. They’ll transform an inscrutable multi-layered archaeological site into a maze. Your shared drive is now a labyrinth made of trash and random horticultural curiosities.

Let’s fix it with a wiki! (Yay. Yet another layer of opinions and obfuscation.)

You could hire somebody with the expertise to organise everything but then you’d miss out on the constant busywork and sense of camaraderie created by a hellscape of abandoned Google Drive folders.

The alternative is to solve it the same way we did with email: shared data, individual organisation.

You don’t need to know how your colleagues organise their email. You only need to know that they get it and respond. The same applies to most work documents. In Personal Information Management (PIM) this is often called “the user-subjective approach”.

From The Science of Managing Our Digital Stuff by Bergman and Whittaker:

Because information consumer differ from each other in multiple ways, the information professional is restricted to exploiting only public (i.e. user-independent) attributes when organizing information. PIM systems, in contrast, are unique in that the person who stores the information and decides on its organization is the same as the person who later retrieves it. (p. 182)


An early PIM study demonstrated the critical role of subjective attributes, inspiring the development of the user-subjective approach. Kwasnick (1991) analyzed the descriptions of eight faculty members who were asked to describe how they organize their personal documents. She found that a minority (30 percent) of the attributes were document-related (e.g. author, form, topic, title). In contrast the majority (70 percent) related to the interactions between the user and the information in the document, in particular how the user perceived and acted upon that information (e.g., situational attributes, disposition, time, cognitive state). Thus, users base their natural organization more on subjective attributes than on general public ones.

If you ever wondered why so many people use their email to organise all of their work, this is why. It’s the only user-subjective information management tool at their disposal.

Most companies would benefit from standardising on a user-subjective information management system but they don’t. Nobody asks for this kind of system so what we get are Google Drives, Dropboxes, shared Notion projects, or weed-like shared Roam spaces.

Meanwhile, individuals cope by overloading their email or by clinging onto software like Zotero or nvAlt that looked ancient even when it was new. Some will just manage copies of everything in their private Dropboxes. Others will cling to DEVONThink for dear life.

Nobody expects their employer to ask for or provide a tool that genuinely works. Because that never happens. To return to the Deming quote at the start:

Nobody gives a hoot about profits.

Read the whole story
162 days ago
Slack will never replace (my) email.
Ojai, CA, US
171 days ago
"People want what’s bad for them; managers want what’s bad for the organisation."

"If you ever wondered why so many people use their email to organise all of their work, this is why. It’s the only user-subjective information management tool at their disposal."
Boulder, CO
Share this story
Next Page of Stories