Behind the Curtain: Moderating Packages on the Chocolatey Community Repository

In previous blog posts, we’ve talked about what the Chocolatey Community Repository is, and the package moderation process that packages go through when they have been submitted to the repository. These posts, and the documentation on the Chocolatey documentation site, mention a “Human Moderation” step; but what does this step actually involve, who does it, and what sort of things are these Moderators looking for?

Moderating the Chocolatey Community Repository is a labor of love for a small group of volunteers and Chocolatey Team Members, who serve as a final check as packages pass through the moderation pipeline looking for issues that the automatic Package Moderation Services cannot. When a moderator approves a package, they are saying “I’ve reviewed this package, and I am willing to vouch that it doesn’t violate any guidelines or rules.”

The Moderators also provide help and guidance to package maintainers when they have questions, or need a deep-dive into the Package Moderation Services testing results.

Throughout this post, I’ll use the term ‘we’ to refer to the moderation team as a whole, however I’m writing from my own perspective and the process I employ when moderating packages.

Why Involve Humans?

Beyond the guidelines, Moderators are focused on one core goal: making sure a package is safe. By the time a package reaches us, the Package Validator service has already checked it against our baseline quality rules, and the Package Verifier service has tested that it actually installs. From there, moderators dig into the package’s metadata, scripts, and other contents to understand what it’s trying to do, confirm that it does what it claims, and ensure that, if approved, users can install it with confidence.

WARNING

While Moderators do their best to ensure that packages on the Chocolatey Community Repository are safe to use, you should always carry out your own due diligence before installing a package.

Since the Chocolatey Community Repository is open to anyone, and because the heart of most Chocolatey packages is just a PowerShell script, a package can theoretically “do” almost anything that PowerShell can do. It’s worth noting that packages are not software, though they often install software. Without a human in the loop, a bad actor could push a package that passes automated checks but does something dangerous or deceptive when run. A smoke test by a human reviewer brings contextual understanding that automated tools can’t replicate and can mean the difference between a trustworthy package and a hidden security risk. But more than just catching malicious intent, Moderators can spot when metadata doesn’t align with the included software, when license terms are misrepresented, or when a package uses creative (or questionable) workarounds to get around restrictions.

Humans also recognize nuance. For example, some software authors / vendors don’t offer silent installation options and so a package maintainer may include a AutoHotkey script to achieve an interaction-free installation. Automation can’t answer why that script has been included and if it does what a casual observer would expect it to do.

Of course, the human touch comes with a cost: time.

The majority of Chocolatey Community Repository Moderators are volunteers that review packages in their spare time. This means that the speed at which a package is reviewed can vary. This makes moderation one of the repository’s strongest features but also its biggest frustration. It’s a delicate balance between expedience and safety. And trust me, we’re just as frustrated when the moderation queue fills.

Ultimately, involving humans in moderation is what allows the Chocolatey Community Repository to remain open to contributions while still offering users a sense of safety, transparency, and quality. We’re the last line of defense, and it’s the human element that makes that line meaningful.

Tips for Package Maintainers: Passing Moderation Smoothly

To make your life (and ours) easier, keep these things in mind when submitting your packages:

• Start from a package template. Using the choco new command will scaffold out a new package for you, which you can complete and add other content your package needs to function, such as scripts or binaries or installers.
• Include all required fields and ensure all URLs are valid. The Package Validator service generally picks up on this particular issue, but sometimes they make it through to the moderation team.
• Install the Chocolatey Community Validation Extension to catch common issues before you push your package.
• Use meaningful tags to help users discover your package, but don’t go overboard. Avoid generic tags, like chocolatey, and expect to be asked to amend a list greater than 10.
• Understand redistribution rights. It is possible to embed a software installer inside your package, but the software’s license must allow for this, and you will need to include the full license and instructions on how to validate those files within your package.
• Package icons should be hosted at a location you control. It is common for this to be the GitHub repository in which you store the contents of your packages, but do note that you should not link directly to the icon on GitHub and instead use a CDN service like jsDeliver, Statically, or Githack.
• Name your package correctly. Use only lowercase letters, split long names with hyphens (-), and don’t use dots.
• Don’t create duplicate packages. If there is an existing package that is out of date, don’t try to push an updated copy of it under a different ID. Instead, volunteer to take over the package by following the Package Triage Process.

Make sure you use all resources available to you, for example:

• Package Creation: Create Packages
• Package Creation: Quick Start
• Creating Chocolatey Packages, Step-By-Step, the Easy Way!

And if you’re unsure about something, ask! Jump onto our Community Hub and ask your question in the #community-maintainers channel.

Time, Effort, and Workload

As mentioned above, the majority of the Chocolatey Moderation Team are volunteers. Even those of us who are members of the Chocolatey Team will moderate packages in our free time. As a small team, the time for a package to be reviewed can vary from a few days to a few weeks. We aim to keep the queue low, but sometimes it fills, especially around major holidays, summer, and year-end, but the volume of packages is not insignificant; there were over 6,000 packages manually reviewed in the last 12 months!

How long does it take a Moderator to review a package? There isn’t one single answer to that question. A brand-new package from a first-time maintainer could take an hour to go through just to get to the point of sending a message asking for some changes to be made. Other packages that are updates to an existing package from a “seasoned” maintainer may only take 5 to 10 minutes.

We maintain full transparency on moderation and publish monthly statistics in the #community-repository channel on our Community Hub, including how many packages were reviewed by which Moderator and the average moderation time.

Becoming a Volunteer Moderator

We love our work, but we always welcome more helpers. If moderation seems interesting to you, you could make a big difference. If this sounds like something you’d enjoy, we encourage you to join us!

There is no formal process for becoming a moderator, but a good first step is to ensure you understand Chocolatey packaging and have published and successfully maintained packages on the Chocolatey Community Repository. Once you feel comfortable packaging, pop onto our Community Hub and let us know that you’re interested. We’re a friendly bunch, and veterans are generally happy to mentor new reviewers.

Wrap Up

Package moderation keeps Chocolatey packages safe and reliable for everyone, but it’s a team effort. By understanding the process and following the guidelines above, maintainers can make approvals smoother. And if you’ve been considering giving back, joining the Moderation Team is a great way to do so.

Whether you’re maintaining packages, reviewing them, or just curious, the Chocolatey Community Repository thrives on shared effort. Come say hi; we’d love to meet you!

Find us on: