title: "DigitalOcean Sponsorship"
date: 2021-05-07T23:30:30-04:00
description: "" # Leave blank to use the auto summary
images: [] # Leave empty to default to the Pidgin logo
- /posts/2021-05-digital-ocean-sponsorship/ Over the last couple of years, Pidgin, and by extension our non-profit
corporation called [
Instant Messaging Freedom, Inc.](
https://imfreedom.org/),
has received sponsorship from a hosting company called
[
DigitalOcean](
https://www.digitalocean.com). DigitalOcean provides a variety
of hosting services, including virtual private servers, managed kubernetes
clusters, and so on. They also provide sponsorship for open source projects,
whereby they provide credits to use to offset the costs of their services.
This sponsorship has been the source of Pidgin and Instant Messaging Freedom's
primary infrastructure. Currently we run this website, our
[
JetBrains Hub](
https://www.jetbrains.com/hub/) centralized authentication
system (running at [
hub.imfreedom.org](
https://hub.imfreedom.org)), our
[
JetBrains YouTrack](
https://www.jetbrains.com/youtrack/) instance (running at
[
issues.imfreedom.org](
https://issues.imfreedom.org/)), our
[
Mercurial](
https://mercurial-scm.org) hosting solution,
[
HGKeeper](
https://keep.imfreedom.org/grim/hgkeeper/), that runs at
[
keep.imfreedom.org](
https://keep.imfreedom.org), and a number of other tools
and services all from the DigitalOcean-provided infrastructure. In the coming
weeks and months, we will have all of our infrastructure, including e-mail and
our previous bug tracking/wiki system, running entirely in DigitalOcean's
datacenters thanks to their generous sponsorship.
We get a lot of people asking why we self-host when there are resources like the
ubiquitous GitHub that would do a lot of the work for us. That's a fair
question, and admittedly we're not always the most patient when answering. The
answer is a bit longer than you'd initially expect.
First and foremost, Pidgin is an end-user application. Sure, we have lots of
developers who use Pidgin, but our target audience is end users, not developers.
End users who aren't developers are going to find the user experience of a tool
like GitHub, GitLab, or similar to be very lacking. In the current landscape,
self-hosting is essentially the only way to get the end-user-facing aspects of
our infrastructure to be as friendly as possible while still being relatively
### Repeatedly Burned, Always Cautious For many years, we used [
SourceForge](
https://sourceforge.net) for our hosting
needs. This included our ancient PHP-based website, our CVS source code
hosting, issue tracking, mailing lists, and our downloads. SourceForge was, and
still is, a gracious host that served us very well. Around the time we changed
our name, we secured alternative hosting for much of those items because the
SourceForge issue trackers were cumbersome and unwieldy and we wanted to move
away from CVS and Subversion and toward distributed version control.
We migrated to [
Edgewall Trac](
https://trac.edgewall.org) for issue tracking and
wiki management, and [
Monotone](
https://www.monotone.ca) for version control.
At the same time, since SourceForge couldn't handle these tools, we migrated to
a hosting provider that our friends at the [
Adium](
https://adium.im) project
were using. This provider donated hosting to Adium and agreed to donate hosting
to us as well---in our case, two virtual private servers. That hosting provider
exited the hosting business in 2020, after both Adium and Pidgin had been with
them for well in excess of 10 years.
We also were using a binary hosting provider to host Continuous Integration
artifacts and our releases as an alternative to SourceForge for those who still
held animosity and distrust for SourceForge for actions under previous ownership
and management. This hosting service has announced it is exiting from the
hosting business in mid 2021.
However, the biggest blow to us was [
Atlassian BitBucket](
https://bitbucket.org)
dropping Mercurial support. We had migrated to Mercurial after our time with
Monotone proved that we needed a different version control tool. After that
migration, in our development workflow for Pidgin 3.0.0, we had become dependent
on the pull request workflow, issue tracking tools, and continuous integration
system provided to us there. Fortunately, Atlassian had the courtesy to
announce the removal of Mercurial support far enough in advance that Gary was
able to write HGKeeper and get our repositories migrated away from BitBucket in
time to prevent the loss of our repositories. (We have no desire to migrate to
git and GitHub, GitLab, or similar, for reasons beyond the scope of this post.)
Most recently, just within the last few days a tool we were using to monitor our
services, [
UptimeRobot](
https://uptimerobot.com), announced and then implemented
severe curtailments to the functionality of their free monitoring offering. We
were using UptimeRobot to monitor and alert on a variety of our infrastructure's
components and services; the changes to the free offering makes it no longer
viable for us, thus we were forced to migrate to another tool.
All of these losses have made us rather wary of becoming too dependent on
specialized hosting providers and services. We're now much more inclined to
build our own infrastructure in a generic, repeatable way that allows us to
migrate to a new generalized hosting provider if we ever need to.
Quite honestly, two reasons. First, they were the only hosting provider we knew
of at the time which offered managed kubernetes clustering. Second, the Open
Source project sponsorship. Gary, in particular, wanted the managed kubernetes
functionality due to his previous experience with it. We became aware of the
sponsorship later and it was essentially a bonus to us.
## How Does Pidgin Use DigitalOcean? Without getting
_too_ technical, we have five "Droplet" virtual private servers,
four of which are a managed kubernetes cluster. The fifth will host our e-mail
and mailing lists when we are able to sort out a few challenges. We also have
a "small" load balancer (DigitalOcean's term) to handle ingress into the various
services running on the cluster, including spreading across multiple instances
of a given service within the cluster.
Our aim is to run everything possible in the cluster, with all the configuration
(except for secrets) well-defined and
[
version-controlled](
https://keep.imfreedom.org/imfreedom/k8s-cluster). This
allows us to have automatic service recovery in the event of a problem or a need
to perform maintenance on one of the Droplets (such as upgrading to new
kubernetes releases or container builds). Where possible, we also aim to run at
least two instances of services within the cluster to provide a measure of high
availability. For example, the container which serves this website runs on at
least two nodes in the cluster at any given time and the load balancer will
spread the traffic across all running instances of the container. If an
instance has a failure, the load balancer stops sending traffic to that instance
until the cluster recycles the container and resolves the problem. This is all
fully automatic, with no need for human intervention. The cluster self-heals
for the vast majority of container failures, which was a huge part of the appeal
to us, especially Gary, due to limited time to deal with administration.
We've even moved our Trac instance, now read-only except to a select few
long-time Pidgin developers, into the cluster to reduce the overall complexity
of our hosting and allow another hosting provider to retire the aging dedicated
physical server currently running Trac.
With the functionality built into the managed kubernetes clustering, TLS
certificates, whether for HTTPS (web services), XMPP messaging, or any other
TLS-capable service, are automatically managed. Using the [
Let'sEncrypt](
https://letsencrypt.org) certificate authority and the integration into
the clustering, all our TLS certificates are automatically issued, renewed, and
replaced dynamically as needed with no intervention required.
Within the cluster we also run the [
website](
https://imfreedom.org) and the
[
protocol documentation wiki](
https://wiki.imfreedom.org) for Instant Messaging
Freedom, Inc. As mentioned before, Hub, YouTrack, and HGKeeper also run within
the cluster, all in their own containers, along with some other tools that we
aren't yet able to make public.
Finally, we also run the [
Prosody IM](
https://prosody.im) XMPP server in the
cluster. This service provides Pidgin and Instant Messaging Freedom, Inc. with
instant messaging services over the XMPP protocol. It also provides the basis
for the [
PidginChat](
{{< ref "/about/pidginchat" >}}) service many of our users
In conclusion, the Pidgin project and Instant Messaging Freedom, Inc. would like
to thank DigitalOcean for their generous sponsorship that allows us to continue
to develop free and open source messaging software for the benefit of the entire
