This blog post complements a live demonstration on our recently announced site: experiments.github.com Motivation Searching code on GitHub is currently limited to keyword search. This assumes either the user knows the syntax, or can anticipate what keywords might be in comments surrounding the code they …
We have recently completed a milestone where we were able to drop jQuery as a dependency of the frontend code for GitHub.com. This marks the end of a gradual, years-long transition of increasingly decoupling from jQuery until we were able to completely remove the library. In this post, we will explain a bit of history …
At GitHub, we serve tens of thousands of requests every second out of our network edge, operating on GitHub’s metal cloud . We’ve previously introduced GLB , our scalable load balancing solution for bare metal datacenters, which powers the majority of GitHub’s public web and git traffic, as well as fronting some of …
GitHub uses MySQL as its main datastore for all things non- git , and its availability is critical to GitHub’s operation. The site itself, GitHub’s API, authentication and more, all require database access. We run multiple MySQL clusters serving our different services and tasks. Our clusters use classic master-replica…
Until last week, GitHub used an Out of Band Garbage Collector (OOBGC) in production. Since removing it, we decreased CPU time across our production machines by 10%. Let’s talk about what an OOBGC is, when to use it, and when not to use it. Then follow up with some statistics about the impact of removing it from …
GitHub recently open sourced Licensed in the hopes that it is as helpful to the OSS community as it has been to us. <disclaimer> 1 of 1 consulted lawyers agree, Licensed is not a replacement for the legal advice of a human. </disclaimer> Glossary Before we go any further, let’s review a few …
On Wednesday, February 28, 2018 GitHub.com was unavailable from 17:21 to 17:26 UTC and intermittently unavailable from 17:26 to 17:30 UTC due to a distributed denial-of-service (DDoS) attack. We understand how much you rely on GitHub and we know the availability of our service is of critical importance to our users. …
Last year we announced the deprecation of several weak cryptographic standards. Then we provided a status update toward the end of last year outlining some changes we’d made to make the transition easier for clients. We quickly approached the February 1, 2018 cutoff date we mentioned in previous posts and, as a …
GitHub’s Spokes system stores multiple distributed copies of Git repositories. This article discusses how we got Spokes replication to span widely separated datacenters. Background: Spokes GitHub developed a system called Spokes to store multiple replicas of our users’ Git repositories and to keep the replicas …
At GitHub, we use MySQL as the main database technology backing our services. We run classic MySQL master-replica setups, where writes go to the master, and replicas replay master’s changes asynchronously. To be able to serve our traffic we read data from the MySQL replicas. To scale our traffic we may add more …
Over the past 18 months we’ve made a significant investment in GitHub’s physical infrastructure. The goal of this work is to improve the redundancy and global availability of our system. In doing so we’ve solidified the foundation upon which we will expand our compute and storage footprint in support of our growing …
GitHub is at a scale that provides exposure to interesting aspects of running a major site and are working to mature and level-up many parts of our infrastructure as we grow. One of the areas where this is evident is in how your requests find their destination using DNS and make their way into our sites over transit …
GitHub is proud to handle thousands of requests per second from our millions of users. The Internet, however, can be a fickle beast of cables and sparks, and sometimes those requests don’t happen very fast (or at all). While we’re happy to help you troubleshoot connection issues to us, we also know our users like …
Earlier this year , we announced the deprecation of several weak cryptographic standards. As noted during our initial announcement, the vast majority of HTTPS clients connect to GitHub using TLSv1.2 and won’t be affected by our disabling of TLSv1 / TLSv1.1 . Since the announcement, we have been focusing on the …
Visibility is essential to effectively operating complex systems. As our network has grown, we’ve had to improve the the way we collect data about it to keep up. Key to these improvements has been the ability to tag metrics. Tagging has allowed us to build dashboards that start at a high level and facilitate drilling …
Over the last year, GitHub has gradually evolved the infrastructure that runs the Ruby on Rails application responsible for github.com and api.github.com . We reached a big milestone recently: all web and API requests are served by containers running in Kubernetes clusters deployed on our metal cloud . Moving a …
We recently launched Topics , a new feature that lets you tag your repositories with descriptive words or phrases, making it easy to discover projects and explore GitHub.com. Topic suggestions on public repositories, provides a quick way to add tags to repositories. These suggestions are the result of recent …
In an effort to increase the adoption of FIDO U2F second factor authentication, we’re releasing Soft U2F : a software-based U2F authenticator for macOS. We’ve long been interested in promoting better user security through two-factor authentication on GitHub.com. Initially, we added support for TOTP …
Our MySQL infrastructure is a critical component to GitHub. MySQL serves GitHub.com, GitHub’s API, authentication and more. Every git request touches MySQL in some way. We are tasked with keeping the data available, and maintaining its integrity. Even while our MySQL clusters serve traffic, we need to be able to …
At GitHub we recently revamped how we do DNS from the ground up. This included both how we interact with external DNS providers and how we serve records internally to our hosts. To do this, we had to design and build a new DNS infrastructure that could scale with GitHub’s growth and across many data centers. …