As an independent open source project, the Jenkins project maintains most of its own infrastructure including services which help to keep the project running. The kind of things that fall into "infrastructure" can span from operating virtual machines, containers, configuring network or developing and maintaining project-specific applications to make the development of Jenkins core and plugins more efficient.
Because we strongly believe in Open Source principles, we also apply them to our infrastructure. As such we consider ourself as an open infrastructure project where everybody is invited to learn, share, contribute.
The Jenkins project has a partnership with Microsoft to run the infrastructure on Azure. This infrastructure is provisioned from the repository jenkins-infra/azure using Terraform.
While we try to stick to Azure, we still have machines sponsored by different organizations such as OSUOSL, Rackspace or CloudBees.
The Jenkins project’s infrastructure is managed and orchestrated via a git repository named jenkins-infra/jenkins-infra using Puppet and we are also experimenting Kubernetes cluster orchestration with jenkins-infra/charts using Helm & Helmfiles.
Infrastructure is monitored via Datadog. Datadog configuration is automated with Terraform on jenkins-infra/datadog
We also provide different dashboard to share the health of our systems.
Another major aspect of the Jenkins Infrastructure project, is the maintenance of all services either provided by us or third parties. Here is a non-exhaustive list of services that we provide and maintain.
Service |
Provided by |
Repository |
Jenkins-infra |
||
Jfrog |
Jfrog |
|
Jenkins-infra |
||
Github |
- |
|
IRC Bot |
Jenkins-infra |
|
Jenkins-infra |
||
Jenkins-infra |
||
Jenkins-infra |
||
Ldap |
Jenkins-infra |
|
Jenkins-infra |
||
Datadog |
||
Vpn |
Jenkins-infra |
|
Pagerduty |
- |
|
Jenkins-infra |
||
Jenkins-infra |
||
Jenkins-infra |
||
Jenkins-infra |
Our infrastructure is an open infrastructure project made by and for the Jenkins community in other words, it’s a contributors driven project. While we can’t share publicly everything like secrets, certificates,… we still try to be as transparent as possible so everybody can read, understand, improve our infrastructure without having privileged accesses. If you have any idea that could help the infra or interest the community, feel free to make suggestions.
In order to contribute to infrastructure project, we ask people to follow the next steps
Pick up a task => Communicate => Implement => Deploy => Review
In order to keep track of the work that need to be done on the Jenkins infrastructure project, we use the project "INFRA" on Jira. Therefor the first thing to do before any contribution is to find the right ticket, assign it to you, then communicate about it.
If you can’t find an appropriate ticket, please create a new ticket with a clear description of what needs to be done and why. Some jenkins-infra git repository references can help to understand the context. You may also specify components and finally you can communicate about it, using the suggestions from the next section.
While a ticket can have different components assigned to it, we also use the label newbie-friendly to identify task which can be done by a new contributor.
Before any implementation, it’s important to verify that first, there is (still) a need for some implementation and then that no work has been done in the past. The best way for that is to either look after similar Jira tickets, ask on IRC or on the mailing list.
When the subject is too broad or hard to explain in few lines, we write an IEP document which stand for "Infrastructure Enhancement Proposal", the purpose of this document is to explain why we need something, how we want to solve it, and why we took a final decision. Finally, once you have your ticket id, you can start looking after knowledgeable people.
Anyway keep in mind that it’s always better to have too more information than too few and in the end you’ll probably be the best person who can work on your case.
+----------------------------------+
| |
| Pick up or Create INFRA Ticket |
| |
+----+----+------------------------+
| | If no responses after few days
| | promote it on
| | +------------------------------------------+
| | | |
| +--------------------> IRC: Freenode #jenkins-infra <----+
| | | | |
| | +------------------------------------------+ |
| | +------------------------------------------+ |
| | | | |
| +--------------------> Mail: jenkins-infra@lists.jenkins-ci.org <----+
| | | |
| +------------------------------------------+ |
| If the topic is too big |
| |
| +-------------------------------------------+ |
| | | |
+--------------------> IEP: https://github.com/jenkins-infra/iep |--------+
| |
+-------------------------------------------+
Once there is an agreement about the approach and before any changes, we ask contributors to respect the following rules.
Those rules are just what we consider "best practices" for a contributors driven project and can be adapted depending on specific git repositories.
So it’s easier for everybody to find/review/audit changes and suggest improvements.
So we always have different people who understand infrastructure changes.
Rem: Non regular contributors are more than welcome to share their expertise or just ask question which also help to spot incoherences.
Code Reviews Purpose:
Educate author and the team about the changes in code being made
Look after logic or security issues not covered by tests
Gather improvement suggestion for code readability or efficiency
So we feel more comfortable when merging PRs and we avoid regression issues.
So we only have one source of truth and we don’t break others people works. And if it can’t do that then it needs to be well communicated and documented cfr rule #1.
Fork project -> Create Feature Branch -> Open Pull Request -> Ask Review -> Merge Pull Request
The deployment step is the only moment where we need approval from someone with elevated permission. As already mentioned, even if we try to be as open as possible, we don’t want to share privileged accesses with every contributors even if we trust them and that mainly for security reasons.