Webinar Q&A: "Scaling Jenkins in the Enterprise"

Thank you to everyone who joined us on our webinar, the recording is now available.

Below are some of the questions we received during the webinar:

Q: How do you implement HA on the data layer (jobs)?  Do you have the data hosted on a network drive?
A: Yes - the 2 masters (primary and failover) share a filesystem visible to both over a network. You can read about HA setup here.

Q: I would like to know how to have a different UI instead of Jenkins UI. If I want to customize the Jenkins UI what needs to be done?
A: There are plugins in the open source community that offer customizable UIs for Jenkins: Simple Theme Plugin is one popular example.

Q: I want to have a new UI for Jenkins. I want to limit certain things for the Jenkins user.
A: Interesting. What types of things? A lot of the Jenkins Enterprise plugins allow admins to exercise stricter limits on different roles’ access to certain functions in Jenkins, whether that be through templating or Role-based access control with Folders. The Jenkins Enterprise templates also allow you to “hide” some configuration parameters.

Q: Let’s take simple example. I want to have a very simple UI for the parameterized build where a user can submit the SRC path and the build script name. He submits that job by specifying the above two values. How we can have a very simple UI instead of Jenkins UI?
A: Okay - this is exactly the use case that the job template was designed for. See the last image in the job template tutorial.

Q: Looks like it will work. How I can get rid of the left hand Jenkins menu?
A: You can remove most of the options in that menu with the role-based access control plugin - you can remove certain roles’ ability to create new jobs, configure the system, kick off builds, delete projects, and see changes/the work space, etc, which will remove most all of the options in that menu.

Q: We use the open source version of Jenkins and we have been facing an issue with parsing the console log. We use curl and there is a limit for console text to be displayed for only 10000 lines. Will this enterprise edition handle that issue?
A: It sounds like you’re seeing Run.doConsoleText being truncated, though it seems there shouldn’t be a 1000-line limit, I just checked sources and it looks to send the full log, regardless of size. 

It has come to our attention that this answer is incorrect. Daniel Beck clarifies:

A: While a build is running, LargeText in Stapler (used by doConsoleText) truncates output after 10k lines since Jenkins 1.447.1. This is not configurable and caused e.g. JENKINS-23660.
https://github.com/stapler/stapler/blob/master/core/src/main/java/org/kohsuke/stapler/framework/io/LargeText.java#L226
https://github.com/stapler/stapler/blob/master/core/src/main/java/org/kohsuke/stapler/framework/io/LargeText.java#L547

Reproduce using e.g. shell build step with the script:

#!/bin/bash
for I in $( seq 1 20000 ) ; do echo $I ; done
sleep 60

Open …/consoleText while the script sleeps. It’ll end around ~9996, depending on the number of log lines before the script is executed.

Q: Is there a customizable workflow capability to allow me to configure some change control and release management process for enterprise?
A: The Jenkins community is currently developing a workflow plugin (0.1-beta at the moment). Jesse Glick, engineer at CloudBees, did a presentation about it at the ‘14 Boston JUC. CloudBees is working on enterprise workflow features such as checkpoints as a part of Jenkins Enterprise by CloudBees.
 
Q: Is there any framework/processes/checklists that you follow to ensure the consistency/security of multi-tenant slaves across multiple masters?
A: Please see the recording of the webinar for the answer
 
Q: Is there a way to version control job configuration?
A: Yes - CloudBees offers a Backup Plugin that allows your to store your job configs in a tar ball. You can set how long to retain these configs and how many to keep, just as you would for a job’s run history. You can also use the Jenkins Job Configuration History plugin.
 
Q: This backup plugin is available with the open source version of Jenkins?
A: The backup plugin that I’m speaking of is only a part of the Jenkins Enterprise package of plugins.
 
Q: How is the environment specific deployment done through same project configuration in Jenkins?
A: You can use CloudBees’ template plugin to define projects and then have a job template take environment variables to pull them from a parent folder with Groovy Scripting, or to take them from user input using the parameterized builds plugin:  
Q: Do we need to purchase additional licenses if we want to set up an upgrade/evaluate validation master and slaves, as you recommend?
A: For testing environments, CloudBees subscription pricing is different, it is cheaper. For evaluation, I recommend just doing a trial of both to see which fits your needs better. You can request a 30 day trial of Jenkins Enterprise here.
 
Q: Is this LDAP group access is only available in the enterprise version? I am asking if I can make it so that some users can only see the jobs of their group.
A: Jenkins OSS supports LDAP authentication. The Role Based Access Control authorization provided by Jenkins Enterprise by CloudBees allows you to apply RBAC security on groups defined in LDAP. You can then put the jobs in folders using the Folders/Folders Plus Plugin and assign read/write/etc permissions over those folders using the CloudBees RBAC plugin.
 
Q: Another question. What’s the difference between having dedicated slaves with your plugin/addon or just add another slave with another label?
A: Dedicated slaves cannot be shared with another master - only with the master that it’s been assigned to - whereas shared slaves with just labels are still open for use between any masters that can connect to it.
 
Q: At this moment my organization is planning to implement Open Source Jenkins. Does CloudBees provide training or consultancy adoc to client environment in order to implement Jenkins with best practices, saving time, money and resources?
A: CloudBees service partners provide consulting and training. The training program is written by CloudBees.
 
Q: Can I use an LDAP for authentication, but create and manage groups (and membership) locally in Jenkins?  For us, creating groups and managing them in the corporate LDAP is a very heavyweight process (plus, support only for static LDAP groups, not dynamic). Clarification - we have a corporate LDAP, and want to use it for authentication. I want to not use the LDAP to host/manage groups and group management.  I want to do that in Jenkins - and not using LDAP whatsoever in any way for groups
A: Yes, with the the Role Based Access Control security provided by Jenkins Enterprise by CloudBees, you can declare users in LDAP and declare groups and associate users in Jenkins. A Jenkins group can combine users and groups declared in LDAP. You can define users in your authentication backend (LDAP, Active Directory, Jenkins internal user database, OpenID SSO, Google Apps SSO …) and manage security groups in Jenkins with the CloudBees RBAC plugin.
 
Q: Is the controlled slaves feature available in the Enterprise version only?
A: Yes - this is a feature of the CloudBees Folders Plus plugin.
 
Q: Is there a way to version control job configuration?
A: Yes - CloudBees offers a Backup Plugin that allows your to store your job configs in a tar ball. You can set how long to retain these configs and how many to keep, just as you would for a job’s run history. You can also use the Jenkins Job Configuration History plugin.
 
Q: Can I start implementing Jenkins Operations Center as a monitoring layer for teams that have set up with Jenkins OSS? Over time I would move them to Jenkins Enterprise, but we need to progress in small iterative stages.
A: Jenkins OSS masters must be converted into Jenkins Enterprise by CloudBees masters. You can do this either by installing the package provided by CloudBees or by installing the “Enterprise by CloudBees” plugin available in the update center of your Jenkins console. Please remember that a Jenkins OSS master must be upgraded to the LTS or to the ‘tip’ before installing the “Enterprise by CloudBees” plugin.
 
Q: What is the purpose of the HA proxy?
A: HA proxy is an example of the load balancer used to setup High Availability of Jenkins Enterprise by CloudBees (JEBC) masters (it could also be another load balancer such as BIG IP F5, Cisco …). More details are available on the JEBC High Availability page and in JEBC User Guide / High Availability.
 
Q: When builds will run on slaves and Jenkins Operation Center will manage, what is use of masters?
A: JOC is the orchestrator. It manages which slaves are in the pool, which masters need a slave, and which masters are connected. The masters are still where the jobs/workflows are configured and where the results are published
 
Q: Is there a functionality for a preflight/proof build - i.e. the build with the local Dev changes grabbed from developer’s desktop?
A: Jenkins Enterprise by CloudBees offers the Validated Merge plugin that allows the developer to validate their code before pushing it to the source code repository.
 
Q: Currently we are using OSS version with 1 Master and 18 slaves with 60 executors and faces performances issues, and a workaround used to bounce the server once in a week. Any clue to debug the issue?
A: We would need more information to help diagnose performance problems, but the CloudBees Support plugin in conjunction with a CloudBees Support plan, you could always create a Support Bundle and send it to our support team along with a description of your performance problem.
 
Q: How do I create dummy users and assign passwords (not using LDAP, AD or any security tool) just for testing my trial Jenkins jobs? (Jenkins open source)
A: Use the “Mock Security Realm” plugin and add dummy users with the syntax “username groupname” under the Global Security Settings
 
Q: Can you have shared slave groups?  For example, slave group “A”  and within it have sub group “A-Linux5”, “A-Linux6”, etc…
A: Yes, you can do this with folders in Jenkins Operation Center.
 
For example with groups “us-east” and “us-west”, you could create folders “us-east” and “us-west”:
  • In the “us-west” folder, you would declare the masters and slave of the West coast (e.g. san-jose-master-1, palo-alto-master-1, san-jose-slave-linux-1, san-francisco-slave-linux-1 …).
  • In the “us-east” folder, you would declare the masters and slave of the East coast (e.g. nyc- master-1…).
Thanks to this, the west coast masters will share the west coast slaves. More subtles scenarios can be implemented with hierarchies of folders as explained in the tutorial.
 
Q: How do you implement HA on the data layer (jobs)?  Do you have the data hosted on a network drive?
A: Yes - the 2 masters (primary and failover) share a filesystem visible to both over a network. You can read about HA setup here.
 
—- Tracy Kennedy & Cyrille Le Clerc
 

Tracy Kennedy
Solutions Architect
CloudBees

As a solutions architect, Tracy’s main focus is reaching out to CloudBees customers on the continuous delivery cloud platform and showing them how to use the platform to its fullest potential.
 
 
 

Cyrille Le Clerc
Elite Architect
CloudBees

Cyrille Le Clerc is an elite architect at CloudBees, with more than 12 years of experience in Java technologies. He came to CloudBees from Xebia, where he was CTO and architect. Cyrille was an early adopter of the “You Build It, You Run It.” He naturally embraced the DevOps culture. 

 
 
 

Add new comment