Skip to content

News and thoughts from the team behind the Eclipse Hudson project

The Hudson Book – Now Available in EPUB Format

Good news! We’ve just altered the Hudson Book build process to generate a copy of the Hudson Book in EPUB format. So you can now download the book and read it offline in your favourite e-book reader, tablet or phone device. Head over to the book page on the wiki for the latest copy of the book in all of the available formats. http://wiki.eclipse.org/The_Hudson_Book

Advertisements

Using Hudson on SOA Projects

I attended two sessions on using Hudson in SOA development projects at the DOAG conference this week. Both were given in German, not my strong suite, but it’s amazing how being present and hearing the enthusiasm in a conference session transcends language barriers – Kontinuierliche Integration – is important in every development language!

The first was a joint presentation from OPITZ Consulting and Continental Automotive. They worked together on  developing a custom testing framework for integration testing of SCA composites as part of Continental’s central IT.  Jurgen Broda, from Continental, spoke in glowing terms of the ease of use and the benefits that his team has encountered from the automation of its testing and integration with MKS through Hudson. In fact, this presentation inspired Lonneke Dikmans, a Managing Partner with Vennster, to blog that she is returning to her current project to install Hudson and implement CI as soon as possible.

The second session was given by Trivadis. Interestingly, the slides for this session were in English but the presenters spoke in German to the predominantly german-speaking audience. They delved deeper into how they use Hudson, Maven, Nexus, SVN, Sonar, MySQL and JIRA together as their CI infrastructure for OSB/SOA client projects.  From this they have a reference project which they quoted as the ‘project team couldn’t live without it anymore‘.

I hope that the DOAG slides will be available soon if you are interested in looking further into these or other sessions.

Susan Duncan

 

 

How Hudson is helping the development of JSF

This week I am at the German Oracle User Conference. It’s great to be out and talking and listening to Hudson users about their experiences with Hudson and how it is being used. I’m going to get some of these customer experiences both from here and the recent EclipseCon (Europe) conference onto this blog in coming weeks.

As I sit working in one of the conference lounges I am joined by Ed Burns, the JavaServer™ Faces Technology lead for the Java Community Process.

After just a few minutes he smiles and says, “We could not be supporting the complex stack that we do without the agility that Hudson gives us” as he clicks the Build Now button, picks up his stuff and moves on to his next appointment after quickly fixing a just found bug in the JSF implementation used by the Oracle JDeveloper team.

JSF are using Hudson with a complex mix of tests to build, test and deploy to a public Maven repository. Now he can update the bug with the build details and the JDeveloper team can grab that patch as soon as they hit the decks later today in the USA.

You can see the JSF2.1  Hudson view here and some more detail about how the JSF team are using Hudson from Ed’s blog 

Susan Duncan

The Hudson Book

We are very excited to announce the publication of this fantastic new book on Hudson. Over the last few months Manfred Moser and Tim O’Brien have worked long and hard to produce a comprehensive guide to using Hudson from initial download to production deployment.

It is packed with 10 chapters of information covering the very latest features in Hudson at the Eclipse Foundation including the Maven 3 integration and even the new Cascading Projects that is so new it’s only a Beta release!

For anyone looking to start with Hudson or those looking to improve their knowledge this book is the place to go.

Manfred says, “Going forward we hope to expand the book’s depth and make it the authoritative and up to date documentation for Hudson users covering the core of Hudson features and moving to documenting more and more use cases, further plugins and advanced usage like plugin creation or scripting console usage.”

The link provided here takes you to the rough format of the book on the Hudson project at Eclipse. Right now, it’s a bit ‘rough and ready’ but we will get a wrapper as soon as possible that provides the single and multi-page html and the pdf. But we wanted to get this information out to you – so enjoy!

http://www.eclipse.org/hudson/hudsonbook

Susan

Cascading Projects released as BETA

Have you ever despaired at having to update multiple Hudson jobs when you need to change properties? Perhaps you use templates or copy jobs to try and alleviate this at job creation, but what about further down the road? What you’ve dreamed of is the ability to have some kind of inheritance of properties throughout the life of jobs.

Dream no longer! Hudson has released a beta of its new Cascading Projects feature. Now you can define a project with a cascading parent and have a child project derive its configurable properties from its parent.

Cascading Projects is not a template, you can configure a job to inherit properties from its parent, whilst overriding those that need to be changed or later reverted back to the parent. Any changes you make to the properties in a parent will cascade down through the child jobs that inherit that property.

Take the example here.  Job3.2 will inherit its properties from Job3.1, which in turn inherits from Job3 and so on.

But what if Job3 overrides a property it inherits from Job1? That change will cascade down to Job3.1 and Job3.2

If the the same property is subsequently changed in Job1 then the change will cascade down to Job2 but the override in Job3 will stay in place for that job and its children

You use the project config page to select an existing saved project to inherit from. Any project can be used as the parent, but you cannot create a cyclic inheritance. Most of the properties are cascadable and if the parent contains Builders, Publishers or Triggers they will be merged with the child.

So what kinds of projects might benefit from Cascading Projects? A few of the most common examples include updating email notification lists, project-based security, reports published locations and polling the source repository.

This new feature comes from a request from the community in HUDSON-3157 and is an example of the work that the Hudson team is engaging in along with bug fixing and all the work associated with the entry to the Eclipse Foundation as detailed in a previous post.

The details of the implementation and the full list of cascadable properties are detailed on the wiki. As with any beta release, your feedback is greatly appreciated. The final release is scheduled for mid-November and we will endeavour to address any feedback and make updates to the feature before then.

You can download the Hudson 2.2.0-Beta war file here. Please be sure to test this with a copy of your production Hudson home. The implementation adds some additional elements to the project configuration XML which will not be recognized by Hudson 2.1.x and previous.

Susan Duncan

IP and License Clean for Eclipse Foundation

Over the past couple of months the Hudson community has been working hard to ensure that its source code and libraries are IP and license clean in order to pass the stringent entry requirements of the Eclipse Foundation. I spoke briefly about this at my Hudson session at JavaOne and the response was so enthusiastic that I feel I need to bring the facts to a wider audience through this blog.

The Hudson code that is moving to Eclipse is made up of its core components and plugins for SSH-Slaves, Git and Subversion. The initial upload was made up of around 1700 Java files, 125 XML files, 600 Jelly files and some 3700 property files. The core elements (core, UITest and UpdateCenter) have now been approved and are around 8 of the external libraries but there is still some way to go. Each license is checked by both Eclipse’s automated system and by hand.

There are a number of different areas that are slowing down the progress and causing us to do major rework:

1. GPL/LGPL libraries
Eclipse does not allow any of these libraries in. We have had to make major code changes to remove 5 libraries. One major example is Java Native Access , like many libraries, when we came to a thorough example of the jars we discovered additional libraries that also had to be removed and worked around. You can see the complete list of libraries here. Notice that this list includes Winstone. This was the bundled web container for standalone Hudson, but the version was obsolete and GPL LGPL licensed so had to be changed to the Eclipse Jetty Container.

Another major code change involved JFreeChart. We have had to abstract this library to a plugin so that it can be provided from outside Eclipse.

2. Obtaining Source Code
Some of the libraries being used in Hudson are so old that obtaining the source code, as required by Eclipse, has proved difficult. One example is the acegi security library. We spent some time trying to procure the source code for the version of this library that Hudson uses. But eventually had to remove it.

3. Forked external libraries
There are a number of external libraries that have been patched within Hudson. These we are having to upgrade to a more recent version and if necessary discuss the patch with the library owner to get it consumed back into the main library . These include jtidy.sourceforge.net, xstream.codehaus.org and others.

4. Duplicate Functionality libraries
Whilst going through the external libraries we have also come across a number that provide duplicate functionality. although not in themselves a problem we do want to analyze and where possible reduce the number of external libraries necessary.

Why is all this necessary? Well, primarily because the Eclipse Foundation require it, and many of our customers are asking for it. But why? And this is really the crux of the matter. Hudson and the Eclipse Foundation are committed to providing a product in which the code and IP are clean, and the process is predictable, stable and open. For some organizations and users this is more important than to others, but for us it is part of our raison d’etre. We believe that if Hudson is to continue along the path of being the de facto standard for CI and to increase its importance to development shops both small and enterprise wide, then it must be absolutely clean. The users, the legal departments, the procurement bods, everyone who has an interest in software and in IP in an organization must be able to categorically say that Hudson is clean, reliable, stable and safe.

Susan Duncan

Hudson at JavaOne

At the recent JavaOne conference in San Francisco Hudson was well represented both through sessions, a lab and on the demo grounds. It was great to talk to so many people about their use of Hudson and hear their excitement about all we had done, are doing and are planning to do in the future.

The session was packed and gave Winston and me the opportunity to not only talk about the successes of the last few months but also to give a sneak preview of some new functionality that Winston is working on around cascading projects (more on that in a later blog). So, what are those successes? After the user survey that was conducted in March 2011 it is clear that our Hudson users have a lot in common. They may come from organizations that vary in size (see fig1)but their major requirements are similar: improved stability and performance and Maven integration

This year Hudson has made great strides in these areas. The introduction of the 6-week release cycle, based on a new development and release process, helped stabilize the releases. Hudson users wanted to see more open-ness in how the project was run and this process was started through the introduction of the development process and the bi-weekly public governance calls that are open to everyone.

Another outcome of the survey was the spread of Hudson upgrade timings. It varies from weeks to well over a quarter, as borne out by the daily ping stats of the last few months. Hudson users appreciate the stability that comes with the 6-week release cycle, they upgrade when they are ready, without worrying so much if the latest build will need to be reverted due to rogue bugs.

Over the last few months we’ve concentrated on clearing the backlog of issues on the Hudson core and the Tier 1 plugins  – SVN, Git, SSh-Slaves. The stats below all follow a similar pattern – large numbers of issues resolved with very few new issues being raised. See how the numbers rose around August. This was greatly influenced by the introduction into the community of a full-time Engineering/QA and Product Management team from Oracle. This team continues to work with and for the community.



Of course, it has been a period of flux over the last few months. We are almost at the end of a period of intense work to pass the stringent requirements needed to join the Eclipse Foundation as a Top-level Technology project. These include a stringent inspection of every line of source code, including 3rd party libraries to ensure clean IP. This has highlighted some problems with the code base, all of which are being or have been addressed, such as the removal of GPL/LGPL libraries and the major code changes those have needed, including switching the bundled web container for standalone Hudson from the obsolete and GPL licensed Winston to Eclipse Jetty and an analysis of multiple 3rd party libraries providing duplicate functionality. In the next blog entry I will go into this work in more detail.

But let’s concentrate on the stats. Even with this upheaval we have continued to release stable and more performant builds and to include major new functionality like the Maven 3 integration (another item high on users’ minds in the March survey).

 
February 2011
September 2011
Project Members
2455
2512
User Mailing List
796
802
Developer Mailing List
487
508

Take a look a the unique pings for the various Hudson releases below. These show a stability of users across the various builds, as do the project members and mailing list users. Anyone who thought that perhaps Hudson would lose share in these times of change is badly mistaken. Of course, there is additional improvement we want to see – and we also want to introduce unique IP recording for each Hudson server – to ensure that we don’t introduce inaccuracies to the figures due to the dynamic nature of IP allocation
Another strong area of Hudson is its plugin infrastructure. There are over 400 plugins available for Hudson. Some of these are maintained by the Hudson community and others by other communities and individuals. We categorize plugins as Tier 1, 2 or 3. The tier 1 plugins I mentioned earlier, they are defined as part of the Hudson core. Tier 2 plugins, along with core and tier 1 plugins are tested as part of the Hudson QA certification process.  A number of the plugins are maintained by their owners within the Jenkins infrastructure, but the owners are as determined as we are that these plugins remain fully functionality for Hudson. As with many aspects of software engineering the 80:20 rule applies. The survey earlier this year showed that only a handful of plugins are considered ‘must have’ by users, with others ‘nice to have’. These include our tier 1/2 plugins and that is where we will focus our efforts in feature development and QA

Hudson Version
Total No. of Pings
Pings from Unique IPs
1.396
1.31 million
19,374
1.398
2.07 million
21, 048
2.0.0
1.09 million
18,697
2.0.1
1.89 million
19,342

As you can see from all this, Hudson continues and will continue to be, as Mik Kersten (Tasktop CEO and Mylyn lead) says, ‘..IP-clean, inviting, API-robust and long-loved de facto standard CI tool.”

Susan Duncan