For the last several months I've been working on migrating the OpenLDAP project infrastructure to new systems. The server housing git-master was the first to be migrated, and was done late last year. The final server to be migrated is a bit more challenging, as it contains all of the public facing aspects of the project (ftp, git (RO), ITS (jitterbugs), FAQ, wiki, mailman, etc).
As a part of the transition to the new hardware, the plan is to retire jitterbugs and migrate it to bugzilla. As a part of this process, I've been working on a development instance so I could work out the details of how I anticipate the migrated bug system to look. I would like to invite list members to look at the new system and provide feedback. It can be found at:
https://quanah.ddns.net/bugzilla
Please note that the dev system is not hooked up to an MTA, so any time it reports that it is sending an email it is not doing so (Instead it writes to a file on disk that I can use to ensure that the generated emails are correct). If you want to know what sort of email was generated by an action, let me know and I will look it up.
Also note that the code to do the initial import has to pick a single email address to record as the person who created the bug (and all follow ups). In part because none of those accounts exist in the DB. I've ensured that the "From" information is preserved, and as time allows and accounts are created, I will work on updating the underlying database to correct the information on who filed the old bug reports. New bug reports will of course be tied to the actual account of the individual who files them. So for all currently existing bugs, I am recorded as the individual who filed the bug report.
----------- Difference highlights -----------
a) Integration with GitHub authentication
When someone arrives at the splash screen, they have a choice -- They can either create a new local account (via the "New Account" link), or they can click the "Sign in" button to create an account that is linked via GitHub. I will be working on customizing this front page so that it's more apparent that you can create an account using the "Sign in" button, as that's not clear right now.
b) product/component design
Bugzilla is organized around the concept of Products and components that make up that product. The jitterbugs system really had no organizing concept. In the process of migrating, I've reworked the jitterbugs categories into components of related products where applicable, and "type" based categories are now migrated to an appropriate severity type.
The Bugzilla instance has the following Products:
JLDAP (Components: JDBC, JLDAP) LMDB (Component: liblmdb) OpenLDAP (Components: backends, build, contrib, documentation, libraries, overlays, slapd) Website (Component: website)
Jitterbugs was comprised of the following categories:
Build (Now OpenLDAP/build) Contrib (Now OpenLDAP/contrib) Development (Now a severity of "development") Documentation (Now OpenLDAP/documentation) Historical (Not tracked as a separate item, unnecessary) Incoming (Not tracked as a separate item, unnecessary) Software Bugs (Not tracked as a separate item, unnecessary) Software Enhancements (Now a severity of "enhancement") Web (Now Website/website)
The re-organization allows a better ability to track bugs while retaining useful type information. With the product/component focus, reports can now be better directed into the appropriate location.
c) Groups support -- Bugzilla has an excellent concept of groups, and the ability to give different groups different permissions. This could allow, for example, a group of privileged users who have the capability to help triage bugs. Currently there is only a single group for the developers, while any other account is a "normal" user account. The ability to have groups allows us to make it possible for people to file "private" bugs that are easily found by the development group, including the ability for those bugs to go to a developers only list. The way the system is currently set up, it will (fake) email a private list for all private bugs & private comments, and (fake) mail all public bugs/comments to openldap-its@openldap.org.
This allows us to preserve the long time functionality of public notifications via openldap-its@openldap.org while adding the ability for the dev team to immediately be updated to new private bugs & comments.
d) Milestones -- Bugzilla allows the creation and use of milestones. This is extremely helpful for both the developers and end users, as it becomes possible to easily query these milestones for information on such things as what was in a release, what will be in the next release, and what is slated for future development. For example, the "Search" link can be used to generate the following queries:
All bugs currently scheduled for OpenLDAP 2.4.46:
All bugs currently scheduled for inclusion with the OpenLDAP 2.5 initial release:
Note that this is all work I've entered by hand, so various bugs may be missing from the above queries.
e) versions -- Bugzilla has a clear concept of versions for each product. This means now that people can select (when filing a bug) what version they are reporting against. You could always enter this information in by hand in jitterbugs, but the major difference here now is that it is possible to use this to generate queries. So you could, for example, do a search to find all bugs reported against 2.4.43.
f) bug states -- Jitterbugs was fairly limited in the state options that could be chosen (Open, Active, Suspended, Feedback, Test, Release, Partial, Closed). This was certainly customizable (and had been customized beyond the defaults), but lacks a level of granularity available with Bugzilla.
With Bugzilla, there is always a primary state, and potentially a secondary state. These states are:
UNCONFIRMED -> Newly filed bug that has not been acted upon by the project. All bug reports start in this state CONFIRMED -> Someone in the project has confirmed this is a valid report. IN_PROGRESS -> Someone in the project is actively working on the report. RESOLVED -> Issue has some type of secondary resolution. These are: FIXED -> Issue is fixed and integrated into the release branch for the next upcoming release INVALID -> Bug report is invalid WONTFIX -> There are no plans to fix this report (Generally to be used with bug reports for historic releases) DUPLICATE -> Duplicate of an existing bug (Which is entered as a part of the process. See https://quanah.ddns.net/bugzilla/show_bug.cgi?id=8765 for an example) WORKSFORME -> Developer(s) unable to reproduce the problem report PARTIAL -> Some of the work in the bug was completed. Other work is still pending. My long term goal is to move all unfinished work for all of the resolved/partial bugs into new bugs to be tracked separately, and then close these out with a See Also reference (More on "See Also" further down) SUSPENDED -> Developer(s) have suspended working on this issue FEEDBACK -> Developer(s) are soliciting feedback from the reporter before continuing to work on the issue TEST -> Developer(s) have committed
VERIFIED -> Issue is considered complete. It has the same secondary resolutions as RESOLVED. One slight difference noted below FIXED -> Issue is fixed, and is part of a released version of OpenLDAP
Note that for the bugs that have been imported, some of these states may not be accurate due to the limitations of how items can be mapped.
g) attachment support -- Currently we require people to upload patches, etc, to the OpenLDAP FTP site. Bugzilla allows attachments to be made directly as a part of submitting or updating a bug. This should allow the FTP server to only be used for large files (such as core dumps), and mostly retired. It also makes it simpler to ensure patches are kept with bug reports (We've lost patches people have provided in the past via URLs that no longer exist). It is possible to make attachments "private" as well, to help protect sensitive data.
h) git integration -- One of the things on my to-do list post migration is to set up git integration. This means that when a developer makes a commit that references an existing bug report, that commit will automatically be added as a comment to the bug. This will make it easy for people who are interested in what the fix to an issue was to find said commit and examine the change(s).
i) Potential future plans -- Mozilla has developed an addon service for Bugzilla that allows GitHub PR reports to be attached to open bug reports (https://github.com/mozilla/github-bugzilla-pr-linker ). I could see how this would be useful to have, as many people do their development work on GitHub and are annoyed at the process of having to directly provide a patch to the project.
For developers, I'm also looking at a utility that allows those in the development group to manipulate bugs via the git command line (https://github.com/mozilla/git-bz-moz/blob/master/git-bz.txt). My goal is to have this working prior to a migration occurring.
j) Major differences -- Currently correspondance between those who report a bug and the project team is done via email, using TO or CC the openldap-its@openldap.org email list. This has been problematic over the course of time, whether from Spam getting into the system (largely mitigated now via making the list moderated), replies not making it into the ITS system due to a failure to CC the openldap-its@openldap.org email list, or people who reported the issue never receiving a reply (thus not knowing the issue was being worked on) because the reply was only sent to the openldap-its@openldap.org email list. It is possible to setup an email->BZ gateway, but the format is a bit arcane, and I'd like to avoid that if possible. I would love to get feedback from those folks on the list as to their thoughts about this.
k) URL field -- Bugzilla has a URL field that can be filled out to point to a URL containing additional relevant information. (Example: https://quanah.ddns.net/bugzilla/show_bug.cgi?id=8766)
l) See Also field -- With Bugzilla, it is possible to provide references to other related bugzilla bug reports, including in Bugzilla instances run by other organizations.
Examples: https://quanah.ddns.net/bugzilla/show_bug.cgi?id=6138 (See Also's to related bugs in the OpenLDAP BZ) https://quanah.ddns.net/bugzilla/show_bug.cgi?id=6304 (See Also to RedHat's BZ)
Thanks for your time, and I look forward to the feedback.
--Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
--On Monday, January 15, 2018 7:56 PM -0800 Quanah Gibson-Mount quanah@symas.com wrote:
Also note that the code to do the initial import has to pick a single email address to record as the person who created the bug (and all follow ups). In part because none of those accounts exist in the DB.
I did not like this aspect of the import script, so I rewrote it to create accounts as necessary and full retain the information as a part of the import process. I think the result is much improved:
https://quanah.ddns.net/bugzilla/show_bug.cgi?id=8127
Now, if someone's account already exists, they can do one of two things:
a) Use the github auth integration, and it just works. or b) Request a password reset, and then follow the instructions from the URL that's provided via email.
As a part of this effort, I have spent time consolidating various email addresses different people have used over the years down into a single address. For example:
quanah@stanford.edu quanah@zimbra.com quanah@symas.com
All got mapped to: quanah@openldap.org
This helped to reduce the number of accounts created in the system. I'm sure there are account mappings I've missed so if you see one, let me know sooner rather than later so I can fix it before the production migration occurs. :)
--Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
-
Quanah Gibson-Mount