Welcome to Orange Factor.
Here we will tell you the chance of getting an orange for any given checkin!
Our goal is to provide a useful tool for measuring the orange in our trees while finding a quantitative method for measuring if we are improving or degrading over time.
find a bug? file in bugzilla under Testing -> Orange Factor
have a complaint or a suggestion? send to auto-tools at mozilla dot com or find us in #ateam on irc.mozilla.org
The Bug Correlations view analyzes what bugs occur together in the same pushes.
Based on the error strings in the buildbot logs, we determine what bugs happened in each push in the given date range. For each bug, we present the count of other bugs, or combination of bugs, happening in the same pushes, along with the percentage of total bug occurrence.
To reduce noise, bugs with only one occurrence and no correlations are omitted. Furthermore, only correlations with counts of 3 or more are listed.
Since the correlations are indexed by bug, the data for two correlated bugs will generally not be the same. For example, if bug B occurs 100% of the time that bug A occurs, it doesn't necessarily follow that bug A occurs 100% of the time that bug B occurs. Examining the tables for both bug A and bug B would be necessary to know the full story. (Suggestions on alternate presentations to make this clearer are welcome.)
The Orange Factor database is built from two main sources: tbpl star data and log parsing. Occasionally these are out of sync, resulting in bug information for a particular date but no test-run information. This means that the test-run count for that day is unknown, and thus the Orange Factor, that is, the number of oranges per test run, is consequently also unknown. These oranges are removed from Orange Factor calculations and graphs, but they are present in graphs and tables of occurrences that do not rely on Orange Factor calculations (e.g. Bug Details, Top Bugs, the graph and table in Bug Count).
All views except Bug Details allow you to filter the data in a variety of ways. There is both include filter, to specify bugs that must be in the presented data, and an exclude filter, to specify bugs that must not be in the presented data. If there is a conflict, generally the exclude filter wins (i.e. the exclude filter is applied after the include filter).
There are two types of filters:
You can use the top N filters in QuickSearch mode in both the include and exclude filters. For example, to see what the Orange Factor would look like if the top 10 bugs were fixed, select "QuickSearch" from the exclude filter dropdown, then select top 10 bugs and apply. You will now see the graph with the top 10 bugs removed beside the real Orange Factor graph.
If necessary, you can get more complicated: if you choose to include the top 10 bugs and exclude the top 15, you will get information on the 16th to 25th highest-ranking bugs, inclusive.