On Mon, Aug 15, 2011 at 9:29 PM, Ian Hinder <span dir="ltr"><<a href="mailto:ian.hinder@aei.mpg.de" target="_blank">ian.hinder@aei.mpg.de</a>></span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>> Applying a patch to the Einstein Toolkit requires commonly several stages:</div><div>
> - A problem or missing feature needs to be identified<br>
> - There is a bit of discussion on the mailing list or in TRAC<br>
> - A patch is created<br>
> - The test cases are run<br>
> - A patch is proposed<br>
> - The patch is reviewed<br>
> - The patch may be modified or updated<br>
> - The test cases are run (again), hopefully on several systems<br>
> - The committer applies the patch to a clean checkout, maybe runs the<br>
> test cases again<br>
> - The committer writes a commit message and commits/pushes the patch<br>
> This is a bit more complicated if several thorns are involved, if the<br>
> original patch author doesn't have commit rights, or if the underlying<br>
> problem only occurs only on a specific system.<br>
><br>
> I suggest to define a self-contained patch format, probably stealing<br>
> the idea from git. This format would define the complete patch,<br>
> possibly for several components, in such a way that a single command<br>
> (PushComponents) can apply the patch, check for conflicts, create<br>
> commit messages, and push the commits upstream. Ideally, it would also<br>
> keep the distinction between author and committer.<br>
<br>
</div>Yes - this is a good idea. �Do you know if it has been done before? �We might be able to re-use an existing format and tools.<br></blockquote><div><br></div><div>Yes, many of these things have been solved by the Android project which is similar to Cactus in that it is comprised of a large number of components pulled from a large number of sources. Their solution was to develop "Gerrit code review" [<a href="http://code.google.com/p/gerrit/" target="_blank">http://code.google.com/p/gerrit/</a>], a system built on top of git. Their system��[<a href="https://review.source.android.com/#/q/status:open,n,z" target="_blank">https://review.source.android.com/#/q/status:open,n,z</a>]�serves as a good example of how this works. This provides a very convenient way of managing the submission and review of patches and also serves as a system for serving git repositories. Along with this, there is the 'repo' tool which is designed to make managing a collection of git repositories easier.</div>
<div><br></div><div>This all relies on the use of git, so I'm not sure whether it would be an option, although any non-git repositories could have git mirrors provided. It also might be a bit too complicated for these purposes, but even then it would still be a good idea to learn from their design choices.</div>
<div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">One could imagine a web interface to the build and test system which accepted an uploaded "super-patch", as described above, and ran the automated tests on a fresh checkout with that patch applied. �This could probably be integrated with TRAC. �When finished, the user would get an email with the results and a URL to the web report for those results. �Perhaps this could also add an automatic comment to the corresponding TRAC ticket. �The email would say "OK to apply" if no tests newly failed. �The user could then run PushComponents on the patch file, which performs the commit. �If we wanted to get very fancy eventually, the test system could sign the commit.<br>
</blockquote><div>�</div><div>�</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div>> To test whether the test suite returns the expected result, I suggest</div>
<div>
> to keep track in which test cases failures may be temporarily ignored.<br>
> This will allow us to use this new system right away, even before we<br>
> have corrected all currently failing test cases. That is, each new<br>
> commit may only reduce the number of failures.<br>
<br>
</div>Do you propose to enforce the system, or just make it easy for people to do the right thing? �Would the shame of an email to the ET list every time a test newly failed, along with the list of commits and authors that had changed since the last successful test run, be enough encouragement to avoid the need for enforcement?</blockquote>
<div><br></div><div>In my opinion, even a webpage which shows the test results and is updated after each commit would probably be sufficient. People would be encouraged to check the webpage after they commit a patch and it would be quite clear who was responsible for breaking anything.</div>
<div><br></div><div>Barry</div></div>