Monday, October 25, 2010

Parrot's Teams: Five Scenarios

Parrot's concept of teams was rushed into service without being entirely fully-formed.  That doesn't make it an automatic disaster, but it does mean that we're figuring out inter-team and intra-team dynamics as we go.  To help this process along, here are some hypothetical (or not) events and my best guess as to how the different teams would interact in addressing them.  After each example, I've tried to list the major advantages and disadvantages that the team structure creates but more are welcome.  Note that these cases are idealized somewhat and are still speculative.  Real life is always messier.

1: Research Paper

We've got a couple developers who keep their eyes peeled for new research papers and we're always glad use relevant research papers to improve our code.  If someone presents us with some research that they think is relevant to Parrot, here's how I'd envision our process working:

  • Someone posts to parrot-dev or #parrot saying that they found a research paper we should consider.
  • The architecture team takes the lead and looks over it, explicitly soliciting feedback from the community and from other teams.
  • If the improvements look viable, the architecture team says so and writes up the algorithm as it's relevant to Parrot on the wiki, along with any relevant notes.
  • The architecture team puts out the call for someone to implement the code.
  • A Parrot hacker picks up the project.
  • Someone from the architecture and product teams follow the progress of the branch and review commits.
  • As the branch stabilizes, the product team benchmarks it (or ensures that it's benchmarked) to demonstrate a meaningful improvement.
  • As the branch stabilizes, QA also makes sure that it has good test coverage and documentation.
  • As the branch gets ready for merging, the product team checks that external projects won't be disrupted by the change.
  • The code is merged, well-documented and tested and doesn't break anything for Parrot's users.

advantages: Teams will ensure that Parrot has a unified direction as new research comes to our attention.  They'll also give us a clear path from paper to mergable code and will help enforce a higher bus number for new code, in addition to ensuring that code is documented and tested before it gets merged.
disadvantages: There will be a higher barrier to entry and increased dependence on the architecture team.


2: Significant Design Change

Say that a Parrot developer proposes a significant design change to address a bug or misfeature.  An example of this is Peter Lobsinger's dynop_mapping merge, which made some small but significant changes to bytecode.  The branch did a good job of solving the problem at hand, but one important test and a significant external project (examples/pir/make_hello_world_pbc.pir and PIRATE, respecitvely, which will be the subject of a later post) broke because of it and have yet to be fixed.  Here's how the process might work with teams in full force:

  •  Someone files a ticket or posts to parrot-dev or #parrot about a design flaw in Parrot that requires some redesigning.
  •  A Parrot hacker steps forward to fix it.
  •  Said hacker figures out a fix and discusses it with the architecture team.
  •  The architecture team reviews it and either gives the ok or helps iterate the design.
  •  The hacker starts implementing his changes.
    • While hacking, he describes the API consequences to the product QA teams, who update the relevant docs and/or add tests.
  • When the code is ready to merge (and ideally while the branch is being developed):
    • the architecture team reviews the code for bugs and to make sure design changes go as planned.
    • the product team reviews the code for user-facing changes.
    • QA makes sure that the changes are well-tested and documented.
  •  The code is merged, the relvant ticket is closed and everyone's happy.

advantages: Parrot maintains a unified direction across design decisions.  The team structure ensures that code is well-reviewed for different aspects while it's being worked on and that when coding is done, the branch will be (mostly) ready to merge.
disadvantages: This process will take more effort from the originator of the fix to explain his thinking and to answer questions during code review.  This will raise the bus number of the code, but will also raise the barrier to entry.


3: API Overhaul

Let's say that we decide that some part of our API needs a massive overhaul.  An example of this may be coming soon: Andrew Whitworth has expressed some distaste at the state of Parrot's embedding API and may soon take a much-needed jackhammer to it.  Here's how I envision the process working with teams:

  • The product team decide that an API needs massive refactoring in order to be useful to users, either through review or due to user feedback.
  • The product team figure out what the API should look like.
  • The product team hacks everything together in a branch.
  • QA looks at the branch to make sure that the new API functions are well-tested and that upcoming deprecations are documented
  • The architecture team does a brief review for sanity.
  • After the proper time for deprecations has passed, the changes are merged into trunk, causing much user jubilation.

advantages: API changes will have more dedicated code review with a specific aim.  More people will be looking over code changes and will be familiar with what will be merged into trunk.
disadvantages: The refactor will be more sensitive to tuit shortages on the part of different teams.



4: Lorito

Lorito is an upcoming major reenvisioning of Parrot at a low level.  Currently most of Parrot is written in C and PIR, and the impedance mismatch between the two is a significant bottleneck.  Lorito will be a very low-level and minimalist set of ops which will provide sufficient power to reimplement most of the C components of Parrot, eliminating the impedance mismatch, among other benefits.  Here's one way Lorito could become a reality:

  • We decide that Lorito is a good idea.
  • The architecture team leads the effort to figure out a rough timeline and order of events.
  • The architecture team leads the design and documentation effort to work out what a Lorito VM will look like.  Everyone is actively encouraged to participate.
  • Volunteers are solicited to implement prototypes to find holes in the design.  These holes are filled in as they're discovered.
  • As the design stabilizes, the product team looks at Lorito from a product perspective, helping further refine the design.
  • Once the design is settled, hacking on the final implementation begins in earnest according to the timeline.
  • The architecture, product and QA teams review major branches for design, test coverage and documentation as they progress.
  • After much effort, we are able to use Lorito overlays* as a replacement for internal Parrot components currently implemented in C.

advantages:  There's a consistent force ensuring that progress is made and a well-defined timeline.  All relevant parties have opportunity to voice their concerns and influence the final product.
disadvantages:  The process depends on having input from different teams and will be sensitive to tuit shortages.

* By "Lorito overlay", I mean anything that compiles down to Lorito ops.


5: Major Security Vulnerability

Let's say that a major security vulnerability is discovered and made known to Parrot's developers.  For this example, say that the latest supported release was 3.9.0 and that the latest developer release was 3.11.0.  Here's how we'd deal with this to ensure a minimal turnaround time:

  • The issue is raised and both 3.9.0 and 3.11.0 are found to be vulnerable.  Consistent with our support policy, the supported 3.9.0 release needs to be fixed.
  • Someone writes a proposed fix, either as a patch or a branch, depending on the vulnerability.
  • Representatives from QA team, product team and architecture teams briefly meet to make sure that the fix is sane (architecture), that the fix is valid, tested and documented as being fixed (QA), and that the fix doesn't negatively impact users (product).
  • The fix is committed to trunk, along with a backported version for 3.9.0 .  QA makes sure that new 3.9.1release is produced and distributed with appropriate notification.

advantages:  We provide a known-good fix in a timely manner, along with a regression test to ensure that the bug doesn't resurface.
disadvantages:  The structure requires some synchronization of schedules.



I hope that this provides a good idea of what I think the teams will look like as they work together to improve Parrot.  Nothing's set in stone yet, but my hope here is to provide a starting point for further discussion.
Internal organization of the architecture is a subject for another day.

2 comments:

  1. Christoph:

    Thank you for this detailed thinking-out-loud about the concept of teams in the Parrot project.

    I have to take exception with one statement, however. You state that "concept of teams was rushed into service without being entirely fully-formed." That implies that we actually have teams 'in service'.

    I would love that to be true, but I don't think it's true yet. The most we've actually accomplished is to identify a number of areas in which people can join task forces (a more project-focused, shorter-term concept than team).

    ReplyDelete
  2. Christoph, in Item #2, "Significant Design Change," you wrote:

    "disadvantages: This process will take more effort from the originator of the fix to explain his thinking and to answer questions during code review. This will raise the bus number of the code, but will also raise the barrier to entry."

    To my way of thinking, almost all of the above would, if implemented, actually be an advantage for the Parrot project over our current ways of working.

    Currently, when we identify an area needing major work, we hope that one of our best hackers steps forward to take on the task. We breathe a huge sigh of relief when he does. But we don't ask the hacker for a plan of attack. We basically trust the hacker to know what he is doing. We then hope that when the hacker says the branch is ready to merge, it is indeed ready and will not break things.

    Does this sound like any problems we've had lately?

    I can anticipate two objections to requiring our developers to articulate a plan of attach when tackling a major problem.

    "Some people are talented at writing code, but not at writing about code."

    To me the best response is to work with the developer to improve his ability to write about code, i.e., to work actively to reduce what you characterize as a "barrier to entry." Being able to explain the reason why code must change is a skill that will not only pay off for the Parrot project in the short-run, but for the developer in the IT business world in the long-run. I, for one, am willing to work with our developers on this.

    "You're proposing too much ceremony. We'll just get bogged down in procedures."

    I would respond that (a) we're not doing that great right now in the absence of good software development procedures; and (b) if our project is ever to scale (say, double the number of currently active contributors), we will need procedures like the ones you describe.

    Of course, we don't know whether any of this will work unless we try it. And that means that team members have to approach contributors and say, "I want you to join this team and work on this project." Let's JFDI.

    ReplyDelete