Skip to content

Development Workflow

Harsh Gupta edited this page Aug 16, 2015 · 4 revisions

Development Workflow

Please make sure you follow the following guidelines while sending pull requests, opening issues, etc. It is your responsibility to make sure you fully understand this documentation.

Create a separate branch

Typically, you will create a new branch to begin work on a new issue. Also pull request related with them.

A branch name should briefly describe the topic of the patch or pull request. If you know the issue number, then the branch name could be, for example, 1234_sequences. To create and checkout (that is, make it the working branch) a new branch ::

$ git branch 1234_sequences
$ git checkout 1234_sequences

or in one command using ::

$ git checkout -b 1234_sequences

To view all branches, with your current branch highlighted, type::

$ git branch

And remember, never type the following commands in master: git merge, git commit, git rebase.

Commit the changes

You can check what files are changed::

$ git status

Add new files to the index if necessary::

$ git add new_file.py

If you want to add only specific changes from a file use the patch flag which will let you choose chunks of patches interactively::

$ git add -p file.py

Check total changes::

$ git diff

You are ready to commit changes locally. A commit also contains a commit message which describes it. See the next section for guidelines on writing good commit messages. Type::

$ git commit

An editor window will appear automatically in this case. In Linux, this is vim by default. You can change what editor pops up by changing the $EDITOR shell variable.

Also with the help of option -a you can tell the command commit to automatically stage files that have been modified and deleted, but new files you have not told git about will not be affected, e.g.,::

$ git commit -a

If you want to stage only part of your changes, you can use the interactive commit feature. Just type::

$ git commit --interactive

and choose the changes you want in the resulting interface.

Writing commit messages

The commit message has two parts: a title (first line) and the body. The two are separated by a blank line.

There are only two formatting rules for commit messages

  • There should be a single line summary of 71 characters or less which allows the one-line form of the log to display the summary without wrapping. A common convention is to not end the summary with a period (full stop).

  • Additional details can be given after the summary. Make sure to leave a blank line after the summary and to keep all lines to 78 characters or less so they can be easily be read in terminals which don't automatically wrap lines.

Here is an example commit message (from the commit [bf0e81e12a2f75711c30f0788daf4e58f72b2a41] <https://github.com/sympy/sympy/commit/bf0e81e12a2f75711c30f0788daf4e58f72b2a41>_, which is part of the SymPy history)::

integrals: Improved speed of heurisch() and revised tests

Improved speed of anti-derivative candidate expansion and solution
phases using explicit domains and solve_lin_sys(). The upside of
this change is that large integrals (those that generate lots of
monomials) are now computed *much* faster. The downside is that
integrals involving Derivative() don't work anymore. I'm not sure
if they really used to work properly or it was just a coincidence
and/or bad implementation. This needs further investigation.

Example:

In [1]: from sympy.integrals.heurisch import heurisch

In [2]: f = (1 + x + x*exp(x))*(x + log(x) + exp(x) - 1)/(x + log(x) + exp(x))**2/x

In [3]: %time ratsimp(heurisch(f, x))
CPU times: user 7.27 s, sys: 0.04 s, total: 7.31 s
Wall time: 7.32 s
Out[3]:
   ⎛ 2        x                 2⋅x      x             2   ⎞
log⎝x  + 2⋅x⋅ℯ  + 2⋅x⋅log(x) + ℯ    + 2⋅ℯ ⋅log(x) + log (x)⎠          1
──────────────────────────────────────────────────────────── + ───────────────
                             2                                      x
                                                               x + ℯ  + log(x)

Previously it took 450 seconds and 4 GB of RAM to compute.

Some things to note about this commit message:

  • The first line gives a brief description of what the commit does. Tools like git shortlog or even GitHub only show the first line of the commit by default, so it is important to convey the most important aspects of the commit in the first line.

  • The first line has integrals:, which gives context to the commit. A commit won't always be seen in the context of your branch, so it is often helpful to give each commit some context. This is not required, though, as it is not hard to look at the commit metadata to see what files were modified or at the commit history to see the nearby related commits.

  • After the first line, there is a paragraph describing the commit in more detail. This is important, as it describes what the commit does, which might be hard to figure out just from looking at the diff. It also gives information that might not be in the diff at all, such as known issues. Such paragraphs should be written in plain English. Commit messages are intended for human readers, both for people who will be reviewing your code right now, and for people who might come across your commit in the future while researching some change in the code. Sometimes, bullet lists are a good format to convey the changes of a commit.

  • Last, there is an example. It is nice to give a concrete example in commits that add new features. This particular example is about improving the speed of something, so the example is a benchmark result.

Try to avoid short commit messages, like "Fix", and commit messages that give no context, like "Found the bug". When in doubt, a longer commit message is probably better than a short one.

Create a patch file or pull request for GitHub

Be sure that you are in your own branch, and run::

$ git push github 1234_sequences

This will send your local changes to your fork of the eklavya-2015 repository. Then navigate to your repository with the changes you want someone else to pull:

https://github.com/mynick/eklavya-2015

Select branch, and press the Pull Request button.

[[img/dev-guide-pull-1-2.png]]

After pressing the Pull Request button, you are presented with a preview page containing

  • a textbox for the title

  • a textbox for the description, also referred to as the opening paragraph (OP)

  • the commits that are included

    img/dev-guide-pull-2.png

The title and description may already have been pre-filled but they can be changed (see Writing pull request title and description_). Markdown is supported in the description, so you can embed images or use preformatted text blocks.

[[img/dev-guide-pull-3.png]]

You can double check that you are committing the right changes by

  • switching to the Commits tab to see which commits are included (sometimes unintended commits can be caught this way)
  • switching to the Files Changed tab to review the diff of all changes

When you are ready, press the Send pull request button. The pull request is sent immediately and you’re taken to the main pull request discussion and review page. Additionally, all repository collaborators and followers will see an event in their dashboard.

If there isn't an issue that the pull request addresses, one should be created so even if the pull request gets closed there is a redundant reference to it in the issues.

See also Updating your pull request_

Writing pull request title and description

You might feel that all your documentation work is done if you have made good commit messages. But a good title and description will help in the review process.

The title should be brief but descriptive.

  • don't write "fixes #1234" there; such references are more useful in the description section.
  • do include the prefix "[WIP]" if you aren't ready to have the pull request merged and remove the prefix when you are ready

The description is a good place to:

  • show what you have done, perhaps comparing output from master with the output after your changes
  • refer to the issue that was addressed like "#1234"; that format will automatically create a link to the corresponding issue or pull request, e.g. "This is similar to the problem in issue #1234...". This format also works in the discussion section of the pull request.
  • use phrases like "closes #1234" or "fixed #1234" (or similar that follow the auto-close syntax <https://help.github.com/articles/closing-issues-via-commit-messages>_) then those other issues or pull requests will be closed when your pull request is merged. Note: this syntax does not work in the discussion of the pull request.

See also github's own guidelines for pull requests <https://github.com/blog/1943-how-to-write-the-perfect-pull-request>_

Updating your pull request

If you need to make changes to a pull request there is no need to close it. The best way to make a change is to add a new commit in you local repository and simply repeat push command::

$ git commit
$ git push github 1234_sequences

Note that if you do any rebasing or in any way edit your commit history, you will have to add the -f (force) option to the push command for it to work::

$ git push -f github

You don't need to do this if you merge, which is the recommended way.

Synchronization with master

Sometimes, you may need to merge your branch with the upstream master. Usually you don't need to do this, but you may need to if

  • Someone tells you that your branch needs to be merged because there are merge conflicts.

  • Github/Travis tells you that your branch could not be merged.

  • You need some change from master that was made after you started your branch.

Note, that after cloning a repository, it has a default remote called origin that points to the parent AGV-IIT-KGP repository. And your fork remote named as github. You can observe the remotes names with the help of this command::

$ git remote -v
github  [email protected]:mynick/eklavya-2015.git (fetch)
github  [email protected]:mynick/eklavya-2015.git (push)
origin  git://github.com/AGV-IIT-KGP/eklavya-2015.git (fetch)
origin  git://github.com/AGV-IIT-KGP/eklavya-2015.git (push)

As an example, consider that we have these commits in the master branch of local git repository::

A---B---C        master

Then we have divergent branch 1234_sequences::

A---B---C           master
         \
          a---b     1234_sequences

In the meantime the remote AGV-IIT-KGP/eklavya-2015 master repository was updated too::

A---B---C---D       origin/master
A---B---C           master
         \
          a---b     1234_sequences

There are basically two ways to get up to date with a changed master: merging and rebasing. **In general, rebasing should only be used before you make a pull request. Once the pull request is made your code is considered public and the history should not be changed through rebasing.

Merging


Merging creates a special commit, called a "merge commit", that joins your
branch and master together::

    A---B---C------D       origin/master
             \      \
              \      M     merge
               \    /
                a--b       1234_sequences


Note that the commits ``A``, ``B``, ``C``, and ``D`` from master and the
commits ``a`` and ``b`` from ``1234_sequences`` remain unchanged. Only the new
commit, ``M``, is added to ``1234_sequences``, which merges in the new commit
branch from master.


Merging

First merge your local repository with the remote::

$ git checkout master
$ git pull

This results in::

A---B---C---D       master
         \
          a---b     1234_sequences

Then merge your 1234_sequences branch from 1234_sequences::

$ git checkout 1234_sequences
$ git merge master

If the last command tells you that conflicts must be solved for a few indicated files.

If that's the case then the marks >>> and <<< will appear at those files. Fix the code with >>> and <<< around it to what it should be. You must manually remove useless pieces, and leave only new changes from your branch.

and commit::

$ git commit

So the result will be like that (automatic merging c)::

A---B---C-------D     master
         \       \
          a---b---M   1234_sequences

Changing of commit messages

The only time when it is recommended to rebase instead of merge is when you need to edit your commit messages, or remove unnecessary commits.

Note, it is much better to get your commit messages right the first time. See the section on writing good commit messages above.

Consider these commit messages::

$ git log --oneline
7bbbc06 bugs fixing
4d6137b some additional corrections.
925d88fx sequences base implementation.

Then run rebase command in interactive mode::

$ git rebase --interactive 925d88fx

Or you can use other ways to point to commits, e.g. git rebase --interactive HEAD^^ or git rebase --interactive HEAD~2.

A new editor window will appear (note that order is reversed with respect to the git log command)::

pick 4d6137b some additional corrections.
pick 7bbbc06 bugs fixing

# Rebase 925d88f..7bbbc06 onto 925d88f
#
# Commands:
#  p, pick = use commit
#  r, reword = use commit, but edit the commit message
#  e, edit = use commit, but stop for amending
#  s, squash = use commit, but meld into previous commit
#  f, fixup = like "squash", but discard this commit's log message

To edit a commit message, change pick to reword (or on old versions of git, to edit) for those that you want to edit and save that file.

To squash two commits together, change pick to squash. To remove a commit, just delete the line with the commit.

To edit a commit, change pick to edit.

After that, git will drop you back into your editor for every commit you want to reword, and into the shell for every commit you wanted to edit::

$ (Change the commit in any way you like.)
$ git commit --amend -m "your new message"
$ git rebase --continue

For commits that you want to edit, it will stop. You can then do::

$ git reset --mixed HEAD^

This will "uncommit" all the changes from the commit. You can then recommit them however you want. When you are done, remember to do::

$ git rebase --continue

Most of this sequence will be explained to you by the output of the various commands of git. Continue until it says: ::

Successfully rebased and updated refs/heads/master.

If at any point you want to abort the rebase, do::

$ git rebase --abort

Warning: this will run git reset --hard, deleting any uncommitted changes you have. If you want to save your uncommitted changes, run git stash first, and then run git stash pop when you are done.

Reviewing patches

Coding's only half the battle in software development: our code also has to be thoroughly reviewed before release. Reviewers thus are an integral part of the development process. Note that you do not have to have any special pull or other privileges to review patches: anyone with Python on his/her computer can review.

When discussing patches, be polite and stick to the point of the patch. GitHub has published an excellent set of guidelines for pull requests <https://github.com/blog/1943-how-to-write-the-perfect-pull-request>_; it is recommended reading for reviewers as well as coders.

Manual testing

If you prefer to test code manually, you will first have to set up your environment as described in the Workflow process section. Then, you need to obtain the patched files. If you're reviewing a pull request, you should get the requested branch into your folder. Go into your folder and execute ( being the username of the pull requester and being the git branch of the pull request)::

$ git remote add <username> git://github.com/<username>/eklavya-2015.git
$ git fetch <username>
$ git checkout -b <branchname> <username>/<branchname>

After obtaining the commits, test them.

If there are any problems, notify the author in the pull request by commenting.

  • Forked from Sympy's development workflow wiki