Work in progress - feedback welcome

[ntoll]

Add documentation-translation workflow.

[deshipu]

I think it would be better to actually make a fork of the microbit repository -- then we can rebase the new repository on top of the main repository to see what changes in the documentation need to be translated.

Alternatively, we could move the docs to a separate repository, and fork that.

[carlosperate]

then we can rebase the new repository on top of the main repository to see what changes in the documentation need to be translated

I am not quite sure I understand how that rebase would be that useful. If I fork this repository, and start editing all the documentation files, then an initial rebase will basically show me all the English text changed to the new language, fair enough. But then if the English documentation is updated any rebase will start getting a bunch of conflicts and even after resolving them, unless you can speak the translated language, there would be no way to see if the text is up to date (as the rebase would be changing the commit timestamp).

[deshipu]

Actually, looking how large the microbit micropython repository is (16MB), I really think we should split the docs off to a separate repository.

[deshipu]

The conflicts are the whole point of the rebase -- they will show you where a new translation is needed. Of course, they will be resolved by people knowing both languages -- the translators themselves.

[mdanka]

I also don't understand why the rebase would help, but maybe I'm missing something. After the first translation, any diffs (changes) on the original English version would not apply well to the newly translated version. Is this wrong?

My feeling is that you'll have to track changes in the English version, and then "human-apply" the English-diff onto the new language.

Forking+rebase is useful to preserve/propagate changes in the setup and configuration.

[deshipu]

Right, we can always switch to something better when we find it.

[deshipu]

I don't have a solution, but I could imagine a simple script that would pull the changes from the master, merge them, and then find the conflict markers and replace them with some Sphinx syntax to make them render nicely (we could add some CSS to make the old translated text appear with some background, and display the updated English text on hover, or something like that). Of course replacing the conflict markers in a way that doesn't mangle the restructured text syntax is a non-trivial task, but I think I could write something that at least handles most of the cases.

Then we can have that script run as a part of config.py or as a commit hook somewhere. (Or, run it manually and then correct the cases where it breaks the rendering, if I can't get it to work properly all the time.) The thing is, you don't need to know the language to which it is translated, to do that, so it can be done by a more technical person.

[carlosperate]

Yeah they sound like feasible solutions, but if it's hard enough to get experienced developers to run test scripts before submitting a PR, then I would expect most PR submissions to not have followed these steps locally, much less to manually configure git hooks. So ideally something on the config.py would be the best way to go, I assume there is no way to "preview" PRs in readthedocs? So if somebody submits something and we need to check if it will render correctly, then the maintainer would have to manually patch a local copy and render the page on their computer?

[deshipu]

You can checkout the pull request with git, and run 'make html' to generate the html locally. There is restructured text preview in github, but it doesn't understand Sphinx directives, so while it will be fine for normal text, it will fail for the reference docs.

[deshipu]

Fortunately, for translations the markup will remain largely the same, only the words will change.

[deshipu]

Not sure if we don't want to have all the projects created under the same user, or at least a "common" user to have access rights to all of them.

[carlosperate]

Does readthedocs have anything similar or parallel to github organisations?

[deshipu]

Not that I know of.

[carlosperate]

Just to clarify, were you suggesting to have all the github repositories for each translation under the same user/"hub"/organisation, or were you referring to a readthedocs user?

[deshipu]

I was thinking about the readthedocs user, but of course it would also make sense to have a github organization for them.

[carlosperate]

I've never publish docs on readthedocs before, so this question comes mostly from my ignorance in this respect: Will this localised repository contain the markdown/restructuredtext files in master, and the Sphinx HTML output on a gh-pages branch with a CNAME file? Or is it enough to have the files in the repository and the build and publish process is then taken care by readthedocs?

[deshipu]

You just put the restructured text sources in there, just like the microbit repository does (see https://github.com/bbcmicrobit/micropython/tree/master/docs) and readthedocs takes care of everything else, including re-building everything on every commit.

[carlosperate]

Oh that's great, thanks for the clarification @deshipu.

[carlosperate]

This bit didn't seem all that clear to me from the https://docs.readthedocs.io/en/latest/localization.html#project-with-multiple-translations page. How is this link between the doc projects created? Is there a configuration page on readthedocs? a project file that needs to go to one or more repositories?

[deshipu]

Yes, there is a configuration page at https://readthedocs.org/dashboard/microbit-micropython/translations/ that we will use to add any translation projects.

[carlosperate]

Right, so the administrators would basically add the repository links for each supported language. Is that configuration page also used to indicate which language the repository represents? Or does the repository creator need to do anything to indicate the language?

[deshipu]

The creator of the repository needs to set the proper language on theirs readthedocs page. We can check if that is set up correctly before linking to it.

[carlosperate]

Okay, got it, thanks for the clarification again :)

[deshipu]

Hmm... so it seems they have a bug in the readthedocs, as this doesn't seem to work as expected.

• I created a fork of the microbit micropython repository: https://github.com/deshipu/microbit-micropython-pl
• I translated the front page of the documentation to Polish: https://github.com/deshipu/microbit-micropython-pl/blob/master/docs/index.rst
• I created a project on readthedocs pointing to that repository, specifying the language as Polish: https://readthedocs.org/projects/microbit-micropython-pl
• I linked that project as the translation of the microbit-micropython repository.

The result is that the default URL for the microbit-micropython-pl changed to http://microbit-micropython.readthedocs.io/pl/latest/ but that page returns a 404. Visiting http://microbit-micropython-pl.readthedocs.io works correctly, and you can switch the language to English in the dropdown, but switching the other way around doesn't work.

[carlosperate]

As it's currently described it sounds like each person that would like to create a translation would have to create/copy/fork a github repository on their username, create a read the docs account or projects, and then submit it for inclusion.

If that's the case I think that would fragment the documentation, it would be a bit weird that people that would want to update or complete the official documentation for https://github.com/bbcmicrobit/micropython/ would have to submit pull request to something like https://github.com/carlosperate/microbit-micropython-es , and that's if they are first able to figure out where to send the PR.

I would suggest to change the process so that the intention to create translation would have to be declared as an issue, and then a maintainer could create a repository inside https://github.com/bbcmicrobit to submit pull requests to. Alternatively we could request for the user to transfer ownership of the repository to the bbcmicrobit GH organisation, but that would also need to be a clear expectation.

[deshipu]

If that's the case I think that would fragment the documentation, it would be a bit weird that people that would want to update or complete the official documentation for https://github.com/bbcmicrobit/micropython/ would have to submit pull request to something like https://github.com/carlosperate/microbit-micropython-es , and that's if they are first able to figure out where to send the PR.

That's not an issue. There is an "edit on Github" button in the top right corner, that will automatically take you to github, let you edit the file, fork the correct repository and let you create a pull request.

[deshipu]

The result is that the default URL for the microbit-micropython-pl changed to http://microbit-micropython.readthedocs.io/pl/latest/ but that page returns a 404.

This issue fixed itself overnight (probably some caching thing somewhere) and the translation now works as expected.

[peconia]

Is there a way of marking a translation to be in progress and highlight it is not finished yet? Should the issue to add the translation be raised as soon as the translation is started, or wait until there is a translation of all the docs? It would be nice to have a way of inviting collaborators too.

I've started the Finnish translation at https://github.com/peconia/micropython-finnish (readthedocs at http://micropython-finnish.readthedocs.io/fi/latest/ ) following what @deshipu did for Polish, but I'm not sure if I should continue with this or wait :)

[deshipu]

Is there a way of marking a translation to be in progress and highlight it is not finished yet? Should the issue to add the translation be raised as soon as the translation is started, or wait until there is a translation of all the docs?

I'm not sure about this. Is it ever "finished"? We are not publishing a book, so there is no "deadline" or a cut-off point after which no corrections can be made. I think it's best to link it as soon as possible, and leave the untranslated part in English -- that may be good for inviting collaboration, too, as people can get annoyed with the English text in the middle of translated work, and go fix it.

It would be nice to have a way of inviting collaborators too.
Since it's on github, you have all the tools for that that github offers, plus all the traditional means, like sending a link with e-mail, sharing it on the social portals, etc.

I think it would be good to agree on a naming convention, such as "microbit-micropython-XX", where XX is the language code, so that it's easier to search for those translations.

[deshipu]

By the way, I just realized that we also need to translate the graphics -- for instance, the comic strip on the first page. Is that available somewhere in some more editable format, like SVG? Or do we have to re-touch the pixels?

[deshipu]

Answering my own question, I just found https://pycomic.github.io/ and I can re-generate the comic in Polish: https://goo.gl/5on66z

Just one small detail: I can't translate the word "by" in the title into Polish (well, OK, I can use the web inspector and edit the page's HTML, but that's not something an average translator can do).