How to preserve cell_markers in a python (percent) script?

[caenrigen]

Hi folks!

First of all thank you very much for this awesome package! 😌

Have a question that I was not able to understand from the docs how to achieve. Hope I am not duplicating an existing issue but I was not able to find a similar one.

I have only a .py script in the percent format with the following header, which I adit as a notebook in Jupyter Lab without having it synced to any other format.

# ---
# jupyter:
#   jupytext:
#     formats: py:percent
#     text_representation:
#       extension: .py
#       format_name: percent
#       format_version: '1.3'
#       jupytext_version: 1.12.0
#       cell_markers: '"""' # <-------- RELEVANT LINE
#   kernelspec:
#     display_name: Python 3 (ipykernel)
#     language: python
#     name: python3
# ---

# %% [raw]
"""
The first time I save the file with the header above all raw cell will be saved like this.
"""

# %% [raw]
# But if I kill the kernel and open it again new raw cell will be saved like this because the `cell_markers` is excluded from the header...

After I save the file in Jupyter Lab the header is processed correctly and the effect of cell_markers is as expected, however the key itself is excluded from the header, is there a way to keep it there?

I tried the notebook_metadata_filter but it seems that the it always filters itself as well, which means the cell_markers won't be ever saved either.

To sum up, the goal is:

• Save and version-control only a .py script.
• Save raw cells inside """ <raw cell contents> """ blocks instead of many # -commented lines.
• Keep the jupytext configuration in the script file itself to allow for consistent format when edited later.

Any ideas of what is happening and how to go about this? Any insight would be appreciated!

PS If you are curious I am building a sphinx extension that combines jupytext and jupyter-sphinx to be able to write tutorial and documentation in sphinx from python scripts directly. The goal is to keep it simple: raw cells are copy-pasted as raw rst blocks and (python) code cells are converted to .. jupyter-execute:: directives.

[mwouts]

Hi @caenrigen , thank you for asking! I think the cell_markers should be directly under jupytext, cf. these tests.

Out of curiosity, what made you think that it should go to the text_representation dict? I am asking because there might be something to improve here...

Also, good luck with your project! If that can spare you some work - did you look at nbsphinx? I think that, in addition to .ipynb files, they also support Jupytext notebooks.

[caenrigen]

Hi @caenrigen , thank you for asking! I think the cell_markers should be directly under jupytext, cf. these tests.

Out of curiosity, what made you think that it should go to the text_representation dict? I am asking because there might be something to improve here...

Also, good luck with your project! If that can spare you some work - did you look at nbsphinx? I think that, in addition to .ipynb files, they also support Jupytext notebooks.

Hi, Marc!

Thank you very much for the reply and tests snippets, it works indeed, I was doing some silly mistake somewhere, and at some point I thought maybe I am placing the key in the wrong place so i tried putting it under text_representation as well.

Since you mention it, I might be expected there because it a somewhat specific setting of a specific text format.

The tip about the nbsphinx is much appreciated! 🤩🤩🤩 I was under the impression that is was less flexible, but just found out that it might indeed do exactly what i need, I will explore this a bit more in depth.

Feel free to close this issue :)

[caenrigen]

@mwouts

PS I had a look at nbsphinx. While it is very nice and actually fully featured, it has some limitation, for example, it is not flexible enough to allow having a notebook cell inside a droppable .. note:: directive or inside some other custom rst directive. In addition rst cells are not supported in Jupyter Lab (only Jupyter Notebook).

In any case, thank you for the tip, it has some nice things in there that I would like to have for my extensions as well (like making the "View source" link to the correct file).

Cheers

[mwouts]

Great! I am glad that we could answer your initial question. Regarding your work on the sphinx extension, please keep us posted, and also do not hesitate to contact the maintainers of nbsphinx, I am sure they will be happy to help.

[caenrigen]

Great! I am glad that we could answer your initial question. Regarding your work on the sphinx extension, please keep us posted, and also do not hesitate to contact the maintainers of nbsphinx, I am sure they will be happy to help.

@mwouts, About the sphinx extensions, I have proposed it to jupyter-sphinx folks in a new issue jupyter/jupyter-sphinx#187, feel free to Subscribe and contribute :)