Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Env variable for extra_template_basedirs #2026

Open
govinda18 opened this issue Aug 1, 2023 · 5 comments · May be fixed by #2028
Open

Env variable for extra_template_basedirs #2026

govinda18 opened this issue Aug 1, 2023 · 5 comments · May be fixed by #2028

Comments

@govinda18
Copy link

Hi team,

I want to be able to set extra_template_basedirs via some env variable. I am using in an internal infrastructure that needs this and some external library seems to use PythonExporter to convert a notebook into python.

Would be helpful to set this globally so this gets automatically picked up.

ndmlny-qs added a commit to ndmlny-qs/nbconvert that referenced this issue Aug 3, 2023
This commit add the following three items.

- Updates the method `_default_extra_template_basedirs` in the module
  `templateexporter.py` to search for a new environment variable called
  `NBCONVERT_EXTRA_TEMPLATE_BASEDIRS`. This environment variable will be
  added to the basedirs list if it is found, and it exists.
- Updates the tests to ensure if the new environment variable exists
  then it is included in the `TemplateExporter.template_paths`.
- Adds a context manager method that modifies the `os.environ`
  dictionary.

Closes jupyter#2026
@ndmlny-qs ndmlny-qs linked a pull request Aug 3, 2023 that will close this issue
@Carreau
Copy link
Member

Carreau commented Aug 10, 2023

I'd like to understand this request a bit more instead of providing a band aid for a single option.

Do you have a way of creating ~/.jupyter/jupyter_nbconvert_config.py (or any other default config file) ? You should be able to also find all the places you can add those default configs using jupyter --paths.

With this you should be able to hook into any logics you want without waiting for a patch/release.

$ cat ~/.jupyter/jupyter_nbconvert_config.py
import os

extra = os.environ.get("NBCONVERT_EXTRA_TEMPLATE_BASEDIR")
if extra:
    # you can even use `.append()` or `.extend()` on `c.Exporter.extra_template_basedirs`
    c.Exporter.extra_template_basedirs = [
        os.environ.get("NBCONVERT_EXTRA_TEMPLATE_BASEDIR")
    ]

Or maybe the software you use explicitly as nbconvert to not read its config, and hence the need for a patch.

This does not means we cannot add the option regardless, but using the default config capabilities should allow you to modify any config option instead of creating and add-hoc version every time.

If there is a global reason to use ENV variable instead of doing on a per-use-case, we could imaging having a mapping from env variable to configuration option, say NBCONVERT__<classname>__<Option_name>=value.

That would make sens I think, but you also should be able to achieve it now with:

$ cat ~/.jupyter/jupyter_nbconvert_config.py
import os

for k, v in os.environ.items():
    if k.startswith("NBCONVERT"):
        # use __ instead of . as separator in env variable.
        # Warning, case sensitive !
        _, klass, attribute = k.split("__")
        # c.Klass.attribute=v
        setattr(getattr(c, klass), attribute, v)

Allows me to set $ export NBCONVERT__Exporter__extra_template_basedirs='mypath', and this will have the expected effect.

@ndmlny-qs
Copy link

Thanks for the thoughtful considerations @Carreau. @govinda18 can chime in on their use case, but I can think of one where environment variables would be useful.

If you have an automated deploy scenario for clients, and want all of them to use a centralized extra nbconvert templates directory that you have custom artifacts in, then I see two options available:

  1. in your deploy scripts, you generate the jupyter_nbconvert_config.py file for each client that gets spun up;
  2. have an env variable that nbconvert recognizes on each client.

I suspect that (since it is easy to add/remove environment variables for newly create clients) having nbconvert read environment variables compared to updating client preferences in a python file is the easiest change from a devops perspective. This is of course is just a guess.

@Carreau
Copy link
Member

Carreau commented Aug 10, 2023

Yes, this is my guess as well, but if that's the case then likely the need is likely not limited to nbconvert, and it might be worth it to patch traitlets to load from env by default in the same way that we have a config file loader and and cli loader ?

@Carreau
Copy link
Member

Carreau commented Aug 28, 2023

@govinda18 I wanted to know if you had a chance to look at my suggestions and clarify your use case.

@Carreau
Copy link
Member

Carreau commented Nov 28, 2023

#2075 might help

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants