diff --git a/learn_the_basics.rst b/learn_the_basics.rst index 76d3eb08..b3b3352b 100644 --- a/learn_the_basics.rst +++ b/learn_the_basics.rst @@ -1,4 +1,4 @@ -Learn the basics +Learn the Basics ---------------- .. grid:: 1 1 3 3 diff --git a/learn_the_basics/03_trace_code.ipynb b/learn_the_basics/03_trace_code.ipynb index f87c5602..56b3a5b6 100644 --- a/learn_the_basics/03_trace_code.ipynb +++ b/learn_the_basics/03_trace_code.ipynb @@ -23,7 +23,7 @@ "source": [ "⚠️ If you are running this notebook in Colab, you will have to install `Ivy` and some dependencies manually. You can do so by running the cell below ⬇️\n", "\n", - "If you want to run the notebook locally but don't have Ivy installed just yet, you can check out the [Get Started section of the docs.](https://unify.ai/docs/ivy/overview/get_started.html)" + "If you want to run the notebook locally but don't have Ivy installed just yet, you can check out the [Get Started section of the docs.](https://www.docs.ivy.dev/overview/get_started.html)" ] }, { @@ -40,24 +40,21 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Firstly, let's pick up where we left off in the [last notebook](02_unify_code.ipynb), with our unified `normalize` function:" + "Let's begin with an implementation of the `normalize` function using `ivy`'s Functional API:" ] }, { "cell_type": "code", - "execution_count": 2, + "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "import ivy\n", - "import torch\n", "\n", "def normalize(x):\n", - " mean = torch.mean(x)\n", - " std = torch.std(x)\n", - " return torch.div(torch.sub(x, mean), std)\n", - "\n", - "normalize = ivy.unify(normalize, source=\"torch\")" + " mean = ivy.mean(x)\n", + " std = ivy.std(x)\n", + " return ivy.divide(ivy.subtract(x, mean), std)" ] }, { @@ -70,47 +67,31 @@ }, { "cell_type": "code", - "execution_count": 4, - "metadata": {}, - "outputs": [], - "source": [ - "# set ivy's backend to jax\n", - "ivy.set_backend(\"jax\")\n", - "\n", - "# Import jax\n", - "import jax\n", - "\n", - "# create random jax arrays for testing\n", - "key = jax.random.PRNGKey(42)\n", - "x = jax.random.uniform(key, shape=(10,))" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "As in the previous example, the Ivy function can be executed like so (in this case it will trigger lazy unification, see the [Lazy vs Eager](05_lazy_vs_eager.ipynb) section for more details):" - ] - }, - { - "cell_type": "code", - "execution_count": 5, + "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "ivy.array([ 0.55563945, -0.65538704, -1.14150524, 1.46951997, 1.30220294,\n", - " -1.14739668, -0.57017946, -0.91962677, 0.51029003, 0.59644395])" + "ivy.array([ 0.58569533, -0.69083852, -1.20325196, 1.5490098 , 1.37264228,\n", + " -1.20946217, -0.60102183, -0.96937162, 0.53789282, 0.62870705])" ] }, - "execution_count": 5, + "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ + "# set ivy's backend to jax\n", + "ivy.set_backend(\"jax\")\n", + "\n", + "# Import jax\n", + "import jax\n", + "\n", + "# create random jax arrays for testing\n", + "key = jax.random.PRNGKey(42)\n", + "x = jax.random.uniform(key, shape=(10,))\n", "normalize(x)" ] }, @@ -137,7 +118,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The traced function can be executed in exactly the same manner as the non-traced function (in this case it will also trigger lazy graph tracing, see the [Lazy vs Eager](05_lazy_vs_eager.ipynb) section for more details):" + "The traced function can be executed in exactly the same manner as the non-traced function:" ] }, { @@ -148,8 +129,8 @@ { "data": { "text/plain": [ - "Array([ 0.5556394 , -0.655387 , -1.1415051 , 1.4695197 , 1.3022028 ,\n", - " -1.1473966 , -0.5701794 , -0.91962665, 0.51028997, 0.5964439 ], dtype=float32)" + "Array([ 0.5856953 , -0.6908385 , -1.203252 , 1.5490098 , 1.3726423 ,\n", + " -1.2094622 , -0.6010218 , -0.9693716 , 0.5378928 , 0.62870705], dtype=float32)" ] }, "execution_count": 9, @@ -171,14 +152,14 @@ }, { "cell_type": "code", - "execution_count": 11, + "execution_count": 10, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "985 µs ± 6.76 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)\n" + "138 ms ± 3.57 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)\n" ] } ], @@ -196,7 +177,7 @@ "name": "stdout", "output_type": "stream", "text": [ - "69.5 µs ± 1.24 µs per loop (mean ± std. dev. of 7 runs, 10,000 loops each)\n" + "122 µs ± 2.02 µs per loop (mean ± std. dev. of 7 runs, 10,000 loops each)\n" ] } ], @@ -210,7 +191,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "As expected, we can see that `normalize` is slower, as it includes all `ivy` wrapping overhead. On the other hand, `traced` has no wrapping overhead and it's more efficient!" + "As expected, we can see that `normalize` is slower, as it includes all `ivy` wrapping overhead. On the other hand, `traced` has no wrapping overhead and it's more efficient!\n", + "\n", + "> Fun Fact: You can use the graph tracer with pretty much any code written in one of the ML frameworks Ivy supports i.e. PyTorch, TensorFlow, Jax, NumPy etc. and speed it up by removing unnecessary computations that don't contribute towards the output by extracting an efficient computation graph stitched together in the set backend framework!" ] }, { @@ -226,13 +209,13 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "That's it, you can now trace `ivy` code for more efficient inference! However, there are several other important topics to master before you're ready to unify ML code like a pro 🥷. Next, we'll be learning how to transpile code from one framework to another in a single line of code 🔄" + "That's it, you can now trace `ivy` code for more efficient inference! However, there are several other [important topics](https://www.docs.ivy.dev/demos/learn_the_basics.html) to master before you're ready to play with ML code like a pro 🥷." ] } ], "metadata": { "kernelspec": { - "display_name": "Python 3", + "display_name": "tracer-transpiler", "language": "python", "name": "python3" }, @@ -246,7 +229,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.8.10" + "version": "3.10.13" }, "orig_nbformat": 4 }, diff --git a/learn_the_basics/05_lazy_vs_eager.ipynb b/learn_the_basics/05_lazy_vs_eager.ipynb index cf129c89..97209869 100644 --- a/learn_the_basics/05_lazy_vs_eager.ipynb +++ b/learn_the_basics/05_lazy_vs_eager.ipynb @@ -23,7 +23,7 @@ "source": [ "⚠️ If you are running this notebook in Colab, you will have to install `Ivy` and some dependencies manually. You can do so by running the cell below ⬇️\n", "\n", - "If you want to run the notebook locally but don't have Ivy installed just yet, you can check out the [Get Started section of the docs.](https://unify.ai/docs/ivy/overview/get_started.html)" + "If you want to run the notebook locally but don't have Ivy installed just yet, you can check out the [Get Started section of the docs.](https://www.docs.ivy.dev/overview/get_started.html)" ] }, { @@ -97,9 +97,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "`ivy.unify`, `ivy.trace_graph` and `ivy.transpile` can all be performed either eagerly or lazily. All previous examples have been performed **lazily**, which means that the unification, tracing, or transpilation process actually occurs during the first call of the **returned** function. \n", + "`ivy.trace_graph` can be performed either eagerly or lazily since it depends on function tracing, which requires function arguments to use for tracing. `ivy.transpile` however doesn't depend on function tracing and so is applied eagerly without requiriing function arguments. One exception though where `ivy.transpile` defaults to lazy transpilation is when transpiling entire modules like `kornia`. It isn't until a function or a class is executed from the lazy transpiled kornia module that the transpilation is actually performed.\n", "\n", - "This is because all three of these processes depend on function tracing, which requires function arguments to use for the tracing. Alternatively, the arguments can be provided during the `ivy.unify`, `ivy.trace_graph` or `ivy.transpile` call itself, in which case the process is performed **eagerly**. We show some simple examples for each case below." + "For `ivy.trace_graph`, the arguments can be provided during the `ivy.trace_graph` call itself, in which case the process is performed **eagerly**. We show some simple examples for each case below." ] }, { @@ -107,7 +107,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## Unify" + "## Trace" ] }, { @@ -115,26 +115,44 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Consider again this simple `torch` function:" + "In the example below, the function is traced **lazily**, which means the first function call will execute slowly, as this is when the tracing process actually occurs." ] }, { "cell_type": "code", - "execution_count": 2, + "execution_count": 1, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "WARNING:root:\tSome binaries seem to be missing in your system. This could be either because we don't have compatible binaries for your system or that newer binaries were available. In the latter case, calling ivy.utils.cleanup_and_fetch_binaries() should fetch the binaries binaries. Feel free to create an issue on https://github.com/ivy-llc/ivy.git in case of the former\n", + "\n", + "WARNING:root:\n", + "Following are the supported configurations :\n", + "compiler : cp38-cp38-manylinux_2_17_x86_64, cp38-cp38-win_amd64, cp39-cp39-manylinux_2_17_x86_64, cp39-cp39-win_amd64, cp310-cp310-manylinux_2_17_x86_64, cp310-cp310-win_amd64, cp310-cp310-macosx_12_0_arm64, cp311-cp311-manylinux_2_17_x86_64, cp311-cp311-win_amd64, cp311-cp311-macosx_12_0_arm64, cp312-cp312-manylinux_2_17_x86_64, cp312-cp312-win_amd64, cp312-cp312-macosx_12_0_arm64\n", + "\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n" + ] + } + ], "source": [ "import ivy\n", - "import torch\n", "\n", "def normalize(x):\n", - " mean = torch.mean(x)\n", - " std = torch.std(x)\n", - " return torch.div(torch.sub(x, mean), std)" + " mean = ivy.mean(x)\n", + " std = ivy.std(x)\n", + " return ivy.divide(ivy.subtract(x, mean), std)" ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -143,7 +161,7 @@ }, { "cell_type": "code", - "execution_count": 3, + "execution_count": 2, "metadata": {}, "outputs": [], "source": [ @@ -156,7 +174,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -165,17 +182,25 @@ }, { "cell_type": "code", - "execution_count": 4, + "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ - "2023-11-01 06:53:37.201733: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.\n", - "To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.\n", - "/workspaces/ivy/ivy/utils/exceptions.py:390: UserWarning: The current backend: 'tensorflow' does not support inplace updates natively. Ivy would quietly create new arrays when using inplace updates with this backend, leading to memory overhead (same applies for views). If you want to control your memory management, consider doing ivy.set_inplace_mode('strict') which should raise an error whenever an inplace update is attempted with this backend.\n", - " warnings.warn(\n" + "2024-10-19 02:11:29.834162: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.\n", + "2024-10-19 02:11:29.965072: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\n", + "2024-10-19 02:11:30.015897: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\n", + "2024-10-19 02:11:30.028543: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\n", + "2024-10-19 02:11:30.107819: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.\n", + "To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.\n", + "2024-10-19 02:11:31.669139: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT\n", + "WARNING: All log messages before absl::InitializeLog() is called are written to STDERR\n", + "I0000 00:00:1729285894.477775 848849 cuda_executor.cc:1001] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node\n", + "Your kernel may have been built without NUMA support.\n", + "2024-10-19 02:11:34.711139: W tensorflow/core/common_runtime/gpu/gpu_device.cc:2343] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.\n", + "Skipping registering GPU devices...\n" ] } ], @@ -187,11 +212,12 @@ ] }, { - "attachments": {}, - "cell_type": "markdown", + "cell_type": "code", + "execution_count": 4, "metadata": {}, + "outputs": [], "source": [ - "In the example below, the function is unified **lazily**, which means the first function call will execute slowly, as this is when the unification process actually occurs." + "norm_trace = ivy.trace_graph(normalize) # Lazy transpilation as no args are provided, executes quicker" ] }, { @@ -199,18 +225,12 @@ "execution_count": 5, "metadata": {}, "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "WARNING:root:To preserve the tracer and transpiler caches across multiple machines, ensure that the relative path of your projects from the .ivy folder is consistent across all machines. You can do this by adding .ivy to your home folder and placing all projects in the same place relative to the home folder on all machines.\n" - ] - }, { "data": { "text/plain": [ - "ivy.array([-0.34431235, 0.51129461, -0.06686894, -0.36452447, -0.98795534,\n", - " 0.15493582, -0.91630631, 1.41939619, 1.78909753, -1.19475674])" + "" ] }, "execution_count": 5, @@ -219,17 +239,7 @@ } ], "source": [ - "norm = ivy.unify(normalize, source=\"torch\")\n", - "norm(x) # slow, lazy unification\n", - "norm(x) # fast, unified on previous call" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "However, in the following example the unification occurs **eagerly**, and both function calls will be fast:" + "norm_trace(x) # lazy -> eager transpilation as args are provided and the function is called for the first time, executes slowly" ] }, { @@ -240,8 +250,9 @@ { "data": { "text/plain": [ - "ivy.array([-0.34431235, 0.51129461, -0.06686894, -0.36452447, -0.98795534,\n", - " 0.15493582, -0.91630631, 1.41939619, 1.78909753, -1.19475674])" + "" ] }, "execution_count": 6, @@ -250,10 +261,7 @@ } ], "source": [ - "ivy.set_backend(\"tensorflow\")\n", - "norm = ivy.unify(normalize, source=\"torch\", args=(x,))\n", - "norm(x) # fast, unified at ivy.unify\n", - "norm(x) # fast, unified at ivy.unify" + "norm_trace(x) # fast, traced on previous call" ] }, { @@ -261,70 +269,59 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## Trace" + "However, in the following example the tracing occurs **eagerly**, and both function calls will be fast:" ] }, { - "attachments": {}, - "cell_type": "markdown", + "cell_type": "code", + "execution_count": 7, "metadata": {}, + "outputs": [], "source": [ - "The same is true for tracing. In the example below, the function is traced **lazily**, which means the first function call will execute slowly, as this is when the tracing process actually occurs." + "norm_tracing = ivy.trace_graph(normalize, args=(x,)) # eager transpilation as args are provided, executes slowly" ] }, { "cell_type": "code", - "execution_count": 7, + "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" + "array([-0.36293708, 0.53895184, -0.07048605, -0.38424253, -1.04139636,\n", + " 0.16331669, -0.96587166, 1.49617496, 1.88587438, -1.25938418])>" ] }, - "execution_count": 7, + "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ - "norm_trace = ivy.trace_graph(norm)\n", - "norm_trace(x) # slow, lazy graph tracing\n", - "norm_trace(x) # fast, traced on previous call" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "However, in the following example the tracing occurs **eagerly**, and both function calls will be fast:" + "norm_tracing(x) # fast, traced at ivy.trace_graph" ] }, { "cell_type": "code", - "execution_count": 8, + "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" + "array([-0.36293708, 0.53895184, -0.07048605, -0.38424253, -1.04139636,\n", + " 0.16331669, -0.96587166, 1.49617496, 1.88587438, -1.25938418])>" ] }, - "execution_count": 8, + "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ - "norm_tracing = ivy.trace_graph(norm, args=(x,))\n", - "norm_tracing(x) # fast, traced at ivy.trace_graph\n", "norm_tracing(x) # fast, traced at ivy.trace_graph" ] }, @@ -336,17 +333,56 @@ "## Transpile" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's redefine the `normalize` function in PyTorch and transpile it to `TensorFlow`:" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [], + "source": [ + "import torch\n", + "\n", + "\n", + "def normalize(x):\n", + " mean = torch.mean(x)\n", + " std = torch.std(x)\n", + " return torch.div(torch.sub(x, mean), std)" + ] + }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ - "The same is true for transpiling. In the example below, the function is transpiled **lazily**, which means the first function call will execute slowly, as this is when the transpilation process actually occurs." + "In the example below, the function is transpiled **eagerly** and so this cell executes slower because this is when the transpilation process actually occurs." ] }, { "cell_type": "code", - "execution_count": 9, + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Transpilation of normalize complete. \n" + ] + } + ], + "source": [ + "norm_trans = ivy.transpile(normalize, source=\"torch\", target=\"tensorflow\")" + ] + }, + { + "cell_type": "code", + "execution_count": 12, "metadata": {}, "outputs": [ { @@ -357,15 +393,13 @@ " 0.15493582, -0.91630631, 1.41939619, 1.78909753, -1.19475674])>" ] }, - "execution_count": 9, + "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ - "norm_trans = ivy.transpile(normalize, source=\"torch\", to=\"tensorflow\")\n", - "norm_trans(x) # slow, lazy transpilation\n", - "norm_trans(x) # fast, transpiled on previous call" + "norm_trans(x) # fast, transpiled at ivy.transpile" ] }, { @@ -373,31 +407,71 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "However, in the following example the transpilation occurs *eagerly*, and both function calls will be fast:" + "However, in the following example the transpilation occurs *lazily* since we are transpiling a module `kornia` to `jax`, hence the first function call will be slower where the actual transpilation will happen while the subsequent function calls will be fast:" ] }, { "cell_type": "code", - "execution_count": 10, + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "import kornia\n", + "\n", + "jax_kornia = ivy.transpile(kornia, source=\"torch\", target=\"jax\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's actually call a function from our transpiled kornia which will trigger the transpilation to happen eagerly, resulting in the first invocation to be slow but the subsequent ones to be faster" + ] + }, + { + "cell_type": "code", + "execution_count": 3, "metadata": {}, "outputs": [ { - "data": { - "text/plain": [ - "" - ] - }, - "execution_count": 10, - "metadata": {}, - "output_type": "execute_result" + "name": "stderr", + "output_type": "stream", + "text": [ + "WARNING:jax._src.xla_bridge:An NVIDIA GPU may be present on this machine, but a CUDA-enabled jaxlib is not installed. Falling back to cpu.\n" + ] } ], "source": [ - "norm_trans = ivy.transpile(normalize, source=\"torch\", to=\"tensorflow\", args=(x,))\n", - "norm_trans(x) # fast, transpiled at ivy.transpile\n", - "norm_trans(x) # fast, transpiled at ivy.transpile" + "import jax\n", + "import numpy as np\n", + "\n", + "input = jax.numpy.array(np.random.rand(2, 3, 4, 5))\n", + "gray = jax_kornia.color.bgr_to_grayscale(input) # 2x1x4x5" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's run another function from `jax_kornia` to finally wrap up this demo!" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n" + ] + } + ], + "source": [ + "gray = jax_kornia.color.bgr_to_grayscale(input) # 2x1x4x5\n", + "print(type(gray))" ] }, { @@ -413,13 +487,13 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "That's it, you now know the difference between lazy vs eager execution for `ivy.unify`, `ivy.trace_graph` and `ivy.transpile`! Next, we'll be exploring how these three functions can all be called as function decorators!" + "That's it, you now know the difference between lazy vs eager execution for `ivy.trace_graph` and `ivy.transpile`! Next, we'll be exploring how `ivy.trace_graph`can be called as a function decorator!" ] } ], "metadata": { "kernelspec": { - "display_name": "Python 3", + "display_name": "tracer-transpiler", "language": "python", "name": "python3" }, @@ -433,7 +507,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.8.10" + "version": "3.10.13" }, "orig_nbformat": 4 }, diff --git a/learn_the_basics/06_how_to_use_decorators.ipynb b/learn_the_basics/06_how_to_use_decorators.ipynb index 1ac680fb..e9c7c22b 100644 --- a/learn_the_basics/06_how_to_use_decorators.ipynb +++ b/learn_the_basics/06_how_to_use_decorators.ipynb @@ -23,7 +23,7 @@ "source": [ "⚠️ If you are running this notebook in Colab, you will have to install `Ivy` and some dependencies manually. You can do so by running the cell below ⬇️\n", "\n", - "If you want to run the notebook locally but don't have Ivy installed just yet, you can check out the [Get Started section of the docs.](https://unify.ai/docs/ivy/overview/get_started.html)" + "If you want to run the notebook locally but don't have Ivy installed just yet, you can check out the [Get Started section of the docs.](https://www.docs.ivy.dev/overview/get_started.html)" ] }, { @@ -39,7 +39,55 @@ "cell_type": "code", "execution_count": 1, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "WARNING:root:\tSome binaries seem to be missing in your system. This could be either because we don't have compatible binaries for your system or that newer binaries were available. In the latter case, calling ivy.utils.cleanup_and_fetch_binaries() should fetch the binaries binaries. Feel free to create an issue on https://github.com/ivy-llc/ivy.git in case of the former\n", + "\n", + "WARNING:root:\n", + "Following are the supported configurations :\n", + "compiler : cp38-cp38-manylinux_2_17_x86_64, cp38-cp38-win_amd64, cp39-cp39-manylinux_2_17_x86_64, cp39-cp39-win_amd64, cp310-cp310-manylinux_2_17_x86_64, cp310-cp310-win_amd64, cp310-cp310-macosx_12_0_arm64, cp311-cp311-manylinux_2_17_x86_64, cp311-cp311-win_amd64, cp311-cp311-macosx_12_0_arm64, cp312-cp312-manylinux_2_17_x86_64, cp312-cp312-win_amd64, cp312-cp312-macosx_12_0_arm64\n", + "\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "2024-10-19 02:30:39.841075: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.\n", + "2024-10-19 02:30:39.891816: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\n", + "2024-10-19 02:30:39.926198: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\n", + "2024-10-19 02:30:39.936573: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\n", + "2024-10-19 02:30:39.984346: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.\n", + "To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.\n", + "2024-10-19 02:30:41.118217: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT\n", + "WARNING: All log messages before absl::InitializeLog() is called are written to STDERR\n", + "I0000 00:00:1729287053.334256 885678 cuda_executor.cc:1001] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node\n", + "Your kernel may have been built without NUMA support.\n", + "2024-10-19 02:30:53.337020: W tensorflow/core/common_runtime/gpu/gpu_device.cc:2343] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.\n", + "Skipping registering GPU devices...\n" + ] + }, + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 1, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "import ivy\n", "import numpy as np\n", @@ -67,7 +115,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 2, "metadata": {}, "outputs": [], "source": [ @@ -80,18 +128,19 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "" + "" ] }, - "execution_count": 9, + "execution_count": 3, "metadata": {}, "output_type": "execute_result" } @@ -111,7 +160,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 4, "metadata": {}, "outputs": [], "source": [ @@ -124,18 +173,19 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "" + "" ] }, - "execution_count": 11, + "execution_count": 5, "metadata": {}, "output_type": "execute_result" } @@ -145,62 +195,17 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ - "## Graph Transpile" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "In the example below, the `ivy.graph_transpile` function is called as a decorator." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "@ivy.graph_transpile(source=\"torch\", to=\"tensorflow\")\n", - "def normalize(x):\n", - " mean = torch.mean(x)\n", - " std = torch.std(x)\n", - " return torch.div(torch.sub(x, mean), std)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "" - ] - }, - "execution_count": 13, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "normalize(x) # transpilation happens here" + "## Transpile 🚧" ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ - "The function can still be called either *eagerly* or *lazily* when calling as a decorator. The example above is *lazy*, whereas the example below is *eager*:" + "In the future, `ivy.transpile` will be able to be used as a decorator like so:" ] }, { @@ -209,69 +214,34 @@ "metadata": {}, "outputs": [], "source": [ - "@ivy.graph_transpile(source=\"torch\", to=\"tensorflow\", args=(x,))\n", + "@ivy.transpile(source=\"torch\", target=\"tensorflow\")\n", "def normalize(x):\n", " mean = torch.mean(x)\n", " std = torch.std(x)\n", - " return torch.div(torch.sub(x, mean), std)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "" - ] - }, - "execution_count": 15, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "normalize(x) # already transpiled" + " return torch.div(torch.sub(x, mean), std)\n", + "\n", + "# use normalize as a tensorflow function\n", + "normalize(tf.random.uniform((10,)))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "## Transpile 🚧" + "## Round Up" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "In the future, `ivy.transpile` will be able to be used as a decorator like so:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "@ivy.transpile(source=\"torch\", target=\"tensorflow\")\n", - "def normalize(x):\n", - " mean = torch.mean(x)\n", - " std = torch.std(x)\n", - " return torch.div(torch.sub(x, mean), std)\n", - "\n", - "# use normalize as a tensorflow function\n", - "normalize(tf.random.uniform((10,)))" + "That's it, now you are equipped with the basics needed to power up your projects with `ivy`! Next up, let's take a look at some practical examples of how to use `ivy` in your projects." ] } ], "metadata": { "kernelspec": { - "display_name": "Python 3", + "display_name": "tracer-transpiler", "language": "python", "name": "python3" }, @@ -285,7 +255,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.10.14" + "version": "3.10.13" }, "orig_nbformat": 4 }, diff --git a/learn_the_basics/torch_to_tf_functions.ipynb b/learn_the_basics/torch_to_tf_functions.ipynb index b3d7a542..8c1aa4f0 100644 --- a/learn_the_basics/torch_to_tf_functions.ipynb +++ b/learn_the_basics/torch_to_tf_functions.ipynb @@ -11,7 +11,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "You can install the dependencies required for this notebook by running the cell below ⬇️, or check out the [Get Started](https://ivy.dev/docs/overview/get_started.html) section of the docs to find out more about installing ivy." + "You can install the dependencies required for this notebook by running the cell below ⬇️, or check out the [Get Started](https://www.docs.ivy.dev/overview/get_started.html) section of the docs to find out more about installing ivy." ] }, { @@ -44,7 +44,40 @@ "cell_type": "code", "execution_count": 1, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "WARNING:root:\tSome binaries seem to be missing in your system. This could be either because we don't have compatible binaries for your system or that newer binaries were available. In the latter case, calling ivy.utils.cleanup_and_fetch_binaries() should fetch the binaries binaries. Feel free to create an issue on https://github.com/ivy-llc/ivy.git in case of the former\n", + "\n", + "WARNING:root:\n", + "Following are the supported configurations :\n", + "compiler : cp38-cp38-manylinux_2_17_x86_64, cp38-cp38-win_amd64, cp39-cp39-manylinux_2_17_x86_64, cp39-cp39-win_amd64, cp310-cp310-manylinux_2_17_x86_64, cp310-cp310-win_amd64, cp310-cp310-macosx_12_0_arm64, cp311-cp311-manylinux_2_17_x86_64, cp311-cp311-win_amd64, cp311-cp311-macosx_12_0_arm64, cp312-cp312-manylinux_2_17_x86_64, cp312-cp312-win_amd64, cp312-cp312-macosx_12_0_arm64\n", + "\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "2024-10-18 14:51:14.248738: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.\n", + "2024-10-18 14:51:14.406321: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\n", + "2024-10-18 14:51:14.472277: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\n", + "2024-10-18 14:51:14.492596: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\n", + "2024-10-18 14:51:14.630290: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.\n", + "To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.\n", + "2024-10-18 14:51:15.705472: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT\n" + ] + } + ], "source": [ "import ivy\n", "import kornia\n", @@ -62,9 +95,35 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 2, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Transpiling rgb_to_grayscale from torch to tensorflow. This could take a few. " + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "WARNING: All log messages before absl::InitializeLog() is called are written to STDERR\n", + "I0000 00:00:1729245116.107734 357721 cuda_executor.cc:1001] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node\n", + "Your kernel may have been built without NUMA support.\n", + "2024-10-18 14:51:56.112350: W tensorflow/core/common_runtime/gpu/gpu_device.cc:2343] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.\n", + "Skipping registering GPU devices...\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Transpilation of rgb_to_grayscale complete. \n" + ] + } + ], "source": [ "tf_rgb_to_grayscale = ivy.transpile(kornia.color.rgb_to_grayscale, source=\"torch\", target=\"tensorflow\")" ] @@ -109,7 +168,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 4, "metadata": {}, "outputs": [ { @@ -118,7 +177,7 @@ "True" ] }, - "execution_count": 10, + "execution_count": 4, "metadata": {}, "output_type": "execute_result" } @@ -136,7 +195,7 @@ ], "metadata": { "kernelspec": { - "display_name": "Python 3", + "display_name": "tracer-transpiler", "language": "python", "name": "python3" }, @@ -150,7 +209,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.10.14" + "version": "3.10.13" } }, "nbformat": 4, diff --git a/learn_the_basics/torch_to_tf_models.ipynb b/learn_the_basics/torch_to_tf_models.ipynb index 246416f6..b5cbadc2 100644 --- a/learn_the_basics/torch_to_tf_models.ipynb +++ b/learn_the_basics/torch_to_tf_models.ipynb @@ -11,7 +11,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "You can install the dependencies required for this notebook by running the cell below ⬇️, or check out the [Get Started](https://ivy.dev/docs/overview/get_started.html) section of the docs to find out more about installing ivy." + "You can install the dependencies required for this notebook by running the cell below ⬇️, or check out the [Get Started](https://www.docs.ivy.dev/overview/get_started.html) section of the docs to find out more about installing ivy." ] }, { @@ -34,13 +34,37 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 1, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "This model is defined as follows:\n", + "\n", + "class SimpleModel(torch.nn.Module):\n", + " def __init__(self):\n", + " super(SimpleModel, self).__init__()\n", + " self.conv1 = torch.nn.Conv2d(1, 3, kernel_size=3)\n", + " self.relu = torch.nn.ReLU()\n", + " self.fc = torch.nn.Linear(3 * 26 * 26, 10)\n", + "\n", + " def forward(self, x):\n", + " x = self.conv1(x)\n", + " x = self.relu(x)\n", + " x = torch.flatten(x, 1)\n", + " x = self.fc(x)\n", + " return x\n", + "\n" + ] + } + ], "source": [ "from example_models import SimpleModel\n", "\n", - "\"\"\"\n", + "print(\"\"\"\n", "This model is defined as follows:\n", "\n", "class SimpleModel(torch.nn.Module):\n", @@ -56,7 +80,8 @@ " x = torch.flatten(x, 1)\n", " x = self.fc(x)\n", " return x\n", - "\"\"\"" + "\"\"\"\n", + ")" ] }, { @@ -68,9 +93,67 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 2, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "WARNING:root:\tSome binaries seem to be missing in your system. This could be either because we don't have compatible binaries for your system or that newer binaries were available. In the latter case, calling ivy.utils.cleanup_and_fetch_binaries() should fetch the binaries binaries. Feel free to create an issue on https://github.com/ivy-llc/ivy.git in case of the former\n", + "\n", + "WARNING:root:\n", + "Following are the supported configurations :\n", + "compiler : cp38-cp38-manylinux_2_17_x86_64, cp38-cp38-win_amd64, cp39-cp39-manylinux_2_17_x86_64, cp39-cp39-win_amd64, cp310-cp310-manylinux_2_17_x86_64, cp310-cp310-win_amd64, cp310-cp310-macosx_12_0_arm64, cp311-cp311-manylinux_2_17_x86_64, cp311-cp311-win_amd64, cp311-cp311-macosx_12_0_arm64, cp312-cp312-manylinux_2_17_x86_64, cp312-cp312-win_amd64, cp312-cp312-macosx_12_0_arm64\n", + "\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "2024-10-18 14:54:00.304303: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.\n", + "2024-10-18 14:54:00.315374: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\n", + "2024-10-18 14:54:00.329079: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\n", + "2024-10-18 14:54:00.332741: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\n", + "2024-10-18 14:54:00.346229: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.\n", + "To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.\n", + "2024-10-18 14:54:01.007774: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Transpiling SimpleModel from torch to tensorflow. This could take a few minu " + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "WARNING: All log messages before absl::InitializeLog() is called are written to STDERR\n", + "I0000 00:00:1729245274.391513 365152 cuda_executor.cc:1001] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node\n", + "Your kernel may have been built without NUMA support.\n", + "2024-10-18 14:54:34.393241: W tensorflow/core/common_runtime/gpu/gpu_device.cc:2343] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.\n", + "Skipping registering GPU devices...\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Transpilation of SimpleModel complete. \n" + ] + } + ], "source": [ "import ivy\n", "\n", @@ -86,7 +169,7 @@ }, { "cell_type": "code", - "execution_count": 9, + "execution_count": 3, "metadata": {}, "outputs": [ { @@ -95,7 +178,7 @@ "TensorShape([1, 10])" ] }, - "execution_count": 9, + "execution_count": 3, "metadata": {}, "output_type": "execute_result" } @@ -116,7 +199,7 @@ }, { "cell_type": "code", - "execution_count": 10, + "execution_count": 4, "metadata": {}, "outputs": [ { @@ -125,7 +208,7 @@ "TensorShape([1, 10])" ] }, - "execution_count": 10, + "execution_count": 4, "metadata": {}, "output_type": "execute_result" } @@ -138,7 +221,7 @@ ], "metadata": { "kernelspec": { - "display_name": "Python 3", + "display_name": "tracer-transpiler", "language": "python", "name": "python3" }, @@ -152,7 +235,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.10.14" + "version": "3.10.13" } }, "nbformat": 4,