Merge pull request #27 from XpressAI/fahreza/build-docs

🏗️ Script to Pull External Markdown Content

README.md

@@ -32,13 +32,13 @@ $ GIT_USER=<Your GitHub username> USE_SSH=true yarn deploy
 
 If you are using GitHub pages for hosting, this command is a convenient way to build the website and push to the `gh-pages` branch.
 
-### Fetching and Displaying External Markdowns
+### Displaying External Markdowns
 
 You can fetch and display specific sections of external markdown files with customizable error placeholders. This is primarily used for fetching component libraries and project templates readmes. 
 
 Here's a sample of how you can do it. 
 ```
-import ExternalMarkdown from '@site/src/components/ExternalMarkdown';
+import ExternalMarkdown from '@site/src/scripts/ExternalMarkdown';
 
 # Example Markdown
 
@@ -47,4 +47,19 @@ import ExternalMarkdown from '@site/src/components/ExternalMarkdown';
   header="Internal Library"
   placeholder="Content is unavailable or the specified section is not found."
 />
-```
+```
+
+### Fetching External Markdowns
+
+Alternatively, you can also include external markdowns with specific headers and save them as mdx. To do this:
+1. Specify the markdown that you wish to fetch using this template syntax:
+```javascript
+{{your-markdown-url.md|your-header-title}}
+```
+If a header is not supplied, it will fetch the whole markdown content.
+
+2. Rename your markdown files with templates as `.mdtemplate`.
+3. Run the script! 
+```bash
+node src/scripts/MarkdownContentFetcher.js
+```

docs/component-library/index.md

@@ -2,8 +2,6 @@
 sidebar_position: 1
 ---
 
-import ExternalMarkdown from '@site/src/components/ExternalMarkdown';
-
 # Xircuits Component Library
 
 Welcome to the Xircuits Component Libraries documentation! With Xircuits serving as the foundation for your projects, Xircuits Component Libraries provides essential tools and features that enable seamless integration with various frameworks. These libraries are like sets of LEGO pieces, ready to be assembled into any structure limited only by your creativity and imagination. With the ability to mix and match different components, you can create customized workflows that result in unique and powerful applications!

docs/component-library/library-guides/index.mdtemplate

@@ -0,0 +1,7 @@
+# Component Library Guides
+
+This section is a dedicated space where we'll share handy guides and interactive tutorials to help you understand our vast array of component libraries better. We've neatly catalogued both our internal libraries - the ones that are added in the base Xircuits, and external libraries sourced from other repositories, making it easier for you to explore. Dive in below!
+
+{{https://raw.githubusercontent.com/XpressAI/xircuits/master/xai_components/readme.md|Internal Library}}
+
+{{https://raw.githubusercontent.com/XpressAI/xircuits/master/xai_components/readme.md|External Library}}

docs/component-library/library-guides/index.mdx

@@ -0,0 +1,82 @@
+# Component Library Guides
+
+This section is a dedicated space where we'll share handy guides and interactive tutorials to help you understand our vast array of component libraries better. We've neatly catalogued both our internal libraries - the ones that are added in the base Xircuits, and external libraries sourced from other repositories, making it easier for you to explore. Dive in below!
+
+## Internal Library
+
+These components will always be included in the Xircuits installation.
+
+<div align="center">
+
+| Name     | Description                                                                                    |                                                                                                                                                            |
+| -------- | ---------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| Tensorflow Keras | Deep learning related components using Tensorflow Keras.                                       | [Link](https://github.com/XpressAI/xircuits/tree/master/xai_components/xai_tensorflow_keras) |
+| Pytorch  | Components that use Pytorch's ML library.                                                      | [Link](https://github.com/XpressAI/xircuits/tree/master/xai_components/xai_pytorch)   |
+| SKlearn | Scikit learn components. | [Link](https://github.com/XpressAI/xircuits/tree/master/xai_components/xai_sklearn) |
+| XGBoost |  Optimized distributed gradient boosting library. | [Link](https://github.com/XpressAI/xircuits/tree/master/xai_components/xai_xgboost) |
+| Spark    | Components built using the Pyspark library. Run these components with the Xircuits Remote Run! | [Link](https://github.com/XpressAI/xircuits/tree/master/xai_components/xai_spark)       |
+| Template | Great components for new learners and to experiment outputs.                                   | [Link](https://github.com/XpressAI/xircuits/tree/master/xai_components/xai_template) |
+| Utils    | Commonly used components for day-to-day workflows.                                             | [Link](https://github.com/XpressAI/xircuits/tree/master/xai_components/xai_utils)       |
+| Controlflow | Components to enable ifs and loops. | [Link](https://github.com/XpressAI/xircuits/tree/master/xai_components/xai_controlflow) |
+
+</div>
+
+
+
+## External Library
+
+Recognized additional component libraries that you can download and install using:
+
+```
+$ xircuits install <component library name>
+```
+
+<div align="center">
+
+
+### 🤖 AI Agents
+| Name | Description | Version |
+| ---- | ----------- | ------- |
+| [GPT Agent Toolkit](https://github.com/XpressAI/xai-gpt-agent-toolkit) | Experiment with and create Collaborative Large Language Model-based automatons (Agents) in the style of BabyAGI and Auto-GPT. | 0.1.0 |
+| [Vecto](https://github.com/XpressAI/xai-vecto) | The power of vector embeddings in Xircuits. | 0.1.0 |
+| [OpenAI](https://github.com/XpressAI/xai-openai) | Easily integrate OpenAI's functionalities, such as retrieving models and generating text, into your Xircuits projects! | 0.1.0 |
+| [Anthropic](https://github.com/XpressAI/xai-anthropic) | Advanced components for anthropic principles. | - |
+| [HF Agent](https://github.com/XpressAI/xai-hfagent) | Advanced components for hugging face agents. | - |
+| [Actors](https://github.com/XpressAI/xai-actors) | Components for creating actor models. | - |
+| [VertexAI](https://github.com/XpressAI/xai-google-vertexai) | Components that interfaces with Google Vertex AI! | - |
+
+### 📊 Data Science and Machine Learning
+| Name | Description | Version |
+| ---- | ----------- | ------- |
+| [Pycaret](https://github.com/XpressAI/xai-pycaret) | A component library built using Pycaret's API! Complete with examples for NLP, anomaly detection, classification, clustering, as well as regression. | 0.1.0 |
+| [RPA](https://github.com/yuenherny/xai-rpa) | Robotic process automation library to automate routine and repetitive tasks! | 0.2.1 |
+| [Streamlit](https://github.com/XpressAI/xai-streamlit) | Turn Xircuits workflows into shareable web apps in minutes. | 0.1.0 |
+
+### 💬 Communication
+| Name | Description | Version |
+| ---- | ----------- | ------- |
+| [Slack](https://github.com/XpressAI/xai-slack) | Xircuits Component Library to interface with Slack! | 0.1.0 |
+| [Discord](https://github.com/XpressAI/xai-discord) | Create Discord Bots in seconds. | 0.1.0 |
+
+### 📭 Message Queues
+| Name | Description | Version |
+| ---- | ----------- | ------- |
+| [RabbitMQ](https://github.com/XpressAI/xai-rabbitmq) | Message broker in Xircuits that just works. | 0.1.0 |
+| [MQTT](https://github.com/XpressAI/xai-mqtt) | Xircuits library that implements the standard for IoT Messaging. | 0.1.0 |
+
+### 🗃 Databases & Servers
+| Name | Description | Version |
+| ---- | ----------- | ------- |
+| [SQLite](https://github.com/XpressAI/xai-sqlite) | Components for interfacing with SQLite databases. | - |
+| [Flask](https://github.com/XpressAI/xai-flask) | Components for developing Flask applications. | - |
+
+
+### 🌍 Cloud Services
+| Name | Description | Version |
+| ---- | ----------- | ------- |
+| [GDrive](https://github.com/XpressAI/xai-gdrive) | Components for interacting with Google Drive. | - |
+| [GSpread](https://github.com/XpressAI/xai-gspread) | Components for interfacing with Google Sheets. | - |
+
+</div>
+
+Want your component library listed here? Just submit a PR to the Xircuits repository. For more information, refer to the [Xircuits component library documentation](https://xircuits.io/docs/component-library/).

docs/component-library/library-guides/streamlit/index.mdtemplate

@@ -0,0 +1,3 @@
+# Streamlit
+
+{{https://raw.githubusercontent.com/XpressAI/xai-streamlit/main/README.md}}

docs/component-library/library-guides/streamlit/index.mdx

@@ -0,0 +1,24 @@
+# Streamlit
+
+# Xircuits Streamlit Component Library
+
+Build your own [Streamlit](https://streamlit.io/) using Xircuits!
+
+## Prerequisites
+
+You will require Xircuits 1.7+ or a build that includes the streamlit run this [PR](https://github.com/XpressAI/xircuits/pull/206).
+
+## Installation
+
+```
+pip install -r requirements.txt
+```
+
+To use this component library, simply copy the directory / clone or submodule the repository to your working Xircuits project directory.
+
+## Preview
+<details>
+
+https://user-images.githubusercontent.com/68586800/219625051-97b75ee6-fe54-4c42-85e8-ebc5abeaf116.mp4
+
+</details>

docs/main/tutorials/running-a-xircuits-project-template.md

@@ -60,4 +60,4 @@ Once you've setup the project, a Xircuits project template becomes a normal Xirc
   <img src="/img/docs/tutorials/iris_template.gif"></img></p>
 </details>
 
-Congratulations you have successfully ran your first Xircuits project template! From here, you should be able to run any project templates. Be sure to checkout the [Xircuits project template section](../../project-template/index.md) if you want to see the various project template workflows made by our engineers and open source contributors.
+Congratulations you have successfully ran your first Xircuits project template! From here, you should be able to run any project templates. Be sure to checkout the [Xircuits project template section](/docs/project-template/) if you want to see the various project template workflows made by our engineers and open source contributors.

docs/project-template/chatbot.mdtemplate

@@ -0,0 +1,10 @@
+---
+sidebar_position: 6
+tags:
+  - project-template
+---
+
+# Chatbot
+
+{{https://raw.githubusercontent.com/XpressAI/x-template-chatbot/main/README.md}}
+

docs/project-template/chatbot.mdx

@@ -0,0 +1,95 @@
+---
+sidebar_position: 6
+tags:
+  - project-template
+---
+
+# Chatbot
+
+# Chatbot Xircuits Project Template
+
+This template allows you to train a chat model and start a chatbot in a terminal.
+
+It consists of 5 components:
+
+- Dataset preparation
+
+  - [LoadData](https://github.com/XpressAI/x-template-chatbot/tree/main/xai_components/xai_chatbot/chatbot.py#L17) : Load CSV file that provides the `patterns`, `intents`, and `responses`. To edit the training data, you may add/delete row/pattern/response in the existing `resource/sample.csv` or provide your own CSV file path. See more at this [section](#training).
+  - [Tokenize](https://github.com/XpressAI/x-template-chatbot/tree/main/xai_components/xai_chatbot/chatbot.py#L85) : Tokenizer that turns the texts into space-separated sequences of words (by default all punctuations will be removed), and these sequences are then split into lists of tokens.
+
+- Model training
+
+  - [CustomModel](https://github.com/XpressAI/x-template-chatbot/tree/main/xai_components/xai_chatbot/chatbot.py#L134) : Custom neural network model that consists of Embedding layer, a Global Average Pooling 1D layer, a number of Dense layers with _relu_ activation and lastly a Dense layer with _softmax_ activation. Optimizer, loss function and metric can be adjusted.
+  - [Train](https://github.com/XpressAI/x-template-chatbot/tree/main/xai_components/xai_chatbot/chatbot.py#L193) : Train model with defined epochs number and save model at `model_output_path`. `training_sentences` and `training_labels` will be used to train the model.
+
+- Inference
+  - [SingleInference](https://github.com/XpressAI/x-template-chatbot/tree/main/xai_components/xai_chatbot/chatbot.py#L282) : Single sentence inference. This component takes one text input and predicts its intent. Also, gives a response if responses data is provided.
+
+## Prerequisites
+
+Python 3.9
+
+## Installation
+
+1. Clone this repository
+
+```
+git clone https://github.com/XpressAI/template-chatbot
+```
+
+2. Run the setup script that creates a virtual environment and install the required python packages
+
+```
+bash setup.sh
+```
+
+3. Run xircuits from the root directory
+
+```
+xircuits
+```
+
+## Workflow in this Template
+
+### Training
+
+#### [TrainChatModel.xircuits](https://github.com/XpressAI/x-template-chatbot/tree/main/xircuits-workflows/TrainChatModel.xircuits)
+
+Train a chat model that try to predict the intents based on sentences patterns.
+
+<details>
+<summary>What are/Why Patterns and Intents?</summary>
+
+In order to answer questions, search from domain knowledge base and perform various other tasks to continue conversations with the user, a chatbot needs to understand what the users say or what they intend to do (identify user’s intention). The strategy here is to define different intents and make training samples for those intents.
+
+Patterns in our case refer to training samples for different intents. Intents in case are the training categories/labels our model will predict. The model would try to match a particular input with its known intent.
+
+See [resource/sample.csv](https://github.com/XpressAI/x-template-chatbot/tree/main/resource/sample.csv) as dataset example.
+
+</details>
+
+<details>
+<summary>How Can I Change the Training Data?</summary>
+
+To edit the training data, you may add/delete row/pattern/response in the existing `resource/sample.csv`. Or, provide your own CSV file and provide its path at `LoadData` input, `csv_file_path`. The input CSV file should provide these three columns `patterns`, `intents`, and `responses`.
+
+Terms We Use
+
+- `Patterns` are training samples/possible user inputs for corresponding intent.
+- `Intents` are user intentions, also training categories/labels.
+- `Responses` are response texts to send to user after getting the predicted tag from model with user input (only used during inference but not model training). See the workflow [Inference.xircuits](https://github.com/XpressAI/x-template-chatbot/tree/main/xircuits-workflows/Inference.xircuits)
+
+</details>
+
+Example:
+![model_training](https://github.com/XpressAI/x-template-chatbot/raw/main/resource/images/model_training.gif)
+
+### Inference
+
+#### [Inference.xircuits](https://github.com/XpressAI/x-template-chatbot/tree/main/xircuits-workflows/Inference.xircuits)
+
+Single prediction on input text.<br/>
+Example:
+![single_inference](https://github.com/XpressAI/x-template-chatbot/raw/main/resource/images/single_inference.gif)
+
+

docs/project-template/face-detection.mdtemplate

@@ -0,0 +1,9 @@
+---
+sidebar_position: 5
+tags:
+  - project-template
+---
+
+# Face Detection
+
+{{https://raw.githubusercontent.com/XpressAI/x-template-face_detection/main/README.md}}