Skip to content

Commit

Permalink
Updated readme
Browse files Browse the repository at this point in the history
  • Loading branch information
animeshchaudhri authored Aug 20, 2024
1 parent e1f2cbf commit cd3a815
Showing 1 changed file with 98 additions and 31 deletions.
129 changes: 98 additions & 31 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,43 +1,110 @@
# Contents
1. [`Backend`](#Backend)
2. [`Project Description (PictoPy)`](#PictoPy)
# PictoPy

# Backend
## Steps to run backend
PictoPy is an advanced desktop gallery application that combines the power of Tauri, React, and Rust for the frontend with a Python backend for sophisticated image analysis and management.

1. Head over to `/backend` after cloning
2. Create a virtual env by running: `python3 -m venv venv`
3. Activate the env by running `source venv/bin/activate` and install dependancies via `pip install requirments.txt`
4. Start the server via `./run.sh` (for development mode add the flag `--test` [RECOMMENDED])
5. The server should now be hosted on port `8000` on localhost.
## Architecture

## Some pointers for the backend
### Frontend
- **Tauri**: Enables building the desktop application
- **React**: Used for creating the user interface
- **Rust**: Powers the backend, which the frontend communicates with through Tauri's API

- Entry point is `main.py` which calls the routes, currently there is a `/test` route for developing, to test an image, send a request to the backend (`localhost:8000/test/return`) with the following Body format:
```
{
"path": "<Path to image>"
}
### Backend (Python)
- **FastAPI**: Serves as the API framework
- **SQLite**: Database for storing metadata and embeddings
- **YOLO**: Used for object detection
- **FaceNet**: Generates face embeddings
- **ONNX Runtime**: Runs the models efficiently
- **DBSCAN**: Performs clustering for face embeddings

### Backend (Rust via Tauri)
Handles file system operations and provides a secure bridge between the frontend and local system.

## Features

- Smart tagging of photos based on detected objects, faces, and their recognition
- Traditional gallery features of album management
- Advanced image analysis with object detection and facial recognition
- Privacy-focused design with offline functionality
- Efficient data handling and parallel processing
- Smart search and retrieval
- Cross-platform compatibility

## Technical Stack

| Component | Technology |
| --------- | ---------- |
| Frontend | React |
| Desktop Framework | Tauri |
| Rust Backend | Rust |
| Python Backend | Python |
| Database | SQLite |
| Image Processing | OpenCV, ONNX Runtime |
| Object Detection | YOLOv8 |
| Face Recognition | FaceNet |
| API Framework | FastAPI |
| State Management | React Hooks |
| Styling | Tailwind CSS |
| Routing | React Router |
| UI Components | Radix UI |
| Build Tool | Vite |
| Type Checking | TypeScript |

## Setup

### Frontend Setup

#### Prerequisites
- Node.js (LTS version recommended)
- npm (comes with Node.js)
- Rust (latest stable version)
- Tauri CLI

#### Installation
1. Navigate to the frontend directory:
```bash
cd frontend
```
2. Install dependencies:
```bash
npm install
```

#### Running the Application
```bash
npm run tauri dev
```
Eg: try it out with path as`backend/tests/inputs/zidane.jpg`
- A window should pop up showing the highlighted detections, simply press `q` to exit, or comment out the `cv2` lines in `test.py`
- Change the type of inference by changing `model_path` variable in `app/test` route, by default there are two models available in the `app/models` directory (face and object deteciton).
# PictoPy

An open source image sorting tool designed to organize your image collection efficiently.
#### Building for Production
```bash
npm run tauri build
```

## Features to be Implemented
### Python Backend Setup

1. **A dedicated Local Database:**
- A dedicated local database for caching tags, significantly improving the speed and efficiency of image sorting.
#### Installation
1. Navigate to the backend directory:
```bash
cd backend
```
2. Set up a virtual environment (recommended)
3. Install requirements:
```bash
pip install -r requirements.txt
```

2. **Image Recognition Library:**
- Uses an advanced image recognition library for evaluating objects in images along with faces, ensuring a smoother and more accurate sorting process.
#### Running the Backend
For UNIX-based systems:
```bash
./run.sh --test
```

3. **Improved UI/UX:**
- A smooth user interface for a more intuitive and pleasant experience.
The backend should now be running on port 8000 by default.

4. **Multiprocessing Support:**
- Pictopy should supports multiprocessing, optimizing the sorting process and improving overall performance.
## Additional Resources
- [Tauri Documentation](https://tauri.app/v1/guides/)
- [React Documentation](https://reactjs.org/docs/getting-started.html)
- [FastAPI Documentation](https://fastapi.tiangolo.com/)


## Troubleshooting
If you encounter any issues, please check the respective documentation for Tauri, React, and FastAPI. For persistent problems, feel free to open an issue in the project repository.

0 comments on commit cd3a815

Please sign in to comment.